HushSearchDocs

Site Crawl

Site Crawl API endpoint — 3 credits per call.

CrawlCrawl & mapLong-running: consider /v1/jobs

POST/v1/crawl/3 credits

curl -X POST https://api.hushsearch.io/v1/crawl/ \
  -H "X-API-Key: hs_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://books.toscrape.com/",
  "max_depth": 1,
  "max_pages": 5
}'

Crawl PipelineCrawl & mapLong-running: consider /v1/jobs

POST/v1/crawl/pipeline4 credits

Request
curl -X POST https://api.hushsearch.io/v1/crawl/pipeline \
  -H "X-API-Key: hs_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://books.toscrape.com/",
  "max_pages": 5,
  "formats": [
    "markdown"
  ]
}'

BFS crawl from a starting URL. Follows internal links with configurable depth and page count.

curl -X POST https://api.hushsearch.io/v1/crawl/ \
  -H "X-API-Key: hs_live_..." \
  -d '{"url":"https://docs.example.com","max_depth":2,"max_pages":10}'
ParameterTypeDefaultDescription
urlstringrequiredStarting URL
max_depthinteger2Maximum link depth (BFS)
max_pagesinteger10Max pages to crawl (≤50)
include_patternsstring[]—Regex: URLs to include
exclude_patternsstring[]—Regex: URLs to skip