HushSearchDocs

Async Jobs

Long-running research, crawl, batch-scrape, and multi-extract operations — poll or webhook on completion.

PlatformLong-running: consider /v1/jobs

POST/v1/jobs/5 credits / job (research 8 · crawl 5 · batch 3 · multi-extract 4)

Request
curl -X POST https://api.hushsearch.io/v1/jobs/ \
  -H "X-API-Key: hs_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "type": "crawl",
  "params": {
    "url": "https://books.toscrape.com/",
    "max_pages": 5
  },
  "webhook_url": null
}'

POST /v1/jobs/ — cost varies by job type

Submit long-running work that returns immediately with a job ID. Poll GET /v1/jobs/{id} or configure a webhook_url for completion notifications.

Job types

TypeCreditsDescription
research8Multi-step research agent
crawl5BFS site crawl
batch_scrape3Scrape up to 20 URLs
multi_extract4Multi-URL structured extract (per URL)

Create job

{
  "type": "multi_extract",
  "params": {
    "urls": ["https://example.com"],
    "schema_fields": ["title"],
    "ground_query": "example domain"
  },
  "webhook_url": "https://your.app/hooks/job-done"
}

Response (202 Accepted)

{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "type": "multi_extract",
  "status": "pending",
  "created_at": 1719859200.0,
  "updated_at": 1719859200.0,
  "progress": 0.0,
  "result": null,
  "error": null,
  "credits_used": 4
}

Poll status

curl https://api.hushsearch.io/v1/jobs/{id} \
  -H "X-API-Key: YOUR_API_KEY"

Lifecycle

pending → running → completed | failed | cancelled

Progress is 0.0–1.0. On completion, result contains the endpoint payload; on failure, error describes the cause.

Notes

  • Jobs are stored in Redis with a 24-hour TTL.
  • Kafka-backed workers handle research, crawl, and batch_scrape when async mode is enabled.
  • multi_extract runs in-process on core-api with background task execution.