HushSearchDocs

Page Scrape

Scrape a URL in multiple formats — markdown, HTML, or plain text.

Scrape & extractFree demo, no key

POST/v1/scrape/2 credits

curl -X POST https://api.hushsearch.io/v1/scrape/ \
  -H "X-API-Key: hs_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://en.wikipedia.org/wiki/Reciprocal_rank_fusion",
  "formats": [
    "markdown"
  ]
}'

Full page scrape with format control. Returns markdown, HTML, and/or plain text along with page metadata. Supports tag inclusion/exclusion for precise content targeting.

Request

curl -X POST https://api.hushsearch.io/v1/scrape/ \
  -H "X-API-Key: hs_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://news.ycombinator.com",
    "formats": ["markdown", "text"],
    "exclude_tags": ["nav", "footer"]
  }'

Parameters

ParameterTypeRequiredDefaultDescription
urlstring (URL)✅—URL to scrape
formatsstring[]["markdown"]Output formats: markdown, html, text
include_tagsstring[]allOnly include these HTML tags
exclude_tagsstring[]noneRemove these HTML tags before scraping
timeoutinteger15Request timeout in seconds

Response

{
  "url": "https://news.ycombinator.com",
  "markdown": "# Hacker News\n\n## Ask HN: What are you working on?...",
  "text": "Hacker News\nAsk HN: What are you working on?...",
  "html": null,
  "metadata": {
    "title": "Hacker News",
    "description": "Hacker News",
    "status_code": 200,
    "word_count": 842
  },
  "credits_used": 2,
  "credits_remaining": 495,
  "response_time_ms": 892
}

Python Example

import requests

response = requests.post(
    "https://api.hushsearch.io/v1/scrape/",
    headers={"X-API-Key": "hs_live_..."},
    json={
        "url": "https://github.com/trending",
        "formats": ["markdown"],
        "exclude_tags": ["script", "style", "nav", "footer", "header"]
    }
)

data = response.json()
print(f"Title: {data['metadata']['title']}")
print(f"Words: {data['metadata']['word_count']}")
print(data["markdown"][:1000])