Page Scrape
Scrape a URL in multiple formats — markdown, HTML, or plain text.
curl -X POST https://api.hushsearch.io/v1/scrape/ \
-H "X-API-Key: hs_live_YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://en.wikipedia.org/wiki/Reciprocal_rank_fusion",
"formats": [
"markdown"
]
}'Full page scrape with format control. Returns markdown, HTML, and/or plain text along with page metadata. Supports tag inclusion/exclusion for precise content targeting.
Request
curl -X POST https://api.hushsearch.io/v1/scrape/ \
-H "X-API-Key: hs_live_..." \
-H "Content-Type: application/json" \
-d '{
"url": "https://news.ycombinator.com",
"formats": ["markdown", "text"],
"exclude_tags": ["nav", "footer"]
}'Parameters
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
url | string (URL) | ✅ | — | URL to scrape |
formats | string[] | ["markdown"] | Output formats: markdown, html, text | |
include_tags | string[] | all | Only include these HTML tags | |
exclude_tags | string[] | none | Remove these HTML tags before scraping | |
timeout | integer | 15 | Request timeout in seconds |
Response
{
"url": "https://news.ycombinator.com",
"markdown": "# Hacker News\n\n## Ask HN: What are you working on?...",
"text": "Hacker News\nAsk HN: What are you working on?...",
"html": null,
"metadata": {
"title": "Hacker News",
"description": "Hacker News",
"status_code": 200,
"word_count": 842
},
"credits_used": 2,
"credits_remaining": 495,
"response_time_ms": 892
}Python Example
import requests
response = requests.post(
"https://api.hushsearch.io/v1/scrape/",
headers={"X-API-Key": "hs_live_..."},
json={
"url": "https://github.com/trending",
"formats": ["markdown"],
"exclude_tags": ["script", "style", "nav", "footer", "header"]
}
)
data = response.json()
print(f"Title: {data['metadata']['title']}")
print(f"Words: {data['metadata']['word_count']}")
print(data["markdown"][:1000])