# Page Scrape

> Scrape a URL in multiple formats — markdown, HTML, or plain text.

Source: http://localhost:3000/docs/endpoints/scrape

## Scrape: `POST /v1/scrape/`

One page as Markdown, HTML or text, with optional query highlights. Cost: 2 credits. Available on the free demo.

```bash
curl -X POST https://api.hushsearch.io/v1/scrape/ \
  -H "X-API-Key: hs_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "url": "https://en.wikipedia.org/wiki/Reciprocal_rank_fusion",
  "formats": [
    "markdown"
  ]
}'
```

Full page scrape with format control. Returns markdown, HTML, and/or plain text along with page metadata. Supports tag inclusion/exclusion for precise content targeting.

## Request

```bash
curl -X POST https://api.hushsearch.io/v1/scrape/ \
  -H "X-API-Key: hs_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://news.ycombinator.com",
    "formats": ["markdown", "text"],
    "exclude_tags": ["nav", "footer"]
  }'
```

### Parameters

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `url` | string (URL) | ✅ | — | URL to scrape |
| `formats` | string[] | | `["markdown"]` | Output formats: `markdown`, `html`, `text` |
| `include_tags` | string[] | | all | Only include these HTML tags |
| `exclude_tags` | string[] | | none | Remove these HTML tags before scraping |
| `timeout` | integer | | 15 | Request timeout in seconds |

## Response

```json
{
  "url": "https://news.ycombinator.com",
  "markdown": "# Hacker News\n\n## Ask HN: What are you working on?...",
  "text": "Hacker News\nAsk HN: What are you working on?...",
  "html": null,
  "metadata": {
    "title": "Hacker News",
    "description": "Hacker News",
    "status_code": 200,
    "word_count": 842
  },
  "credits_used": 2,
  "credits_remaining": 495,
  "response_time_ms": 892
}
```

## Python Example

```python

response = requests.post(
    "https://api.hushsearch.io/v1/scrape/",
    headers={"X-API-Key": "hs_live_..."},
    json={
        "url": "https://github.com/trending",
        "formats": ["markdown"],
        "exclude_tags": ["script", "style", "nav", "footer", "header"]
    }
)

data = response.json()
print(f"Title: {data['metadata']['title']}")
print(f"Words: {data['metadata']['word_count']}")
print(data["markdown"][:1000])
```
