Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Scrape any webpage as clean markdown via Bright Data Web Unlocker API. Bypasses bot detection and CAPTCHA. Requires BRIGHTDATA_API_KEY and BRIGHTDATA_UNLOCKER_ZONE environment variables.
| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-03 | ✗→✓ | ▲ Improved | -64% | 0% |
| case-05 | ✗→✓ | ▲ Improved | -38% | 0% |
| case-08 | ✗→✓ | ▲ Improved | -37% | 0% |
| case-09 | ✗→✓ | ▲ Improved | -48% | 0% |
| case-11 | ✗→✓ | ▲ Improved | -48% | 0% |
Scrape any webpage and get clean markdown content using Bright Data's Web Unlocker API. Automatically bypasses bot detection and CAPTCHA.
1. Get your API Key: Get a key from Bright Data Dashboard.
2. Create a Web Unlocker zone: Create a zone at brightdata.com/cp by clicking "Add" (top-right), selecting "Unlocker zone".
3. Set environment variables:
bashexport BRIGHTDATA_API_KEY="your-api-key" export BRIGHTDATA_UNLOCKER_ZONE="your-zone-name"
bashbash scripts/scrape.sh "url"
Parameters:
url (required): The webpage URL to scrapeExamples:
bash# Scrape a news article bash scripts/scrape.sh "https://example.com/article" # Scrape a product page bash scripts/scrape.sh "https://shop.example.com/product/123"
Returns clean markdown content extracted from the webpage:
markdown# Page Title Main content of the page converted to markdown format... ## Section Heading More content...
curl - For API requestsOther measured skills in the registry, with their headline benchmark lift.