Bright Data Deploy Integration

Overview

Deploy Bright Data scraping applications to cloud platforms with proper secrets management. Key consideration: Bright Data proxy connections require outbound TCP to brd.superproxy.io on ports 33335 (proxy) and 9222 (Scraping Browser).

Prerequisites

• Bright Data production zone credentials
• Platform CLI installed (vercel, fly, or gcloud)
• Application tested in staging

Instructions

Step 1: Vercel Deployment (Serverless)

bash
# Add secrets
vercel env add BRIGHTDATA_CUSTOMER_ID production
vercel env add BRIGHTDATA_ZONE production
vercel env add BRIGHTDATA_ZONE_PASSWORD production
vercel env add BRIGHTDATA_API_TOKEN production

json
// vercel.json
{
  "functions": {
    "api/scrape.ts": {
      "maxDuration": 60
    }
  }
}

typescript
// api/scrape.ts — Vercel serverless function
import type { VercelRequest, VercelResponse } from '@vercel/node';
import axios from 'axios';
import https from 'https';

export default async function handler(req: VercelRequest, res: VercelResponse) {
  const { url } = req.body;
  if (!url) return res.status(400).json({ error: 'url required' });

  const proxy = {
    host: 'brd.superproxy.io',
    port: 33335,
    auth: {
      username: `brd-customer-${process.env.BRIGHTDATA_CUSTOMER_ID}-zone-${process.env.BRIGHTDATA_ZONE}`,
      password: process.env.BRIGHTDATA_ZONE_PASSWORD!,
    },
  };

  try {
    const response = await axios.get(url, {
      proxy,
      httpsAgent: new https.Agent({ rejectUnauthorized: false }),
      timeout: 55000, // Leave 5s buffer for Vercel's 60s limit
    });
    res.json({ status: response.status, length: response.data.length });
  } catch (error: any) {
    res.status(502).json({ error: error.message });
  }
}

Step 2: Fly.io Deployment (Long-Running)

toml
# fly.toml — better for Scraping Browser (needs WebSocket)
app = "my-scraper"
primary_region = "iad"

[env]
  NODE_ENV = "production"

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

bash
# Set secrets
fly secrets set BRIGHTDATA_CUSTOMER_ID=c_abc123
fly secrets set BRIGHTDATA_ZONE=web_unlocker_prod
fly secrets set BRIGHTDATA_ZONE_PASSWORD=z_prod_pass
fly secrets set BRIGHTDATA_API_TOKEN=prod_token

# Deploy
fly deploy

Step 3: Google Cloud Run

bash
# Store secrets in Secret Manager
echo -n "c_abc123" | gcloud secrets create brightdata-customer-id --data-file=-
echo -n "web_unlocker_prod" | gcloud secrets create brightdata-zone --data-file=-
echo -n "z_prod_pass" | gcloud secrets create brightdata-zone-password --data-file=-

# Deploy with secret mounts
gcloud run deploy scraper \
  --image gcr.io/$PROJECT_ID/scraper \
  --region us-central1 \
  --set-secrets=BRIGHTDATA_CUSTOMER_ID=brightdata-customer-id:latest \
  --set-secrets=BRIGHTDATA_ZONE=brightdata-zone:latest \
  --set-secrets=BRIGHTDATA_ZONE_PASSWORD=brightdata-zone-password:latest \
  --timeout=120 \
  --memory=512Mi

Step 4: Platform Comparison

| Feature | Vercel | Fly.io | Cloud Run | |---------|--------|--------|-----------| | Max timeout | 60s (Pro: 300s) | No limit | 3600s | | WebSocket (Scraping Browser) | No | Yes | No | | Outbound TCP | Yes | Yes | Yes | | Best for | Web Unlocker API | Scraping Browser | Batch scraping | | Cold start | Fast | Configurable | Medium |

Step 5: Health Check Endpoint

typescript
// api/health.ts — works on all platforms
export async function GET() {
  try {
    const proxy = { host: 'brd.superproxy.io', port: 33335, auth: { username: `brd-customer-${process.env.BRIGHTDATA_CUSTOMER_ID}-zone-${process.env.BRIGHTDATA_ZONE}`, password: process.env.BRIGHTDATA_ZONE_PASSWORD! } };
    const start = Date.now();
    const res = await axios.get('https://lumtest.com/myip.json', { proxy, httpsAgent: new (await import('https')).Agent({ rejectUnauthorized: false }), timeout: 15000 });
    return Response.json({ status: 'healthy', proxy_ip: res.data.ip, latency_ms: Date.now() - start });
  } catch {
    return Response.json({ status: 'degraded' }, { status: 503 });
  }
}

Output

• Platform-specific deployment with secrets management
• Health check endpoint verifying proxy connectivity
• Timeout and memory configured for scraping workloads

Error Handling

| Issue | Cause | Solution | |-------|-------|----------| | Vercel timeout | 60s limit too short | Upgrade to Pro (300s) or use Fly.io | | WebSocket refused | Platform blocks WS | Use Fly.io for Scraping Browser | | Cold start timeout | Proxy handshake slow | Configure min instances | | Secrets not found | Wrong env name | Verify with platform CLI |

Resources

• Vercel Serverless Limits
• Fly.io Documentation
• Cloud Run Docs

Next Steps

For webhook handling, see brightdata-webhooks-events.