Scrapinghub Reference Documentation

This is the documentation for Scrapinghub services and products.

Note

For help guides and other articles please check our Help Center.

Crawlera

Crawlera Basics

Start with the basics

Crawlera Proxy API

API to make the most out of the Crawlera smart proxy

Crawlera Sessions

Understand how to use Crawlera sessions to use the same IP across multiple requests

Crawlera Errors Reference

Understand Crawlera errors and their meaning

Crawlera Residential

Learn about the Crawlera Residential addon

Crawlera Stats API

HTTP API to access Crawlera usage data

Crawlera Fetch API

Learn about Crawlera Fetch API to download pages with server-side browser execution

AutoExtract

API

HTTP API to automatically extract structured data from web pages

Scrapy Cloud

HTTP API

HTTP API to interact with spiders, jobs, and other Scrapy Cloud resources

Entry Point API

Write custom Docker images that are compatible with Scrapy Cloud

Unified Schema

Schemas

Our proposal for standard schemas for commonly extracted types of data