Scrapinghub Documentation

Scrapinghub Documentation

Scrapinghub is the most advanced platform for deploying and running web crawlers (also known as spiders or scrapers). It allows you to build crawlers easily, deploy them instantly and scale them on demand, without having to manage servers, backups or cron jobs. Everything is stored in a highly available database and retrievable using an API.

Spiders can be written in Python using Scrapy, built visually using Portia tool or both. Spiders are grouped into projects. Each spider run is known as a job.

Here you will find reference documentation. For more articles, guides and other help resources please visit our Knowledge Base in our Support center.

Platform Concepts, Features & Tools