Scrapinghub Reference Documentation

Crawlera Old Fetch API (obsolete)

Warning

The Crawlera Old Fetch API is obsolete and only kept to support legacy users. Use instead the Crawlera Proxy API.

Crawlera’s fetch API let’s you request URLs as an alternative to Crawlera’s proxy interface.

Fields

Note

Field values should always be encoded.

Field

Required

Description

Example

url

yes

URL to fetch

http://www.food.com/

headers

no

Headers to send in the outgoing request

header1:value1;header2:value2

Basic example:

curl -u <API key>: http://proxy.crawlera.com:8010/fetch?url=https://twitter.com

Headers example:

curl -u <API key>: 'http://proxy.crawlera.com:8010/fetch?url=http%3A//www.food.com&headers=Header1%3AVal1%3BHeader2%3AVal2'