Custom Cookies and Headers

This example shows how to include your own custom cookies and/or header values when making a request.

Sometimes you need to pass in certain cookie or header values (sessions, site configuration or captcha tokens for example) in order for a website to render it’s content correctly. Being able to set these is a powerful feature for advanced users. An API response will include and headers and cookies generated by the website you are targeting so you can easily re-use them in subsequent requests. Cookies can also automatically be applied to subsequent requests for you if you when using Session ID’s.

If you are not using a browser to make the request then quite a few additional headers are sent automatically to ensure that these requests do not fail fingerprinting checks.

Example Request

Both cookies and headers are supplied as a dictionary of key/value pairs.

GET https://api.scrapi.tech/v1/scrape?url=http://httpbin.org/cookies&cookies=cookie1=value1;cookie2=value2

GET https://api.scrapi.tech/v1/scrape?url=http://httpbin.org/headers&headers=header1=value1;header2=value2

POST https://api.scrapi.tech/v1/scrape

Example With Cookies

{
  "url": "http://httpbin.org/cookies",
  "cookies": {
    "My-Cookie": "Test 1",
    "Session-ID": "Test 2"
  }
}

Example With Headers

{
  "url": "http://httpbin.org/headers",
  "headers": {
    "My-Header": "Test Header 1",
    "X-SITE-KEY": "Test Header 2"
  }
}

Testing

If you want to test this request or a variation of it, head to the Playground or the OpenAPI page.