Using Sessions

Using a session identifier will reuse the same contextual information for multiple requests. This means the same IP address, user agent and any cookies collected will apply on each request.

Sessions are useful to avoid having bypass multiple captchas or remain logged into a website because the request does not change and clearance and session cookies will remain. If using a proxy server, your IP address will remain the same for 10 minutes before rotating. This is to ensure IP addresses in the pool are not banned.

If you are creating a “crawler” then sessions are very useful because as you navigate through a site (paging for example) it will appear to be the same user and dramatically reduce your chances of being detected.

To use a session you need to include the sessionId parameter in the request with any value. Any subsequent requests need to use the same session ID value to continue using the same contextual information.

Example Request

GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&sessionId=any_unique_identifier

POST https://api.scrapi.tech/v1/scrape

{
  "url": "https://deventerprise.com",
  "sessionId": "any_unique_identifier"
}

Testing

If you want to test this request or a variation of it, head to the Playground or the OpenAPI page.