Using a Callback/Webhook

This example shows how to use the callback/webhook feature to send API requests asynchronously and have the results sent to your own end-point when processing is complete.

Once the website scraping operation is complete the provided callback URL will hve the response data POST’ed to it. You can use various webhook testing providers such as Webhook.site or Webhook Tester to see exactly what the POST data will look like.

Example Request

GET https://api.scrapi.tech/v1/scrape?url=https://deventerprise.com&callbackUrl=https://webhook-test.com/f661e0e4c08dc1cf2449bc9d507b2d8d

POST https://api.scrapi.tech/v1/scrape

{
  "url": "https://deventerprise.com",
  "callbackUrl": "https://webhook-test.com/f661e0e4c08dc1cf2449bc9d507b2d8d"
}

Check Callback Status

When using the callbackUrl parameter the API will return a 202 (Accepted) status with a reference value that can be used to check the status of the callback.

The status is cached for 10 minutes after which it will always return “Unknown” for a reference. Nothing about the callback is permanently stored except the status as it’s executing.

{
  "reference": "2c1e5c6d-d00d-47f1-bab0-de92ed2c81d5"
}

This status check is usually not necessary but can be useful to check if your callback/webhook failed. Another use case would be if you need to poll for completion from a system calling the API that is separate to where the webhook exists.

Check Callback Endpoint

GET https://api.scrapi.tech/v1/scrape/status/{reference}?apiKey=00000000-0000-0000-0000-000000000000

Check Callback Response

The response data contains the status text and the code that represents the status.

{
  "status": "Complete",
  "code": 4
}

Possible Response Codes

Testing

If you want to test this request or a variation of it, head to the Playground or the OpenAPI page.