VeriScrape

API Documentation

Integrate VeriScrape powerful scraping capabilities into your application with our simple REST API.

Getting Started

Your VeriScrape application acts as a secure gateway to your own scraping service. Your clients will make requests to this app's API, which will then validate their key, check their usage limits, and forward the request to your scraping infrastructure.

To make this work, you must set the `SCRAPING_SERVICE_URL` in your .env.local file.

Authentication

Authenticate your API requests by providing your unique API key in the X-API-Key header. You can find your API key in your dashboard.

X-API-Key: YOUR_API_KEY
Endpoints

POST
/api/v1/scrape

The core endpoint for scraping web pages.

Request Body (JSON)

The body of your POST request should be a JSON object with the parameters your scraping service expects.

{
    "url": "https://www.bbc.com/news/articles/cpvjlj3n1vmo",
    "include_raw_html": false,
    "max_age_hours": 24
}
Code Example
Here's a quick example in Python.
import requests
import json

API_KEY = "YOUR_API_KEY"
# The API URL is the URL of your VeriScrape application
API_URL = "YOUR_VERISCRAPE_APP_URL/api/v1/scrape"

payload = {
    "url": "https://www.bbc.com/news/articles/cpvjlj3n1vmo",
    "include_raw_html": False,
    "max_age_hours": 24
}

headers = {
    "X-API-Key": API_KEY,
    "Content-Type": "application/json"
}

response = requests.post(API_URL, headers=headers, data=json.dumps(payload))

if response.status_code == 200:
    data = response.json()
    print(data)
else:
    print(f"Error: {response.status_code}")
    print(response.text)
Example Response (Success)
{
  "status": "success",
  "url": "https://quotes.toscrape.com/",
  "results": [
    "“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”",
    "“It is our choices, Harry, that show what we truly are, far more than our abilities.”",
    ...
  ],
  "credits_remaining": 999
}
Error Codes
Status CodeMeaning
400 Bad RequestA required parameter is missing or invalid, or the request body contains invalid JSON.
401 UnauthorizedYour API key is missing or invalid.
402 Payment RequiredYou have exceeded your monthly API call limit.
500 Internal Server ErrorSomething went wrong on our end. Please try again later.