Rate Limiting Policy

Guidelines for accessing the 1990 Web Archive API responsibly. Our infrastructure runs on preserved legacy systems, so we enforce strict limits to ensure stability and fair access for all researchers.

Overview

The 1990 Web Archive API uses a sliding window counter algorithm to enforce rate limits per API key and per IP address. Limits are applied per endpoint group (e.g., `/crawl`, `/render`, `/search`) rather than globally, allowing flexible usage patterns while protecting critical archival endpoints.

💡 Why we rate limit: Our storage consists of restored 1990s-era hardware and emulated environments. Excessive requests can cause latency spikes or temporary degradation of historical rendering services. Rate limits protect the integrity of the archive.

Tier Limits

Plan	Requests / Minute	Requests / Day	Burst Allowance
Free	30	5,000	5 extra
Researcher	200	100,000	20 extra
Enterprise	1,000	Unlimited	Custom

Response Headers

Every API response includes rate limit metadata. Use these headers to track your quota and avoid hitting limits.

GET /api/v1/pages/1995/geocities-land-1292
HTTP/1.1 200 OK
X-RateLimit-Limit: 30
X-RateLimit-Remaining: 24
X-RateLimit-Reset: 1718492340
Retry-After: 45
            

X-RateLimit-Limit: Maximum requests allowed in the current window
X-RateLimit-Remaining: Requests left before the window resets
X-RateLimit-Reset: Unix timestamp when the current window expires
Retry-After: Seconds to wait before sending the next request (only present when limited)

Exceeding Limits (429 Response)

When you exceed your rate limit, the API returns HTTP 429 Too Many Requests. The response body contains a detailed error object:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Retry-After: 30

{
  "error": {
    "code": "RATE_LIMIT_EXCEEDED",
    "message": "You have exceeded the allowed request rate for this endpoint.",
    "limit": 30,
    "window": "1m",
    "reset_at": 1718492340,
    "docs_url": "https://1990webarchive.dev/docs/rate-limiting"
  }
}
            

⚠️ Important: The Retry-After header is mandatory. Ignoring it and continuing to send requests may result in temporary IP suspension to protect archive infrastructure.

Best Practices

Implement Exponential Backoff: On 429 responses, wait Retry-After seconds, then retry. If errors persist, double the wait time (max 5 minutes).
Cache Aggressively: Archived pages rarely change. Cache responses using ETag or Last-Modified headers to reduce API calls.
Batch Requests: Use the /api/v1/batch endpoint to fetch multiple pages in a single call (counts as 1 request).
Respect Window Boundaries: Don't attempt to "pre-fetch" before X-RateLimit-Reset. Our sliding window resets dynamically based on traffic patterns.
Monitor Headers: Log X-RateLimit-Remaining to predict when you'll hit limits and adjust crawl intervals automatically.

Requesting a Limit Increase

Need higher throughput for academic research or large-scale digitization projects? We offer custom rate limit upgrades for verified institutions.

Submit a request via api-support@1990webarchive.dev
Include your use case, expected QPS, and data retention policy
Enterprise plans include dedicated archival nodes and SLA-backed uptime

📖 Related Docs: Authentication · Pagination · Error Handling