Documentation / Rate Limits

Rate Limits & Usage Policies

Understand how our preservation API handles request throttling, quota allocation, and fair usage across all tiers.

Overview

To ensure the stability of our archival infrastructure and protect fragile historical datasets, all API endpoints are subject to rate limiting. Limits are enforced per API key and are evaluated using a sliding window algorithm.

ℹ️ Why Rate Limits?

Early web preservation requires careful bandwidth management. Rate limits prevent overload on our retro-compatible rendering engines and ensure equitable access for researchers, developers, and institutions.

Tier Allocation

Rate limits scale according to your subscription tier. All limits are enforced per key, not per user or IP address.

Metric Free Tier Researcher Enterprise
Requests per minute 60 600 6,000
Requests per hour 1,000 15,000 150,000
Concurrent connections 2 10 Unlimited
Batch payload size 10 URLs 100 URLs 1,000 URLs
Historical depth 1995–2000 1990–2005 1990–Present

Need a custom allocation for large-scale archival projects? Contact our partnership team for institutional licensing.

Response Headers

Every API response includes standard rate-limit headers to help you monitor your quota in real-time.

HTTP Headers 
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1698765432
Retry-After: 15 (only on 429 responses)
  • X-RateLimit-Limit: Maximum requests allowed in the current window
  • X-RateLimit-Remaining: Requests left before throttle triggers
  • X-RateLimit-Reset: Unix timestamp when the window resets
  • Retry-After: Seconds to wait before retrying (only sent with 429)

Handling 429 Too Many Requests

When you exceed your allocated quota, the API returns a 429 status code. Your application should implement exponential backoff with jitter to avoid thundering herd problems.

⚠️ Important

Repeated 429 responses within a short window may trigger temporary key suspension. Always respect the Retry-After header.

Recommended Retry Strategy

JavaScript 
async function fetchWithRetry(url, retries = 3) {
  try {
    const res = await fetch(url);
    if (res.status === 429) {
      const delay = parseInt(res.headers.get('Retry-After')) * 1000;
      await new Promise(r => setTimeout(r, delay));
      return fetchWithRetry(url, retries - 1);
    }
    return res.json();
  } catch (err) {
    if (retries <= 0) throw err;
    const backoff = Math.min(1000 * Math.pow(2, 3 - retries) + Math.random() * 500, 5000);
    await new Promise(r => setTimeout(r, backoff));
    return fetchWithRetry(url, retries - 1);
  }
}

Best Practices

✅ Pro Tips

Optimize your archival workflows to maximize quota efficiency and reduce unnecessary requests.

  • Cache aggressively: Archive snapshots are immutable. Cache responses by sha256 hash to avoid duplicate fetches.
  • Use batch endpoints: Group up to 100 URLs per request to reduce overhead and count as a single API call.
  • Prefilter with metadata: Use the /metadata endpoint before downloading full pages to check availability and size.
  • Enable webhooks: For large crawls, subscribe to async job completion via webhooks instead of polling.
  • Monitor headers: Always log rate-limit headers in your telemetry to predict quota exhaustion.

Frequently Asked Questions

Can I request a temporary limit increase?

Yes. Submit a request via your dashboard or email api-support@1990archive.io with your project scope. We typically respond within 24 hours.

Are rate limits shared across my team?

Rate limits are tied to the API key, not individual users. If multiple developers use the same key, they share the quota. Generate team-specific keys in the developer portal to isolate usage.

What happens if I hit the daily hard cap?

Requests are gracefully rejected with a 429 response. No data is lost, and your key remains active. Limits reset at 00:00 UTC daily.

}