🕷️ Crawler Tool

Deep-crawl and archive web pages from the early internet era

url
geocities.com angelfire.com tripod.com yahoo.com (1995) altavista.com amazon.com (1995)
🌐

First Web Page

CERN's original 1990 page

📚

GeoCities Recovery

Mass crawl of GeoCities pages

🔗

WebRing Archive

Crawl a WebRing chain

📄

W3C Specifications

HTML 2.0 / 3.2 specs

🎯 Era Detection (Optional)

📟
Pre-Web
1989–1993
💾
Web 1.0 Early
1994–1996
🌈
GeoCities Era
1997–1999
💿
Dot-Com Boom
2000–2001
📱
Post Crash
2002–2004
🔮
Any Era
No filter

⚙ Crawl Settings

▾
Crawl Depth Max link depth to follow
Max Pages Cap on total pages to crawl
Politeness Delay Seconds between requests
User Agent Browser identity for requests

📄 Content Filters

▾
Content Types
☑ HTML
☑ GIF
☑ JPEG
☐ MIDI
☐ AVI
☐ PDF
☐ CSS
☐ JS
Follow Redirects
Respect robots.txt
Archive Subdomains
Detect Era Metadata

⚙ Crawl in Progress

0
Pages
0 KB
Size
0
Errors
0s
Elapsed
0% complete Est. remaining: calculating...
Recent Crawled URLs
crawler-log — 1990webarchive
[--:--:--] INFO Crawler initialized. Ready to accept crawl jobs.
[--:--:--] INFO Configure your settings above and enter a URL to begin archiving.
[--:--:--] INFO Supports: HTML, GIF, JPEG, MIDI, AVI, and legacy formats.
[--:--:--] INFO Tip: Use batch mode to crawl multiple URLs at once. 📥

📊 Crawl Results 0

URL Status Format Size Detected Era Crawled At Actions
🕸️

No crawl results yet

Enter a URL above and start a crawl to see archived pages appear here. Each page is preserved with its original format and metadata.