Comprehensive Digital Preservation Standards for Early Web Artifacts

§1.0 Overview

Digital preservation of early web artifacts requires a multidisciplinary approach combining archival science, web technology expertise, and forward-compatible storage solutions. This guideline establishes the standards for capturing, preserving, and providing access to web content from the formative decade of the World Wide Web.

The primary objective is to ensure that the cultural, historical, and technological significance of early web content remains accessible to researchers, historians, and the public for generations to come. This includes everything from personal GeoCities pages to institutional websites, commercial portals, and community-driven content platforms.

§2.0 Acquisition Protocols

2.1 Crawling Methodology

All acquisition must follow the Heritrix 3.x-based crawling framework configured with era-appropriate user-agent strings and protocol support. Crawlers must be configured to handle the following early web characteristics:

1. Table-based layouts without CSS
2. Frames and iframe implementations
3. CGI-based dynamic content where static snapshots are possible
4. External resource dependencies (inline images, MIDI files, animated GIFs)
5. Non-standard character encodings (ISO-8859-1, Shift_JIS, EUC-KR)

2.2 Capture Frequency

Capture intervals are determined by the volatility classification of the target site:

Classification	Frequency	Examples
Static Archives	Annually	Museum pages, personal homepages
Community Sites	Quarterly	WebRings, fan sites, discussion boards
Active Commerce	Monthly	E-commerce stores, news portals
High-Volatility	Weekly	Chat rooms, message boards, blogs
At-Risk	Continuous	Sites with shutdown notices

§3.0 Format Standards

3.1 HTML Preservation

All HTML documents must be preserved in their original source form, including whitespace, comments, and non-standard markup. Additionally, a normalized version should be generated for rendering purposes:

3.2 Media File Preservation

All media files must be preserved in their original format. Transcoders should not be applied during the initial preservation phase. Supported formats include:

• Images: GIF (including animated), JPEG, BMP, TIFF, PNG (late 90s)
• Audio: MIDI/SMF, AU, WAV, early MP3
• Documents: PDF (1.0–1.3), PostScript, plain text
• Scripts: JavaScript (pre-ES3), VBScript, CGI scripts
• Plugins: ActiveX controls, Java applets, NPR plugins

§4.0 Metadata Schema

Every archived artifact must be accompanied by metadata following the WARP-MD (Web Artifact Preservation Metadata) schema, version 3.2. This schema extends PREMIS and Dublin Core with web-specific fields:

§5.0 Long-Term Storage

5.1 Storage Architecture

The archive employs a triple-redundancy storage model across geographically distributed data centers. Each artifact exists in three independent copies, with regular integrity verification:

1. Primary Storage: High-performance SSD arrays for active access
2. Secondary Storage: Enterprise HDD arrays for bulk preservation
3. Tertiary Storage: M-DISC optical media (250-year rated) for cold storage

5.2 Integrity Verification

All stored artifacts undergo automated fixity checks using SHA-256 checksums. Verification occurs at the following intervals:

• Daily: Random 5% sample of recently ingested content
• Monthly: Full scan of active-access collection
• Annually: Complete collection-wide integrity audit
• Per-access: Checksum validation before serving archived content

§6.0 Access and Rendering

6.1 Authentic Rendering

The archive provides three rendering modes for viewing archived content:

1. Authentic Mode: Renders using era-appropriate browser engines (Netscape 3.0, IE 4.0) via emulation layers, preserving the original visual experience
2. Normalized Mode: Renders using modern standards while preserving original layout intent, making content readable in contemporary browsers
3. Source View: Displays the raw HTML source code with syntax highlighting and line-by-line annotation

6.2 API Access

Researchers and developers can access archived content programmatically through our RESTful API. The API supports filtering by era, technology, platform origin, and content characteristics:

§7.0 Ethical Considerations

The preservation of personal web content raises important ethical and privacy considerations. This guideline establishes the following principles:

• Informed Archival: Where feasible, content creators are notified of archival activities and provided with opt-out mechanisms
• Privacy Redaction: Personally identifiable information (PII) such as home addresses and phone numbers can be redacted upon request
• Cultural Sensitivity: Content from indigenous communities and culturally significant sites is handled in consultation with community representatives
• Right to Be Forgotten: Legitimate requests for content removal are processed within 30 business days, except where content has been cited in published research

§8.0 References & Related Standards

This guideline references and builds upon the following established standards and best practices:

1. PREMIS Data Dictionary for Preservation Metadata (Version 3.0)
2. Dublin Core Metadata Element Set (DCMI)
3. ISO 14721:2012 — Reference model for an Open Archival Information System (OAIS)
4. W3C Web Archiving Community Group — Best Practices
5. Library of Congress — Guidelines for Digitizing Born-Digital Formats
6. International Internet Preservation Consortium (IIPC) — Recommendations