03: FTP & File Transfer Documentation

File Transfer Protocol (FTP) was the backbone of content distribution and web publishing during the formative years of the internet. The 1990 Web Archive maintains specialized crawlers and emulators to capture, preserve, and restore FTP-based sites and file repositories exactly as they existed between 1990 and 1999.

This guide details how our archival infrastructure handles FTP connections, directory indexing, binary vs. ASCII transfer modes, and authentication methods used during the era.

Historical Context

Before HTTP became the dominant protocol for public-facing content, FTP served as the primary method for sharing software, documentation, and early web assets. Many universities, research institutions, and independent publishers hosted read-only FTP mirrors that functioned as de facto websites.

Our archive preserves these FTP namespaces by generating static directory listings (often styled with classic HTML index templates) and maintaining the original file hierarchy. Directories are indexed using legacy `INDEX.html` generators that mimic tools like ftppage and webgen.

Connection & Transfer Modes

FTP operates using a two-channel architecture: a control connection for commands and a data connection for file transfer. During the 1990s, two primary modes were used:

Active Mode (PORT): Client opens a data port and informs the server. Firewalls often blocked this due to incoming connection requirements.
Passive Mode (PASV): Server opens the data port and tells the client to connect. Became standard as NAT and early firewall deployments increased.

Archival Note: Our emulation layer forces PASV mode for all modern retrieval requests to ensure compatibility behind NAT, while preserving the original PORT/PASV behavior in historical snapshots.

Authentication Methods

Method	Usage Era	Archive Handling
`anonymous`	1990–1999	Preserved with original email prompts. Captured as public read-only.
Standard Username/Password	1992–1999	Credentials redacted. Directory structure & files preserved if accessible via mirrors.
Guest Accounts	1994–1999	Indexed separately. Session logs archived where available.

Preserved File Types & Transfer Modes

FTP distinguishes between ASCII (text) and Binary (IMAGE) transfer modes. Misconfigurations often led to corrupted executables or broken image files. Our archive automatically detects and corrects transfer mode mismatches during restoration.

ASCII Mode: HTML, TXT, PL, C, SH, LOG
Binary Mode: EXE, ZIP, GZ, TAR, GIF, JPG, MIDI, WAV, PDF
Auto-Detect: Archives with unknown extensions are stored in both modes and validated against CRC32 checksums where available.

Terminal Examples

Below are typical FTP sessions archived from the period. Commands and server responses have been preserved exactly as logged.

                        ftp> open archive.example.edu 21

                        Connected to archive.example.edu.

                        220 (vsFTPd 1.1.3)

                        Name (archive.example.edu:guest): anonymous

                        331 Please specify the password.

                        Password: user@archive.edu

                        230 Login successful. Have fun.

                        ftp> passive

                        Passive mode on.

                        ftp> ls /pub/www/1997/

                        227 Entering Passive Mode (198,51,100,23,19,42)

                        150 Opening ASCII mode data connection.

                        drwxr-xr-x    2 ftp      ftp          4096 Oct 12 14:22 graphics

                        drwxr-xr-x    3 ftp      ftp          4096 Sep 05 09:11 html_src

                        -rw-r--r--    1 ftp      ftp         14288 Nov 01 1997  index.html

                        -rw-r--r--    1 ftp      ftp          4096 Oct 30 1997  under-construction.gif

                        226 Transfer complete.

                        ftp> get /pub/www/1997/index.html

                        local: index.html remote: /pub/www/1997/index.html

                        227 Entering Passive Mode (198,51,100,23,19,45)

                        150 Opening ASCII mode data connection.

                        226 Transfer complete.

Archival snapshots automatically convert these directory listings into navigable HTML interfaces while preserving the original path structure and metadata.

Troubleshooting & Common Issues

503 Bad Sequence of Commands

Often occurs when attempting to change transfer mode after a data connection is already open. Ensure type a or type i is issued before get or put.

Passive Mode Firewall Drops

Early FTP servers sometimes used ephemeral ports outside the expected range. If connections stall during data transfer, verify that your network allows outbound connections to ports 1024–65535, or use our HTTP-proxy fallback which converts FTP directories to web-accessible endpoints.

Corrupted Binary Files

If text files contain ^M or binary files show garbled headers, the wrong transfer mode was likely used. Our archive stores dual-mode copies where checksums are unavailable, allowing automatic fallback during restoration.