Providence

Overview

Providence is the foundational data layer that powers Dictionary's global language services. It orchestrates the ingestion, verification, normalization, and distribution of over 15 million lexical entries across 100+ languages.

Designed for developers, researchers, and enterprise integrations, Providence guarantees deterministic outputs, versioned datasets, and transparent lineage tracking for every word, definition, and translation.

Data Provenance

Every entry in Providence carries a verifiable lineage trail. We source from licensed academic corpora, peer-reviewed linguistic journals, and validated community contributions. All data undergoes a multi-stage verification process before entering the production index.

Lineage metadata is exposed via the ?include=lineage query parameter. Each record contains source IDs, review timestamps, confidence scores, and revision hashes.

Processing Pipeline

Providence uses a deterministic, batch-and-stream hybrid architecture. Ingestion occurs via secure SFTP, API webhook, and licensed data feeds. The pipeline enforces strict schema validation, deduplication, and semantic normalization.

Failed records are quarantined in the /pipeline/quarantine endpoint for manual review. SLA requires 99.2% successful processing within 4 hours of ingestion.

API Specifications

Providence exposes a RESTful API with OpenAPI 3.0 documentation. All endpoints return JSON, support pagination, and include rate-limit headers. Authentication uses Bearer tokens scoped to dataset access levels.

Example Response Structure

{
  "word": "ephemeral",
  "phonetic": "/əˈfem.ər.əl/",
  "pos": "adjective",
  "definitions": [
    {
      "text": "Lasting for a very short time",
      "source_id": "ACAD-CORP-8821",
      "confidence": 0.98,
      "verified": true
    }
  ],
  "lineage_hash": "sha256:a3f1c9d...",
  "updated": "2025-03-12T08:14:00Z"
}

Reliability & Scale

Providence is deployed across three geographically distributed regions with active-active replication. All data is served via edge caching with a 99.99% uptime SLA for Pro and Enterprise tiers.

Failover is automatic. If a region exceeds error thresholds, traffic is rerouted via DNS failover and edge proxy rules. Status pages and incident reports are published at status.dictionary.com.

Compliance & Ethics

Providence adheres to international data protection standards and maintains rigorous ethical guidelines for linguistic data curation. All processing activities are logged, auditable, and reversible upon request.

We maintain a public Data Ethics Charter outlining our commitment to linguistic diversity, non-commercialization of endangered languages, and equitable representation across global dialects.

Technical FAQ