System Architecture
Technical overview of the .git platform infrastructure, component topology, data flow, and scaling primitives. Updated for v2.4 release.
Component Registry
Routes, authenticates, and rate-limits all inbound traffic. Implements circuit breakers, request ID injection, and protocol translation.
Configuration Primitives
- Rate limiting: sliding window + leaky bucket
- Auth: JWT validation + service mesh mTLS
- Routing: path-prefix + header-based
- Retry: exponential backoff (max 3)
Stateless build agents orchestrated by a queue service. Each job runs in an isolated ephemeral container with network & volume isolation.
Isolation & Security
- Sandbox: gVisor + seccomp profiles
- Network: per-job VPC endpoints
- Storage: ephemeral tmpfs + persistent artifact mount
- Logs: streamed to centralized pipeline
High-performance git operations layer. Handles packfile generation, delta compression, smart HTTP protocol, and concurrent read/write locking.
Performance Tuning
- Storage: ZFS + compression (zstd)
- Caching: multi-level (CPU cache → Redis → SSD)
- Concurrency: async I/O + lock-free queues
- Replication: eventual consistency across zones
Primary datastore for metadata, user accounts, permissions, and audit logs. Sharded by tenant_id with automated failover.
Schema & Partitioning
- Sharding: hash(tenant_id) across 8 shards
- Replication: synchronous primary + 2 async replicas
- Backups: incremental + PITR via WAL archiving
- Migrations: zero-downtime via expand/contract pattern
Request Lifecycle
Client Initiation
SDK or CLI establishes mTLS connection to edge endpoint. Request carries API key or OAuth token.
Edge & Gateway Processing
WAF inspects payload, CDN terminates TLS, Envoy validates JWT against Auth Service, applies rate limits, and forwards to microservice mesh.
Service Orchestration
Deploy service validates permissions, creates job in Redis queue, and returns 202 Accepted with tracking ID. Async processing begins.
Worker Execution
Kubernetes scheduler provisions ephemeral pod. Worker pulls source, runs pipeline steps, streams logs via WebSocket, and uploads artifacts.
Finalization & Observability
Results written to PostgreSQL, cache invalidated, webhook fired, and metrics exported to Prometheus. Client polls or receives SSE update.
Scaling & High Availability
Stateless services scale based on CPU/memory and custom queue depth metrics. Worker pool scales to 0 during idle periods and ramps to 500+ nodes under load.
All services implement graceful degradation. Database uses synchronous replication with automatic failover. Redis cluster maintains quorum across availability zones.
Critical metadata backed up every 15 minutes with 7-day PITR. Git objects stored in geo-redundant object storage with versioning and lifecycle policies.
Technology Stack
| Layer | Component | Version | Purpose |
|---|---|---|---|
| Language | Rust, Go, TypeScript | Latest LTS | Core engine, services, CLI |
| Runtime | Docker, gVisor, WASM | 24.x+ | Isolation & sandboxing |
| Orchestration | Kubernetes, ArgoCD | 1.29 | Deployment & GitOps |
| Networking | Envoy, Linkerd | 1.28 / 2.13 | Proxy & service mesh |
| Database | PostgreSQL, Citus | 16 / 12 | Metadata & sharding |
| Cache/Queue | Redis, NATS | 7.2 / 2.9 | Sessions, pub/sub, jobs |
| Storage | S3-compatible, ZFS | — | Artifacts, LFS, packs |
| Observability | Prometheus, Grafana, ELK | Stable | Metrics, logs, traces |
| CI/CD | GitHub Actions, Bazel | — | Internal build pipelines |