Real-Time AI Streams via WebSockets

⚡ Stream Architecture

NexusAI exposes a persistent WebSocket channel that delivers token-by-token inference results, heartbeat signals, and system events without HTTP polling overhead. Ideal for:

• Real-time conversational AI & live translation
• Multi-agent orchestration & swarm coordination
• Live sensor/telemetry processing with on-the-fly ML inference
• Interactive code generation & IDE integrations

🔗 Connection Setup

Initialize a persistent connection using your API key. The server validates credentials within 200ms and establishes a dedicated session channel.

📦 Message Protocol

All messages are JSON-encoded. The stream uses a strict envelope format to differentiate between system events and payload data.

{
  "type": "stream|heartbeat|error|close",
  "id": "msg_uuid",
  "timestamp": 1718905200123,
  "payload": {
    "tokens": ["..."],
    "metadata": { "latency_ms": 42 }
  }
}

Event Types

• stream — Contains generated tokens or inference chunks
• heartbeat — Keep-alive ping (default: 30s interval)
• error — Validation or quota limits
• close — Graceful session termination with final summary

💻 Implementation Examples

const ws = new WebSocket('wss://api.nexusai.io/stream/v3', {
  headers: { 'Authorization': 'Bearer YOUR_API_KEY' }
});

ws.addEventListener('message', (event) => {
  const data = JSON.parse(event.data);
  if (data.type === 'stream') {
    process.stdout.write(data.payload.tokens.join(''));
  }
});

ws.onopen = () => ws.send(JSON.stringify({
  type: 'prompt',
  content: 'Analyze real-time market sentiment'
}));

import asyncio
import websockets
import json

async def stream_inference():
    uri = "wss://api.nexusai.io/stream/v3"
    async with websockets.connect(uri) as ws:
        await ws.send(json.dumps({
            "type": "prompt",
            "content": "Real-time data analysis"
        }))
        async for msg in ws:
            data = json.loads(msg)
            if data["type"] == "stream":
                print("".join(data["payload"]["tokens"]), end="")

asyncio.run(stream_inference())

wscat -c wss://api.nexusai.io/stream/v3 \
  --header "Authorization: Bearer YOUR_API_KEY" \
  --header "X-Nexus-Model: nexus-v3-ultra"

📊 Live Performance

Real-time telemetry from the edge network. Metrics update every 2 seconds.

🛡️ Error Handling & Retries

The client SDK implements exponential backoff with jitter. Network drops trigger automatic reconnection within 500ms. Server-initiated closes include a retry_after_ms field when applicable.

{
  "type": "error",
  "code": "RATE_LIMIT_EXCEEDED",
  "retry_after_ms": 1200,
  "message": "Stream quota reached for tier"
}

⏱️ Rate Limits & Quotas

Exceeding limits returns a 429 close code with precise backoff instructions.