Load Balancers

Overview

CloudNexus Load Balancers automatically distribute incoming traffic across healthy backend instances, providing high availability, fault tolerance, and scalability. Every load balancer is provisioned with a static ANYCAST IP address that resolves to the nearest edge location.

Load Balancer Types

CloudNexus supports two types of load balancers, each optimized for different use cases:

Key Features

• Health Checking — Configurable HTTP, TCP, and custom script-based health checks
• SSL/TLS Termination — Managed certificates with automatic renewal
• Layer 7 Routing — Path-based and host-based routing rules
• Rate Limiting — Per-IP and global rate limiting with custom policies
• Sticky Sessions — Cookie-based and IP-hash session persistence
• WebSocket Support — Automatic protocol upgrade for WebSocket connections
• HTTP/3 & QUIC — Native support for the latest protocols
• Real-time Metrics — Latency, request counts, and error rates via the Observability API

Quick Start

Create a load balancer in under a minute using the CLI, API, or Console. Here's how to get started with the CLI:

# Create a new HTTP load balancer
cnx lb create --name production-web --type regional \
  --region us-east-1 \
  --algorithm round-robin \
  --protocol https \
  --backend-port 8080 \
  --health-check-path /healthz \
  --health-check-interval 10 \
  --ssl-mode edge

# Output:
# ✓ Load balancer "production-web" created
#   ID:        lb-8f4a2c1e
#   IP:        203.0.113.42
#   Algorithm: round-robin
#   Status:    provisioning (≈30s)

Once the load balancer is provisioned, attach backend instances:

# Add backend instances to the load balancer
cnx lb backends add --lb production-web \
  --instances vm-a1b2c3,vm-d4e5f6,vm-g7h8i9 \
  --port 8080 \
  --weight 1

# Verify backend status
cnx lb backends list --lb production-web

# Output:
# +-------------+-------------+---------+---------+--------+
# | Instance    | Address     | Port    | Weight  | Status +
# +-------------+-------------+---------+---------+--------+
# | vm-a1b2c3   | 10.0.1.10   | 8080    | 1       | healthy+
# | vm-d4e5f6   | 10.0.1.11   | 8080    | 1       | healthy+
# | vm-g7h8i9   | 10.0.1.12   | 8080    | 1       | healthy+
# +-------------+-------------+---------+---------+--------+

Configuration

Routing Algorithm

The load balancing algorithm determines how traffic is distributed among backends. CloudNexus supports the following algorithms:

Health Checks

Health checks monitor the status of your backend instances. Unhealthy backends are automatically removed from the rotation.

{
  "protocol": "HTTPS",
  "path": "/healthz",
  "port": 8080,
  "interval_seconds": 10,
  "timeout_seconds": 5,
  "unhealthy_threshold": 3,
  "healthy_threshold": 2,
  "expected_status": 200,
  "follow_redirects": true,
  "headers": {
    "X-Health-Check": "true"
  }
}

SSL/TLS Configuration

CloudNexus supports three SSL termination modes:

# Enable SSL with managed certificate
cnx lb ssl enable --lb production-web \
  --domains app.example.com,www.example.com \
  --mode edge \
  --auto-renew

# Upload a custom certificate
cnx lb ssl upload --lb production-web \
  --cert-file ./cert.pem \
  --key-file ./key.pem \
  --chain-file ./chain.pem

Layer 7 Routing Rules

Define fine-grained routing rules to direct traffic to different backends based on the request path, host header, or custom headers.

{
  "rules": [
    {
      "description": "API requests to api-pool",
      "match": {
        "path": {
          "prefix": "/api/v2"
        },
        "methods": ["GET", "POST", "PUT", "DELETE"]
      },
      "action": {
        "type": "forward",
        "backend_pool": "api-pool",
        "rewrite_path": "/"
      }
    },
    {
      "description": "Static assets to storage pool",
      "match": {
        "path": {
          "prefix": "/static/"
        },
        "host": "cdn.example.com"
      },
      "action": {
        "type": "forward",
        "backend_pool": "storage-pool"
      }
    },
    {
      "description": "Default rule for web app",
      "match": {
        "path": {
          "prefix": "/"
        }
      },
      "action": {
        "type": "forward",
        "backend_pool": "web-pool"
      }
    }
  ]
}

API Reference

Manage load balancers programmatically using the REST API. All requests require a valid API key passed via the Authorization header.

Create a Load Balancer

{
  "name": "production-web",
  "type": "regional",
  "region": "us-east-1",
  "algorithm": "round-robin",
  "protocol": "https",
  "ssl": {
    "mode": "edge",
    "domains": ["app.example.com"]
  },
  "health_check": {
    "protocol": "HTTPS",
    "path": "/healthz",
    "port": 8080,
    "interval_seconds": 10,
    "unhealthy_threshold": 3
  }
}

{
  "id": "lb-8f4a2c1e",
  "name": "production-web",
  "type": "regional",
  "ip": "203.0.113.42",
  "status": "provisioning",
  "region": "us-east-1",
  "created_at": "2025-10-14T10:30:00Z",
  "backends": [],
  "stats": {
    "requests_per_second": 0,
    "active_connections": 0
  }
}

List Load Balancers

Update a Load Balancer

Delete a Load Balancer

Manage Backends

Rate Limiting

Protect your backends from traffic spikes and abuse with configurable rate limiting policies. Rules are evaluated in order, and the first matching rule is applied.

{
  "rate_limits": [
    {
      "name": "api-global",
      "type": "global",
      "requests_per_second": 10000,
      "burst_size": 15000,
      "action": "reject",
      "reject_code": 429
    },
    {
      "name": "per-ip-limit",
      "type": "per-source-ip",
      "requests_per_second": 100,
      "burst_size": 200,
      "action": "reject",
      "reject_code": 429
    },
    {
      "name": "trusted-partners",
      "type": "per-source-ip",
      "source_whitelist": [
        "198.51.100.0/24"
      ],
      "requests_per_second": 1000,
      "action": "queue"
    }
  ]
}

Sticky Sessions

For applications that require session persistence, CloudNexus supports two sticky session modes:

Cookie-Based Stickiness

The load balancer injects a cookie (CNXLB by default) into the response. Subsequent requests with the same cookie are routed to the same backend.

{
  "sticky_sessions": {
    "type": "cookie",
    "cookie_name": "CNXLB",
    "ttl_seconds": 3600,
    "secure": true,
    "http_only": true
  }
}

IP Hash Stickiness

Uses the client's IP address to determine the backend. Useful for applications that don't support cookies or require IP-based affinity.

Monitoring & Observability

All load balancer metrics are automatically sent to the CloudNexus Observability pipeline. Key metrics include:

Set up alerts using the Alerting API to be notified when metrics exceed thresholds:

# Create an alert for high error rate
cnx alerts create --name "LB High Error Rate" \
  --metric cnx.lb.errors_total \
  --condition "rate[5m] > 100" \
  --resource lb-8f4a2c1e \
  --channel slack:production-alerts \
  --severity critical

# Create an alert for backend health
cnx alerts create --name "Backend Degraded" \
  --metric cnx.lb.backend_health \
  --condition "healthy < total * 0.5" \
  --resource lb-8f4a2c1e \
  --channel pagerduty:oncall \
  --severity critical

Best Practices

1. Use Health Checks Appropriately

Configure health checks with realistic thresholds. A 10s interval with 3 unhealthy thresholds means a backend is removed after ~30s of continuous failures. Adjust based on your service's RTO requirements.

2. Separate Concerns with Backend Pools

Use named backend pools and Layer 7 routing rules to direct traffic to the right service. This makes it easier to scale individual components and perform rolling deployments.

3. Enable SSL at the Edge

Using edge SSL termination reduces CPU load on your backends and simplifies certificate management. CloudNexus manages all certificate renewals automatically.

4. Set Up Rate Limiting Early

Even if you don't expect high traffic, enable baseline rate limiting to protect against accidental misconfigurations or abuse. You can always adjust limits later.

5. Monitor Backend Health Proactively

Set up alerts for backend health metrics so you're notified before users experience issues. Combine with auto-scaling policies for fully automated recovery.

Troubleshooting

Backend Shows as Unhealthy

Common causes and solutions:

• Health check path returns non-200 — Verify your health endpoint returns HTTP 200. The default expected status is 200, but this can be configured.
• Firewall blocking health checks — Ensure your instance firewall allows traffic from CloudNexus health check IPs. Use the Firewall rules to allow health check ranges.
• Wrong backend port — Confirm the health check port matches your application's listening port.

# From within your backend instance, verify the health endpoint
curl --max-time 5 -v https://localhost:8080/healthz \
  -H "X-Health-Check: true" \
  -H "User-Agent: CloudNexus-HealthChecker/2.0"

# Expected output:
# HTTP/2 200
# content-type: application/json
# {"status": "healthy", "timestamp": "2025-10-14T12:00:00Z"}

High Latency or Timeouts

• Check backend CPU and memory usage — overprovisioned backends respond slower to health checks
• Review connection pool settings — increase keepalive_timeout on your application server
• Enable HTTP/2 multiplexing for more efficient connections

Traffic Not Reaching Backends

• Verify DNS A/CNAME records point to the load balancer's IP address
• Check that routing rules have a catch-all default rule
• Confirm SSL certificate covers the requested domain
• Review firewall and security group rules