Usage & Log Data Management

Overview

CloudNexus provides comprehensive visibility into your infrastructure through real-time usage metrics and centralized log aggregation. This guide covers how to monitor resource consumption, structure log data, query historical events, and optimize costs.

Understanding Usage Metrics

All cloud resources generate telemetry data that powers billing, scaling decisions, and performance insights. Metrics are categorized by resource type:

Metrics exceeding 90% utilization for 15 consecutive minutes trigger automated scaling recommendations in the dashboard.

Log Data Structure

All logs are standardized to JSON format and enriched with metadata before storage. The base schema ensures consistency across compute, network, and service layers.

{
  "timestamp": "2025-11-12T14:32:18.429Z",
  "level": "INFO",
  "service": "cloudnexus.compute.v2",
  "trace_id": "tx-8f3a9c21-4b5d-4e8a-9c12",
  "resource_id": "i-0a1b2c3d4e5f",
  "region": "us-east-1",
  "message": "Auto-scaling triggered: CPU avg 87% over 5m window",
  "metadata": {
    "cpu_percent": 87.4,
    "instance_type": "c6g.xlarge",
    "action": "scale_up",
    "tags": ["production", "web-tier"]
  }
}

Core Fields

• timestamp: ISO 8601 UTC format with millisecond precision
• level: DEBUG, INFO, WARN, ERROR, CRITICAL
• trace_id: Distributed tracing identifier (correlates across services)
• metadata: Arbitrary key-value pairs for custom instrumentation

Querying & Filtering Logs

Use the CloudNexus Log Query Language (CLQL) to filter, aggregate, and analyze log streams. The API endpoint accepts JSON payloads with query operators.

curl -X POST https://api.cloudnexus.io/v2/logs/query \
  -H "Authorization: Bearer $CLOUDNEXUS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "time_range": {
      "start": "2025-11-12T00:00:00Z",
      "end": "2025-11-12T23:59:59Z"
    },
    "filters": [
      { "field": "level", "op": "eq", "value": "ERROR" },
      { "field": "resource_id", "op": "starts_with", "value": "i-0a1b" }
    ],
    "limit": 100,
    "sort_by": "timestamp",
    "sort_order": "desc"
  }'

Exporting & Integrations

Stream or export usage and log data to external systems for advanced analytics, SIEM integration, or compliance archival.

• S3-Compatible Export: Hourly compressed JSONL dumps to designated buckets
• Webhooks: Real-time event forwarding to HTTPS endpoints
• Public Integrations: Datadog, Splunk, Grafana Cloud, AWS CloudWatch
• CLI Export: cnx logs export --format csv --date-range last-7d

Data Retention Policies

Log and usage data retention is tiered by plan and configurable per resource group. Default retention ensures compliance while managing storage costs.

Data exceeding retention windows is permanently purged. Export before deletion if archival is required.

Best Practices

Frequently Asked Questions

How is network egress calculated?

Egress is measured at the edge gateway after compression and deduplication. Intra-region traffic is free; cross-region transfers apply standard tiered rates.

Can I customize log retention for specific resources?

Yes. Use the retention_policy attribute in the resource configuration or the Dashboard → Logs → Retention Settings page. Minimum is 7 days.

Why are some logs delayed in the dashboard?

Real-time ingestion targets <2s latency, but high-volume bursts may experience 5-15s buffering. The API query endpoint always reflects committed data.

Are logs encrypted at rest?

Yes. All log data is encrypted using AES-256-GCM with per-tenant rotation. Transit uses TLS 1.3 with mutual authentication for API access.