Monitoring
Operational metrics are exposed on a separate HTTP endpoint configured with CHRONACTA_METRICS_ADDRESS (default 127.0.0.1:9090).
Endpoints
Section titled “Endpoints”| Path | Purpose |
|---|---|
/metrics |
Prometheus exposition format |
/healthz |
Liveness probe |
/readyz |
Readiness probe (engine disk/WAL; cluster leader also requires replication_lag ≤ configured max, default 0) |
Metrics are operational only. Stream storage APIs remain gRPC-only.
RED / USE catalog
Section titled “RED / USE catalog”RED (request-driven paths)
Section titled “RED (request-driven paths)”| Dimension | gRPC append/read | Metric |
|---|---|---|
| Rate | Requests per second | rate(chronacta_append_total[5m]), rate(chronacta_read_total[5m]) |
| Errors | Failed RPCs | rate(chronacta_rpc_errors_total[5m]), rate(chronacta_auth_failures_total[5m]) |
| Duration | Latency | chronacta_append_latency_seconds, chronacta_read_latency_seconds (histogram) |
USE (resource saturation)
Section titled “USE (resource saturation)”| Dimension | Signal | Metric |
|---|---|---|
| Utilization | Active consumers | chronacta_active_subscriptions, chronacta_active_projections |
| Saturation | Backlog / lag | chronacta_replication_lag_positions, chronacta_projection_lag, chronacta_subscription_lag |
| Errors | Engine / projection health | chronacta_engine_ready, chronacta_failed_projections, chronacta_stalled_projections |
Disk and WAL pressure: chronacta_wal_bytes, chronacta_active_segment_bytes (compare with node filesystem via node_exporter).
SLO catalog
Section titled “SLO catalog”| SLI | Source | Target (baseline) | Alert rule |
|---|---|---|---|
| Write availability | chronacta_engine_ready |
Ready ≥ 99.9% monthly | ChronactaNotReady |
| Append latency p99 | chronacta_append_latency_seconds |
< 500ms steady-state | ChronactaAppendLatencyHigh |
| HA leadership | chronacta_cluster_leader |
Exactly one leader in cluster | ChronactaNoClusterLeader |
| Replication lag | chronacta_replication_lag_positions |
< 10k positions | ChronactaReplicationLagHigh |
| Projection freshness | chronacta_stalled_projections |
0 stalled | ChronactaStalledProjections |
| Auth abuse | chronacta_auth_failures_total |
< 1/s sustained | ChronactaAuthFailuresHigh |
| On-disk growth | chronacta_wal_bytes + segments |
< 10 GiB or site policy | ChronactaStorageBytesHigh |
Numeric baselines and workload tiers: performance-methodology.md. HA RPO/RTO: replication.md.
Key metrics
Section titled “Key metrics”chronacta_append_total,chronacta_append_latency_secondschronacta_read_total,chronacta_read_latency_secondschronacta_subscribe_totalchronacta_backup_totalchronacta_wal_sync_latency_seconds,chronacta_segment_sync_latency_secondschronacta_auth_failures_total,chronacta_rate_limit_rejections_total,chronacta_resource_rejections_totalchronacta_rpc_errors_total{method=...}chronacta_global_position,chronacta_wal_bytes,chronacta_active_segment_byteschronacta_active_subscriptions,chronacta_subscription_lagchronacta_active_projections,chronacta_projection_lag,chronacta_failed_projections,chronacta_stalled_projectionschronacta_engine_readychronacta_cluster_leader,chronacta_cluster_epochchronacta_replication_lag_positions,chronacta_replication_lag_by_follower{follower_id}chronacta_replication_append_entries_total,chronacta_replication_append_entries_rejected_total
Disable metrics with CHRONACTA_METRICS_ENABLED=false.
Dashboards and alerts
Section titled “Dashboards and alerts”- Grafana: import grafana/chronacta-overview.json (validation:
make ops-validate) - Prometheus rules: prometheus/chronacta-alerts.yaml
- Structured logs: logging.md
- OpenTelemetry:
CHRONACTA_OTEL_ENABLED,CHRONACTA_OTEL_ENDPOINT
Rate limiting
Section titled “Rate limiting”Enable with CHRONACTA_RATE_LIMIT_ENABLED=true and set CHRONACTA_RATE_LIMIT_RPM (default 600). Limits apply per authenticated username, or per X-Forwarded-For / anonymous key when auth is disabled. Rejections increment chronacta_rate_limit_rejections_total and return RESOURCE_EXHAUSTED.
Example scrape config
Section titled “Example scrape config”scrape_configs: - job_name: chronacta static_configs: - targets: ['127.0.0.1:9090']Rule file example:
rule_files: - /etc/prometheus/chronacta-alerts.yaml
