How to Configure Metrics Persistence and Retention in CCX: Complete Guide

Set METRICS_PERSISTENCE_ENABLED=true and METRICS_RETENTION_DAYS=30 in your environment variables to enable SQLite-backed metric storage with automatic cleanup in the BenedictKing/ccx repository.

The CCX project provides an optional SQLite backend for persisting operational metrics including request success rates, token usage, and circuit breaker states. You control both the persistence toggle and data retention window through environment variables read at startup, with automatic garbage collection removing expired records hourly according to the configuration.

Architecture Overview

CCX implements metrics persistence through a layered architecture defined in the backend-go directory. The system uses environment variable parsing in backend-go/internal/config/env.go to read configuration, initializes the storage backend in backend-go/main.go, and manages the SQLite lifecycle through backend-go/internal/metrics/sqlite_store.go. The persistence interface is abstracted in backend-go/internal/metrics/persistence.go, allowing the metrics manager to operate agnostic of the underlying storage mechanism.

Enabling and Disabling Persistence

By default, CCX creates a SQLite database at .config/metrics.db relative to the working directory and writes metrics to disk. To run in memory-only mode without persistence, set the environment variable before starting the application:

METRICS_PERSISTENCE_ENABLED=false

The application checks this variable in backend-go/main.go during initialization (lines 90‑99):

if envCfg.MetricsPersistenceEnabled {
    metricsStore, err = metrics.NewSQLiteStore(&metrics.SQLiteStoreConfig{
        DBPath:        ".config/metrics.db",
        RetentionDays: envCfg.MetricsRetentionDays,
    })
}

When disabled, the metrics manager operates without the SQLite store, and no disk writes occur.

Configuring Data Retention Days

The retention period determines how long historical metrics remain available before automatic deletion. CCX clamps this value between 3 days (minimum) and 90 days (maximum), defaulting to 30 days.

Set your desired retention window using:

METRICS_RETENTION_DAYS=60

The configuration parsing occurs in backend-go/internal/config/env.go (lines 78‑80), where the clampInt function ensures the value stays within bounds:

MetricsRetentionDays: clampInt(getEnvAsInt("METRICS_RETENTION_DAYS", 30), 3, 90),

This validation prevents configuration errors from causing either immediate data loss (values too low) or excessive disk usage (values too high).

Understanding the Automatic Cleanup Mechanism

The SQLiteStore struct implements two background goroutines for maintenance. The flushLoop writes buffered metrics to disk every 30 seconds, while the cleanupLoop runs hourly to purge expired records.

The cleanup logic resides in backend-go/internal/metrics/sqlite_store.go (lines 71‑79) within the doCleanup method:

func (s *SQLiteStore) doCleanup() {
    cutoff := time.Now().AddDate(0, 0, -s.retentionDays)
    // Executes DELETE FROM metrics WHERE timestamp < cutoff
    s.CleanupOldRecords(cutoff)
}

The retentionDays field, populated from your environment variable during store initialization, calculates the cutoff timestamp using time.Now().AddDate(0, 0, -s.retentionDays).

Customizing SQLite Storage Settings

For advanced deployments requiring custom database paths or programmatic configuration, instantiate the store directly using SQLiteStoreConfig defined in backend-go/internal/metrics/sqlite_store.go (lines 44‑47):

store, err := metrics.NewSQLiteStore(&metrics.SQLiteStoreConfig{
    DBPath:        "/var/lib/ccx/metrics.db",
    RetentionDays: 45,  // Will be clamped to 3-90 range
})

The DBPath accepts any writable file system path, allowing you to place the database on mounted volumes or dedicated storage devices separate from the application root.

Runtime Storage Behavior

When persistence is enabled, CCX outputs initialization logs confirming the storage backend. The database file grows according to your traffic volume and the configured retention window. Upon graceful shutdown, the Close() method ensures all buffered writes flush to disk and the database connection terminates cleanly.

Summary

• Enable persistence by setting METRICS_PERSISTENCE_ENABLED=true (default) or disable it with false to run in memory-only mode.
• Control retention via METRICS_RETENTION_DAYS (3‑90 day range, default 30), parsed in backend-go/internal/config/env.go.
• Automatic cleanup runs hourly via cleanupLoop in backend-go/internal/metrics/sqlite_store.go, deleting records older than the retention window.
• Storage location defaults to .config/metrics.db but accepts custom paths through SQLiteStoreConfig.
• Flush interval batches writes every 30 seconds to balance I/O performance with data durability.

Frequently Asked Questions

Where does CCX store metrics when persistence is enabled?

CCX writes metrics to a SQLite database file located at .config/metrics.db relative to the working directory by default. You can customize this path by providing a different DBPath value when constructing the SQLiteStoreConfig struct in backend-go/internal/metrics/sqlite_store.go or by modifying the initialization code in backend-go/main.go.

How does CCX handle expired metrics data?

The SQLiteStore runs a cleanupLoop goroutine that executes every hour, calling doCleanup to calculate a cutoff timestamp based on retentionDays and deleting all records older than that threshold from the database. This process runs automatically without requiring manual intervention or restart.

Can I change the retention period without restarting CCX?

No. CCX reads METRICS_RETENTION_DAYS once at startup in backend-go/internal/config/env.go and initializes the SQLiteStore with a fixed retention value. To apply a new retention window, you must restart the application after updating the environment variable. The cleanup routine uses the initially configured value for the duration of the process lifecycle.

What is the performance impact of enabling metrics persistence?

Enabling persistence introduces minimal overhead through batch writing (30‑second flush intervals) and hourly cleanup queries. However, high-throughput deployments should monitor the SQLite file size and consider placing the .config/metrics.db file on fast SSD storage or a dedicated volume to prevent I/O contention with the application logs or temporary files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}