Storage Backends for the Memory Core Service: COS, SQLite, FS, and Memory Explained

The Memory Core service supports four pluggable storage backends: COS (Tencent Cloud Object Storage) for distributed production environments, SQLite for local file persistence, FS for plain filesystem storage, and Memory for ephemeral in-process storage.

The Memory Core service, part of the TencentCloud/TencentDB-Agent-Memory repository, persists conversational memory and state data through a flexible storage abstraction shared with the Memory Proxy component. Understanding these storage backends is essential for deploying the system correctly across development, testing, and production environments. All backend options are centrally defined in the shared type declarations used by both services.

Available Storage Backends for Memory Core

The storage backend is declared in MemoryProxy/src/types.ts through the StorageConfig interface, which accepts one of four string literals for the backend field: "cos", "sqlite", "fs", or "memory". Each backend offers distinct trade-offs between durability, scalability, and operational complexity.

COS (Tencent Cloud Object Storage)

COS is the only distributed backend capable of sharing data across multiple service instances. It stores data as objects in a Tencent Cloud Object Storage bucket, providing durable persistence and horizontal scalability. According to the source code in MemoryProxy/src/storage/factory.ts (lines 32-165), COS is the default choice for production clusters because it maintains consistency across multi-node deployments.

SQLite

SQLite provides local file-based persistence using a single SQLite database file. This backend is process-local and suitable for single-node development environments, small-scale deployments, or scenarios requiring ACID compliance without external dependencies. Data survives process restarts but cannot be shared across nodes.

FS (Filesystem)

FS writes data directly to ordinary files under a configurable directory path. Like SQLite, this is a process-local backend best suited for debugging scenarios or legacy setups requiring simple file-based access patterns. The backend depends on the underlying filesystem for durability.

Memory

Memory maintains all data in-process using RAM only. This ephemeral backend provides the fastest access but offers no durability after process termination, making it ideal for unit tests, continuous integration pipelines, and throw-away instances.

How Backend Selection Works

The backend instantiation logic resides in MemoryProxy/src/storage/factory.ts. The createStorage() function implements a factory pattern that materializes the requested backend according to the configuration's priority list.

If the requested backend cannot be initialized—for example, if COS credentials are missing—the factory degrades gracefully to the next available process-local backend. This fallback mechanism at lines 100-148 ensures that accidental fallback to non-distributed backends does not silently compromise data consistency. When degradation occurs, the factory logs warnings that the effective backend is process-local and unsafe for multi-node clusters.

Configuration Examples

You configure the storage backend through YAML or JSON configuration files consumed by both Memory Core and Memory Proxy.

COS Configuration

storage:
  enabled: true
  backend: cos
  cos:
    bucket: my-memory-bucket
    region: ap-guangzhou
    pathPrefix: memory/

SQLite Configuration

storage:
  backend: sqlite
  sqlite:
    file: /var/lib/memory/memory.sqlite

Memory Configuration

storage:
  backend: memory
  # No additional settings required

Backend Comparison

Backend	Persistence	Multi-node Support	Typical Use Case
cos	Durable (object storage)	Yes (shared across nodes)	Production clusters
sqlite	Durable (local file)	No (single-node only)	Development, small-scale
fs	Durable (filesystem)	No (depends on shared disk)	Debugging, legacy setups
memory	Ephemeral (RAM)	No (process-local)	Unit tests, CI/CD

Programmatic Backend Usage

While configuration files handle most deployments, you can instantiate backends directly in code for custom integrations.

Using the Factory Method

import { createStorage } from "./src/storage/factory";
import { loadConfig } from "./src/config";

async function initialize() {
  const cfg = await loadConfig();
  const storage = await createStorage(cfg.storage);
  // storage implements the unified ProxyStorage interface
}

Direct COS Backend Initialization

import { CosStorage } from "./src/storage/cos-storage";
import { _kernelStsFactory } from "./src/kernels/kernel-sts";

const cosBackend = _kernelStsFactory(cfg.cos);
const storage = new CosStorage(cosBackend);

await storage.putObject("team/12345/memory.json", JSON.stringify(data));
const retrieved = await storage.getObject("team/12345/memory.json");

Direct SQLite Backend Initialization

import { SqliteStorage } from "./src/storage/sqlite-storage";

const storage = new SqliteStorage("/tmp/memory.sqlite");
await storage.init(); // Creates DB if needed

Summary

• The Memory Core service supports four storage backends: COS, SQLite, FS, and Memory, defined in MemoryProxy/src/types.ts.
• COS is the only backend that supports multi-node deployments; all others are process-local and unsuitable for scaled-out production clusters.
• The factory at MemoryProxy/src/storage/factory.ts handles backend instantiation with graceful degradation to local backends when distributed storage is unavailable (lines 100-148).
• Configure backends through the storage.backend field in your service configuration, with additional parameters required for COS and SQLite.
• Process-local backends provide faster access for development but lack the durability and consistency required for production multi-node setups.

Frequently Asked Questions

Which storage backend should I use for production?

Use COS (Tencent Cloud Object Storage) for all production deployments. It is the only backend in TencentDB-Agent-Memory that safely shares data across multiple service instances. SQLite, FS, and Memory backends are process-local and will cause data inconsistencies if multiple nodes attempt to coordinate state.

Can I switch from SQLite to COS without losing data?

No direct migration path is implemented in the current codebase. The backends use different storage formats—SQLite stores structured relational data while COS stores JSON objects. To migrate, you must export data from your SQLite file and upload it to the COS bucket using the expected key structure defined in MemoryProxy/src/storage/cos-storage.ts.

Why does my service fall back to SQLite when I configured COS?

The factory logic in MemoryProxy/src/storage/factory.ts implements graceful degradation at lines 100-148. If COS initialization fails due to missing credentials, network issues, or invalid bucket configuration, the service attempts to start with the next available process-local backend to prevent total service failure. Check your logs for warnings indicating the effective backend has changed, and verify your COS configuration including bucket, region, and authentication tokens.

Is the Memory backend suitable for any production use cases?

No. The Memory backend stores all data in RAM and loses everything when the process terminates or restarts. It is strictly for unit tests, development debugging, and CI/CD pipelines where data persistence is not required and maximum performance is desired.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}