How Daemon Coordination Works Across Multiple Agent Sessions in codebase-memory-mcp

The codebase-memory-mcp daemon uses a Unix-domain socket rendezvous point, six coordination lock files, and a version-cohort system to ensure only one daemon instance owns a given cache root while allowing multiple agent sessions to share that single daemon.

The codebase-memory-mcp repository implements a singleton per-account daemon that multiplexes requests from multiple concurrent agent sessions (CLI tools or IDE language servers). This architecture relies on precise daemon coordination across multiple agent sessions to prevent resource conflicts, enforce cache consistency, and ensure binary compatibility through file-based locking and Unix-domain socket IPC.

The Rendezvous Socket Architecture

Each user account operates under a unique rendezvous key (c888acc3ae367a1b) that deterministically generates the socket and lock file paths. The Unix-domain socket resides in a temporary runtime directory scoped to the user ID:

/tmp/cbm-daemon-<uid>/cbm-c888acc3ae367a1b.sock

The socket is created lazily: the first client requesting a connection triggers the daemon startup sequence and socket creation, while subsequent clients discover the existing socket path and attach directly. This behavior is validated in tests/test_daemon_smoke.py (lines 1070–1085), where the test suite constructs the socket_path and verifies lock file states before daemon initialization.

Lock File Coordination Strategy

Six distinct lock files protect the daemon lifecycle and enforce the version-cohort contract. All locks reside in the same temporary runtime directory as the socket:

Lock file	Purpose	Lock type
cbm-<key>.lock	Startup lock – ensures only one process can create the daemon socket	Non-record (fcntl)
cbm-<key>.lifetime.lock	Lifetime reservation – held exclusively while the daemon runs	Record-lock
cbm-version-cohort-admission-v1.lock	Admission cohort – guarantees joining processes use the same cache root	Record-lock
cbm-version-cohort-lifetime-v1.lock	Lifetime cohort – prevents a second daemon from starting while the first is alive	Record-lock
cbm-version-cohort-maintenance-v1.lock	Maintenance cohort – reserves exclusive access for updates	Record-lock
cbm-version-cohort-daemon-v1.lock	Daemon cohort – protects the daemon’s build fingerprint and version	Record-lock

The test suite validates these locks via the lock_status() helper function, which inspects file mode, ownership, and lock state to ensure resources are free before daemon startup (see tests/test_daemon_smoke.py, lines 1071–1085).

Client Connection Flow

The coordination protocol follows a strict three-phase attachment process:

1. First Client Initialization – When client c1 sends an initialize request, the binary acquires the startup lock, writes the lifetime lock, spawns the daemon process, and creates the socket. The daemon records a daemon.start event to cbm-daemon.log.

2. Subsequent Client Attachment – Clients c2, c3, and beyond discover the existing socket and obtain a session on the running daemon without triggering a new instance. The smoke tests assert that only one daemon.start event exists even after multiple simultaneous clients connect (lines 1129–1134 in test_daemon_smoke.py).

3. Session Tracking – Each client receives a unique client ID. The daemon maintains counters for daemon_active_clients and daemon_active_connections, which are verified in the daemon.stop audit record (see run_successful_activation() validation at lines 1265–1272).

Version-Cohort Enforcement

The daemon fingerprints its binary using a SHA-256 hash and version string to enforce version cohorts. When a new process attempts to join:

• Cache Root Mismatch: If the CBM_CACHE_DIR environment variable differs from the running daemon’s cache root, the connection is rejected with the error active account daemon uses a different cache directory. A durable conflict record is written to daemon-conflicts.ndjson (validated in lines 1170–1185).

• Binary Build Mismatch: If the SHA-256 fingerprint or version string differs, the daemon emits a daemon.version_conflict event containing both the active and requested versions, then rejects the connection (see the future-generation probe at lines 1146–1154 and 1178–1185).

The cohort lock files (*_cohort_*.lock) ensure that only one version cohort exists for a given cache root, eliminating race conditions where incompatible daemons might compete for resources.

Graceful Shutdown and Recovery

The daemon implements lifecycle management that distinguishes between individual requests and session termination:

• Request Cancellation: When a client sends a cancel notification, the daemon terminates only that request’s worker tree; the client session remains active. The smoke tests verify that invalid cancellation tokens do not close the frontend connection.

• Daemon Termination: When the last client disconnects, the daemon releases the lifetime lock, removes the Unix socket, and writes a daemon.stop event. This allows a fresh daemon to start on the next connection request.

• Crash Recovery: If the daemon process is killed (e.g., SIGKILL), stale lock files are cleared on the next startup attempt. The system supports launching a cold one-shot daemon without error when the cleanup logic detects orphaned locks.

Auditing and Observability

All coordination actions emit JSON-Lines events for forensic analysis:

• daemon.start – Daemon process creation
• daemon.stop – Process termination (includes daemon_active_clients and daemon_active_connections counts)
• daemon.version_conflict – Version or build fingerprint mismatch

These logs are stored under the cache directory at logs/cbm-daemon.log and logs/daemon-conflicts.ndjson with strict 0600 filesystem permissions (verified in tests/test_daemon_smoke.py, lines 1247–1251).

Practical Implementation Examples

Starting Two Concurrent Clients

The following pattern from tests/test_daemon_smoke.py (lines 1060–1130) demonstrates how two clients share a single daemon:

# Initialize two thin clients pointing to the same binary

c1 = McpClient(binary, env, tmpdir / "client-1.err")
c2 = McpClient(binary, env, tmpdir / "client-2.err")
clients.extend([c1, c2])

# Send initialize requests

c1.send({"jsonrpc":"2.0","id":101,"method":"initialize","params":init_params})
c2.send({"jsonrpc":"2.0","id":201,"method":"initialize","params":init_params})
assert_rpc_success(c1.wait_response(101))
assert_rpc_success(c2.wait_response(201))

# Verify single daemon ownership

wait_until(lambda: socket_path.exists() and lock_status(lifetime_lock, True) == "held",
           START_TIMEOUT, "daemon endpoint and lifetime reservation")
assert len(json_events(daemon_log, "daemon.start")) == 1

Handling Cache Root Conflicts

This example from lines 1170–1185 shows the error handling when a client attempts to connect with a mismatched cache directory:

mismatched_env = env.copy()
mismatched_env["CBM_CACHE_DIR"] = "/tmp/other-cache"

result = subprocess.run(
    [str(binary)],
    input=json.dumps({"jsonrpc":"2.0","id":203,
                     "method":"initialize","params":init_params},
                    separators=(",",":")) + "\n",
    env=mismatched_env,
    stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)

assert result.returncode != 0
assert "active account daemon uses a different cache directory" in result.stderr

Detecting Version Conflicts

The test suite probes version compatibility using a future-generation mock (lines 1120–1150):

future_version = "9.0.0-future-wire-v2"
future_fingerprint = "c"*64
probe_future_generation_rendezvous(
    socket_path,
    current_version,
    current_fingerprint,
    future_version,
    future_fingerprint,
)

# Verify conflict logging

assert any(
    ev.get("reason") == "version"
    and ev.get("requested_version") == future_version
    for ev in json_events(conflict_log, "daemon.version_conflict")
)

Summary

• codebase-memory-mcp implements a singleton daemon model where one process serves multiple agent sessions through a Unix-domain socket at /tmp/cbm-daemon-<uid>/cbm-<key>.sock.
• Six lock files coordinate startup, lifetime, and version cohorts, with record-locking semantics ensuring exclusive daemon ownership.
• Version-cohort enforcement rejects connections mismatched on cache root (CBM_CACHE_DIR) or binary build fingerprint, logging conflicts to daemon-conflicts.ndjson.
• Client multiplexing allows simultaneous sessions while tracking active counts in daemon.stop audit records.
• Crash recovery automatically clears stale locks, enabling seamless daemon restarts without manual intervention.

Frequently Asked Questions

What happens if two agents try to start the daemon simultaneously?

The first agent to acquire the startup lock (cbm-<key>.lock) creates the socket and spawns the daemon. The second agent blocks briefly on the lock, then discovers the existing socket and attaches as a secondary client. The test suite verifies this race condition handling in tests/test_daemon_smoke.py, ensuring only one daemon.start event is recorded.

How does the daemon handle version mismatches between clients?

The daemon maintains a SHA-256 fingerprint and version string. Incoming clients must match both the cache root and binary build; otherwise, the connection is rejected with a daemon.version_conflict event. This prevents data corruption from incompatible protocol versions accessing the same index files.

Where are daemon coordination events logged?

All events are written as JSON-Lines to logs/cbm-daemon.log (for lifecycle events) and logs/daemon-conflicts.ndjson (for version and cache conflicts). These files are created with 0600 permissions to protect potentially sensitive path information contained in the logs.

Can the daemon recover automatically after a crash?

Yes. If the daemon terminates unexpectedly (e.g., kill -9), the lifetime lock remains held by the dead process. On the next connection attempt, the startup logic detects the stale lock, clears the orphaned files, and allows a new daemon to bind to the socket. The system also supports cold one-shot daemon modes for recovery scenarios.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}