Single-Committer Git Pattern: How to Prevent .git/index.lock Corruption in Multi-Agent Systems

The single-committer git pattern is a concurrency design where only one dedicated process writes to a Git repository, eliminating lock contention by ensuring all git add and git commit operations run serially while other agents write only to workspace files.

In the chaitanyagiri/munder-difflin repository, this pattern solves a critical reliability issue when multiple autonomous agents share a single Git working directory. By isolating Git operations to a solitary "committer" process, the system prevents the race conditions and stale lock files that typically corrupt repositories under concurrent access.

What Is the Single-Committer Git Pattern?

The single-committer pattern enforces a strict write-serialization boundary around Git operations. In this architecture:

• Worker agents write plain files to their own workspace directories but never invoke Git commands
• Committer process (implemented in src/main/hive.ts) exclusively handles staging and committing
• Queue semantics ensure mutations accumulate as filesystem changes before atomic commit

This design treats Git as a single-threaded resource, even when the surrounding application is highly concurrent. The committer serializes all index mutations, preventing the .git/index.lock contention that occurs when multiple processes simultaneously attempt to stage changes.

Why .git/index.lock Corruption Happens

Git acquires an exclusive lock on .git/index.lock during index-modifying operations to ensure repository integrity. When concurrent processes collide:

1. Process A creates index.lock to stage files
2. Process B attempts git add simultaneously and fails with "fatal: Unable to create '/path/.git/index.lock'"
3. If Process A crashes, the lock file persists indefinitely
4. Subsequent Git commands fail until manual intervention

The single-committer pattern eliminates this class of failures by guaranteeing that only one process ever attempts to create the lock file, with built-in recovery logic for handling orphaned locks from external tools or crashes.

Implementation in munder-difflin

The implementation in src/main/hive.ts combines retry logic, stale-lock detection, and deterministic configuration to create a bulletproof commit pipeline.

The Commit Loop with Retry Logic

The committer uses an exponential back-off retry mechanism to handle transient lock conflicts. As shown in the source code at lines 1991-1999, the loop attempts the commit up to five times:

// src/main/hive.ts – attempts a commit up to 5 times
for (let attempt = 0; attempt < 5; attempt++) {
  this.clearStaleLock(root);                     // remove old lock files
  const add = this.git(['add', '-A'], root);    // stage everything
  const commit = this.git(['commit', '-q', '-m', message], root);
  if (commit.ok) return;                        // success
  if (/nothing to commit/i.test(commit.out + commit.err)) return;
  // If add failed or we see a lock error, back-off then retry
  if (!add.ok || /index\.lock/i.test(commit.err)) {
    sleepSync(50 * (attempt + 1));
    continue;
  }
  return; // non-lock failure – give up; next mutation will retry
}

This loop checks for index.lock errors in stderr and progressively increases the delay between attempts (50ms, 100ms, 150ms, etc.), allowing any external Git process to complete and release its lock.

Stale-Lock Recovery Mechanism

Before each commit attempt, the clearStaleLock helper (lines 2002-2006) implements automatic recovery from orphaned locks:

private clearStaleLock(root: string): void {
  const lock = join(root, '.git', 'index.lock');
  try {
    // Remove the lock if it's older than 10 seconds
    if (existsSync(lock) && Date.now() - statSync(lock).mtimeMs > 10_000) rmSync(lock);
  } catch { /* ignore errors */ }
}

By checking the modification time against a 10-second threshold, the committer distinguishes between active Git operations and stale lock files left by crashed processes. This prevents the "infinite lock" scenario that would otherwise require manual rm .git/index.lock intervention.

Deterministic Git Identity Configuration

The committer supplies fixed identity parameters and disables interactive prompts to ensure commits never block:

git -c commit.gpgsign=false -c user.name=Hive -c user.email=hive@local \
    commit -m "…"

This configuration prevents GPG signing prompts and ensures consistent user.name and user.email values across all commits, eliminating another potential source of hanging operations that could delay the serial commit queue.

Key Benefits of the Single-Committer Approach

• Eliminates lock races by guaranteeing only one writer to the Git index at any time
• Self-healing repositories automatically clear stale locks without human intervention
• Deterministic behavior through fixed Git configuration and non-interactive operation
• Simplified error handling as all Git failures concentrate in one well-tested code path
• Scalable read concurrency since worker agents read the filesystem without locking while only the committer writes to Git

Summary

• The single-committer git pattern restricts all Git write operations to one dedicated process in src/main/hive.ts, preventing the concurrent access that causes .git/index.lock corruption.
• The implementation combines a retry loop with exponential back-off, automatic stale-lock detection (10-second threshold), and deterministic Git identity settings.
• Worker agents write files to workspace directories without invoking Git, while the committer serializes all git add and git commit operations.
• This approach eliminates the "fatal: Unable to create index.lock" errors common in multi-agent systems and enables automatic recovery from orphaned lock files.

Frequently Asked Questions

What causes .git/index.lock corruption in multi-process systems?

Concurrent Git processes attempting to modify the staging index simultaneously trigger race conditions for the exclusive lock file. If one process crashes while holding the lock, or if multiple processes compete during high-frequency operations, the repository enters a corrupted state where no further Git commands can execute until manual lock removal occurs.

How does the single-committer pattern differ from Git's built-in locking?

Git's internal locking prevents simultaneous index modifications but does not handle lock cleanup or retry logic. The single-committer pattern adds an orchestration layer (the committer process) that guarantees only one Git operation starts at a time, implements stale-lock recovery via clearStaleLock(), and retries failed attempts with back-off delays, effectively managing the locking lifecycle rather than merely asserting the lock.

Can the single-committer pattern scale to high-throughput environments?

Yes, because the bottleneck is constrained to Git operations specifically, not filesystem writes. Worker agents can write files to disk concurrently at full speed; only the final commit step serializes. The retry logic with back-off (50ms increments) and stale-lock cleanup ensures the committer maintains throughput even under contention, as demonstrated in the five-attempt loop in src/main/hive.ts.

Where is the single-committer logic implemented in munder-difflin?

The core implementation resides in src/main/hive.ts, specifically the commit-retry loop at lines 1991-1999 and the clearStaleLock helper at lines 2002-2006. Architecture documentation appears in docs/blog/single-committer-git-pattern/index.html, with additional context in HIVE.md and README.md line 110 explaining the pattern's role in preventing repository corruption.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}