# How to Debug the Managed-Workstream Ledger for Cross-Harness Continuity

> Debug the managed-workstream ledger for cross-harness continuity. Learn how this append-only SQLite log enables seamless session resumption and consistent transcripts across tool invocations.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: how-to-guide
- Published: 2026-08-27

---

**The managed-workstream ledger is an append-only SQLite log that records every event from native harnesses, enabling different sessions to resume runs and maintain consistent transcripts across tool invocations.**

The `akitaonrails/ai-memory` repository implements this ledger as a portable event store that bridges CLI tools, MCP servers, and custom plugins. When cross-harness continuity breaks—whether due to lost leases, duplicate events, or stalled checkpoints—you need systematic debugging techniques to inspect the raw SQLite tables, validate lease files, and verify event sequences. This guide walks through the exact source locations, diagnostic queries, and remediation steps required to restore ledger integrity.

## Understanding the Managed-Workstream Ledger Architecture

The ledger operates as a durable event stream that persists in SQLite and synchronizes across process boundaries. Understanding its five-phase lifecycle is essential before debugging.

### Core Components and Data Flow

According to the source code in [`crates/ai-memory-core/src/workstream.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/workstream.rs), the ledger relies on three primary structures:

- **`PrepareManagedRunRequest`** – Initiates a managed run by acquiring a lease and selecting (or creating) a workstream. Returns `workstream_id` and high-water marks (`sync_after`, `sync_through`).
- **`NewWorkstreamEvent`** – Wraps every native transcript event (messages, tool calls, checkpoints) into a JSON payload posted to `/workstream/event`.
- **`FinishManagedRunRequest`** – Signals completion, triggering event import, compaction records, and retention policy enforcement.

During execution, the native harness emits events through the hooks layer defined in [`crates/ai-memory-hooks/src/workstream.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/workstream.rs), which forwards them to the MCP server. The server writes these to the `workstream_events` table managed by [`crates/ai-memory-store/src/migrations.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/migrations.rs).

### The SQLite Persistence Layer

Each ledger row corresponds to a `NewWorkstreamEvent` with the following critical columns:

- `event_id` – Must be deterministic to prevent duplicates
- `workstream_id` – Links events to a specific run stream
- `kind` – Event type (`message`, `tool_call`, `tool_result`, `checkpoint`, `compaction`)
- `occurred_at` – ISO 8601 timestamp for ordering
- `metadata` – JSON blob containing repo fingerprints and checkpoint data

The lease mechanism creates a temporary [`managed-workstream.md`](https://github.com/akitaonrails/ai-memory/blob/main/managed-workstream.md) file on disk containing `run_id`, `lease_owner`, and expiration data. The server validates this lease before accepting events to prevent race conditions between harnesses.

## Debugging Common Cross-Harness Continuity Issues

When workstreams fail to sync across sessions or show inconsistent state, investigate these four specific failure modes.

### Missing Events and Lease Failures

**Symptom:** A run appears incomplete in the ledger despite the harness finishing successfully.

**Likely Cause:** The child process crashed before sending `FinishManagedRunRequest`, or the lease file was deleted prematurely, causing the server to reject final event imports.

**Debug Steps:**

1. Verify the lease file exists in the temporary run directory: [`managed-workstream.md`](https://github.com/akitaonrails/ai-memory/blob/main/managed-workstream.md)
2. Check for a `compaction` event in the ledger (search `event_kind = "compaction"` in `workstream_events`)
3. Query the MCP admin endpoint `/admin/workstream/leases` for stale or orphaned lease records

If the lease exists but events are missing, the import phase likely failed. Check the MCP server logs for SQLite constraint violations during the `FinishManagedRunRequest` handler.

### Duplicate Event Entries

**Symptom:** The same tool call or message appears multiple times in search results.

**Likely Cause:** The client retried a POST after a network timeout, or the `event_id` was not derived deterministically from the native record ID.

**Resolution:**

Query the SQLite database directly to confirm duplicates:

```sql
SELECT event_id, COUNT(*) AS dup
FROM workstream_events
WHERE workstream_id = 'your-workstream-id'
GROUP BY event_id
HAVING dup > 1;

```

Prune duplicates via the admin endpoint:

```bash
ai-memory admin workstream purge --id <workstream_id>

```

Alternatively, manually delete specific rows:

```sql
DELETE FROM workstream_events 
WHERE workstream_id = 'your-workstream-id' 
AND event_id = 'duplicate-id';

```

### Orphaned Tool Calls

**Symptom:** `tool_call` events lack corresponding `tool_result` entries, leaving async operations hanging.

**Debug Procedure:**

1. Search the ledger for `event_kind = "tool_call"` without subsequent `tool_result` entries
2. Verify the harness correctly forwards results by inspecting [`crates/ai-memory-hooks/src/workstream.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/workstream.rs) 
3. Check that the native tool execution actually completed and emitted the result payload before the process exited

This often indicates the harness was terminated (SIGTERM) before the tool finished, or the hook implementation failed to capture STDERR/STDOUT properly.

### Stalled Checkpoints

**Symptom:** The workstream refuses to advance past a specific point despite new events being emitted.

**Root Cause:** Checkpoint validation failed due to a `repo_fingerprint` mismatch between the checkpoint metadata and the `PrepareManagedRunRequest.repo_fingerprint`.

**Fix:**

Query the latest checkpoint:

```sql
SELECT metadata->>'repo_fingerprint' as fingerprint, 
       metadata->>'head' as git_head 
FROM workstream_events 
WHERE workstream_id = 'ws-01' 
AND kind = 'checkpoint' 
ORDER BY rowid DESC 
LIMIT 1;

```

Compare the output against your current repository state using `git rev-parse HEAD`. If mismatched, the ledger is protecting against cross-repository contamination—verify you are running the harness in the correct working directory.

## Step-by-Step Debugging Procedures

Use this systematic checklist when investigating ledger inconsistencies:

1. **Identify the target workstream** – Run `ai-memory admin workstream list` and note the `workstream_id`

2. **Inspect the lease file** – Open [`managed-workstream.md`](https://github.com/akitaonrails/ai-memory/blob/main/managed-workstream.md) in the temporary run directory and confirm `run_id` and `lease_owner` match expected values

3. **Query raw events** – Execute:
   ```bash
   ai-memory workstream-search --id <workstream_id> --raw
   ```

   Look for gaps in `sync_after`/`sync_through` sequences or missing `event_id` fields

4. **Validate database integrity** – Connect to SQLite and verify monotonic rowid increase:
   ```sql
   SELECT rowid, event_id, kind, occurred_at 
   FROM workstream_events 
   WHERE workstream_id = 'ws-01' 
   ORDER BY rowid;
   ```

5. **Check compaction boundaries** – Ensure a `compaction` event appears after event batches. Missing compaction indicates the ledger is stuck in an un-compacted state, potentially consuming excess disk space

6. **Review admin logs** – Query `/admin/workstream/logs` for lease acquisition errors or HTTP 409 conflict responses indicating duplicate event rejections

7. **Enable debug logging** – Re-run with verbose output to trace every HTTP request:
   ```bash
   RUST_LOG=debug ai-memory run -- <your-command>
   ```

8. **Manual lease cleanup** – If the lease is stale, release it via:
   ```bash
   curl -X POST http://127.0.0.1:49374/api/v1/admin/workstream/lease/release \
        -d '{"workstream_id":"ws-01","run_id":"run-abcde"}'
   ```

## Diagnostic Code Examples

### Manual Event Injection for Testing

Use this Rust snippet to reproduce events directly against the store API, bypassing the harness:

```rust
use ai_memory_core::{NewWorkstreamEvent, WorkstreamEventKind, AgentKind};
use ai_memory_store::StoreClient;

let event = NewWorkstreamEvent {
    event_id: "debug-123".into(),
    agent: AgentKind::Cli,
    native_session_id: "test-session".into(),
    source_record_id: None,
    kind: WorkstreamEventKind::Message,
    role: Some("assistant".into()),
    content: "Diagnostic message".into(),
    occurred_at: Some(chrono::Utc::now().to_rfc3339()),
    metadata: serde_json::json!({ "debug": true }),
};

let client = StoreClient::connect("http://127.0.0.1:49374").await?;
client.post_workstream_event(event).await?;

```

### Live Ledger Tailing

Monitor events as they arrive in real-time:

```bash
RUST_LOG=debug ai-memory workstream-search \
    --id my-workstream \
    --tail

```

### Lease File Inspection

Parse the lease metadata directly:

```bash
cat /tmp/ai-memory-run-$(uuid)/managed-workstream.md

# lease_owner: "ai-memory-cli:12345"

# run_id: "run-abcde"

# workstream_id: "ws-01"

# expires_at: "2026-09-01T12:00:00Z"

```

### Administrative Purge

When corruption is irreparable, purge and restart:

```bash
ai-memory admin workstream purge --id <workstream_id>

```

## Summary

- The **managed-workstream ledger** in [`crates/ai-memory-core/src/workstream.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/workstream.rs) provides cross-harness continuity through an append-only SQLite event log
- **Lease management** via [`managed-workstream.md`](https://github.com/akitaonrails/ai-memory/blob/main/managed-workstream.md) files prevents concurrent write conflicts; stale leases cause event import failures
- **Duplicate events** stem from non-deterministic `event_id` generation or HTTP retries—query `workstream_events` grouped by `event_id` to detect them
- **Checkpoint stalls** occur when `repo_fingerprint` in the checkpoint metadata mismatches the current repository state
- **Debug tooling** includes `ai-memory workstream-search`, raw SQLite queries against `workstream_events`, and MCP admin endpoints in [`crates/ai-memory-mcp/src/admin.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/admin.rs)

## Frequently Asked Questions

### How do I verify that my workstream ledger is consistent across two different harness sessions?

Query the `workstream_events` table for the specific `workstream_id` in both sessions using `SELECT rowid, event_id, kind FROM workstream_events WHERE workstream_id = 'ws-01' ORDER BY rowid`. The `event_id` sequence should be identical and monotonically increasing in both environments. If discrepancies exist, check for un-compacted events or lease conflicts in the MCP admin logs.

### What causes the "lease already held" error when starting a managed run?

This error occurs when a previous run crashed without sending `FinishManagedRunRequest`, leaving the lease file [`managed-workstream.md`](https://github.com/akitaonrails/ai-memory/blob/main/managed-workstream.md) orphaned on disk or a stale entry in the server's lease registry. Release the lease via the admin endpoint `/admin/workstream/lease/release` with the appropriate `workstream_id` and `run_id`, or wait for the lease expiration timestamp to pass.

### Can I manually edit the SQLite database to fix corrupted ledger entries?

Yes, but exercise caution. Connect directly to the SQLite file used by `crates/ai-memory-store` and execute standard SQL `UPDATE` or `DELETE` statements against `workstream_events`. After manual edits, run `ai-memory admin workstream purge` to trigger a compaction, ensuring that the high-water marks in `PrepareManagedRunResponse` stay synchronized with the actual table state.