# How the SDK Worker Processes Transcripts in the Background in letta-ai/claude-subconscious

> Discover how the letta-ai/claude-subconscious SDK worker in send_worker_sdk.ts efficiently processes transcripts in the background. Learn about its non-blocking approach to sending messages and updating sync states.

- Repository: [Letta/claude-subconscious](https://github.com/letta-ai/claude-subconscious)
- Tags: internals
- Published: 2026-03-26

---

**The SDK worker ([`send_worker_sdk.ts`](https://github.com/letta-ai/claude-subconscious/blob/main/send_worker_sdk.ts)) is a detached Node.js process that reads a JSON payload from the main Claude-Code hook, sends transcript messages to a Letta agent via the Letta Code SDK, streams the response, and updates the sync state file—all without blocking the main process.**

The `letta-ai/claude-subconscious` repository implements an asynchronous bridge between Claude-Code and Letta agents. The SDK worker located at [`scripts/send_worker_sdk.ts`](https://github.com/letta-ai/claude-subconscious/blob/main/scripts/send_worker_sdk.ts) handles background transcript processing to ensure the main Claude-Code stop-hook remains non-blocking while maintaining accurate incremental sync with Letta agents.

## Architecture Overview

The SDK worker operates as a **detached child process** spawned by the main stop-hook. This architecture decouples the Letta SDK communication from the Claude-Code lifecycle, preventing network latency or LLM processing time from delaying the user's editor experience. The worker receives its instructions through a temporary JSON payload file, executes the SDK operations asynchronously, and manages persistent state to enable incremental transcript synchronization.

## Step-by-Step Processing Flow

The transcript processing follows a precise eight-step pipeline:

### Payload Creation in the Main Hook

In [`scripts/send_messages_to_letta.ts`](https://github.com/letta-ai/claude-subconscious/blob/main/scripts/send_messages_to_letta.ts), the hook constructs a `sdkPayload` object containing the agent ID, conversation ID, formatted Letta message XML, sync state file path, last processed index, working directory, and tool mode configuration (lines 16-25). This payload is serialized to a temporary JSON file that serves as the inter-process communication mechanism.

### Worker Spawning

The hook invokes `spawnSilentWorker()` from [`scripts/conversation_utils.ts`](https://github.com/letta-ai/claude-subconscious/blob/main/scripts/conversation_utils.ts) to launch [`send_worker_sdk.ts`](https://github.com/letta-ai/claude-subconscious/blob/main/send_worker_sdk.ts) as a detached child process, passing the payload file path as the sole CLI argument (lines 29-31). The spawn configuration uses `detached: true` with stdio redirected to `/dev/null`, ensuring the worker survives independently of the parent process.

### Payload Loading

Upon startup, the worker reads the JSON payload from the temporary file path provided in `process.argv[2]`, logs the session identifier, and delegates to the `sendViaSdk()` function (lines 28-30 in [`send_worker_sdk.ts`](https://github.com/letta-ai/claude-subconscious/blob/main/send_worker_sdk.ts)).

### SDK Session Setup

Inside `sendViaSdk()`, the Letta Code SDK (`@letta-ai/letta-code-sdk`) is **dynamically imported** at line 44 to prevent parsing errors if the SDK is not installed. The session options configure tool permissions based on the `sdkToolsMode` parameter:

- **`off`**: Blocks all client-side tools, creating a read-only mode
- **`read-only`**: Whitelists only `Read`, `Grep`, `Glob`, `web_search`, and `fetch_webpage` tools (lines 47-65)
- **`full`**: Places no restrictions on available tools

### Message Dispatch

The worker creates a Letta SDK session using `resumeSession(conversationId, sessionOptions)`, then transmits the payload message via `session.send(payload.message)` (lines 73-78). This establishes the connection to the Letta agent with the appropriate conversation context.

### Response Streaming

The worker iterates over `session.stream()` to capture real-time assistant output, logging each chunk as it arrives. The stream handling loop (lines 83-93) also captures any tool calls or errors that occur during Letta-side processing, providing complete observability into the agent's execution.

### State File Update

After successful message delivery, the worker reads the persistent sync-state file specified in the payload, updates `lastProcessedIndex` to the `newLastProcessedIndex` value from the payload, and writes the updated state back to disk (lines 34-38). This guarantees that subsequent hook executions only process new transcript entries, preventing duplicate message sending.

### Cleanup

The worker removes the temporary payload file using `fs.unlinkSync()`, closes the SDK session, and logs completion status (lines 41-45). This ensures no ephemeral files accumulate and resources are properly released.

## Configuration and Tool Modes

The **SDK tool mode** (`sdkToolsMode`) provides granular control over agent capabilities during background processing. When set to `read-only`, the worker specifically restricts the Letta agent to information-retrieval tools only, preventing file system modifications during automated transcript analysis. The `off` mode provides maximum safety for audit scenarios, while `full` enables complete agent autonomy for complex refactoring tasks.

## Code Implementation

### Creating the SDK Payload

The main hook prepares the worker instructions:

```typescript
const sdkPayload = {
  agentId,
  conversationId,
  sessionId: hookInput.session_id,
  message: userMessage,                     // XML with new transcript entries
  stateFile: getSyncStateFile(hookInput.cwd, hookInput.session_id),
  newLastProcessedIndex: messages.length - 1,
  cwd: hookInput.cwd,
  sdkToolsMode,                            // 'off' | 'read-only' | 'full'
};

fs.writeFileSync(payloadFile, JSON.stringify(sdkPayload), 'utf-8');

```

### Spawning the Background Worker

The hook initiates the detached process:

```typescript
const workerScript = path.join(__dirname, 'send_worker_sdk.ts');
const child = spawnSilentWorker(workerScript, payloadFile, hookInput.cwd);
log(`Spawned SDK worker (PID: ${child.pid})`);

```

### Processing in the Worker

The worker executes the SDK communication and state management:

```typescript
async function main(): Promise<void> {
  const payloadFile = process.argv[2];
  const payload: SdkPayload = JSON.parse(fs.readFileSync(payloadFile, 'utf-8'));
  const success = await sendViaSdk(payload);

  if (success) {
    // Update sync state so next hook run knows the last line processed
    const state = JSON.parse(fs.readFileSync(payload.stateFile, 'utf-8'));
    state.lastProcessedIndex = payload.newLastProcessedIndex;
    fs.writeFileSync(payload.stateFile, JSON.stringify(state, null, 2));
  }

  fs.unlinkSync(payloadFile);   // clean up temp file
}

```

## Summary

- The **detached process architecture** prevents Letta SDK network calls from blocking the Claude-Code stop-hook execution.
- **Dynamic SDK importing** ensures the codebase remains parseable even when `@letta-ai/letta-code-sdk` is not installed.
- Three-tier **tool permission system** (`off`, `read-only`, `full`) controls agent capabilities during background processing.
- **Incremental synchronization** via `lastProcessedIndex` tracking prevents duplicate transcript processing and ensures message ordering.
- **Streaming response handling** captures real-time agent output and tool execution logs for comprehensive debugging.

## Frequently Asked Questions

### What happens if the Letta Code SDK is not installed?

The worker uses dynamic imports (`import('@letta-ai/letta-code-sdk')`) at line 44 of [`send_worker_sdk.ts`](https://github.com/letta-ai/claude-subconscious/blob/main/send_worker_sdk.ts) to conditionally load the SDK only when needed. If the SDK is missing, the import fails gracefully, allowing the rest of the codebase to function for users who do not require Letta integration.

### How does the worker handle transcript formatting?

The worker does not directly read the transcript file. Instead, [`scripts/transcript_utils.ts`](https://github.com/letta-ai/claude-subconscious/blob/main/scripts/transcript_utils.ts) reads the JSONL transcript and converts entries into Letta-compatible XML via `formatMessagesForLetta()` before the worker spawns. The worker receives pre-formatted XML in the payload's `message` field.

### What is the purpose of the sync state file?

The sync state file (managed by `getSyncStateFile()` in [`conversation_utils.ts`](https://github.com/letta-ai/claude-subconscious/blob/main/conversation_utils.ts)) persists the `lastProcessedIndex` between hook executions. This index tracks which transcript lines have already been sent to Letta, enabling the system to send only new messages during incremental updates and preventing duplicate processing.

### Why use a detached process instead of asynchronous calls in the main thread?

The detached process ensures survival independent of the Claude-Code lifecycle. If the main hook process exits, the worker continues processing the Letta communication to completion. Additionally, this isolation prevents any SDK errors or network timeouts from affecting the main editor's stop-hook performance.