# How to Debug Rowboatlabs/Rowboat: A Complete Troubleshooting Guide

> Debug rowboatlabs/rowboat effectively by enabling verbose logging and analyzing logs/services.jsonl to trace events across subsystems. Your comprehensive troubleshooting guide.

- Repository: [RowBoat Labs/rowboat](https://github.com/rowboatlabs/rowboat)
- Tags: how-to-guide
- Published: 2026-02-16

---

**Enable verbose logging with PrefixLogger and inspect `logs/services.jsonl` to trace events through the Electron app, agent runtime, and knowledge graph subsystems.**

Debugging rowboatlabs/rowboat requires understanding its multi-process architecture. This local-first AI coworker stitches together an Electron desktop app, core agent services, and a knowledge-graph builder that communicate via events and hierarchical logging. By leveraging the built-in `PrefixLogger` and `ServiceLogger` utilities, you can isolate failures across the runtime, service bus, and file-change detection layers.

---

## Enable Verbose Logging with PrefixLogger and ServiceLogger

Rowboat provides two complementary logging mechanisms: console-based hierarchical tracing and persistent JSON-L event logs.

### Hierarchical Tracing with PrefixLogger

The `PrefixLogger` class in [`apps/x/packages/shared/src/prefix-logger.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/shared/src/prefix-logger.ts) wraps `console.log` and supports child loggers for granular traceability. Each logical unit prepends its identifier to create a hierarchical log chain.

```typescript
import { PrefixLogger } from "@x/shared";

const logger = new PrefixLogger(`run-${runId}-${state.agentName}`);
const loopLogger = logger.child(`iter-${loopCounter}`);

loopLogger.log("processing step");

```

Output appears as:

```

2026-02-16T12:34:56.789Z [run-123-copilot] [iter-2] processing step

```

### Persistent Event Logging with ServiceLogger

For audit trails, `ServiceLogger` in [`apps/x/packages/core/src/services/service_logger.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/core/src/services/service_logger.ts) writes structured events to `logs/services.jsonl` and publishes them on the `serviceBus`.

```typescript
await serviceLogger.log({
  type: "progress",
  service: "graph",
  runId,
  level: "info",
  message: "Processing batch 1/3",
});

```

To inspect recent events:

```bash
tail -n 20 logs/services.jsonl | jq .

```

---

## Debug the Agent Runtime in rowboatlabs/rowboat

The agent execution loop resides in [`apps/x/packages/core/src/agents/runtime.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/core/src/agents/runtime.ts). Each run instantiates a run-specific `PrefixLogger` at line 641, providing visibility into the decision flow.

Key debug points to monitor:

- **Tool execution start**: Look for `_logger.log('executing tool')` inside the `pendingToolCalls` processing loop
- **Permission denied**: Check for `_logger.log('returning denied tool message')` when user approval is withheld
- **Abort detection**: Watch for `_logger.log('skipping, reason: aborted')` before tool invocation
- **LLM turn start**: Identify `loopLogger.log('running llm turn')` preceding the `streamLlm` call

When a tool fails, grep the console output for `executing tool` to find the run ID, then filter `logs/services.jsonl` by that `runId` to reconstruct the event timeline.

---

## Troubleshoot Knowledge Graph Issues

The knowledge graph builder in [`apps/x/packages/core/src/knowledge/build_graph.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/core/src/knowledge/build_graph.ts) manages markdown ingestion and entity extraction. When notes appear missing, duplicated, or stale, use these targeted debugging techniques.

### Reset and Rebuild the Graph State

To force a full reprocessing of all markdown files, invoke `resetGraphState()`:

```typescript
import { resetGraphState } from "@x/core/knowledge/build_graph";

resetGraphState(); // wipes knowledge_graph_state.json

```

This function clears the `processedFiles` map in [`graph_state.ts`](https://github.com/rowboatlabs/rowboat/blob/main/graph_state.ts), causing the next `buildGraph` run to treat every file as new. After resetting, restart the dev server:

```bash
cd apps/x
npm run deps
npm run dev

```

### Inspect File Change Detection

The change-detection logic in [`apps/x/packages/core/src/knowledge/graph_state.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/core/src/knowledge/graph_state.ts) uses mtime and SHA-256 hash comparisons (lines 66-86).

To verify why a file is being skipped:

1. Open [`WorkDir/knowledge_graph_state.json`](https://github.com/rowboatlabs/rowboat/blob/main/WorkDir/knowledge_graph_state.json)
2. Locate the file path key
3. Compare the stored `mtime` with the current file's `stat.mtime`
4. Verify the `hash` field matches a fresh SHA-256 of the file content

Force reprocessing by deleting the file's entry from the JSON or running `touch` on the markdown file to update its mtime.

### Debug Voice Memo Indexing

Voice memos are handled by `processVoiceMemosForKnowledge` in [`build_graph.ts`](https://github.com/rowboatlabs/rowboat/blob/main/build_graph.ts) (lines 72-84), which logs with a `[GraphBuilder]` prefix.

Common issues and diagnostic steps:

- **No files detected**: Confirm `WorkDir/knowledge/Voice Memos` exists and contains `voice-memo-*.md` files
- **Files skipped as "still recording"**: Open the markdown file and remove the placeholder `*Recording in progress...*`
- **Transcription failures**: Search for `*Transcription failed*` in the file content; check `logs/services.jsonl` for transcription service errors

---

## Practical Debugging Workflow for rowboatlabs/rowboat

Use this diagnostic script to surface runtime state without manually grepping logs. Save as [`debug.ts`](https://github.com/rowboatlabs/rowboat/blob/main/debug.ts) in the project root:

```typescript
import { serviceLogger } from "@x/core/services/service_logger";
import { loadState, getFilesToProcess } from "@x/core/knowledge/graph_state";
import { WorkDir } from "@x/core/config/config";

// Show recent service events
async function dumpRecentEvents(limit = 20) {
  const { execSync } = await import("child_process");
  const raw = execSync(`tail -n ${limit} logs/services.jsonl`).toString();
  console.log(JSON.parse(`[${raw.trim().split("\n").join(",")}]`));
}

// List files pending processing
function listPendingFiles() {
  const state = loadState();
  const source = `${WorkDir}/gmail_sync`; // adjust path as needed
  const pending = getFilesToProcess(source, state);
  console.log(`Pending files (${pending.length}):`);
  pending.forEach(p => console.log(`- ${p}`));
}

// Reset graph state (use with caution)
async function resetGraph() {
  const { resetGraphState } = await import("@x/core/knowledge/build_graph");
  resetGraphState();
  console.log("Graph state reset – next run will re-process everything.");
}

// Execute diagnostics
await dumpRecentEvents();
listPendingFiles();
// resetGraph(); // uncomment only when full rebuild required

```

Run the script:

```bash
node -r ts-node/register debug.ts

```

---

## Key Source Files for Debugging

| File | Purpose | Location |
|------|---------|----------|
| **PrefixLogger** | Hierarchical console logging | [`apps/x/packages/shared/src/prefix-logger.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/shared/src/prefix-logger.ts) |
| **ServiceLogger** | Persistent JSON-L event logs | [`apps/x/packages/core/src/services/service_logger.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/core/src/services/service_logger.ts) |
| **ServiceBus** | Event publishing backbone | [`apps/x/packages/core/src/services/service_bus.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/core/src/services/service_bus.ts) |
| **AgentRuntime** | Main agent execution loop | [`apps/x/packages/core/src/agents/runtime.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/core/src/agents/runtime.ts) |
| **GraphState** | File change detection (mtime + hash) | [`apps/x/packages/core/src/knowledge/graph_state.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/core/src/knowledge/graph_state.ts) |
| **BuildGraph** | Knowledge graph orchestration | [`apps/x/packages/core/src/knowledge/build_graph.ts`](https://github.com/rowboatlabs/rowboat/blob/main/apps/x/packages/core/src/knowledge/build_graph.ts) |
| **services.jsonl** | Runtime event log (generated) | `logs/services.jsonl` |
| **knowledge_graph_state.json** | Processed file tracking (generated) | [`WorkDir/knowledge_graph_state.json`](https://github.com/rowboatlabs/rowboat/blob/main/WorkDir/knowledge_graph_state.json) |

---

## Summary

- **Use PrefixLogger** to add hierarchical context to console output when tracing specific runs or agent iterations.
- **Inspect `logs/services.jsonl`** for persistent, searchable event histories across all subsystems.
- **Debug the agent runtime** by monitoring key log points in [`runtime.ts`](https://github.com/rowboatlabs/rowboat/blob/main/runtime.ts) around tool execution and LLM turns.
- **Reset graph state** via `resetGraphState()` when knowledge files appear stale or duplicated, forcing a full reprocessing cycle.
- **Verify file changes** by examining [`knowledge_graph_state.json`](https://github.com/rowboatlabs/rowboat/blob/main/knowledge_graph_state.json) mtime and hash entries when markdown updates aren't detected.

---

## Frequently Asked Questions

### How do I enable debug logging in rowboatlabs/rowboat?

Import `PrefixLogger` from `@x/shared` and instantiate it with a descriptive prefix related to your component or run ID. For system-wide event tracking, the `ServiceLogger` in `@x/core/services/service_logger` automatically writes structured JSON-L logs to `logs/services.jsonl` without additional configuration.

### Where are persistent logs stored in rowboatlabs/rowboat?

Persistent service logs are written to `logs/services.jsonl` in line-delimited JSON format. Additionally, the knowledge graph maintains state in [`WorkDir/knowledge_graph_state.json`](https://github.com/rowboatlabs/rowboat/blob/main/WorkDir/knowledge_graph_state.json), which tracks processed files, their modification times, and SHA-256 hashes to detect changes.

### How do I fix missing or duplicate notes in rowboatlabs/rowboat?

Missing or duplicate notes usually indicate stale graph state. Import and call `resetGraphState()` from `@x/core/knowledge/build_graph` to clear the `processedFiles` map in [`knowledge_graph_state.json`](https://github.com/rowboatlabs/rowboat/blob/main/knowledge_graph_state.json). This forces the next `buildGraph` execution to treat all markdown files as new, rebuilding the knowledge graph from scratch.

### Why are my voice memos not being indexed?

Voice memos are processed by `processVoiceMemosForKnowledge` in [`build_graph.ts`](https://github.com/rowboatlabs/rowboat/blob/main/build_graph.ts). Common causes include: the `WorkDir/knowledge/Voice Memos` directory missing or containing no `voice-memo-*.md` files; markdown files still containing the placeholder `*Recording in progress...*` which marks them as incomplete; or transcription failures indicated by `*Transcription failed*` in the file content. Check `logs/services.jsonl` for transcription service errors if the content shows failure markers.