How to Debug Issues When Running Prime Agent Agents in Detached Background Mode

To debug detached background agents in Prime Agent, enable verbose logging with PI_DEBUG_REDRAW=1, inspect supervisor state via prime-agent daemon ps, and trace the session_detached event through the daemon protocol.

Prime Agent runs many sub-agents—coding-agent workers, daemon supervisors, and external tool processes—as detached child processes that survive after the parent terminal closes. This architecture, implemented in the PrimeIntellect-ai/prime-agent repository, creates unique debugging challenges because detached processes inherit no controlling terminal and typically have stdio set to ignore. This guide walks through the source-level mechanics of detached execution and provides a systematic workflow for diagnosing failures.

Architecture of Detached Agent Execution

Prime Agent's detached execution stack spans several interconnected components:

Component	Role	Key Source File
Daemon Supervisor	Launches, monitors, and evicts detached sub-agents	packages/coding-agent/src/modes/daemon/daemon-supervisor.ts
Daemon Mode	Entry point for the background daemon keeping the supervisor alive	packages/coding-agent/src/modes/daemon/daemon-mode.ts
Agent Session	Tracks lifecycle including detachedDeletion state	packages/coding-agent/src/core/agent-session.ts
Shell Utility	Creates detached processes for external tools	packages/coding-agent/src/utils/shell.ts
Daemon Protocol	Defines session_detached event and JSON message format	packages/coding-agent/src/modes/daemon/daemon-protocol.ts
Daemon Launcher	CLI command starting daemon with detached: true	prime-agent.sh, packages/coding-agent/src/cli/daemon-launch.ts
Ownership Lock	Ensures single supervisor instance via lockfile	packages/coding-agent/src/modes/daemon/daemon-supervisor-ownership.ts
Debug Logging	Writes diagnostics to configurable path	packages/coding-agent/src/config.ts

The daemon protocol coordinates all detached execution. When spawn() creates a child with detached: true, the supervisor registers a session_detached entry and continues monitoring via heartbeats, exit codes, and protocol messages.

Common Failure Points in Detached Mode

Symptom	Likely Cause	Investigation Target
Sub-agent missing from prime-agent daemon ps	Spawn failure (missing executable, permission error)	shell.ts spawn configuration
Daemon crash after sub-agent exit	Attempted write to closed stdio pipe	daemon-supervisor.ts writePipe / unref handling
Empty or missing debug logs	PI_DEBUG_REDRAW unset or permission denied	config.ts debug flag handling
Stack overflow during cleanup	Recursive detachedDeletion resolution	agent-session.ts state machine
Orphaned child after parent crash	Missed exit event (especially on Windows)	daemon-supervisor-ownership.ts cleanup logic

Step-by-Step Debugging Workflow for Detached Agents

Enable Verbose Debug Logging

Set environment variables before starting the daemon:

export PI_DEBUG_REDRAW=1      # Forces full render logging

export PI_LOG_LEVEL=debug     # Verbose internal logs

prime-agent daemon start

The debug log location is determined by getDebugLogPath() in packages/coding-agent/src/config.ts, defaulting to ~/.prime/agent/pi-debug.log. Search for session_detached to identify when children successfully detach.

Inspect Supervisor State

Verify the supervisor recognizes detached children:

prime-agent daemon ps

Missing entries indicate the detach event never reached the supervisor—typically a spawn or protocol registration failure.

Validate Child Process Exit Codes

Use the repository's minimal reproduction helper:

npx tsx scripts/tool-stats.ts

This script spawns a detached process with unref() and logs its behavior, providing a baseline for comparison.

Check Platform-Specific Spawn Behavior

In packages/coding-agent/src/utils/shell.ts, the detached flag behavior varies by platform:

• macOS/Linux: detached: true always enabled
• Windows: detached disabled when process.platform === "win32" due to platform limitations

Confirm your OS matches the code path being executed.

Verify Ownership Lock Acquisition

The supervisor uses acquireDaemonSupervisorOwnership() in daemon-supervisor-ownership.ts to guarantee single-instance operation. Lock failures cause silent aborts. Check for acquisition errors in the debug log or examine /tmp/daemon-supervisor.lock directly.

Monitor Heartbeat Patterns

The supervisor emits periodic heartbeats controlled by idleEvictionSweepIntervalMs in daemon-supervisor.ts. Missing heartbeats suggest event loop blockage in the detached child. Temporarily increase the interval for finer-grained observation:

// In daemon-supervisor.ts
const idleEvictionSweepIntervalMs = 5000; // Default: increase for debugging

Reproduce with Minimal Test Code

Create examples/debug-detached.ts to isolate the behavior:

import { spawn } from "node:child_process";

const child = spawn(process.execPath, ["-e", "setTimeout(()=>{}, 60000)"], {
  detached: true,
  stdio: "ignore",
});
child.unref();
console.log("Detached PID:", child.pid);

Run under the daemon:

prime-agent daemon run examples/debug-detached.ts

Then verify the session_detached event appears in the log.

Inspect Raw Daemon Protocol Messages

All parent-child communication uses JSON messages defined in daemon-protocol.ts. Add temporary logging to the protocol handlers:

// In daemon-supervisor.ts or protocol implementation
console.log("Protocol write:", JSON.stringify(message));
console.log("Protocol read:", JSON.stringify(parsed));

Or use npm run check to compile TypeScript definitions and examine message structures.

Handle Windows-Specific Cleanup

On Windows, the supervisor must explicitly terminate orphaned children. Around line 3800 of daemon-supervisor.ts, verify this cleanup executes:

// Windows cleanup path
execSync(`taskkill /PID ${childPid} /T /F`);

Failure here leaves zombie processes after unexpected parent exits.

Run Regression Test Suite

Execute targeted tests to confirm behavior:

npx vitest run packages/coding-agent/test/suite/regressions/4656-resume-active-session.test.ts

The packages/coding-agent/test/suite/regressions/ directory contains tests specifically validating detached session behavior.

Code Examples for Common Debugging Scenarios

Spawning a Detached Sub-Agent

Derived from packages/coding-agent/src/utils/shell.ts (lines 190-210):

import { spawn } from "node:child_process";

function launchDetachedAgent(script: string): number | undefined {
  const child = spawn(process.execPath, ["-r", "esbuild-register", script], {
    detached: true,
    stdio: "ignore",
  });
  child.unref(); // Critical: allows parent to exit independently
  return child.pid;
}

const pid = launchDetachedAgent("./my-agent.ts");
console.log(`Detached agent PID=${pid}`);

Listening for session_detached Events

From packages/coding-agent/src/modes/daemon/daemon-protocol.ts:

import { DaemonSupervisor } from "./daemon-supervisor.js";

const supervisor = new DaemonSupervisor();
supervisor.on("session_detached", ({ activeSessionId }) => {
  console.info(`Session ${activeSessionId} successfully detached`);
});
supervisor.start();

Reading Debug Log Configuration

From packages/coding-agent/src/config.ts:

import { getDebugLogPath } from "./config.js";

const logPath = getDebugLogPath();
console.log(`Debug logs available at: ${logPath}`);

Key Source Files Reference

File	Purpose
packages/coding-agent/src/modes/daemon/daemon-supervisor.ts	Core supervisor: tracks detached agents, emits session_detached, handles heartbeats
packages/coding-agent/src/modes/daemon/daemon-mode.ts	Daemon entry point and supervisor wiring
packages/coding-agent/src/modes/daemon/daemon-protocol.ts	JSON message definitions including session_detached
packages/coding-agent/src/utils/shell.ts	Detached process creation for external tools
packages/coding-agent/src/core/agent-session.ts	Session lifecycle and detachedDeletion state
packages/coding-agent/src/config.ts	getDebugLogPath() and PI_DEBUG_REDRAW handling
packages/coding-agent/src/modes/daemon/daemon-supervisor-ownership.ts	Lockfile-based single-instance enforcement
prime-agent.sh	Main launcher with detached: true passthrough
scripts/tool-stats.ts	Minimal detached spawn reproduction

Summary

• Detached agents in Prime Agent use child_process.spawn({ detached: true, stdio: "ignore" }) followed by unref() to survive parent termination.

• The daemon supervisor in daemon-supervisor.ts tracks these agents via the session_detached event and heartbeat monitoring.

• Debug effectively by setting PI_DEBUG_REDRAW=1, running prime-agent daemon ps, and tracing protocol messages through daemon-protocol.ts.

• Platform differences matter: Windows disables detached in some paths and requires explicit taskkill cleanup.

• Lockfile ownership in daemon-supervisor-ownership.ts prevents duplicate supervisors—check for stale locks when agents fail to register.

Frequently Asked Questions

How do I know if a detached agent started successfully?

Check ~/.prime/agent/pi-debug.log for the session_detached event and run prime-agent daemon ps to verify the supervisor's process list includes the child PID. Absence from either indicates a spawn or protocol registration failure in shell.ts or daemon-supervisor.ts.

Why does my detached agent disappear from the process list immediately?

The child likely exited before completing initialization. Examine the exit code using a minimal reproduction script like scripts/tool-stats.ts and verify the executable path and permissions in the spawn() call within shell.ts.

How do I clean up orphaned detached processes on Windows?

Prime Agent's supervisor should execute taskkill /PID <pid> /T /F for Windows children, located around line 3800 of daemon-supervisor.ts. If this fails, manually terminate orphaned processes and verify the cleanup code path executes during supervisor shutdown.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}