# How the Probe Module Verifies Backend Command Executability in Agent-Reach

> Discover how Agent-Reach's probe module verifies backend command executability using subprocess.run to identify ok missing broken timeout or error states for robust system management.

- Repository: [Pnant/Agent-Reach](https://github.com/Panniantong/Agent-Reach)
- Tags: internals
- Published: 2026-07-12

---

**The probe module executes commands via `subprocess.run` and classifies the outcome into five distinct states—`ok`, `missing`, `broken`, `timeout`, or `error`—distinguishing between missing binaries, broken installations, and transient runtime failures.**

The [`agent_reach/probe.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/probe.py) module provides a robust health-check mechanism that validates whether backend commands are actually executable, not merely present on the filesystem. Unlike simple path lookups, this verification runs the command and interprets exceptions, timeouts, and exit codes to deliver accurate diagnostics. This approach powers the doctor command and channel health checks throughout the Agent-Reach codebase.

## The Five States of Command Health

The `probe_command` function categorizes command health into five distinct statuses:

- **`ok`** — Command exists and runs successfully.
- **`missing`** — Command is not found on `PATH` (`shutil.which` returns `None`).
- **`broken`** — Command is found but cannot be executed (e.g., stale virtualenv, missing interpreter).
- **`timeout`** — Command hangs or exceeds the allotted timeout.
- **`error`** — Command runs but returns a non-zero exit code not covered by the "broken" codes.

This granular classification allows the system to provide specific remediation hints, such as reinstalling a package when a command is broken rather than merely missing.

## The Verification Flow in probe_command

The core verification logic resides in `probe_command` (lines 47–76 in [`agent_reach/probe.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/probe.py)). The implementation follows a four-phase execution model:

### Path Discovery

The process begins with `shutil.which` to locate the command on the system `PATH`. If the command is not found, the function immediately returns a `ProbeResult` with status `"missing"` (lines 64–66), avoiding unnecessary execution attempts.

### Execution Attempt

When the command exists, the helper `_run_once` (lines 79–103) executes it via `subprocess.run` with specific safety constraints:

- Output is captured with `capture_output=True` and UTF-8 encoding
- A configurable timeout prevents indefinite hangs (default 10 seconds)
- The environment is sanitized using `utf8_subprocess_env()` from [`agent_reach/utils/process.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/utils/process.py) to ensure consistent encoding

### Exception Handling

The function catches specific exceptions to classify execution failures:

- **`FileNotFoundError` or `OSError`** — Indicates a broken installation where the shim exists but its interpreter is missing. Returns `ProbeResult("broken")` with a helpful reinstall hint (lines 88–93).
- **`subprocess.TimeoutExpired`** — Returns `ProbeResult("timeout")` when the command exceeds the time limit (lines 94–96).

### Exit Code Analysis

If the process completes, the return code determines the final status:

- Exit codes `126` or `127` (defined in `_BROKEN_EXIT_CODES` at lines 24–25) indicate a broken installation, yielding `ProbeResult("broken")` (lines 97–99).
- Any other non-zero exit code is classified as `error` with combined stdout/stderr attached (lines 100–103).
- A zero exit code produces `ProbeResult("ok")`.

## Practical Usage Examples

Here is how to use the probe module to verify command executability:

```python
from agent_reach.probe import probe_command

# Simple health check for `git`

result = probe_command("git")
print(result.status)   # → "ok" if git is functional

print(result.output)   # Version string or error details

print(result.hint)     # Helpful reinstall hint for broken installs

```

```python

# Checking a tool that may be missing or broken

result = probe_command("mycli", args=["--help"], timeout=5, retries=1, package="mycli")
if not result.ok:
    print(f"Problem: {result.status}")
    if result.hint:
        print("Hint:", result.hint)

```

## Integration with Channel Health Checks

Channels invoke `probe_command` inside their `check()` methods to validate backend dependencies. This integration allows the doctor command ([`agent_reach/doctor.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/doctor.py)) to report accurate health diagnostics rather than merely confirming file presence.

```python
def check(self):
    probe = probe_command(self.command_name, package=self.package_name)
    if not probe.ok:
        self.logger.error(f"{self.command_name} probe failed: {probe.status}")
    return probe.ok

```

The `probe_command` function optionally retries transient failures via the `retries` argument, but stops early for permanent issues such as `missing` or `broken` statuses to avoid unnecessary overhead.

## Summary

- The probe module in [`agent_reach/probe.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/probe.py) verifies executability by running commands, not just locating them.
- **Five distinct states** (`ok`, `missing`, `broken`, `timeout`, `error`) provide precise diagnostics.
- **Exception handling** distinguishes between missing interpreters (broken) and missing binaries (missing).
- **Exit code analysis** treats codes 126 and 127 as broken installations.
- Channels use `probe_command` in their `check()` methods to power the doctor command's health reports.

## Frequently Asked Questions

### What is the difference between "missing" and "broken" status in the probe module?

A `"missing"` status means the command is not found on the system `PATH` (detected via `shutil.which` returning `None`). A `"broken"` status means the command file exists but cannot execute, typically due to a missing interpreter, stale virtualenv, or permission issues detected via `OSError` or exit codes 126/127.

### How does the probe module handle commands that hang indefinitely?

The `_run_once` helper passes a configurable `timeout` argument (default 10 seconds) to `subprocess.run`. If the command exceeds this limit, `subprocess.TimeoutExpired` is caught and the function returns a `ProbeResult` with status `"timeout"`.

### Why does the probe module use exit codes 126 and 127 to identify broken installations?

Exit code 126 indicates "command found but not executable" (permission denied), while 127 indicates "command not found" at the shell level despite the file existing. The probe module defines these in `_BROKEN_EXIT_CODES` (lines 24–25) to catch cases where wrapper scripts exist but their underlying interpreters are missing.

### Can the probe module retry failed health checks?

Yes, the `probe_command` function accepts a `retries` parameter for transient failures. However, it stops early for permanent conditions (`missing` or `broken`) to avoid unnecessary overhead, as these states typically require manual intervention rather than retry attempts.