# How the Probe Module Validates Backend Health Beyond Command Existence Checks

> Learn how the probe module validates backend health beyond basic checks. Explore its five statuses ok broken timeout error missing and get remediation hints.

- Repository: [Pnant/Agent-Reach](https://github.com/Panniantong/Agent-Reach)
- Tags: internals
- Published: 2026-07-16

---

**The `agent_reach.probe` module validates backend health by executing binaries with `--version` and classifying outcomes into five explicit statuses—`ok`, `broken`, `timeout`, `error`, or `missing`—while providing actionable remediation hints for broken installations.**

The Agent-Reach framework depends on external CLI tools to interface with platforms like Twitter and YouTube. While basic existence checks only verify that a file is present on PATH, the probe module implements a comprehensive validation pipeline that confirms binaries are actually executable and responsive.

## Command Discovery and Initial Validation

The validation process begins in [`agent_reach/probe.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/probe.py) with the `probe_command()` function. Rather than assuming availability, it first calls `shutil.which` to locate the binary on the system PATH. If the executable cannot be found, the function immediately returns a `ProbeResult` with status **`missing`**, bypassing any further execution attempts.

This initial check serves as a fast-fail gate. However, the module recognizes that file existence alone does not guarantee a working backend, so it proceeds to the next phase when the binary is located.

## Live Execution and Health Classification

When the binary is found, the probe module performs a **live execution** test using `subprocess.run` (by default invoked with the `--version` argument). This execution captures the process exit code, stdout, and stderr to determine the true health state. The result is classified into one of five explicit statuses based on the outcome.

### The Five Status Categories

The `probe_command()` function maps execution results to the following statuses:

- **`ok`** – The process exits with code 0 and returns readable output. The backend is fully operational.
- **`broken`** – The binary exists but cannot be executed, typically due to stale virtual environments, missing shebang interpreters, or corrupted installations. Detected via `FileNotFoundError`, generic `OSError`, or exit codes 126 and 127.
- **`timeout`** – The command fails to respond within the configured timeout period (default 10 seconds).
- **`error`** – The command runs but returns a non-zero exit code not classified as "broken." The raw stdout and stderr are preserved for diagnostic purposes.
- **`missing`** – The command was not found on the system during the initial `shutil.which` check.

### Broken Binary Detection

The **`broken`** status specifically addresses scenarios where a binary file exists but the runtime environment cannot execute it. In [`agent_reach/probe.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/probe.py), the module catches `FileNotFoundError` and `OSError` exceptions during subprocess execution, and also interprets shell exit codes 126 (command not executable) and 127 (command not found at execution time).

When a broken installation is detected, the module generates a remediation hint using the `reinstall_hint()` function. This hint references the package name (passed via the `package` parameter) to suggest specific reinstallation commands for the affected backend.

## Retry Logic for Transient Failures

Unlike static existence checks, the probe module implements **intelligent retry logic** for transient failures. The `probe_command()` function accepts configurable `retries` and `timeout` parameters.

Retries are only attempted for **`timeout`** and **`error`** statuses because these conditions may resolve spontaneously due to temporary system load or network issues. The module **does not** retry **`missing`** or **`broken`** statuses, as these represent persistent configuration problems that require manual intervention.

```python
from agent_reach.probe import probe_command

# Simple health check for the `yt-dlp` executable

result = probe_command("yt-dlp")
if result.ok:
    print("yt‑dlp is ready:", result.output.splitlines()[0])
elif result.status == "broken":
    print("Installation is broken:", result.hint)
elif result.status == "missing":
    print("yt‑dlp not installed")
else:
    print(f"{result.status.title()} – details: {result.output}")

```

```python

# Probe with custom arguments and retries

result = probe_command(
    cmd="git",
    args=("--version",),
    timeout=5,
    retries=2,
    package="git"  # used in hint if broken

)

print(f"Status: {result.status}")
if result.hint:
    print("Hint:", result.hint)

```

## Integration with the Doctor CLI

The probe module integrates directly with the system's health monitoring through [`agent_reach/doctor.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/doctor.py). Each channel's `check()` method (defined in files like [`agent_reach/channels/twitter.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/channels/twitter.py)) calls `probe_command()` to validate required CLI tools before operations proceed.

This integration ensures that the `agent_reach.doctor` CLI reports **true backend health**—confirming that executables are present *and* runnable—rather than merely confirming file existence. The subprocess execution utilizes `utf8_subprocess_env()` from [`agent_reach/utils/process.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/utils/process.py) to ensure a clean UTF-8 environment, preventing locale-related execution failures that might otherwise be misclassified as broken binaries.

## Summary

- **Live execution validation**: The probe module runs binaries with `--version` to verify they execute correctly, not just that they exist on PATH.
- **Five-status classification**: Results are categorized as `ok`, `broken`, `timeout`, `error`, or `missing` for precise diagnostics.
- **Broken detection**: Identifies corrupted installations via exit codes 126/127 and `OSError`, providing `reinstall_hint()` guidance.
- **Smart retries**: Only retries transient `timeout` and `error` conditions, avoiding wasted cycles on permanent `missing` or `broken` states.
- **Doctor integration**: [`agent_reach/doctor.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/doctor.py) leverages `probe_command()` through channel `check()` methods to ensure all backends are truly operational.

## Frequently Asked Questions

### What happens if the binary exists but is corrupted?

The probe module detects corrupted or unexecutable binaries as **`broken`**. When `subprocess.run` raises `FileNotFoundError` or `OSError`, or when the process returns exit codes 126 or 127, the module sets the status to `broken` and generates a remediation hint via `reinstall_hint()`. This specifically handles cases like stale virtual environments or missing interpreter shebangs that cause file existence without executability.

### How does the probe module handle slow-responding backends?

Slow responses are classified as **`timeout`** when they exceed the default 10-second limit (configurable via the `timeout` parameter). Because timeouts may indicate temporary system congestion rather than permanent failure, the `probe_command()` function can retry these attempts based on the configured `retries` count, giving the backend multiple chances to respond before finalizing the status.

### Why doesn't the probe module retry missing or broken statuses?

The module skips retries for **`missing`** and **`broken`** statuses because these represent **persistent configuration issues** that cannot self-heal. A missing binary requires installation, and a broken binary requires manual repair or reinstallation. Retrying these conditions would waste resources, whereas `timeout` and `error` conditions may resolve spontaneously due to temporary environmental factors.

### How does the doctor command use probe results?

The `agent_reach.doctor` CLI calls each channel's `check()` method (found in files like [`agent_reach/channels/twitter.py`](https://github.com/Panniantong/Agent-Reach/blob/main/agent_reach/channels/twitter.py)), which internally invokes `probe_command()`. This allows the doctor to report comprehensive health status for all backends, distinguishing between fully operational tools (`ok`), non-functional installations (`broken`), and missing dependencies (`missing`), complete with remediation hints for repair.