# What Information Does `celld diagnose` Provide and How It Probes Peers

> Understand what celld diagnose reveals about your Celld nodes. This command performs signed HTTP health checks, returning real-time metrics like memory, CPU, and connection counts.

- Repository: [Deno/celld](https://github.com/denoland/celld)
- Tags: how-to-guide
- Published: 2026-08-09

---

**The `celld diagnose` command performs signed HTTP health checks against Celld nodes stored in an S3 bucket, returning real-time metrics including resident cells, memory usage, CPU load, and connection counts while validating TLS handshakes and protocol versions.**

The `celld diagnose` command is a fleet health inspection tool in the [denoland/celld](https://github.com/denoland/celld) repository that verifies node availability and resource utilization across distributed deployments. By leveraging S3-backed node leases and cryptographic peer authentication, it provides operators with a secure, real-time view of cluster status without requiring direct shell access to individual instances.

## Information Provided by `celld diagnose`

When executing against a live fleet, the command aggregates critical health indicators from each responsive peer. The output format follows a structured line protocol beginning with `ok peer`, making it suitable for automated monitoring pipelines.

### Runtime Resource Metrics

For every successfully probed node, `celld diagnose` exposes the following operational data points:

- **Node identification**: Node ID and advertised address
- **Protocol version**: The celld protocol version the peer is running
- **Cell metrics**: Count of `resident_cells` and `shed_cells`
- **Connection state**: Number of active `websockets`
- **System resources**: `rss_bytes` (memory), `cpu_percent` utilization, and file descriptor usage (`fds`)
- **Pressure indicators**: Boolean `pressured` flag and `load_age_ms` showing sample freshness

### Fleet Enumeration Metadata

Beyond individual node statistics, the command reports:
- Total node leases discovered in the S3 bucket
- Count of expired leases skipped during the run
- Bucket validation status

## How `celld diagnose` Probes Peers

The probing workflow implemented in [[`crates/celld/fleet.rs`](https://github.com/denoland/celld/blob/main/crates/celld/fleet.rs)](https://github.com/denoland/celld/blob/main/crates/celld/fleet.rs) executes a multi-stage validation pipeline that ensures only authorized, reachable, and healthy nodes are reported.

### Peer Discovery via S3 Bucket Leases

The command requires the `--bucket` argument to locate the fleet state. If specific `--peer` IDs are not provided, the system enumerates all available nodes by calling `diagnostic_node_ids` against the S3 storage backend. When explicit peer IDs are supplied via `--peer`, the command targets only those specific nodes, bypassing full enumeration.

### Secure HTTP Client Configuration

Before initiating probes, `celld diagnose` constructs a hardened HTTP client with strict timeouts to prevent hanging on unresponsive hosts:

```rust
let http = reqwest::Client::builder()
    .connect_timeout(Duration::from_secs(3))
    .timeout(Duration::from_secs(5))
    .redirect(reqwest::redirect::Policy::none())
    .build()?;

```

This configuration enforces a 3-second connection timeout and 5-second total request timeout, with redirects explicitly disabled to prevent request hijacking.

### Cryptographic Authentication

Each probe request is signed using `PeerAuth::new(..., "diagnostic")` as implemented in [[`crates/celld/peer_auth.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs)](https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs). This generates a diagnostic authentication token signed with the node's private key, allowing the target peer to verify that the request originates from a legitimate diagnostic client rather than an unauthorized third party.

### Direct Probe Execution

The actual health check occurs in [[`crates/celld/peer_probe.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_probe.rs)](https://github.com/denoland/celld/blob/main/crates/celld/peer_probe.rs) via the `probe(&http, &node, &auth).await` function. This performs a signed HTTP request to the peer's internal diagnostic endpoint, validating:

- TLS handshake integrity
- Protocol version compatibility
- Node identity confirmation

### Address Validation and Safety Controls

Before probing, the command parses the advertised address in [[`crates/celld/startup.rs`](https://github.com/denoland/celld/blob/main/crates/celld/startup.rs)](https://github.com/denoland/celld/blob/main/crates/celld/startup.rs). If the address resolves to a public IP address and the `--unsafe-public-advertise` flag is not set, the probe is rejected to prevent accidental exposure of diagnostic endpoints to the internet. Expired node leases are automatically skipped and counted in the final summary.

## Practical Usage Examples

Diagnose an entire fleet stored in S3:

```bash
celld diagnose --bucket s3://my-celld-bucket

```

Check specific nodes only:

```bash
celld diagnose --bucket s3://my-celld-bucket --peer node-a --peer node-b

```

Allow probing of publicly advertised IPs (use with caution):

```bash
celld diagnose --bucket s3://my-celld-bucket --unsafe-public-advertise

```

### Sample Output

```

ok bucket s3://my-celld-bucket
ok fleet 3 node lease(s) enumerated
ok peer node-a at 10.0.0.2:8000 (signed direct probe) protocol=2 resident_cells=120 websockets=3 rss_bytes=45MiB cpu_percent=12.34 fds=45/1024 pressured=false shed_cells=0 load_age_ms=1500
ok peer node-b at 10.0.0.3:8000 (signed direct probe) protocol=2 resident_cells=98 websockets=1 rss_bytes=32MiB cpu_percent=8.10 fds=23/1024 pressured=false shed_cells=0 load_age_ms=900
ok fleet skipped 1 expired node lease(s)

```

## Summary

- **`celld diagnose`** requires an S3 bucket (`--bucket`) to locate node leases and performs signed HTTP health checks against discovered peers.
- The command reports detailed runtime metrics including **resident cells**, **RSS memory**, **CPU percentage**, **websocket counts**, and **pressure status** for each responsive node.
- Probes use a **3-second connection timeout** and **5-second total timeout** with redirects disabled, utilizing `PeerAuth::new` for cryptographic request signing.
- By default, probes targeting **public IP addresses are rejected** unless `--unsafe-public-advertise` is explicitly enabled.
- Expired node leases are automatically skipped during enumeration but reported in the final summary statistics.

## Frequently Asked Questions

### What happens if a node lease is expired?

When `celld diagnose` encounters an expired lease during enumeration via `diagnostic_node`, it increments the expired lease counter and skips the probe for that node. The command continues processing remaining peers and reports the total skipped count in the final summary line (e.g., `ok fleet skipped 1 expired node lease(s)`), exiting successfully unless active probes fail.

### Why does `celld diagnose` reject public IP addresses by default?

As implemented in [`crates/celld/startup.rs`](https://github.com/denoland/celld/blob/main/crates/celld/startup.rs), the command rejects advertised addresses that resolve to public IPs unless the `--unsafe-public-advertise` flag is provided. This security measure prevents accidental exposure of diagnostic endpoints to untrusted networks, ensuring that health checks remain within private infrastructure boundaries unless explicitly overridden by the operator.

### How is the diagnostic request authenticated?

The command generates a signed authentication token using `PeerAuth::new(..., "diagnostic")` from [`crates/celld/peer_auth.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs). This token is cryptographically signed with the probing node's private key, allowing the target peer to verify the request's legitimacy through the `probe` function in [`crates/celld/peer_probe.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_probe.rs) before returning sensitive runtime metrics.

### Can I diagnose a single node instead of the entire fleet?

Yes. Rather than relying on automatic enumeration via `diagnostic_node_ids`, you can target specific nodes by passing one or more `--peer` arguments with explicit node IDs. This bypasses full fleet discovery and probes only the specified peers, reducing execution time and S3 API calls when checking individual node health.