What Information Does `celld diagnose` Provide and How It Probes Peers

The celld diagnose command performs signed HTTP health checks against Celld nodes stored in an S3 bucket, returning real-time metrics including resident cells, memory usage, CPU load, and connection counts while validating TLS handshakes and protocol versions.

The celld diagnose command is a fleet health inspection tool in the denoland/celld repository that verifies node availability and resource utilization across distributed deployments. By leveraging S3-backed node leases and cryptographic peer authentication, it provides operators with a secure, real-time view of cluster status without requiring direct shell access to individual instances.

Information Provided by celld diagnose

When executing against a live fleet, the command aggregates critical health indicators from each responsive peer. The output format follows a structured line protocol beginning with ok peer, making it suitable for automated monitoring pipelines.

Runtime Resource Metrics

For every successfully probed node, celld diagnose exposes the following operational data points:

• Node identification: Node ID and advertised address
• Protocol version: The celld protocol version the peer is running
• Cell metrics: Count of resident_cells and shed_cells
• Connection state: Number of active websockets
• System resources: rss_bytes (memory), cpu_percent utilization, and file descriptor usage (fds)
• Pressure indicators: Boolean pressured flag and load_age_ms showing sample freshness

Fleet Enumeration Metadata

Beyond individual node statistics, the command reports:

• Total node leases discovered in the S3 bucket
• Count of expired leases skipped during the run
• Bucket validation status

How celld diagnose Probes Peers

The probing workflow implemented in [crates/celld/fleet.rs](https://github.com/denoland/celld/blob/main/crates/celld/fleet.rs) executes a multi-stage validation pipeline that ensures only authorized, reachable, and healthy nodes are reported.

Peer Discovery via S3 Bucket Leases

The command requires the --bucket argument to locate the fleet state. If specific --peer IDs are not provided, the system enumerates all available nodes by calling diagnostic_node_ids against the S3 storage backend. When explicit peer IDs are supplied via --peer, the command targets only those specific nodes, bypassing full enumeration.

Secure HTTP Client Configuration

Before initiating probes, celld diagnose constructs a hardened HTTP client with strict timeouts to prevent hanging on unresponsive hosts:

let http = reqwest::Client::builder()
    .connect_timeout(Duration::from_secs(3))
    .timeout(Duration::from_secs(5))
    .redirect(reqwest::redirect::Policy::none())
    .build()?;

This configuration enforces a 3-second connection timeout and 5-second total request timeout, with redirects explicitly disabled to prevent request hijacking.

Cryptographic Authentication

Each probe request is signed using PeerAuth::new(..., "diagnostic") as implemented in [crates/celld/peer_auth.rs](https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs). This generates a diagnostic authentication token signed with the node's private key, allowing the target peer to verify that the request originates from a legitimate diagnostic client rather than an unauthorized third party.

Direct Probe Execution

The actual health check occurs in [crates/celld/peer_probe.rs](https://github.com/denoland/celld/blob/main/crates/celld/peer_probe.rs) via the probe(&http, &node, &auth).await function. This performs a signed HTTP request to the peer's internal diagnostic endpoint, validating:

• TLS handshake integrity
• Protocol version compatibility
• Node identity confirmation

Address Validation and Safety Controls

Before probing, the command parses the advertised address in [crates/celld/startup.rs](https://github.com/denoland/celld/blob/main/crates/celld/startup.rs). If the address resolves to a public IP address and the --unsafe-public-advertise flag is not set, the probe is rejected to prevent accidental exposure of diagnostic endpoints to the internet. Expired node leases are automatically skipped and counted in the final summary.

Practical Usage Examples

Diagnose an entire fleet stored in S3:

celld diagnose --bucket s3://my-celld-bucket

Check specific nodes only:

celld diagnose --bucket s3://my-celld-bucket --peer node-a --peer node-b

Allow probing of publicly advertised IPs (use with caution):

celld diagnose --bucket s3://my-celld-bucket --unsafe-public-advertise

Sample Output

ok bucket s3://my-celld-bucket
ok fleet 3 node lease(s) enumerated
ok peer node-a at 10.0.0.2:8000 (signed direct probe) protocol=2 resident_cells=120 websockets=3 rss_bytes=45MiB cpu_percent=12.34 fds=45/1024 pressured=false shed_cells=0 load_age_ms=1500
ok peer node-b at 10.0.0.3:8000 (signed direct probe) protocol=2 resident_cells=98 websockets=1 rss_bytes=32MiB cpu_percent=8.10 fds=23/1024 pressured=false shed_cells=0 load_age_ms=900
ok fleet skipped 1 expired node lease(s)

Summary

• celld diagnose requires an S3 bucket (--bucket) to locate node leases and performs signed HTTP health checks against discovered peers.
• The command reports detailed runtime metrics including resident cells, RSS memory, CPU percentage, websocket counts, and pressure status for each responsive node.
• Probes use a 3-second connection timeout and 5-second total timeout with redirects disabled, utilizing PeerAuth::new for cryptographic request signing.
• By default, probes targeting public IP addresses are rejected unless --unsafe-public-advertise is explicitly enabled.
• Expired node leases are automatically skipped during enumeration but reported in the final summary statistics.

Frequently Asked Questions

What happens if a node lease is expired?

When celld diagnose encounters an expired lease during enumeration via diagnostic_node, it increments the expired lease counter and skips the probe for that node. The command continues processing remaining peers and reports the total skipped count in the final summary line (e.g., ok fleet skipped 1 expired node lease(s)), exiting successfully unless active probes fail.

Why does celld diagnose reject public IP addresses by default?

As implemented in crates/celld/startup.rs, the command rejects advertised addresses that resolve to public IPs unless the --unsafe-public-advertise flag is provided. This security measure prevents accidental exposure of diagnostic endpoints to untrusted networks, ensuring that health checks remain within private infrastructure boundaries unless explicitly overridden by the operator.

How is the diagnostic request authenticated?

The command generates a signed authentication token using PeerAuth::new(..., "diagnostic") from crates/celld/peer_auth.rs. This token is cryptographically signed with the probing node's private key, allowing the target peer to verify the request's legitimacy through the probe function in crates/celld/peer_probe.rs before returning sensitive runtime metrics.

Can I diagnose a single node instead of the entire fleet?

Yes. Rather than relying on automatic enumeration via diagnostic_node_ids, you can target specific nodes by passing one or more --peer arguments with explicit node IDs. This bypasses full fleet discovery and probes only the specified peers, reducing execution time and S3 API calls when checking individual node health.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}