# How celld Implements the Peer Probe Mechanism for Secure Health Checking

> Discover how celld secures health checks with its signed peer probe mechanism. Nodes cryptographically verify each other using Ed25519 keys over HTTP for robust, decentralized validation.

- Repository: [Deno/celld](https://github.com/denoland/celld)
- Tags: internals
- Published: 2026-08-09

---

**Celld uses a signed, challenge-response peer probe mechanism where nodes verify each other's health by exchanging cryptographically signed challenges over HTTP, proving possession of Ed25519 private keys without relying on external services.**

The `denoland/celld` repository implements a robust **peer probe mechanism** to validate that remote diagnostic nodes are both reachable and correctly configured. This cryptographic health-checking system ensures cluster nodes can verify each other's identities using Ed25519 signatures and canonical challenge payloads, eliminating reliance on mutable state or third-party services.

## How the Peer Probe Mechanism Works in celld

### Signer Initialization and Key Management

When a celld instance starts, it initializes its cryptographic identity through `PeerProbeSigner::generate` in [`crates/celld/peer_probe.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_probe.rs) (lines 30-36). The system either generates a fresh Ed25519 signing key or loads an existing one from the `CELLD_REEXEC_PROBE_SIGNING_KEY` environment variable (lines 68-76). The resulting public key is hex-encoded and stored in the global `INSTALLED_SIGNER`, then advertised to peers via the lease metadata field `probe_public_key`.

### Generating and Sending Cryptographic Challenges

The probing node initiates health checks by generating a 32-byte random challenge using `random_challenge()` (lines 96-100). This value is transmitted in the HTTP header `x-cells-probe-challenge` to the target node's `/__celld/probe` endpoint. The request itself is authenticated using celld's standard peer-authentication scheme via `PeerAuth::sign` (lines 27-34), ensuring only authorized cluster members can issue probes.

### Canonical Payload Signing and Response

Upon receiving a probe request, the target node invokes `PeerProbeSigner::respond` (lines 47-64) to validate the incoming challenge and generate a signed response. The signer constructs a canonical payload with the following strict format:

```text
cells‑peer‑probe‑v1
<node>
<advertise>
<challenge>

```

The resulting `PeerProbeResponse` contains the version, node identifier, advertised address, challenge, and a hex-encoded signature (lines 21-28).

### Response Verification and Health Determination

The probing node verifies responses through the `verify` method (lines 66-90), which validates that the version, node name, advertise address, and challenge match expectations, and cryptographically verifies the signature against the public key stored in the node lease (`probe_public_key`). Any signature mismatch, field discrepancy, or oversized body triggers an error, marking the node as unhealthy. Successful verification confirms the target possesses the correct private key and matches its lease advertisement.

## Security Integration with Peer Authentication

The peer probe mechanism leverages the existing peer authentication layer implemented in [`crates/celld/peer_auth.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs). According to the source code, probe requests utilize `PeerAuth` for signing, while response validation checks the `x-cells-peer-version` header through `validate_response` (lines 108-119). This integration provides replay protection, timestamp validation, and nonce guarantees that protect against man-in-the-middle and replay attacks.

## Implementing Peer Probes in Practice

### Installing the Probe Signer at Startup

During node initialization, install the signer and expose the public key in the lease metadata:

```rust
// Executed during startup sequence
let public_key = celld::peer_probe::install_signer()?;
// Store `public_key` in node lease as `probe_public_key` for peer verification

```

### Initiating Health Checks from Client Nodes

To check a remote node's health, use the high-level `probe` function which handles the full request-response cycle:

```rust
use reqwest::Client;
use celld::{peer_probe, peer_auth::PeerAuth};

async fn verify_node_health(
    client: &Client, 
    node: &DiagnosticNode, 
    auth: &PeerAuth
) -> anyhow::Result<()> {
    // Performs challenge generation, request signing, and response verification
    peer_probe::probe(client, node, auth).await?;
    println!("Node {} is healthy and authentic", node.node);
    Ok(())
}

```

### Handling Incoming Probe Requests

Target nodes expose the `/__celld/probe` endpoint to respond to health checks:

```rust
use axum::extract::State;
use celld::peer_probe::PeerProbeResponse;

async fn probe_handler(
    headers: axum::http::HeaderMap,
) -> Result<axum::Json<PeerProbeResponse>, axum::http::StatusCode> {
    let challenge = headers
        .get("x-cells-probe-challenge")
        .ok_or(axum::http::StatusCode::BAD_REQUEST)?
        .to_str()
        .map_err(|_| axum::http::StatusCode::BAD_REQUEST)?;
    
    let response = celld::peer_probe::respond(
        &node_name,
        &advertise_addr,
        challenge,
    ).map_err(|_| axum::http::StatusCode::INTERNAL_SERVER_ERROR)?;
    
    Ok(axum::Json(response))
}

```

## Summary

- **Ed25519 Cryptography**: The peer probe mechanism uses Ed25519 signatures to bind node identity to health check responses, with keys generated via `PeerProbeSigner::generate` in [`crates/celld/peer_probe.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_probe.rs).
- **Challenge-Response Protocol**: Each probe sends a 32-byte random challenge via the `x-cells-probe-challenge` header and verifies the signed canonical payload containing node metadata.
- **Lease-Bound Verification**: Responses are validated against the `probe_public_key` stored in the node lease, ensuring the responding node matches its advertised identity.
- **Layered Security**: Probe requests are signed via `PeerAuth::sign` and responses validated through `validate_response` in [`crates/celld/peer_auth.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs), providing replay protection and timestamp verification.

## Frequently Asked Questions

### What cryptographic algorithm does the peer probe mechanism use?

The celld peer probe mechanism uses **Ed25519** for digital signatures. When a node starts, `PeerProbeSigner::generate` creates an Ed25519 signing key (lines 30-36), or loads one from the `CELLD_REEXEC_PROBE_SIGNING_KEY` environment variable. The corresponding public key is distributed through the lease metadata, allowing peers to verify probe responses cryptographically.

### How does celld prevent replay attacks during peer health checks?

Replay protection comes from the integration with celld's peer authentication layer. According to [`crates/celld/peer_auth.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs) (lines 108-119), the `validate_response` function checks the `x-cells-peer-version` header and enforces timestamp and nonce validation. Additionally, the 32-byte random challenge in each probe request ensures responses cannot be reused across sessions.

### What happens if a node fails the peer probe verification?

If verification fails—whether due to an invalid signature, mismatched node identity, incorrect challenge response, or oversized body—the `verify` method in [`crates/celld/peer_probe.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_probe.rs) (lines 66-90) returns an error. The probing node interprets this as a health check failure, marking the target as unhealthy and preventing it from participating in cluster operations until it passes subsequent checks.

### Does the peer probe mechanism require external services or databases?

No. The peer probe mechanism is self-contained and does not rely on external services or mutable state. It operates purely through cryptographic proof—specifically, possession of the Ed25519 private key corresponding to the public key advertised in the node lease. This design allows health checks to function even during network partitions or third-party service outages.