Celld Peer Authentication HMAC Model: Securing Node-to-Node Communication with Fleet Secrets

Celld authenticates peer-to-peer HTTP requests using a shared fleet secret to generate HMAC-SHA256 signatures over canonical request strings, with replay protection via timestamps and nonces.

The peer authentication HMAC model in the denoland/celld repository enables distributed nodes to establish trust without external certificate authorities. Every node shares a single fleet secret stored in a central bucket, allowing each peer to cryptographically sign outgoing requests and verify incoming ones using symmetric key cryptography.

How the Fleet Secret Establishes Trust

The foundation of Celld’s security model is a 32-byte HMAC key shared across the entire fleet. This secret is persisted in the bucket at fleet/peer-auth.json and is automatically initialized if absent.

In crates/celld/peer_auth.rs, the load_or_create function (lines 59–79) handles this initialization:

• If the file exists, it loads the existing key.
• If missing, it generates a new 32-byte secret and stores it in the bucket for other nodes to access.

// Load existing secret or create a new one if missing
let key = PeerAuth::load_or_create(&bucket).await?;   // ↗ https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs#L59-L79
let auth = PeerAuth::new(key, "node-01")?;

Canonical Request Signing Process

When a node initiates an HTTP request to a peer, it constructs a deterministic signature using the fleet secret. The process involves three distinct steps implemented in crates/celld/peer_auth.rs.

Deriving the Canonical Request String

The PeerAuth::mac method (lines 38–51) builds a canonical string that concatenates:

• A fixed domain identifier
• Protocol version (2)
• HTTP method (e.g., POST)
• Request path and query string
• SHA-256 hash of the request body
• Source and target node identities
• Unix-epoch timestamp (milliseconds)
• Random 16-byte nonce

This canonicalization ensures that any modification to the request—whether in transit or by an intermediary—invalidates the signature.

Computing the HMAC-SHA256 Signature

Using the fleet secret stored in self.key, the system computes the signature:

// HMAC object created from the fleet secret
Hmac<Sha256>::new_from_slice(&self.key)  // ↗ https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs#L52-L55

The resulting MAC is serialized as a lowercase hexadecimal string via encode_hex and placed in the x-cells-peer-signature header.

Required HTTP Headers

The PeerAuth::signed_headers method (lines 30–63) assembles the complete set of authentication headers:

• x-cells-peer-version: Protocol version (2)
• x-cells-peer-source: Identity of the calling node
• x-cells-peer-target: Identity of the receiving node
• x-cells-peer-timestamp: Unix-epoch milliseconds when signed
• x-cells-peer-nonce: 16-byte random nonce, hex-encoded
• x-cells-peer-body-sha256: SHA-256 hash of the request body

let client = reqwest::Client::new();
let builder = client.post("https://peer.example/api/do");
// PeerAuth produces the required signed headers
let signed = auth.sign(
    builder,
    "POST",
    "/api/do?param=1",
    b"{\"payload\":42}",
    "node-02",
)?;                                                // ↗ https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs#L12-L22
let resp = signed.send().await?;

Verification and Replay Protection

The receiving node reverses the signing process to validate authenticity and integrity. Verification is implemented in the same peer_auth.rs module and includes multiple defensive layers.

Signature Verification Steps

The verification routine performs the following checks in sequence:

1. Protocol version validation: Ensures the x-cells-peer-version header matches the expected value (2).
2. Identity validation: Validates source and target identities using celld_logic::peer::valid_identity (lines 25–33).
3. Timestamp validation: Confirms the request timestamp falls within CLOCK_WINDOW_MS = 30000 milliseconds (30 seconds) of the current time to prevent delayed replay attacks.
4. MAC recomputation: Rebuilds the canonical request string from the received headers and body, then recomputes the HMAC using the fleet secret.
5. Signature comparison: Verifies the received signature against the computed MAC using mac.verify_slice(&signature).

// Inside an Axum handler
let method = req.method();
let path = req.uri().path_and_query().map(|pq| pq.as_str()).unwrap_or("/");
let body = hyper::body::to_bytes(req.into_body()).await?;
let headers = req.headers();

auth.verify(
    method,
    path,
    headers,
    &body,
    "node-02",               // expected target identity
)?;                                               // ↗ https://github.com/denoland/celld/blob/main/crates/celld/peer_auth.rs#L65-L73

Nonce-Based Replay Prevention

Even within the valid time window, Celld prevents replay attacks using an in-memory ReplayCache. After successful verification, the nonce is stored for at least REPLAY_RETENTION_MS (twice the clock window, i.e., 60 seconds). If the same nonce appears again before expiration, verification fails with VerifyError::Replay.

The cache enforces a hard memory limit via MAX_REPLAY_ENTRIES = 1000000 entries, ensuring the system cannot be exhausted by malicious traffic (see lines 22–30 and verification logic at lines 17–31).

match auth.verify(...) {
    Ok(_) => println!("request accepted"),
    Err(PeerAuth::VerifyError::Replay) => eprintln!("replay detected"),
    Err(err) => eprintln!("auth failure: {}", err.message()),
}

Summary

• Fleet Secret: A 32-byte shared key stored in fleet/peer-auth.json, loaded via load_or_create in crates/celld/peer_auth.rs.
• Canonical Requests: Deterministic string construction in PeerAuth::mac concatenates method, path, body hash, identities, timestamp, and nonce.
• HMAC-SHA256: Symmetric signing using Hmac<Sha256>::new_from_slice with fleet secret, outputting hex-encoded signatures in x-cells-peer-signature.
• Required Headers: Six custom headers communicate version, identities, timestamp, nonce, and body hash.
• Replay Protection: 30-second clock window plus nonce caching with ReplayCache and MAX_REPLAY_ENTRIES limit.

Frequently Asked Questions

What is the fleet secret in Celld's peer authentication?

The fleet secret is a 32-byte symmetric key shared by all nodes in a Celld cluster. Stored in the fleet/peer-auth.json bucket file, it serves as the HMAC key for signing and verifying inter-node HTTP requests. Each node loads this secret via PeerAuth::load_or_create in crates/celld/peer_auth.rs (lines 59–79).

How does Celld prevent replay attacks in peer-to-peer communication?

Celld employs a dual-layer defense: timestamps must fall within a 30-second window (CLOCK_WINDOW_MS), and each 16-byte nonce is tracked in an in-memory ReplayCache for 60 seconds (REPLAY_RETENTION_MS). If a nonce reappears before expiration, verification fails with VerifyError::Replay, preventing attackers from retransmitting captured requests.

Which HTTP headers are required for Celld peer authentication?

Nodes must transmit six headers: x-cells-peer-version (protocol version 2), x-cells-peer-source (caller identity), x-cells-peer-target (callee identity), x-cells-peer-timestamp (Unix ms), x-cells-peer-nonce (16-byte hex), and x-cells-peer-body-sha256 (body hash). The signature itself resides in x-cells-peer-signature.

Where is the peer authentication logic implemented in the Celld codebase?

The core implementation resides in crates/celld/peer_auth.rs, containing PeerAuth::mac for signing, PeerAuth::verify for validation, and ReplayCache for nonce tracking. Auxiliary identity validation functions are imported from crates/logic/peer.rs, while the bucket abstraction for secret storage is defined in crates/celld/fleet.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}