Memory Pressure Shedding in Deno celld: How Nodes Release Cells Under Load

Memory pressure shedding in celld is a pure classifier system that uses configurable watermarks and hysteresis latches to trigger cell eviction when memory usage exceeds safe thresholds.

The celld runtime—Deno's distributed cell-based execution system—implements a deterministic, side-effect-free mechanism for deciding when a node must relinquish cells back to the cluster. This article examines how the memory pressure shedding mechanism works, its architectural design, and how operators can configure it.

Core Components of the Shedding Classifier

The entire mechanism resides in crates/logic/pressure.rs. It operates as a pure function: given a memory snapshot and previous state, it returns a decision without performing I/O or reading the clock.

PressureConfig

PressureConfig stores the two optional watermarks that drive shedding decisions:

• high_bytes — the ordinary memory ceiling for cell-in-use bytes
• rss_hard_bytes — an absolute cap on resident-set size (RSS)

Configuration is built once at startup via PressureConfig::from_limits, which parses environment values and handles missing or zero inputs gracefully【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L19-L43】【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L97-L105】.

Load

The Load struct contains the only inputs the classifier reads:

• resident_cells — current cell count on the node
• rss_bytes — actual resident-set size
• in_use_bytes — memory actively used by cells【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L71-L84】

Latches and Hysteresis

Two boolean latches—memory and rss_hard—remember whether each ceiling has been crossed. This hysteresis prevents oscillation: once latched, a ceiling stays engaged until the sample drops below 80% of the threshold (sample > ceiling * 4 / 5)【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L53-L58】【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L66-L73】.

Metric Selection

When shedding triggers, PressureConfig::walk_metric selects which measurement guides eviction:

• Metric::InUse — evict based on in_use_bytes (ordinary memory pressure)
• Metric::Rss — evict based on rss_bytes (hard RSS cap)【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L66-L69】【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L84-L97】

How Memory Pressure Shedding Works

1. Configuration at Startup

The node's main entry point calls pressure_config_from_environment to build PressureConfig. Operators control behavior through two inputs:

• Total machine memory (detected automatically)
• Optional CELLD_MAX_RSS_MB environment variable

From these, from_limits computes high_bytes and rss_hard_bytes with safe defaults【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L97-L105】.

2. Classification on Each Sample

The state machine invokes PressureConfig::classify in State::load_sampled【/cache/repos/github.com/denoland/celld/main/crates/logic/lib.rs#L3661-L3665】. The method:

1. Compares in_use_bytes against high_bytes and rss_bytes against rss_hard_bytes
2. Applies the over closure to implement 80% hysteresis
3. Returns updated Latches and an optional shedding reason

Shedding reasons follow priority order: ** SHED_RSS_HARD > SHED_MEMORY**【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L86-L94】.

3. Target Calculation and Pin Guarding

Once shedding activates:

• release_target computes eviction volume as roughly 10% of resident cells (minimum 1 cell)【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L99-L104】
• may_pin_outbound limits WebSocket-pinned cells to preserve the eviction pool【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L108-L125】

Code Example: Using the Pressure Classifier

use celld_logic::pressure::{PressureConfig, Load, Latches};

// Build configuration once at startup (from environment)
let config = PressureConfig::from_limits(
    Some(total_memory_bytes),
    std::env::var("CELLD_MAX_RSS_MB")
        .ok()
        .and_then(|s| s.parse::<u64>().ok()),
);

// Memory sample gathered by the runtime
let load = Load {
    resident_cells: 10_000,
    rss_bytes: 2_147_483_648,      // 2 GB
    in_use_bytes: 1_800_000_000,   // ~1.8 GB active
};

// Previous latch state from last classification
let old_latches = Latches { memory: false, rss_hard: false };

// Classify and decide
let (new_latches, reason) = config.classify(load, old_latches);

match reason {
    Some(celld_logic::pressure::SHED_RSS_HARD) => {
        // Hard RSS cap exceeded — use Rss metric for eviction
        let metric = PressureConfig::walk_metric(new_latches);
        let target = PressureConfig::release_target(load.resident_cells);
        evict_cells(metric, target);
    }
    Some(celld_logic::pressure::SHED_MEMORY) => {
        // Ordinary memory ceiling exceeded — use InUse metric
        let metric = PressureConfig::walk_metric(new_latches);
        let target = PressureConfig::release_target(load.resident_cells);
        evict_cells(metric, target);
    }
    None => {
        // No shedding required
    }
}

Architectural Design Principles

The celld memory pressure shedding mechanism follows three key design principles:

• Purity — classify has no side effects, making it trivial to unit test and deterministic under replay
• Separation of concerns — The classifier decides whether to shed; walk_down elsewhere implements how to evict
• Operator tunability — Watermarks are externalized via environment variables without requiring recompilation

Summary

• Memory pressure shedding in celld is implemented as a pure classifier in crates/logic/pressure.rs
• Two watermarks control behavior: high_bytes for ordinary memory and rss_hard_bytes for RSS limits
• Hysteresis latches prevent thrashing with an 80% release threshold
• Eviction targeting uses release_target (~10% of cells) and walk_metric to select the appropriate measurement
• Integration point is State::load_sampled in crates/logic/lib.rs, where classification results feed the state machine

Frequently Asked Questions

How do I configure the RSS hard limit for celld memory pressure shedding?

Set the CELLD_MAX_RSS_MB environment variable before starting the node. The value is parsed in PressureConfig::from_limits and converted to rss_hard_bytes. If unset or zero, no hard RSS cap applies. The total machine memory is auto-detected and used to compute high_bytes【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L97-L105】.

Why does celld use 80% hysteresis in its memory pressure latches?

The 80% threshold (sample > ceiling * 4 / 5) prevents rapid oscillation. Once a node crosses a memory ceiling and latches "on," it stays in shedding mode until usage drops substantially below the threshold. This dampens noise from natural memory fluctuations without requiring complex history tracking【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L66-L73】.

What happens when both SHED_MEMORY and SHED_RSS_HARD conditions trigger?

SHED_RSS_HARD takes priority. The shedding reasons are ordered, and the classifier returns the highest-priority reason found. This ensures catastrophic RSS exhaustion is handled before ordinary memory pressure. The walk_metric function then selects Metric::Rss for the eviction walk-down【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L86-L94】【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L84-L97】.

How does celld prevent outbound WebSocket connections from blocking memory pressure shedding?

The may_pin_outbound method caps pin allocations based on current latches and resident cells. When memory pressure is latched, this guard restricts how many cells outbound connections can pin, preserving sufficient cells in the eviction pool. The implementation scales the cap with available headroom【/cache/repos/github.com/denoland/celld/main/crates/logic/pressure.rs#L108-L125】.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}