How celld Implements Pressure Shedding Using Resident-Cell Watermarks

Celld uses dynamic resident-cell watermarks calculated at runtime to determine exactly how many cells to evict when memory or CPU pressure exceeds configured thresholds, preventing oscillation through proportional reduction rather than static caps.

The denoland/celld repository implements a sophisticated pressure shedding mechanism that separates resource monitoring from eviction targeting. Unlike static cell limits, celld employs a two-tier watermark system where high-level resource thresholds (RSS/CPU) trigger shedding mode, while a derived resident-cell watermark dictates precisely how many cells must be released to relieve pressure.

Understanding celld’s Two-Level Watermark System

Celld distinguishes between resource watermarks and resident-cell watermarks to decouple pressure detection from eviction quantity.

Resource watermarks define the trigger thresholds. In crates/logic/pressure.rs, the PressureConfig struct stores these at lines 21-23:

pub struct PressureConfig {
    pub rss_high_bytes: Option<usize>,
    pub cpu_high_x100: Option<usize>,
}

These values represent the memory and CPU utilization levels that initiate shedding.

Resident-cell watermarks function differently. Defined in the Load struct at lines 30-33 of the same file, this watermark tracks the current number of cells resident in memory:

pub struct Load {
    pub resident_cells: usize,
    pub rss_bytes: usize,
    pub cpu_percent_x100: usize,
}

This is not a hard admission cap but a dynamic baseline used to calculate reduction targets when pressure triggers fire.

How Pressure Detection Triggers Shedding

When the system samples resource utilization, the load_sampled function in crates/logic/lib.rs (lines 3557-3561) evaluates whether to enter shedding state:

let state = self.config.pressure.state(load, self.shedding);
self.shedding = state.shedding;
self.shed_reason = self
    .config
    .pressure
    .shedding_trigger(load, state.shedding)
    .or(state.trigger);

The PressureConfig::state method compares the current Load sample against the high watermarks. If RSS or CPU exceeds the configured thresholds, state.shedding returns true, latching the node into shedding mode. This latch persists until resource levels drop below relief thresholds (typically 80% of the high watermarks), preventing rapid oscillation.

Calculating the Resident-Cell Watermark Target

Once shedding activates, celld calculates how many cells to retain using the resident-cell watermark. At line 3594 of crates/logic/lib.rs, the code computes a floor value:

self.shed_floor = self.config.pressure.release_target(load.resident_cells);

The release_target function in crates/logic/pressure.rs (lines 64-70) implements a proportional reduction strategy:

pub fn release_target(self, resident_cells: usize) -> usize {
    resident_cells.saturating_sub((resident_cells / 10).max(1))
}

This formula reduces the resident set by approximately 10 percent, ensuring at least one cell is always evicted (the .max(1) guard), but scales the reduction with the current memory pressure. A node holding 10,000 cells will shed roughly 1,000, while a node with 10 cells sheds only 1.

The Eviction Loop: From Watermark to Action

With the target floor established, the shed_toward_floor function (lines 3622-3629 in crates/logic/lib.rs) drives the actual eviction:

while self.hibernation_permits.len() < self.config.max_hibernations
    && self.occupied()
        .saturating_sub(self.hibernation_permits.len())
    > self.shed_floor
{
    // select idle candidate and begin eviction...
}

The loop continues issuing hibernation permits until the occupied cell count drops to or below the calculated shed_floor. This ensures celld meets the resident-cell watermark target before exiting the pressure relief cycle.

Configuring Pressure Thresholds

To implement this behavior, instantiate PressureConfig with your resource boundaries:

use celld_logic::pressure::{PressureConfig, Load};

let cfg = PressureConfig {
    rss_high_bytes: Some(4 * 1024 * 1024 * 1024), // 4 GiB memory limit
    cpu_high_x100: Some(8000),                    // 80% CPU limit
};

let current_load = Load {
    resident_cells: state.occupied(),
    rss_bytes: get_current_rss(),
    cpu_percent_x100: get_current_cpu(),
};

if cfg.state(current_load, false).shedding {
    let target = cfg.release_target(current_load.resident_cells);
    println!("Reducing resident cells to ≤ {}", target);
}

The high watermarks act as the trigger mechanism, while release_target dynamically derives the resident-cell watermark based on current occupancy.

Summary

• Celld separates concerns between resource watermarks (RSS/CPU triggers) and resident-cell watermarks (eviction quantity targets).
• Dynamic proportional reduction via release_target scales shedding volume with current load, shedding roughly 10% of resident cells per trigger.
• Hysteresis protection prevents oscillation by requiring resource levels to drop to relief thresholds (80% of highs) before clearing the shedding latch.
• File locations: Configuration resides in crates/logic/pressure.rs, while orchestration logic appears in crates/logic/lib.rs.
• Eviction follows the floor: The system evicts cells only until reaching the calculated shed_floor, ensuring minimal necessary disruption.

Frequently Asked Questions

What is the difference between resource watermarks and resident-cell watermarks in celld?

Resource watermarks are static thresholds configured in PressureConfig (rss_high_bytes and cpu_high_x100) that determine when to start shedding. Resident-cell watermarks are dynamic values derived at runtime from the current Load::resident_cells count that determine how many cells to evict. The former acts as a binary switch, while the latter provides a quantitative target.

How does celld determine how many cells to evict during pressure shedding?

The eviction count is calculated by release_target in crates/logic/pressure.rs, which subtracts roughly 10% (minimum 1) from the current resident cell count. This value becomes shed_floor, and the system evicts cells until occupancy falls to this level or below.

Why does celld use a proportional reduction rather than a fixed number for shedding?

Proportional reduction ensures that small deployments (few cells) shed minimally while large deployments shed sufficiently to relieve pressure. A fixed number might be catastrophic for small nodes (shedding 100% of 10 cells) or ineffective for large nodes (shedding 10 cells when holding 100,000). The (resident_cells / 10).max(1) formula scales appropriately with deployment size.

How does celld prevent oscillation between shedding and normal operation?

Celld implements hysteresis through separate high and low watermarks. The state method only clears the shedding latch when resource usage drops below relief thresholds (approximately 80% of the high watermark values). This ensures the node remains in shedding mode long enough to stabilize rather than rapidly toggling states when hovering near the threshold.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}