# How CubeSandbox Implements Resource Limits Using cgroups: A Deep Dive

> Discover how CubeSandbox uses cgroups and rustjail to implement robust resource limits for CPU, memory, PIDs, and I/O. Learn about cgroup v1 and v2 abstraction.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: deep-dive
- Published: 2026-07-05

---

**CubeSandbox enforces resource limits by isolating each workload in a Linux cgroup using the Rust-based `rustjail` component and the `cgroups-rs` crate, which abstracts both cgroup v1 and v2 hierarchies to apply CPU, memory, PID, and I/O constraints.**

CubeSandbox, TencentCloud's open-source sandbox runtime, leverages Linux **cgroups** (control groups) to provide hard resource boundaries for containerized workloads. The implementation resides primarily in the `rustjail` crate within the `agent` directory, where the `cgroups-rs` library bridges the gap between OCI container specifications and kernel-level resource controllers. This architecture ensures compatibility with both legacy cgroup v1 systems and modern unified cgroup v2 hierarchies.

## Cgroup Creation and Hierarchy Detection

When a sandbox starts, CubeSandbox must first determine whether the host kernel uses the legacy cgroup v1 or the unified cgroup v2 mode before creating the appropriate control group structure.

### Detecting Host Cgroup Version

The `FsManager::new` function in [`agent/rustjail/src/cgroups/fs/mod.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/rustjail/src/cgroups/fs/mod.rs) auto-detects the hierarchy type by calling `hierarchies::auto()`, which returns a boxed trait object implementing the `Hierarchy` trait. This detection happens at lines 45-58, where the code checks `h.v2()` to determine the execution path.

### Creating the Cgroup Object

Based on the detected version, the `new_cgroup` helper function instantiates the cgroup differently:

- **For cgroup v2**: Calls `cgroups::Cgroup::new()` with the hierarchy and path, allowing the library to translate kernel controller names into its unified model.
- **For cgroup v1**: Calls `cgroups::Cgroup::new_with_specified_controllers()`, explicitly passing `CUBE_CONTROLLER` (containing `cpu`, `memory`, `pids`, etc.) to create the legacy hierarchy.

```rust
// agent/rustjail/src/cgroups/fs/mod.rs
fn new_cgroup(h: Box<dyn cgroups::Hierarchy>, path: &str) -> Result<Cgroup> {
    let valid_path = path.trim_start_matches('/').to_string();
    if h.v2() {
        // unified cgroup v2
        return cgroups::Cgroup::new(h, &valid_path).context("create cgroup v2");
    }
    // legacy cgroup v1
    cgroups::Cgroup::new_with_specified_controllers(
        h,
        &valid_path,
        Some(CUBE_CONTROLLER.to_vec()),
    )
    .context("create cgroup v1")
}

```

## Mapping OCI Resources to Kernel Controls

CubeSandbox translates the OCI `LinuxResources` specification into `cgroups::Resources` structs through specialized helper functions in [`mod.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/mod.rs). Each resource type requires specific handling for v1 and v2 differences.

### CPU Limits and Shares

The `set_cpu_resources` function configures CPU constraints by writing to the `cpu` controller. For **cgroup v2**, it converts traditional share values to the kernel's "weight" format using `convert_shares_to_v2_value()`. For both versions, it sets CFS quota, period, and realtime limits.

```rust
// agent/rustjail/src/cgroups/fs/mod.rs (lines 30-41)
if let Some(shares) = cpu.shares {
    let shares = if cg.v2() {
        convert_shares_to_v2_value(shares)
    } else { shares };
    if shares != 0 { cpu_controller.set_shares(shares)?; }
}
set_resource!(cpu_controller, set_cfs_quota, cpu, quota);
set_resource!(cpu_controller, set_cfs_period, cpu, period);

```

### Memory Constraints and Swap Handling

The `set_memory_resources` function handles hard limits, soft limits, swap, kernel memory, and OOM-killer settings. It manages the order-dependent operations required by the kernel—specifically that memory limits must be set before swap limits when decreasing values. For **cgroup v2**, it converts swap values using `convert_memory_swap_to_v2_value()`.

```rust
// agent/rustjail/src/cgroups/fs/mod.rs (lines 51-92)
if memory.limit.is_some() && swap != 0 {
    // Order-dependent: set swap first when decreasing
    mem_controller.set_memswap_limit(swap)?;
    set_resource!(mem_controller, set_limit, memory, limit);
} else {
    set_resource!(mem_controller, set_limit, memory, limit);
    // v2 conversion for swap value
    swap = if cg.v2() { convert_memory_swap_to_v2_value(swap, ...) } else { swap };
    if swap != 0 { mem_controller.set_memswap_limit(swap)?; }
}

```

### PID Limits and Process Control

The `set_pids_resources` function uses the `pids` controller to enforce maximum process counts. It converts the OCI limit to either a specific value or `MaxValue::Max` when the limit is zero (unlimited).

```rust
// agent/rustjail/src/cgroups/fs/mod.rs (lines 20-27)
let v = if pids.limit > 0 {
    MaxValue::Value(pids.limit)
} else {
    MaxValue::Max
};
pid_controller.set_pid_max(v).context("failed to set pids resources")

```

### Block I/O and Device Access

Additional helper functions—`set_block_io_resources()`, `set_hugepages_resources()`, and `set_devices_resources()`—populate the corresponding fields of the `cgroups::Resources` struct to enforce throttling limits on disk I/O, restrict hugepage usage, and control device access permissions.

## Applying Resource Limits at Runtime

After the `FsManager` initializes the cgroup structure, the actual enforcement occurs in [`agent/rustjail/src/container.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/rustjail/src/container.rs). The container struct holds an optional `cgroup_manager` field, and during container creation, it invokes `set()` to push the limits into the kernel.

```rust
if let Some(ref mut cgm) = self.cgroup_manager {
    // `r` is the OCI CreateContainerRequest carrying the LinuxResources
    cgm.set(&r, true)?;           // pushes all limits into the cgroup
}

```

The `set` method internally calls the resource-specific helpers described above, finally invoking `cg.apply(&cg_res)` to atomically apply all constraints to the kernel's cgroup filesystem.

## Pre-flight Validation and Controller Setup

Before workloads start, the [`deploy/one-click/install.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/deploy/one-click/install.sh) script performs pre-flight checks to ensure cgroup v2 controllers are available. It verifies that required controllers (`cpu`, `memory`, `cpuset`) exist in `cgroup.controllers` and attempts to enable the `cpu` controller in `cgroup.subtree_control` if missing.

```bash

# deploy/one-click/install.sh – cgroup v2 CPU preflight

if [[ -r "${cgroot}/cgroup.controllers" ]]; then
    controllers=$(cat "${cgroot}/cgroup.controllers")
    # fail if 'cpu' missing …

fi

# try to enable '+cpu' in subtree_control

if printf '+cpu\n' >"${cgroot}/cgroup.subtree_control"; then
    log "enabled '+cpu' on ${cgroot}/cgroup.subtree_control"
fi

```

This validation prevents runtime failures on Ubuntu/Debian cloud images where services like `multipathd` might block the `cpu` controller.

## Summary

- CubeSandbox uses the `cgroups-rs` crate in [`agent/rustjail/src/cgroups/fs/mod.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/rustjail/src/cgroups/fs/mod.rs) to abstract cgroup v1 and v2 hierarchies.
- The `new_cgroup` function detects the host version and creates appropriate controllers using either `Cgroup::new()` or `Cgroup::new_with_specified_controllers()`.
- **CPU**, **memory**, **PID**, and **I/O** limits are translated from OCI specs to kernel-specific values, handling v2 conversions for shares and swap.
- Resource application occurs atomically via `cgroup_manager.set()` in [`agent/rustjail/src/container.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/rustjail/src/container.rs).
- Installation scripts in [`deploy/one-click/install.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/deploy/one-click/install.sh) verify that required v2 controllers are enabled in `cgroup.subtree_control` before deployment.

## Frequently Asked Questions

### How does CubeSandbox handle the difference between cgroup v1 and v2?

CubeSandbox detects the host hierarchy at runtime using `hierarchies::auto()`. For **cgroup v2**, it creates a unified cgroup using `cgroups::Cgroup::new()` and converts legacy share values to v2 "weight" values. For **cgroup v1**, it explicitly specifies required controllers (`cpu`, `memory`, `pids`) via `new_with_specified_controllers()` to create the legacy hierarchy. This dual-path approach ensures compatibility across different kernel configurations.

### What happens if the CPU controller is not enabled in cgroup v2?

The pre-flight script in [`deploy/one-click/install.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/deploy/one-click/install.sh) checks for the `cpu` controller in `cgroup.controllers` and attempts to enable it by writing `+cpu` to `cgroup.subtree_control`. If this fails, the sandbox initialization will fail with a clear error message, preventing workloads from starting without proper CPU isolation. This commonly occurs on cloud images where system services like `multipathd` hold the controller.

### How are OCI container specifications translated to actual kernel limits?

The `FsManager` converts OCI `LinuxResources` into `cgroups::Resources` through specialized setters: `set_cpu_resources()` handles shares and CFS quotas, `set_memory_resources()` manages limits and swap with order-dependent logic, and `set_pids_resources()` configures process limits. Each function accounts for v1/v2 differences, such as converting CPU shares to v2 weight values or adjusting swap calculations for unified hierarchies.

### Where does the actual resource application occur in the codebase?

The final application happens in [`agent/rustjail/src/container.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/rustjail/src/container.rs) where the container struct calls `cgroup_manager.set(&r, true)`. This method invokes the resource-specific helpers in [`agent/rustjail/src/cgroups/fs/mod.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/agent/rustjail/src/cgroups/fs/mod.rs), which write the constraints to the cgroup filesystem via the `cgroups-rs` crate's `apply()` method, atomically enforcing all limits in the kernel.