# CEM Solver Configuration Parameters for World Models: Understanding num_samples, horizon, and receding_horizon

> Master CEM solver configuration for world models. Understand num_samples, horizon, and receding_horizon to optimize action sampling and prediction for better planning.

- Repository: [GalilAI-group/stable-worldmodel](https://github.com/galilai-group/stable-worldmodel)
- Tags: how-to-guide
- Published: 2026-05-30

---

**The `num_samples`, `horizon`, and `receding_horizon` configuration parameters in the stable-worldmodel CEM solver respectively control the breadth of action sequence sampling, the number of future steps to predict, and how many steps are executed before triggering a replan.**

In model-based reinforcement learning, the Cross-Entropy Method (CEM) generates optimal action sequences by sampling candidates and refining them against a learned world model. In the `galilai-group/stable-worldmodel` repository, these three parameters defined in [`stable_worldmodel/solver/cem.py`](https://github.com/galilai-group/stable-worldmodel/blob/main/stable_worldmodel/solver/cem.py) and [`stable_worldmodel/policy.py`](https://github.com/galilai-group/stable-worldmodel/blob/main/stable_worldmodel/policy.py) govern the trade-off between planning accuracy, computational cost, and control reactivity.

## Where CEM Solver Configuration Parameters Are Defined

Understanding the source location of each parameter clarifies how they flow from configuration to execution.

### num_samples: Controlling Search Breadth

The **`num_samples`** parameter is defined in `CEMSolver.__init__` at line 35 of [`stable_worldmodel/solver/cem.py`](https://github.com/galilai-group/stable-worldmodel/blob/main/stable_worldmodel/solver/cem.py). It specifies the number of candidate action sequences drawn from the Gaussian distribution during each CEM iteration. Inside the optimization loop (`for step in range(self.n_steps)`), this value determines the allocation of the candidates tensor with shape `(batch, num_samples, horizon, action_dim)` (lines 90-99).

### horizon: Setting the Planning Depth

The **`horizon`** parameter lives in the **`PlanConfig`** dataclass at lines 29-30 of [`stable_worldmodel/policy.py`](https://github.com/galilai-group/stable-worldmodel/blob/main/stable_worldmodel/policy.py). It represents the total number of future model steps the solver predicts when evaluating action sequences. The policy exposes this to the solver via `self.solver.configure(..., config=self.cfg)`, and the CEM solver accesses it through its `horizon` property (lines 84-86 of [`cem.py`](https://github.com/galilai-group/stable-worldmodel/blob/main/cem.py)). This value directly determines the temporal dimension of the mean and variance tensors maintained by the optimizer.

### receding_horizon: Managing Replanning Frequency

The **`receding_horizon`** parameter is also defined in **`PlanConfig`** at lines 30-31 of [`policy.py`](https://github.com/galilai-group/stable-worldmodel/blob/main/policy.py). Unlike `horizon`, this parameter governs execution rather than planning. The **`WorldModelPolicy`** converts it to environment steps using `flatten_receding_horizon` (lines 26-28), accounting for any action repeat (`action_block`). This calculation determines the size of the internal action buffer (`self._action_buffer`) initialized in `set_env` (lines 42-44), which stores planned actions awaiting execution.

## How num_samples, horizon, and receding_horizon Interact

These three parameters form a pipeline that balances exploration, foresight, and responsiveness.

**Planning Horizon (`horizon`)** dictates how far into the future the world model predicts. The CEM optimizer maintains a mean-variance distribution over an array of shape `(batch, horizon, action_dim)`. When solving, the sampler generates `num_samples` sequences spanning this full horizon, and the world model evaluates the cumulative cost across all future steps.

**Number of Samples (`num_samples`)** determines the granularity of the search. Higher values provide a more accurate estimate of the elite action region but increase memory consumption proportional to `batch × num_samples × horizon × action_dim`. This is the primary knob for trading computation against optimization quality.

**Receding Horizon (`receding_horizon`)** controls replanning frequency. After the policy extracts the first `flatten_receding_horizon` actions from the solver’s output mean, it deposits them into a deque buffer. The policy consumes these buffered actions for subsequent environment steps, only triggering a fresh call to `solver.solve` when the buffer empties. If `warm_start=True`, the previous plan initializes the new optimization.

**Action Blocking (`action_block`)** scales both `horizon` and `receding_horizon` from model steps to environment steps. As implemented in `PlanConfig.plan_len` and `WorldModelPolicy.flatten_receding_horizon`, an `action_block` of 2 means each model step represents two environment frames, effectively doubling the physical time horizon without increasing the model rollout length.

## Practical Configuration Examples

### Complete Workflow Setup

```python
import torch
from stable_worldmodel.solver.cem import CEMSolver
from stable_worldmodel.policy import WorldModelPolicy, PlanConfig
from stable_worldmodel.world.world import WorldModel

# Initialize trained world model

wm = WorldModel.load("path/to/checkpoint.pt")

# Configure planning parameters

cfg = PlanConfig(
    horizon=15,            # Look-ahead 15 model steps

    receding_horizon=5,    # Replan every 5 environment steps

    action_block=2,        # Model step = 2 env frames

    warm_start=True,
)

# Instantiate solver with search breadth

solver = CEMSolver(
    model=wm,
    batch_size=4,
    num_samples=400,       # Evaluate 400 candidates per iteration

    var_scale=1.0,
    n_steps=30,            # 30 CEM refinement iterations

    topk=30,
    device="cuda",
)

# Create policy wrapper

policy = WorldModelPolicy(solver=solver, config=cfg)

# Attach to environment

import gymnasium as gym
env = gym.make("CartPole-v1")
policy.set_env(env)

# Execution loop

obs, _ = env.reset()
for _ in range(200):
    action = policy.get_action(obs)
    obs, reward, done, truncated, info = env.step(action)
    if done or truncated:
        obs, _ = env.reset()

```

### Inspecting Solver State

After calling `solve()`, you can verify the parameter effects on tensor shapes:

```python
out = solver.solve(info_dict)  # Returns dict with 'mean', 'var', 'actions'

print("Planned horizon:", out['actions'].shape[1])  # Matches cfg.horizon

print("Samples used:", solver.num_samples)

```

### Runtime Parameter Adjustment

Modify planning depth without reconstructing the solver:

```python

# Reduce horizon mid-execution for faster computation

policy.cfg = PlanConfig(
    horizon=8, 
    receding_horizon=4, 
    action_block=1
)

# Next call to policy.get_action() reconfigures automatically

```

### Deterministic Debugging Configuration

Disable sampling stochasticity for reproducible debugging:

```python
solver = CEMSolver(
    model=wm,
    num_samples=1,    # Only evaluate current mean

    var_scale=0.0,      # Zero variance

    n_steps=1,          # Single iteration

)

```

## Summary

- **`num_samples`** is defined in `CEMSolver.__init__` (line 35) and controls candidate action sequence generation, directly impacting memory usage via tensors of shape `(batch, num_samples, horizon, action_dim)`.
- **`horizon`** is defined in `PlanConfig` (lines 29-30) and determines the temporal dimension of the optimization problem, accessed by the solver through its `horizon` property.
- **`receding_horizon`** is defined in `PlanConfig` (lines 30-31) and governs how many steps are buffered before replanning, converted to environment steps via `flatten_receding_horizon`.
- **Interaction**: `horizon` sets prediction depth, `num_samples` sets search breadth, and `receding_horizon` sets execution latency, with `action_block` scaling the latter two to environment timesteps.

## Frequently Asked Questions

### What does num_samples control in the CEM solver?

The `num_samples` parameter controls how many candidate action sequences are drawn from the Gaussian distribution during each iteration of the CEM optimization loop. According to [`stable_worldmodel/solver/cem.py`](https://github.com/galilai-group/stable-worldmodel/blob/main/stable_worldmodel/solver/cem.py) lines 90-99, increasing this value improves the estimate of the elite action region but raises computational cost proportional to the product of batch size, horizon length, and action dimension.

### How does receding_horizon differ from horizon?

While `horizon` defines how many future steps the world model predicts during planning, `receding_horizon` determines how many of those planned steps are actually executed before the policy triggers a new solve call. As implemented in [`stable_worldmodel/policy.py`](https://github.com/galilai-group/stable-worldmodel/blob/main/stable_worldmodel/policy.py), the `WorldModelPolicy` converts `receding_horizon` to environment steps using `flatten_receding_horizon` and buffers exactly that many actions in `self._action_buffer` (lines 42-44).

### Can I adjust horizon at runtime without recreating the solver?

Yes. You can modify [`policy.cfg`](https://github.com/galilai-group/stable-worldmodel/blob/main/policy.cfg) with a new `PlanConfig` containing a different `horizon` value. The `WorldModelPolicy` passes this configuration to the solver via `self.solver.configure()`, which updates the internal planning tensors. The next call to `policy.get_action()` will automatically use the new horizon without requiring you to reinstantiate the `CEMSolver` object.

### How does action_block affect these parameters?

The `action_block` parameter (defined in `PlanConfig`) scales both `horizon` and `receding_horizon` from model steps to environment steps. For example, with `action_block=2`, a `horizon` of 15 model steps represents 30 environment frames. This allows you to plan at a lower temporal resolution than the environment requires while maintaining the physical time horizon, reducing computational load while preserving long-horizon reasoning.