# How the fell_over Termination is Handled in the VelStand Task Curriculum

> Learn how the fell_over termination is managed in the VelStand task curriculum. Discover the strategy for enabling, disabling, and removing this termination condition during training and evaluation.

- Repository: [Pollen Robotics/microduck_rl](https://github.com/pollen-robotics/microduck_rl)
- Tags: deep-dive
- Published: 2026-09-01

---

**The VelStand task curriculum manages the `fell_over` termination by activating it with a 70° tilt limit during the first 500 training iterations, then progressively disabling it by raising the limit to π radians, and finally removing it entirely during evaluation (play mode).**

The VelStand environment in the `pollen-robotics/microduck_rl` repository extends the base walking configuration to teach bipedal robots stable locomotion followed by fall recovery. Understanding how the `fell_over` termination is handled across training phases is critical for implementing similar curriculum learning strategies in robotic reinforcement learning.

## Phase 1: Active Termination for Clean Walking (0–500 Iterations)

During the initial training phase, the environment enforces strict stability criteria. The `fell_over` termination remains **active** with a tilt limit of **70 degrees** (approximately 1.22 radians), forcing episodes to end immediately when the robot's body tilt exceeds this threshold.

This design choice prioritizes learning **stable walking patterns** before exposing the policy to complex recovery scenarios. By terminating episodes that result in falls, the agent receives clear negative signals early in training, preventing the accumulation of noisy experiences from uncontrolled ground contact.

The configuration for this phase is defined in the module docstring of [`src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py), specifically noting that Phase 1 focuses on "clean walking first" before transitioning to recovery behaviors.

## Disabling the Termination via Curriculum at Iteration 500

Once the policy demonstrates competent walking at iteration 500, the curriculum **disables** the `fell_over` termination by manipulating its angle parameter. Rather than removing the termination check entirely, the system raises the `limit_angle` from 70° to **π radians (180°)**, effectively preventing it from ever triggering during normal operation.

This gradual disablement is implemented using a `CurriculumTermCfg` that interfaces with the `termination_param_curriculum` function from the MDP utilities:

```python
cfg.curriculum["fell_over_disable"] = CurriculumTermCfg(
    func=microduck_mdp.termination_param_curriculum,
    params={
        "term_name": "fell_over",
        "param_stages": [
            {"step": 0,
             "params": {"limit_angle": math.radians(70.0)}},
            {"step": FELL_OVER_DISABLE_ITER * NUM_STEPS_PER_ENV,
             "params": {"limit_angle": math.pi}},
        ],
    },
)

```

The `FELL_OVER_DISABLE_ITER` constant (set to 500) multiplied by `NUM_STEPS_PER_ENV` determines the exact environment step at which the parameter transition occurs. This approach allows the agent to learn **fall recovery behaviors** without episode resets, treating falls as recoverable states rather than terminal failures.

## Play Mode: Complete Termination Removal

During evaluation or deployment, the curriculum system does not execute, requiring a separate handling mechanism for the termination condition. When initializing the environment with `play=True`, the configuration explicitly **removes** the `fell_over` termination from the termination dictionary:

```python
if play:
    cfg.terminations.pop("fell_over", None)

```

This code appears in [`src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py) and ensures that evaluation episodes continue regardless of body orientation. Removing the termination entirely in play mode prevents unintended episode truncation when testing the final policy's robustness.

## Implementation Architecture

The VelStand task builds upon the proven walking recipe defined in `make_microduck_velocity_env_cfg` from [`src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py). The `make_microduck_velstand_env_cfg` function adds the recovery layer by configuring the curriculum parameters and conditional termination logic.

Key implementation details include:

- **Source File**: [`src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py) (lines 62–70 and 221–231)
- **Curriculum Function**: `termination_param_curriculum` located in [`src/mjlab_microduck/tasks/mdp.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/tasks/mdp.py)
- **Configuration Class**: `CurriculumTermCfg` manages staged parameter updates
- **Base Environment**: `make_microduck_velocity_env_cfg` provides the underlying locomotion task structure

## Summary

- The `fell_over` termination starts with a strict **70° tilt limit** during the first 500 iterations to enforce clean walking.
- A **curriculum parameter stage** at iteration 500 raises the limit to π radians, functionally disabling the termination without removing the check.
- In **play mode**, the termination is completely removed from the configuration to prevent evaluation interruptions.
- The implementation resides in [`microduck_velstand_env_cfg.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/microduck_velstand_env_cfg.py) and utilizes `termination_param_curriculum` for smooth parameter transitions.

## Frequently Asked Questions

### What is the initial tilt angle limit for the fell_over termination?

The initial tilt limit is **70 degrees** (approximately 1.22 radians). This value is set at environment initialization and remains active until the curriculum intervenes at iteration 500.

### Why does the curriculum raise the angle to π instead of removing the termination?

Raising the limit to π radians (180 degrees) **preserves the termination infrastructure** while ensuring it never triggers during normal operation. This approach maintains code consistency with the base velocity environment and allows for potential reactivation or debugging without structural changes to the termination dictionary.

### How does play mode affect the fell_over termination differently from training?

In play mode (`play=True`), the termination is **completely popped from the configuration** using `cfg.terminations.pop("fell_over", None)`, whereas during training it remains present but with a disabled angle threshold. This distinction exists because the curriculum system does not execute during evaluation, requiring hard removal to prevent unintended episode terminations.

### Where is the curriculum stage timing defined for the fell_over disablement?

The timing is controlled by the `FELL_OVER_DISABLE_ITER` constant (set to 500) multiplied by `NUM_STEPS_PER_ENV`, calculated in the `CurriculumTermCfg` parameters within [`src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py) at lines 221–231.