What Is the `fallen_too_long` Termination in the VelStand Task?

The fallen_too_long termination acts as a back‑stop that forces an episode reset when the robot remains fallen for more than 8 seconds, preventing the policy from exploiting zero‑cost waiting states during recovery training in the VelStand task.

In the pollen-robotics/microduck_rl repository, the VelStand task trains a quadruped robot to recover from falls and stand up. During Phase 2 training (after iteration 500), the standard fell_over termination is disabled to allow the robot to learn recovery behaviors while prone. Without the fallen_too_long termination, the agent could exploit infinite episode lengths by remaining motionless on the ground, avoiding action‑rate and torque penalties while accumulating zero cost.

Why the VelStand Task Requires a Recovery Back‑Stop

During Phase 2 of the training curriculum, the robot must learn to recover from a fallen position. To facilitate this, the environment disables the immediate fell_over termination that would otherwise end the episode when the robot topples. While this allows the policy to explore recovery strategies, it creates an exploitation vulnerability: the agent could simply wait out the episode duration while fallen, incurring no penalties for inaction.

The fallen_too_long termination solves this by establishing a maximum allowable duration for the fallen state. According to the source code documentation, this serves as a "Failed‑recovery back‑stop: continuously fallen this long → terminate/reset." This ensures that the recovery reward layer only receives data from genuine attempts to get up, rather than passive waiting.

How fallen_too_long Works Under the Hood

The termination triggers when three specific conditions are met simultaneously for a sustained period. These parameters are defined in src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py:

• Trunk height threshold: gate_z_below set to TERM_GATE_Z = 0.08 m
• Tilt angle threshold: gate_tilt_above_deg set to TERM_GATE_TILT_DEG = 40°
• Maximum duration: max_duration_s set to FALLEN_TIMEOUT_S = 8.0 s

When the robot's trunk height drops below 0.08 meters and its tilt exceeds 40 degrees for more than 8 consecutive seconds, the termination fires. This logic is implemented in the microduck_mdp.fallen_too_long function and referenced in the environment configuration.

Source Code Reference

The termination is registered in microduck_velstand_env_cfg.py at lines 303‑315:

cfg.terminations["fallen_too_long"] = TerminationTermCfg(
    func=microduck_mdp.fallen_too_long,
    time_out=False,
    params={
        "gate_z_below": TERM_GATE_Z,
        "gate_tilt_above_deg": TERM_GATE_TILT_DEG,
        "max_duration_s": FALLEN_TIMEOUT_S,
    },
)

The underlying MDP logic resides in src/mjlab_microduck/tasks/mdp.py, where the fallen_too_long function evaluates the height, tilt, and temporal conditions to determine if the episode should reset.

Practical Configuration Examples

Enabling the Termination (Default Behavior)

The termination is automatically added when creating a training configuration. No additional code is required:

from mjlab_microduck.tasks.microduck_velstand_env_cfg import (
    make_microduck_velstand_env_cfg,
)

cfg = make_microduck_velstand_env_cfg(play=False)   # training mode

assert "fallen_too_long" in cfg.terminations          # termination present

Disabling the Termination

For debugging or pure play mode, remove the termination to allow indefinite fallen states:

cfg = make_microduck_velstand_env_cfg(play=False)
cfg.terminations.pop("fallen_too_long", None)   # remove back-stop

Adjusting the Timeout

Modify the recovery window by changing the max_duration_s parameter:

cfg = make_microduck_velstand_env_cfg()
cfg.terminations["fallen_too_long"].params["max_duration_s"] = 5.0  # 5 seconds

Summary

• The fallen_too_long termination prevents policy exploitation during Phase 2 recovery training by forcing a reset after 8 seconds of continuous fallen state.
• It triggers only when the robot is truly toppled (height < 0.08 m and tilt > 40°) and remains motionless for the timeout duration.
• Configuration resides in microduck_velstand_env_cfg.py (lines 303‑315), while the logic is implemented in mdp.py.
• The termination can be disabled or customized by adjusting the max_duration_s parameter.

Frequently Asked Questions

When exactly does fallen_too_long trigger during training?

The termination triggers when the robot maintains a trunk height below 0.08 meters and a tilt angle above 40 degrees continuously for 8 seconds. This ensures the robot is genuinely fallen and unresponsive before forcing a reset.

Why is the standard fell_over termination disabled in Phase 2?

The standard termination is disabled to allow the robot to remain on the ground and learn active recovery behaviors. Without this change, the episode would end immediately upon falling, preventing the agent from learning to stand up.

Can the timeout duration be customized?

Yes, modify the max_duration_s parameter in the termination configuration. The default is 8.0 seconds, but you can adjust this value to enforce faster recovery attempts or allow more time for difficult maneuvers.

Where is the termination logic implemented?

The configuration and parameters are defined in src/mjlab_microduck/tasks/microduck_velstand_env_cfg.py (lines 303‑315), while the underlying MDP evaluation function fallen_too_long is implemented in src/mjlab_microduck/tasks/mdp.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}