# How Microduck RL Simulates Actuator Physics for Dynamixel Servos: Voltage-Controlled BAM Modeling

> Microduck RL accurately simulates Dynamixel servos with voltage-controlled BAM modeling. Achieve high-fidelity sim-to-real transfer by replicating PD control, voltage sag, and friction.

- Repository: [Pollen Robotics/microduck_rl](https://github.com/pollen-robotics/microduck_rl)
- Tags: deep-dive
- Published: 2026-09-08

---

**Microduck RL simulates Dynamixel XL330 servos using a voltage-controlled BAM (Bayesian Actuator Model) actuator that replicates firmware-level PD control loops, load-dependent voltage sag, and Coulomb/Stribeck friction to ensure high-fidelity sim-to-real transfer.**

Microduck RL is an open-source reinforcement learning framework developed by Pollen Robotics for training quadruped locomotion policies. To bridge the reality gap for hardware deployment, the library implements sophisticated actuator physics simulation that mirrors the actual firmware behavior of Dynamixel XL330 servos, utilizing voltage-based control rather than idealized torque commands.

## BAM Actuator Backbone and Voltage Control

The foundation of Microduck RL's actuator physics lies in the `bam.mjlab.BamActuator` class imported in [`src/mjlab_microduck/actuator/friction_dr_bam.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/actuator/friction_dr_bam.py). Unlike standard MuJoCo actuators that accept direct torque commands, this model implements a **voltage-controlled architecture** that reproduces the Dynamixel firmware's internal control loop.

According to the source code in [`friction_dr_bam.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/friction_dr_bam.py) (lines 23-26), the BAM actuator computes output torque through a PD controller operating on position error, then applies a **load-dependent voltage sag** calculation. The model internally handles **Coulomb and Stribeck friction** dynamics, capturing the static friction (stiction) and velocity-dependent friction transitions characteristic of real servo gearboxes.

This voltage-controlled approach is critical for accurate simulation because physical XL330 servos receive voltage commands from their onboard controllers, not raw torque setpoints. By modeling the electrical and mechanical limitations at the voltage level, Microduck RL captures the compliance and power limitations that affect real robot dynamics.

## Per-Environment Friction Randomization

Real-world servos exhibit stochastic friction variations due to temperature changes, lubrication degradation, and manufacturing tolerances. Microduck RL addresses this through **per-environment friction scaling** implemented in the `FrictionDRBamActuator` class.

The source code defines a `friction_scale` scalar (lines 31-48 of [`friction_dr_bam.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/friction_dr_bam.py)) that multiplies the internal friction budget computed by `_compute_friction_budget()`. During each episode reset, the `randomize_bam_friction` event defined in [`src/mjlab_microduck/tasks/mdp.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/tasks/mdp.py) (lines 3220-3235) samples a new friction scale value from the configured distribution, injecting domain randomization directly into the actuator physics.

This randomization ensures that policies trained in simulation encounter the same friction variability present in physical hardware, preventing overfitting to idealized dynamics.

## Encoder-Through-Backlash Modeling

For robot variants that model gear play, Microduck RL implements `BacklashEncoderBamActuator`, a subclass defined in [`friction_dr_bam.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/friction_dr_bam.py) (lines 64-80). This specialized actuator addresses the **encoder-through-backlash** phenomenon where the magnetic encoder reads position from the output shaft side of the gearbox, measuring the accumulated backlash angle along with the actual joint position.

The implementation inserts passive joints named `passive_<joint>_backlash` between the motor output and the driven link. The `BacklashEncoderBamActuator` overrides the `get_command` method (lines 101-104) to add the backlash joint's angle to the position feedback sent to the PD controller. This faithfully reproduces the control instability and hysteresis observed in real hardware when motors switch direction across gear play zones.

## Actuator Configuration in Robot Specifications

Robot specifications in Microduck RL instantiate these actuator models through configuration objects defined in [`src/mjlab_microduck/robot/microduck_constants.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/robot/microduck_constants.py). The default configurations use `FrictionDRBamActuatorCfg`, which constructs `FrictionDRBamActuator` instances with specific motor models (`xl330`, `m6`) and enables per-environment friction randomization.

The `_BAM_ACTUATOR_KWARGS` dictionary (lines 31-38) specifies default parameters including voltage limits and friction coefficients, while the `actuators` mapping (lines 45-46) associates joint names with their respective BAM configurations. For backlash-enabled robot variants, the configuration switches to `BacklashEncoderBamActuatorCfg` (lines 47-51), ensuring the encoder reads position from the output side of the gear play.

## Integration with the MuJoCo Simulation Loop

During environment initialization, actuator objects attach to the `EntityArticulationInfoCfg` of the robot entity. At each simulation step, MuJoCo invokes the actuator's `compute()` method, which executes the voltage controller logic, applies the current `friction_scale`, and incorporates backlash corrections when applicable.

The [`tasks/mdp.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/tasks/mdp.py) file (lines 3170-3190) contains validation logic that ensures BAM actuators are properly configured before training begins. The resulting joint torques integrate with MuJoCo's physics engine, delivering realistic servo dynamics that account for electrical saturation, friction nonlinearities, and mechanical compliance.

## Practical Code Examples

### Creating a Walking Environment with BAM Actuators

```python
from mjlab_microduck.robot.microduck_constants import MICRODUCK_WALK_ROBOT_CFG
from mjlab.scene import Scene, SceneCfg

# Configure scene with default BAM actuators

scene_cfg = SceneCfg(
    entities={"robot": MICRODUCK_WALK_ROBOT_CFG},
    terrain=None,
)
scene = Scene(scene_cfg, device="cpu")
scene.compile()  # Instantiates FrictionDRBamActuator for all joints

```

### Enabling Friction Randomization

```python

# In task configuration (e.g., microduck_velocity_env_cfg.py)

ENABLE_JOINT_FRICTION_RANDOMIZATION = True

# This triggers randomize_bam_friction events that sample friction_scale 

# per episode in src/mjlab_microduck/tasks/mdp.py

```

### Using Backlash-Aware Actuators

```python
from mjlab_microduck.robot.microduck_constants import MICRODUCK_BACKLASH_ROBOT_CFG

# Backlash variant uses BacklashEncoderBamActuator

scene_cfg = SceneCfg(entities={"robot": MICRODUCK_BACKLASH_ROBOT_CFG})
scene = Scene(scene_cfg, device="cpu")
scene.compile()

# Now encoder feedback includes backlash joint angles

```

## Summary

- **Voltage-controlled modeling**: Microduck RL uses `BamActuator` from the BAM library to simulate voltage-based control rather than ideal torque control, matching Dynamixel XL330 firmware behavior.
- **Stochastic friction**: The `friction_scale` parameter and `randomize_bam_friction` event in [`tasks/mdp.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/tasks/mdp.py) provide per-episode domain randomization of Coulomb and viscous friction.
- **Backlash fidelity**: `BacklashEncoderBamActuator` models encoder-through-backlash effects by reading position from passive backlash joints inserted between motor and link.
- **Configuration-driven**: Actuator specifications in [`microduck_constants.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/microduck_constants.py) allow easy switching between standard and backlash-aware actuator models via configuration objects.
- **MuJoCo integration**: The actuator `compute()` method integrates with the simulation step to apply physically accurate torques based on voltage limits and friction states.

## Frequently Asked Questions

### What makes the BAM actuator different from standard MuJoCo actuators?

Standard MuJoCo actuators typically accept direct torque or position commands, assuming ideal transmission. The BAM actuator in Microduck RL implements a **voltage-controlled model** that includes PD control loops, voltage sag under load, and nonlinear friction curves. This matches the actual control architecture of Dynamixel XL330 servos, where the onboard microcontroller converts voltage inputs to motor currents while handling friction and compliance internally.

### How does friction randomization improve sim-to-real transfer?

The `randomize_bam_friction` event samples a `friction_scale` factor for each environment every episode, varying the Coulomb and Stribeck friction parameters. This **domain randomization** prevents policies from overfitting to specific friction coefficients present during training. When deployed on physical hardware, the policy has already encountered similar friction variations in simulation, making it robust to temperature changes, wear, and manufacturing differences in real servos.

### Why is backlash modeling necessary for accurate simulation?

Physical gearboxes exhibit backlash (dead zones) between driving and driven gears. On Dynamixel servos, the encoder measures position after the gearbox, so it reads the accumulated backlash angle when the motor reverses direction. Without modeling this **encoder-through-backlash** behavior, simulated policies learn aggressive oscillatory control that becomes unstable on real hardware. The `BacklashEncoderBamActuator` class adds the backlash joint angle to the control feedback, replicating the hysteresis and control delays present in physical systems.

### Where are the actuator parameters configured in the codebase?

All actuator specifications reside in [`src/mjlab_microduck/robot/microduck_constants.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/robot/microduck_constants.py). The `_BAM_ACTUATOR_KWARGS` dictionary defines default voltage limits and friction coefficients, while `FrictionDRBamActuatorCfg` and `BacklashEncoderBamActuatorCfg` classes associate specific joint names with actuator models. During scene compilation, these configurations instantiate the appropriate Python classes from [`friction_dr_bam.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/friction_dr_bam.py) for each joint in the robot model.