How Microduck RL Simulates Actuator Physics for Dynamixel Servos: Voltage-Controlled BAM Modeling

Microduck RL simulates Dynamixel XL330 servos using a voltage-controlled BAM (Bayesian Actuator Model) actuator that replicates firmware-level PD control loops, load-dependent voltage sag, and Coulomb/Stribeck friction to ensure high-fidelity sim-to-real transfer.

Microduck RL is an open-source reinforcement learning framework developed by Pollen Robotics for training quadruped locomotion policies. To bridge the reality gap for hardware deployment, the library implements sophisticated actuator physics simulation that mirrors the actual firmware behavior of Dynamixel XL330 servos, utilizing voltage-based control rather than idealized torque commands.

BAM Actuator Backbone and Voltage Control

The foundation of Microduck RL's actuator physics lies in the bam.mjlab.BamActuator class imported in src/mjlab_microduck/actuator/friction_dr_bam.py. Unlike standard MuJoCo actuators that accept direct torque commands, this model implements a voltage-controlled architecture that reproduces the Dynamixel firmware's internal control loop.

According to the source code in friction_dr_bam.py (lines 23-26), the BAM actuator computes output torque through a PD controller operating on position error, then applies a load-dependent voltage sag calculation. The model internally handles Coulomb and Stribeck friction dynamics, capturing the static friction (stiction) and velocity-dependent friction transitions characteristic of real servo gearboxes.

This voltage-controlled approach is critical for accurate simulation because physical XL330 servos receive voltage commands from their onboard controllers, not raw torque setpoints. By modeling the electrical and mechanical limitations at the voltage level, Microduck RL captures the compliance and power limitations that affect real robot dynamics.

Per-Environment Friction Randomization

Real-world servos exhibit stochastic friction variations due to temperature changes, lubrication degradation, and manufacturing tolerances. Microduck RL addresses this through per-environment friction scaling implemented in the FrictionDRBamActuator class.

The source code defines a friction_scale scalar (lines 31-48 of friction_dr_bam.py) that multiplies the internal friction budget computed by _compute_friction_budget(). During each episode reset, the randomize_bam_friction event defined in src/mjlab_microduck/tasks/mdp.py (lines 3220-3235) samples a new friction scale value from the configured distribution, injecting domain randomization directly into the actuator physics.

This randomization ensures that policies trained in simulation encounter the same friction variability present in physical hardware, preventing overfitting to idealized dynamics.

Encoder-Through-Backlash Modeling

For robot variants that model gear play, Microduck RL implements BacklashEncoderBamActuator, a subclass defined in friction_dr_bam.py (lines 64-80). This specialized actuator addresses the encoder-through-backlash phenomenon where the magnetic encoder reads position from the output shaft side of the gearbox, measuring the accumulated backlash angle along with the actual joint position.

The implementation inserts passive joints named passive_<joint>_backlash between the motor output and the driven link. The BacklashEncoderBamActuator overrides the get_command method (lines 101-104) to add the backlash joint's angle to the position feedback sent to the PD controller. This faithfully reproduces the control instability and hysteresis observed in real hardware when motors switch direction across gear play zones.

Actuator Configuration in Robot Specifications

Robot specifications in Microduck RL instantiate these actuator models through configuration objects defined in src/mjlab_microduck/robot/microduck_constants.py. The default configurations use FrictionDRBamActuatorCfg, which constructs FrictionDRBamActuator instances with specific motor models (xl330, m6) and enables per-environment friction randomization.

The _BAM_ACTUATOR_KWARGS dictionary (lines 31-38) specifies default parameters including voltage limits and friction coefficients, while the actuators mapping (lines 45-46) associates joint names with their respective BAM configurations. For backlash-enabled robot variants, the configuration switches to BacklashEncoderBamActuatorCfg (lines 47-51), ensuring the encoder reads position from the output side of the gear play.

Integration with the MuJoCo Simulation Loop

During environment initialization, actuator objects attach to the EntityArticulationInfoCfg of the robot entity. At each simulation step, MuJoCo invokes the actuator's compute() method, which executes the voltage controller logic, applies the current friction_scale, and incorporates backlash corrections when applicable.

The tasks/mdp.py file (lines 3170-3190) contains validation logic that ensures BAM actuators are properly configured before training begins. The resulting joint torques integrate with MuJoCo's physics engine, delivering realistic servo dynamics that account for electrical saturation, friction nonlinearities, and mechanical compliance.

Practical Code Examples

Creating a Walking Environment with BAM Actuators

from mjlab_microduck.robot.microduck_constants import MICRODUCK_WALK_ROBOT_CFG
from mjlab.scene import Scene, SceneCfg

# Configure scene with default BAM actuators

scene_cfg = SceneCfg(
    entities={"robot": MICRODUCK_WALK_ROBOT_CFG},
    terrain=None,
)
scene = Scene(scene_cfg, device="cpu")
scene.compile()  # Instantiates FrictionDRBamActuator for all joints

Enabling Friction Randomization

# In task configuration (e.g., microduck_velocity_env_cfg.py)

ENABLE_JOINT_FRICTION_RANDOMIZATION = True

# This triggers randomize_bam_friction events that sample friction_scale 

# per episode in src/mjlab_microduck/tasks/mdp.py

Using Backlash-Aware Actuators

from mjlab_microduck.robot.microduck_constants import MICRODUCK_BACKLASH_ROBOT_CFG

# Backlash variant uses BacklashEncoderBamActuator

scene_cfg = SceneCfg(entities={"robot": MICRODUCK_BACKLASH_ROBOT_CFG})
scene = Scene(scene_cfg, device="cpu")
scene.compile()

# Now encoder feedback includes backlash joint angles

Summary

• Voltage-controlled modeling: Microduck RL uses BamActuator from the BAM library to simulate voltage-based control rather than ideal torque control, matching Dynamixel XL330 firmware behavior.
• Stochastic friction: The friction_scale parameter and randomize_bam_friction event in tasks/mdp.py provide per-episode domain randomization of Coulomb and viscous friction.
• Backlash fidelity: BacklashEncoderBamActuator models encoder-through-backlash effects by reading position from passive backlash joints inserted between motor and link.
• Configuration-driven: Actuator specifications in microduck_constants.py allow easy switching between standard and backlash-aware actuator models via configuration objects.
• MuJoCo integration: The actuator compute() method integrates with the simulation step to apply physically accurate torques based on voltage limits and friction states.

Frequently Asked Questions

What makes the BAM actuator different from standard MuJoCo actuators?

Standard MuJoCo actuators typically accept direct torque or position commands, assuming ideal transmission. The BAM actuator in Microduck RL implements a voltage-controlled model that includes PD control loops, voltage sag under load, and nonlinear friction curves. This matches the actual control architecture of Dynamixel XL330 servos, where the onboard microcontroller converts voltage inputs to motor currents while handling friction and compliance internally.

How does friction randomization improve sim-to-real transfer?

The randomize_bam_friction event samples a friction_scale factor for each environment every episode, varying the Coulomb and Stribeck friction parameters. This domain randomization prevents policies from overfitting to specific friction coefficients present during training. When deployed on physical hardware, the policy has already encountered similar friction variations in simulation, making it robust to temperature changes, wear, and manufacturing differences in real servos.

Why is backlash modeling necessary for accurate simulation?

Physical gearboxes exhibit backlash (dead zones) between driving and driven gears. On Dynamixel servos, the encoder measures position after the gearbox, so it reads the accumulated backlash angle when the motor reverses direction. Without modeling this encoder-through-backlash behavior, simulated policies learn aggressive oscillatory control that becomes unstable on real hardware. The BacklashEncoderBamActuator class adds the backlash joint angle to the control feedback, replicating the hysteresis and control delays present in physical systems.

Where are the actuator parameters configured in the codebase?

All actuator specifications reside in src/mjlab_microduck/robot/microduck_constants.py. The _BAM_ACTUATOR_KWARGS dictionary defines default voltage limits and friction coefficients, while FrictionDRBamActuatorCfg and BacklashEncoderBamActuatorCfg classes associate specific joint names with actuator models. During scene compilation, these configurations instantiate the appropriate Python classes from friction_dr_bam.py for each joint in the robot model.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}