# How to Deploy Policies on the Real Microduck Robot: A Complete Pipeline Guide

> Deploy policies to the real Microduck robot using a three-stage pipeline. Export to ONNX, publish manifests to Hugging Face Hub, and execute commands on hardware. Learn the complete pipeline guide.

- Repository: [Pollen Robotics/microduck_rl](https://github.com/pollen-robotics/microduck_rl)
- Tags: how-to-guide
- Published: 2026-09-08

---

**Policies are deployed to the real Microduck robot through a three-stage pipeline that exports the trained network to ONNX, publishes a manifest to the Hugging Face Hub, and executes generated robotctl commands on the target hardware.**

Deploying reinforcement learning policies from simulation to physical hardware requires rigorous tooling. In the `pollen-robotics/microduck_rl` repository, the path from trained model to real robot execution follows a structured workflow involving ONNX serialization, JSON manifests, and the `robotctl` daemon. This guide explains exactly how policies are deployed on the real Microduck robot using the source implementation.

## Understanding the Policy Deployment Pipeline

The deployment system consists of three coordinated stages that bridge machine learning training and embedded execution. Each stage is implemented in specific source files that handle the transition from Python training code to on-device inference.

- **ONNX export and manifest generation** handled in [`src/mjlab_microduck/export.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/export.py)
- **Publishing to remote repositories** managed by [`src/mjlab_microduck/publish/cli.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/publish/cli.py)
- **Robot-side loading and execution** controlled through `robotctl` commands generated by [`src/mjlab_microduck/publish/manifest.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/publish/manifest.py)

## Step 1: ONNX Export and Manifest Generation

Before deploying policies on the real Microduck robot, the trained network must be converted into a hardware-agnostic format. The [`src/mjlab_microduck/export.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/export.py) module performs **ONNX export** while baking the observation normalizer directly into the network graph to eliminate runtime preprocessing overhead.

The [`scripts/export.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/scripts/export.py) CLI wrapper orchestrates this process, creating two critical artifacts in your output directory:

- `policy.onnx`: The serialized neural network with fused normalizers
- [`manifest.json`](https://github.com/pollen-robotics/microduck_rl/blob/main/manifest.json): Runtime metadata describing policy shape, type, and execution parameters

The manifest is constructed by `build_manifest` and validated by `validate_manifest`, both defined in [`src/mjlab_microduck/publish/manifest.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/publish/manifest.py). Key configuration fields include:

- `kind`: The execution type (`episodic`, `perpetual`, or `held`)
- `slot`: The target actuator slot for perpetual gaits (e.g., `walk`, `stand`)
- `duration_s`: Runtime limit in seconds for episodic skills

```python
from src.mjlab_microduck.publish.manifest import build_manifest, render_readme

manifest = build_manifest(
    name="walk",
    kind="perpetual",
    description="A walking gait for Microduck",
    slot="walk",
    duration_s=None,  # perpetual policies omit duration

)
repo_id = "user/microduck-walk"
print(render_readme(manifest, repo_id))

```

The `render_readme` function automatically generates deployment documentation that stays synchronized with the manifest, ensuring users always have the correct `robotctl` syntax.

## Step 2: Publishing to the Hugging Face Hub

Once exported, the policy package is pushed to the **Hugging Face Hub** using the `uv run publish` command. This CLI tool is implemented in [`src/mjlab_microduck/publish/cli.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/publish/cli.py) and handles authentication, repository creation, and artifact upload.

The command pushes both the ONNX file and the JSON manifest to a remote repository (e.g., `user/microduck-walk`). The `robotctl` daemon running on the physical Microduck robot can subsequently pull these artifacts directly from the hub using standard Git LFS protocols.

## Step 3: Loading and Executing Policies on the Robot

The `robotctl` daemon handles the final stage of deploying policies on the real Microduck robot. Based on the manifest's `kind` field, the `install_commands` function in [`src/mjlab_microduck/publish/manifest.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/publish/manifest.py) generates specific shell commands for each execution mode.

### Episodic Skills

For one-shot behaviors that terminate automatically after a fixed duration, use the `episodic` kind. The generated commands install the policy temporarily and execute it:

```bash
sudo robotctl policy add <name> <repo>
robotctl robot do <name>

```

The skill runs for exactly `duration_s` seconds as specified in the manifest, then automatically unwinds and returns control to the default controller.

### Perpetual Gaits

Continuous behaviors like walking use the `perpetual` kind and load into fixed actuator slots. The `install_commands` function generates:

```bash
sudo robotctl policy load <slot> <repo>

```

Valid slots include `walk`, `stand`, and other predefined gait categories defined in the robot's configuration. The gait persists until explicitly replaced by another policy or the robot powers down.

### Held Poses

For static configurations that must remain active indefinitely until explicitly released, use the `held` kind. The generated command includes the `--hold` flag:

```bash
sudo robotctl policy add <name> <repo> --hold <seconds>

```

This loads a pose that stays locked until the control system issues a release command or the specified timeout expires.

## Robot-Side Validation and Execution

When `robotctl` receives a deployment command to load a policy on the real Microduck robot, it performs several validation steps before issuing motor commands:

1. Pulls the policy repository from the Hugging Face Hub to local storage
2. Validates the ONNX input/output tensor shapes (typically `61 → 14`) using the `check_onnx` utility
3. Confirms the observation normalizer is properly baked into the network graph
4. Registers the policy under the requested slot or name in the daemon's registry
5. Executes the policy according to the manifest's timing rules (`duration_s`, `unwind_s`, or perpetual)

## Summary

- **ONNX export** converts trained policies into hardware-agnostic format using [`src/mjlab_microduck/export.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/export.py), embedding observation normalizers directly into the network
- **Manifest generation** defines runtime behavior through `build_manifest` in [`src/mjlab_microduck/publish/manifest.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/publish/manifest.py), specifying execution kind and timing parameters
- **Remote publishing** pushes artifacts via `uv run publish` implemented in [`src/mjlab_microduck/publish/cli.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/src/mjlab_microduck/publish/cli.py) for version-controlled distribution
- **Robot loading** uses auto-generated `robotctl` commands from `install_commands` to execute episodic, perpetual, or held behaviors on the physical hardware
- **Runtime validation** ensures ONNX shape compatibility (`61 → 14`) and normalizer presence before real robot execution

## Frequently Asked Questions

### What file format does Microduck use for policy deployment?

Microduck deployments use **ONNX** for the neural network weights and a JSON manifest for runtime metadata. This combination ensures cross-platform compatibility and self-describing parameters that the `robotctl` daemon can interpret without additional configuration files.

### How does the robot validate downloaded policies before execution?

The `robotctl` daemon validates policies by checking ONNX tensor shapes against expected dimensions (typically `61` inputs to `14` outputs) and verifying that observation normalizers are baked into the network graph. This validation occurs in `check_onnx` before any motor commands are issued, preventing runtime shape mismatches.

### What distinguishes a perpetual gait from an episodic skill?

**Perpetual gaits** load into persistent slots like `walk` or `stand` using `robotctl policy load` and run indefinitely until replaced. **Episodic skills** use `robotctl policy add` followed by `robotctl robot do`, executing for a specific `duration_s` before automatically terminating and returning to the default controller.

### Where is the deployment command generation tested?

Unit tests for the `install_commands` function and manifest schema validation reside in [`tests/test_publish_manifest.py`](https://github.com/pollen-robotics/microduck_rl/blob/main/tests/test_publish_manifest.py). These tests confirm that generated `robotctl` syntax matches expected patterns for all three policy kinds (`episodic`, `perpetual`, and `held`) and that the `render_readme` output contains valid bash commands.