How to Publish Microduck RL Policies to HuggingFace Hub

Publish trained Microduck RL policies to HuggingFace Hub using uv run publish or the helper scripts in scripts/hf/ to export ONNX files, generate manifests, and upload with a single command.

Microduck RL provides a complete publishing pipeline that converts PyTorch checkpoints into deployment-ready ONNX models and pushes them to the HuggingFace Hub. This article walks through the exact commands and source code paths used to publish policies from the pollen-robotics/microduck_rl repository.

Overview of the Publishing Pipeline

The publishing workflow consists of three stages: export, manifest generation, and upload. These stages are orchestrated by the CLI in src/mjlab_microduck/publish/cli.py or can be executed separately using the lower-level helper scripts.

• Export: Converts a training checkpoint to ONNX with embedded observation normalization
• Manifest: Creates a JSON contract describing the policy's interface
• Upload: Pushes artifacts to a HuggingFace repository using huggingface_hub

Method 1: Quick Publish with the CLI

The fastest way to publish a policy is the builtin publish command, implemented in [src/mjlab_microduck/publish/cli.py](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/publish/cli.py).

Required Parameters

Parameter	Description	Example
--task	Task identifier used during training	microduck_velocity
--wandb-run-path	Full W&B run path (entity/project/run_id)	myteam/microduck/rl-run-123
--checkpoint	Checkpoint step number to export	10
--repo	Target HuggingFace repository	myusername/microduck-walk
--kind	Policy runtime mode: episodic or perpetual	episodic
--duration-s	Simulated rollout duration in seconds	4.0

Command Example

uv run publish \
    --task microduck_velocity \
    --wandb-run-path myteam/microduck/rl-run-123 \
    --checkpoint 10 \
    --repo myusername/microduck-walk \
    --kind episodic \
    --duration-s 4.0

This command performs all three pipeline stages automatically and requires HF_TOKEN to be set in your environment.

Method 2: Step-by-Step Manual Publishing

For custom workflows or debugging, execute each stage independently using the underlying Python modules.

Step 1: Export the Checkpoint to ONNX

The exporter in [scripts/export.py](https://github.com/pollen-robotics/microduck_rl/blob/develop/scripts/export.py) bakes the observation normalizer directly into the ONNX graph—a required invariant for the runtime.

uv run scripts/export.py microduck_velocity \
    --wandb-run-path myteam/microduck/rl-run-123 \
    --checkpoint 10 \
    --output out.onnx

The normalizer parameters are retrieved from the W&B run artifacts and fused into the model as constant tensors. This eliminates runtime normalization overhead and ensures deterministic behavior.

Step 2: Generate the Policy Manifest

The manifest describes the ONNX input/output shapes, observation contract, and policy metadata. Use [src/mjlab_microduck/publish/manifest.py](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/publish/manifest.py):

from mjlab_microduck.publish.manifest import make_manifest

manifest = make_manifest(
    onnx_path="out.onnx",
    kind="episodic",           # or "perpetual"

    duration_s=4.0,            # rollout duration for validation

    task_id="microduck_velocity",
)
manifest.save("manifest.json")

The manifest includes:

• Input tensor shapes and dtype
• Observation keys and their semantic mapping
• Policy kind (episodic resets each rollout; perpetual runs indefinitely)
• Task metadata for runtime validation

Step 3: Upload to HuggingFace Hub

Use the low-level uploader in [scripts/hf/uploader.py](https://github.com/pollen-robotics/microduck_rl/blob/develop/scripts/hf/uploader.py):

python scripts/hf/uploader.py \
    --onnx out.onnx \
    --manifest manifest.json \
    --repo myusername/microduck-walk

The uploader requires HF_TOKEN in the environment. It creates the repository if it does not exist and uploads both files with appropriate Git LFS handling for the ONNX binary.

Method 3: Programmatic Publishing in Python

Integrate publishing into training loops or CI pipelines using the Python API directly:

from mjlab_microduck.publish.manifest import make_manifest
from mjlab_microduck.publish.uploader import upload_policy  # internal API

# After training or checkpoint selection

onnx_path = export_checkpoint(task_id, wandb_run_path, checkpoint_step)

# Build and save manifest

manifest = make_manifest(
    onnx_path=onnx_path,
    kind="episodic",
    duration_s=4.0,
    task_id=task_id,
)
manifest_path = "manifest.json"
manifest.save(manifest_path)

# Upload with explicit token handling

upload_policy(
    onnx_path=onnx_path,
    manifest_path=manifest_path,
    repo_id="myusername/microduck-walk",
    token=os.environ["HF_TOKEN"],
)

For training workflows that publish automatically, see [scripts/hf/train_hf.py](https://github.com/pollen-robotics/microduck_rl/blob/develop/scripts/hf/train_hf.py)—this script runs training and triggers publishing after each checkpoint save.

Verified Repository Structure After Upload

A successfully published policy repository contains:

• policy.onnx — The exported policy with embedded normalizer
• manifest.json — Runtime contract with shapes and observation mapping
• README.md — Auto-generated from task description (optional but recommended)

Verify by visiting:

https://huggingface.co/<username>/microduck-<policy-name>

Deploying the Published Policy

Once on HuggingFace Hub, deploy to a physical robot with:

robotctl policy add <username>/microduck-<policy-name>

The runtime downloads the ONNX file and manifest, validates the observation contract against the robot's sensor configuration, and loads the policy for inference. No additional conversion is needed—the normalizer is already embedded in the model graph.

Authentication and Environment Setup

All publishing methods require HuggingFace authentication. Set your token:

export HF_TOKEN="hf_..."

The publish CLI and uploader.py both abort with a clear error if this variable is missing. Never commit tokens to version control; use repository secrets for CI/CD.

CI/CD Integration Example

Automate publishing in GitHub Actions:

name: Publish Policy

on:
  workflow_dispatch:
    inputs:
      checkpoint:
        description: 'Checkpoint step to publish'
        required: true
        type: string

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Install uv
        uses: astral-sh/setup-uv@v3
      
      - name: Publish to HuggingFace
        run: |
          uv run publish \
            --task ${{ vars.TASK_ID }} \
            --wandb-run-path ${{ vars.WANDB_RUN_PATH }} \
            --checkpoint ${{ inputs.checkpoint }} \
            --repo ${{ secrets.HF_USERNAME }}/microduck-${{ vars.POLICY_NAME }} \
            --kind episodic \
            --duration-s 4.0
        env:
          HF_TOKEN: ${{ secrets.HF_TOKEN }}
          WANDB_API_KEY: ${{ secrets.WANDB_API_KEY }}

Summary

• Primary command: uv run publish handles export, manifest generation, and upload in one step via [src/mjlab_microduck/publish/cli.py](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/publish/cli.py)
• Manual pipeline: Use export.py → make_manifest() → uploader.py for custom workflows
• Key invariant: The ONNX file must contain embedded observation normalizers for runtime compatibility
• Authentication: HF_TOKEN environment variable required for all upload methods
• Deployment: robotctl policy add <repo> pulls from HuggingFace Hub and validates the manifest

Frequently Asked Questions

What file format are Microduck RL policies published in?

Policies are published as ONNX files with embedded observation normalizers. The ONNX format provides portability across runtime environments and hardware accelerators. The embedded normalizer ensures consistent input preprocessing without runtime dependencies on training artifacts.

Why does the manifest include a duration parameter?

The duration_s field in the manifest specifies the expected rollout length for episodic policies. This metadata allows the runtime to validate that the policy completes full trajectories and helps debugging when episode termination behavior differs from training. Perpetual policies ignore this field.

Can I publish without using Weights & Biases?

The default publish command requires W&B for checkpoint retrieval. However, you can bypass this by manually exporting checkpoints with a custom script, then using [scripts/hf/uploader.py](https://github.com/pollen-robotics/microduck_rl/blob/develop/scripts/hf/uploader.py) with local file paths. The core requirement is a valid ONNX file with the normalizer baked in—W&B is not strictly required for the upload stage itself.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}