Where to Find the Hugging Face Jobs Submission Script for Microduck RL

The Hugging Face Jobs submission script for Microduck RL is located at scripts/hf/train_hf.py, with core submission logic in src/mjlab_microduck/hf_jobs.py.

Submitting distributed reinforcement learning training runs to Hugging Face Jobs requires navigating a small but focused codebase in the Microduck RL repository. Whether you need to launch jobs from the terminal or automate submissions programmatically, understanding the relationship between the wrapper script and the underlying module is essential.

Main Submission Files

Core Module: src/mjlab_microduck/hf_jobs.py

The heart of the Hugging Face Jobs submission system is src/mjlab_microduck/hf_jobs.py. This file implements the submit() function that orchestrates the entire workflow:

• Parses command-line arguments (task ID, hardware flavor, Docker image, namespace, timeout, etc.)
• Creates a gzipped tarball of the repository source code
• Uploads the tarball to a private Hugging Face dataset
• Creates a checkpoint model repository for run artifacts
• Launches the job via the huggingface_hub API
• Streams logs back to your terminal

The submit() function returns an exit code indicating success or failure, making it suitable for programmatic use in pipelines or CI/CD systems.

CLI Wrapper: scripts/hf/train_hf.py

The primary entry point for users is scripts/hf/train_hf.py. This thin wrapper script imports hf_jobs.submit() and forwards your command-line arguments.

When you execute:

uv run scripts/hf/train_hf.py <TASK_ID> --hf-jobs …

The wrapper handles argument parsing, validates the --hf-jobs flag, and delegates to the core module. This design separates user interface concerns from submission implementation.

How to Submit Training Jobs

Method 1: Command-Line Submission (Recommended)

For most use cases, invoke the wrapper script directly with your task configuration:

uv run scripts/hf/train_hf.py Mjlab-Kick-Flat-MicroDuck \
    --env.scene.num-envs 4096 \
    --agent.max_iterations 4000 \
    --hf-jobs \
    --flavor l4x1 \
    --namespace my-hf-username \
    --image pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime \
    --timeout 12h \
    --run-name microduck-demo-run

Required flags when using --hf-jobs:

• --flavor: Hardware specification (e.g., l4x1, a10g, a100)
• --namespace: Your Hugging Face username or organization
• --image: Docker image for the training environment
• --timeout: Maximum job duration

Method 2: Programmatic Submission

For custom automation or integration with experiment tracking systems, import and call submit() directly:

from src.mjlab_microduck.hf_jobs import submit

# Example arguments you would normally pass on the command line

argv = [
    "Mjlab-Kick-Flat-MicroDuck",   # task id

    "--flavor", "l4x1",
    "--image", "pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime",
    "--timeout", "12h",
    "--namespace", "my-hf-username",
    "--run-name", "microduck-demo-run",
    "--uv-cache",
]

exit_code = submit(argv)
print(f"Submission exited with code {exit_code}")

This pattern is useful for hyperparameter sweeps, scheduled jobs, or CI pipelines where shell execution is less convenient.

Dry-Run Mode

Before consuming GPU quota, verify your job specification:

uv run scripts/hf/train_hf.py Mjlab-King-Flat-MicroDuck \
    --hf-jobs --dry-run

The dry-run prints:

• Target namespace
• Docker image URL
• Hardware flavor
• Volume mounts for dataset and checkpoint access
• Environment variables injected into the container
• URL of the checkpoint repository to be created

No actual job is launched, and no compute resources are allocated.

Testing and Validation

The repository includes tests/test_hf_jobs_flag.py, a unit test that validates the --hf-jobs interception logic. This test ensures that when the flag is present, the wrapper correctly routes execution to hf_jobs.submit() rather than proceeding with local training.

Run this test to confirm your installation handles the submission pathway correctly:

uv run pytest tests/test_hf_jobs_flag.py -v

File Reference Summary

File	Purpose
src/mjlab_microduck/hf_jobs.py	Core submission implementation—tarball creation, HF repo management, job launch, log streaming
scripts/hf/train_hf.py	User-facing CLI wrapper; parses arguments and invokes hf_jobs.submit()
tests/test_hf_jobs_flag.py	Unit test verifying --hf-jobs flag interception

Summary

• Primary script: Use scripts/hf/train_hf.py for command-line submissions to Hugging Face Jobs
• Core logic: src/mjlab_microduck/hf_jobs.py contains the submit() function that handles authentication, packaging, and API calls
• Required flags: --hf-jobs, --flavor, --namespace, --image, and --timeout are mandatory for remote execution
• Safety feature: --dry-run previews job configuration without launching
• Testing: tests/test_hf_jobs_flag.py validates the submission flag path

Frequently Asked Questions

What is the difference between train_hf.py and hf_jobs.py?

scripts/hf/train_hf.py is a thin wrapper that handles argument parsing and user interaction. src/mjlab_microduck/hf_jobs.py contains the actual submission logic—building tarballs, creating Hugging Face repositories, and calling the Hub API. The wrapper exists so users don't need to import Python modules manually for standard workflows.

Can I submit jobs without using the uv run command?

Yes, but you must ensure the huggingface_hub package and all dependencies are available in your Python environment. The repository uses uv for dependency management and isolation. If you install dependencies via pip install -e ., you can invoke python scripts/hf/train_hf.py directly with the same arguments.

How do I specify custom environment variables or secrets for my training job?

The submit() function in hf_jobs.py supports environment variable injection through additional command-line arguments. Check the source in src/mjlab_microduck/hf_jobs.py for the --env or --secret flags, or modify the environment_variables dictionary passed to the create_inference_endpoint call for permanent customizations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}