# How to Train or Fine-Tune moeru-ai/airi Models: A Complete Guide

> Master fine-tuning moeru-ai/airi models. Learn to collect data, train with Ultralytics YOLO, and deploy ONNX files seamlessly. Get the complete guide now.

- Repository: [Moeru AI/airi](https://github.com/moeru-ai/airi)
- Tags: how-to-guide
- Published: 2026-03-08

---

**Fine-tune moeru-ai/airi models by collecting domain-specific data with the Factorio YOLO collector mod, converting annotations to YOLO format, training with Ultralytics YOLO in Python, and deploying the exported ONNX file into the AIRI TypeScript runtime.**

The moeru-ai/airi platform orchestrates multimodal AI agents for desktop and web automation, relying on fine-tuned computer vision and speech models. While speech models like Whisper and Kokoro load directly from upstream checkpoints, the object detection pipeline supports custom training to recognize domain-specific entities in games like Factorio or Minecraft. This guide walks you through the complete workflow to train or fine-tune moeru-ai/airi models using the YOLOv11n pipeline.

## Step 1: Collect Domain-Specific Data with the Factorio YOLO Mod

AIRI provides a TypeScript-to-Lua mod for Factorio that automates data collection. When you pause the game, a "Start YOLO Data Collection" button appears, triggering the collection script defined in [`packages/factorio-rcon-snippets-for-node/src/factorio_yolo_dataset_collector_v0.ts`](https://github.com/moeru-ai/airi/blob/main/packages/factorio-rcon-snippets-for-node/src/factorio_yolo_dataset_collector_v0.ts).

The mod records rendered frames and generates JSON annotations whenever entities of interest (such as `enemy`, `ore_copper`, or `ore_iron`) appear on screen. These files are saved to your chosen directory (e.g., `./dataset/raw/`).

## Step 2: Convert Annotations to YOLO Format

YOLO requires a specific folder structure with separate `images/` and `labels/` directories. Each image needs a corresponding `.txt` file containing normalized bounding box coordinates in the format `<class_id> <x_center> <y_center> <width> <height>`.

Convert the Factorio JSON annotations using the conversion script referenced in [`docs/content/en/blog/DevLog-2025.08.26/index.md`](https://github.com/moeru-ai/airi/blob/main/docs/content/en/blog/DevLog-2025.08.26/index.md) (lines 82-112):

```bash
node scripts/convert-factorio-to-yolo.js ./dataset/raw ./dataset/detect

```

This generates a [`data.yaml`](https://github.com/moeru-ai/airi/blob/main/data.yaml) descriptor:

```yaml
train: ./dataset/detect/images
val: ./dataset/detect/images
nc: 3
names: ['enemy', 'ore_copper', 'ore_iron']

```

## Step 3: Fine-Tune the YOLO Model with Ultralytics

AIRI uses the **Ultralytics** Python package for training. Install it and fine-tune the pre-trained YOLOv11n checkpoint as shown in [`docs/content/en/blog/DevLog-2025.08.26/index.md`](https://github.com/moeru-ai/airi/blob/main/docs/content/en/blog/DevLog-2025.08.26/index.md) (lines 108-118):

```python
from ultralytics import YOLO

# Load pre-trained weights

model = YOLO("yolo11n.pt")

# Fine-tune on your dataset

model.train(
    data="./dataset/detect.yaml",
    epochs=100,
    imgsz=640,
    device="mps"  # Use "cuda" for NVIDIA, "cpu" for CPU-only

)

# Export to ONNX for AIRI runtime

model.export(format="onnx")

```

The training produces `best.onnx` in the `runs/train/exp*/weights/` directory.

## Step 4: Deploy the Fine-Tuned Model in AIRI

Copy the exported ONNX file into the AIRI runtime's model directory (e.g., `packages/stage-ui/models/`). Then register it in the provider store at [`packages/stage-ui/src/stores/providers.ts`](https://github.com/moeru-ai/airi/blob/main/packages/stage-ui/src/stores/providers.ts) (lines 650-670):

```typescript
{
  id: 'custom-yolo',
  name: 'Custom YOLO (Factorio)',
  description: 'Fine-tuned YOLO model for Factorio object detection',
  model_id: 'path/to/best.onnx'
}

```

Load the model in a TypeScript worker using the same pattern as other multimodal models in [`packages/stage-ui/src/libs/workers/worker.ts`](https://github.com/moeru-ai/airi/blob/main/packages/stage-ui/src/libs/workers/worker.ts) (lines 32-42):

```typescript
import { AutoModel } from "@xenova/transformers";

const model = await AutoModel.from_pretrained(modelPath, {
  config: { model_type: "yolo" } as any,
});

```

Now your fine-tuned model is available for real-time inference within the AIRI agent stack.

## Summary

- **Data Collection**: Use the Factorio YOLO mod ([`packages/factorio-rcon-snippets-for-node/src/factorio_yolo_dataset_collector_v0.ts`](https://github.com/moeru-ai/airi/blob/main/packages/factorio-rcon-snippets-for-node/src/factorio_yolo_dataset_collector_v0.ts)) to generate annotated screenshots.
- **Format Conversion**: Convert JSON annotations to YOLO text format and create [`data.yaml`](https://github.com/moeru-ai/airi/blob/main/data.yaml) for training.
- **Training**: Fine-tune YOLOv11n using the Ultralytics Python API, then export to ONNX.
- **Deployment**: Copy the ONNX file to `packages/stage-ui/models/` and register a provider in [`packages/stage-ui/src/stores/providers.ts`](https://github.com/moeru-ai/airi/blob/main/packages/stage-ui/src/stores/providers.ts) to load it via `AutoModel.from_pretrained`.

## Frequently Asked Questions

### Can I fine-tune the speech models (Whisper, Kokoro) within AIRI?

No. According to the source code in [`packages/stage-ui/src/libs/workers/worker.ts`](https://github.com/moeru-ai/airi/blob/main/packages/stage-ui/src/libs/workers/worker.ts) and [`packages/stage-ui/src/workers/kokoro/worker.ts`](https://github.com/moeru-ai/airi/blob/main/packages/stage-ui/src/workers/kokoro/worker.ts), AIRI loads Whisper and Kokoro models directly from upstream Hugging Face checkpoints using `from_pretrained`. These models are consumed as-is without a training pipeline in the repository. To fine-tune them, use external tools like OpenAI's Whisper fine-tuning scripts or the Kokoro training repository, then convert the results to ONNX for AIRI consumption.

### What hardware do I need to train the YOLO models?

The Ultralytics training script shown in [`docs/content/en/blog/DevLog-2025.08.26/index.md`](https://github.com/moeru-ai/airi/blob/main/docs/content/en/blog/DevLog-2025.08.26/index.md) supports CPU (`device="cpu"`), NVIDIA CUDA (`device="cuda"`), and Apple Silicon (`device="mps"`). For the small YOLOv11n model used by AIRI, training on a modern GPU with 8GB VRAM completes in minutes for small datasets. CPU training works for experimentation but is significantly slower for the 100 epochs recommended in the configuration.

### How do I convert my existing dataset to the Factorio JSON format?

You don't need to. The conversion script referenced in the DevLog documentation converts Factorio's specific JSON format (produced by [`factorio_yolo_dataset_collector_v0.ts`](https://github.com/moeru-ai/airi/blob/main/factorio_yolo_dataset_collector_v0.ts)) into standard YOLO text format. If you have an existing dataset in COCO or Pascal VOC format, write a reverse converter to generate the Factorio-style JSON, or bypass the Factorio-specific tooling and create the YOLO `labels/*.txt` files directly using standard conversion tools like Roboflow or Labelme.

### Where does the ONNX model get loaded at runtime?

The runtime loading happens in [`packages/stage-ui/src/libs/workers/worker.ts`](https://github.com/moeru-ai/airi/blob/main/packages/stage-ui/src/libs/workers/worker.ts) (lines 32-42) using the `@xenova/transformers` library. The code calls `AutoModel.from_pretrained(modelPath, {config: {model_type: "yolo"}})`, which downloads or loads the local ONNX file and wraps it for inference. For custom models, ensure the ONNX file is copied to the public models directory and referenced correctly in the provider configuration at [`packages/stage-ui/src/stores/providers.ts`](https://github.com/moeru-ai/airi/blob/main/packages/stage-ui/src/stores/providers.ts).