# How LlamaFactory Integrates with Experiment Tracking Tools: A Complete Technical Guide

> Discover how LlamaFactory integrates with experiment tracking tools. Learn to route metrics and hyperparameters effortlessly for better ML model development with our technical guide.

- Repository: [Yaowei Zheng/LlamaFactory](https://github.com/hiyouga/LlamaFactory)
- Tags: how-to-guide
- Published: 2026-03-04

---

**LlamaFactory routes training metrics and hyperparameters to external experiment tracking platforms through a unified `report_to` option that connects Web UI selections, configuration validation, and runtime callbacks.**

LlamaFactory provides a seamless bridge between fine-tuning workflows and experiment tracking ecosystems. The open-source repository implements a **three-layer architecture**—spanning user interface components, argument parsers, and training callbacks—to ensure that how LlamaFactory integrates with experiment tracking tools remains consistent across Weights & Biases, TensorBoard, Trackio, and other supported backends.

## The Three-Layer Integration Architecture

LlamaFactory decouples the user-facing selection from runtime execution through discrete validation and callback layers. This design makes the system extensible: adding a new tracker requires updating the dropdown, extending the parser, and implementing a hook in the callback class.

### UI Selection Layer

The Web UI exposes experiment tracking configuration through the *Training* panel in **[`src/llamafactory/webui/components/train.py`](https://github.com/hiyouga/LlamaFactory/blob/main/src/llamafactory/webui/components/train.py)**. Users select their preferred backend from a dropdown populated with the following choices:

- `none`
- `wandb`
- `mlflow`
- `neptune`
- `tensorboard`
- `trackio`
- `all`

This component stores the selection in the `report_to` field, which propagates through the training configuration pipeline.

### Configuration Validation

Before training begins, the `report_to` argument undergoes normalization and validation in **[`src/llamafactory/hparams/parser.py`](https://github.com/hiyouga/LlamaFactory/blob/main/src/llamafactory/hparams/parser.py)**. The parser converts the input to a list format and enforces stage-specific constraints. For example, **PPO training** only accepts `wandb`, `tensorboard`, `trackio`, or `none`; other stages like SFT or DPO additionally permit `mlflow` and `neptune`. This validation step prevents runtime errors by catching incompatible tracker configurations early in the execution flow.

### Runtime Callback Execution

The **`ReporterCallback`** class in **[`src/llamafactory/train/callbacks.py`](https://github.com/hiyouga/LlamaFactory/blob/main/src/llamafactory/train/callbacks.py)** handles the actual integration. As a subclass of `TrainerCallback`, it implements the `on_train_begin` method to initialize external loggers immediately before training starts. When `report_to` contains `"wandb"`, the callback imports the **wandb** SDK, sets the project name from the `WANDB_PROJECT` environment variable (defaulting to `"llamafactory"`), and pushes the full configuration—including `model_args`, `data_args`, `finetuning_args`, and `generating_args`—via `wandb.config.update()`.

Similarly, when `"trackio"` is specified, the callback imports the **trackio** SDK and updates the configuration object with the same argument dictionaries. If `finetuning_args.use_swanlab` evaluates to true, the logger initializes **SwanLab** using an identical pattern.

## Platform-Specific Implementations

Each supported tracking platform leverages distinct initialization patterns while maintaining consistent configuration export.

### Weights & Biases (W&B)

For **W&B** integration, LlamaFactory relies on environment variables and automatic SDK initialization. The `ReporterCallback` checks for the `"wandb"` string in `report_to`, then invokes `wandb.init()` implicitly through the SDK. All hyperparameters and model configurations serialize automatically through the callback's dictionary export, making them browsable in the W&B dashboard without additional instrumentation code.

### TensorBoard Integration

**TensorBoard** support requires no custom callback code. LlamaFactory inherits from Hugging Face's `Seq2SeqTrainingArguments` (defined in **[`src/llamafactory/hparams/training_args.py`](https://github.com/hiyouga/LlamaFactory/blob/main/src/llamafactory/hparams/training_args.py)**), which exposes native `report_to=["tensorboard"]` functionality. When this flag is active, the underlying Transformers `Trainer` writes scalar logs to the specified `logging_dir`, typically `./logs/`, enabling visualization via `tensorboard --logdir`.

### Trackio and SwanLab

For **Trackio**, the Web UI renders additional fields under *Trackio Settings*: *Project Name* and *Space ID*. These values propagate to `trackio.config` during callback initialization, routing metrics to private Hugging Face Spaces or public Trackio dashboards. **SwanLab** follows an analogous path when users enable `use_swanlab` in the fine-tuning arguments, piggybacking on the same configuration export mechanism.

### MLflow and Neptune

**MLflow** and **Neptune** currently appear in the UI dropdown but remain restricted to specific training stages. As of the current implementation, these backends are available in the Web UI but require manual validation extension in [`parser.py`](https://github.com/hiyouga/LlamaFactory/blob/main/parser.py) for full CLI support across all training paradigms.

## Configuring Experiment Tracking in Practice

### Enable Weights & Biases via CLI

Set the project environment variable and pass the `report_to` flag:

```bash
export WANDB_PROJECT=my-llamafactory-run
llamafactory-cli train \
  --model_name_or_path meta-llama/Meta-Llama-3-8B \
  --train_file data/train.json \
  --report_to wandb \
  --output_dir ./output

```

The `ReporterCallback` automatically triggers `wandb.init()` and uploads the complete configuration object.

### Configure TensorBoard Logging

TensorBoard requires only the report target and logging directory:

```bash
llamafactory-cli train \
  --report_to tensorboard \
  --logging_dir ./tb_logs \
  --output_dir ./output

```

Visualize results after training:

```bash
tensorboard --logdir ./tb_logs

```

### Set Up Trackio from the Web UI

In the interface, expand **Trackio Settings** and configure:

```python

# Project Name: "my-hf-space"

# Trackio Space ID: "username/trackio-space"

# Enable external logger: select "trackio"

```

When training starts, the callback executes:

```python
trackio.config.update({
    "model_args": model_args.to_dict(),
    "data_args": data_args.to_dict(),
    "finetuning_args": finetuning_args.to_dict(),
    "generating_args": generating_args.to_dict(),
})

```

All metrics stream to the specified Trackio dashboard automatically.

## Summary

- **Three-layer architecture**: LlamaFactory separates UI selection ([`train.py`](https://github.com/hiyouga/LlamaFactory/blob/main/train.py)), configuration validation ([`parser.py`](https://github.com/hiyouga/LlamaFactory/blob/main/parser.py)), and runtime execution ([`callbacks.py`](https://github.com/hiyouga/LlamaFactory/blob/main/callbacks.py)) to ensure robust experiment tracking integration.
- **ReporterCallback**: The core integration point lives in [`src/llamafactory/train/callbacks.py`](https://github.com/hiyouga/LlamaFactory/blob/main/src/llamafactory/train/callbacks.py), handling initialization for W&B, Trackio, and SwanLab via `on_train_begin`.
- **Native TensorBoard support**: Inherited from Transformers' `Seq2SeqTrainingArguments`, requiring no custom callback code when `report_to` includes `"tensorboard"`.
- **Extensible design**: Adding new trackers involves updating the dropdown in the Web UI, extending validation rules in the parser, and adding initialization blocks to `ReporterCallback`.

## Frequently Asked Questions

### Which experiment tracking tools does LlamaFactory support natively?

LlamaFactory supports **Weights & Biases**, **TensorBoard**, **Trackio**, **SwanLab**, **MLflow**, and **Neptune** through the unified `report_to` interface. However, **PPO training** restricts valid options to `wandb`, `tensorboard`, `trackio`, and `none`, while supervised fine-tuning (SFT) and direct preference optimization (DPO) stages accept the full suite including `mlflow` and `neptune`.

### How do I configure Weights & Biases logging from the command line?

Export the `WANDB_PROJECT` environment variable to define your project name, then append `--report_to wandb` to your `llamafactory-cli train` command. The `ReporterCallback` in [`src/llamafactory/train/callbacks.py`](https://github.com/hiyouga/LlamaFactory/blob/main/src/llamafactory/train/callbacks.py) handles the `wandb.init()` call and automatically pushes `model_args`, `data_args`, and other configuration dictionaries to the dashboard.

### Can I use MLflow with LlamaFactory's CLI interface?

Currently, **MLflow** appears in the Web UI dropdown but requires the training stage to support it beyond the UI selection. To enable full CLI support, you must extend the `_verify_trackio_args` function in [`src/llamafactory/hparams/parser.py`](https://github.com/hiyouga/LlamaFactory/blob/main/src/llamafactory/hparams/parser.py) to accept `"mlflow"` for your specific training stage, then add the corresponding initialization block in `ReporterCallback.on_train_begin` following the existing W&B pattern.

### Where does LlamaFactory initialize the tracking callbacks during training?

The initialization occurs in **[`src/llamafactory/train/callbacks.py`](https://github.com/hiyouga/LlamaFactory/blob/main/src/llamafactory/train/callbacks.py)** within the `ReporterCallback` class. Specifically, the `on_train_begin` method checks the `report_to` list for supported strings like `"wandb"` or `"trackio"`, imports the respective SDK, and executes configuration updates before the first training step begins.