# How to Configure AFAB vs 1F1B Pipeline Schedules in Nanotron

> Configure AFAB vs 1F1B pipeline schedules in Nanotron Nanotron using YAML, Python API, or CLI. Optimize your training flow with All-Forward-All-Backward or Interleaved-Forward-Interleaved-Backward execution.

- Repository: [Hugging Face/nanotron](https://github.com/huggingface/nanotron)
- Tags: how-to-guide
- Published: 2026-03-03

---

**Set `pipeline_parallel.schedule` to `"afab"` for All‑Forward‑All‑Backward execution or `"1f1b"` for Interleaved‑Forward‑Interleaved‑Backward via YAML configs, Python APIs, or CLI overrides in Hugging Face Nanotron.**

Nanotron is Hugging Face's lightweight framework for large‑scale transformer pretraining, offering configurable pipeline parallelism to optimize distributed training. Selecting the appropriate pipeline schedule directly impacts GPU memory consumption and computational throughput when scaling across multiple devices. This guide explains how to configure AFAB vs 1F1B pipeline schedules in Nanotron using the `PipelineParallelConfig` dataclass and runtime interfaces.

## Understanding AFAB and 1F1B Pipeline Schedules

### AFAB (All‑Forward‑All‑Backward)

The **AFAB** schedule executes the forward pass for every micro‑batch across all pipeline stages before initiating any backward passes. In [`nanotron/config/config.py`](https://github.com/huggingface/nanotron/blob/main/nanotron/config/config.py), this strategy is activated by setting `schedule="afab"` within the `PipelineParallelConfig` dataclass. While this approach simplifies reasoning about execution order and aids debugging, it requires storing the full set of activations for the entire pipeline depth simultaneously, increasing peak memory usage.

### 1F1B (Interleaved‑Forward‑Interleaved‑Backward)

The **1F1B** schedule (also documented as IFIB) interleaves forward and backward operations, immediately processing the backward pass for a micro‑batch once its forward pass completes. Configure this by setting `schedule="1f1b"` in the same configuration object. This pipelining strategy minimizes the activation memory footprint by releasing tensors as soon as they are consumed by the gradient computation, making it ideal for training large models on memory‑constrained hardware.

## Configuration Methods

### YAML Configuration

Define the schedule within the `pipeline_parallel` block of your training configuration file:

```yaml
pipeline_parallel:
  schedule: "1f1b"  # Use "afab" for All‑Forward‑All‑Backward

  # Additional pipeline parallel settings...

```

Switching strategies requires only changing the string value between `"afab"` and `"1f1b"`.

### Programmatic Configuration

When constructing training configurations dynamically, modify the `NanotronConfig` object before instantiating the `Trainer`:

```python
from nanotron.config import NanotronConfig

cfg = NanotronConfig.load_from_yaml("examples/config_tiny_llama.yaml")
cfg.pipeline_parallel.schedule = "afab"  # Options: "afab" or "1f1b"

trainer = Trainer(cfg, model, optimizer)
trainer.fit()

```

### Command‑Line Override

Override the YAML configuration at runtime using the `--pipeline_parallel_schedule` flag when executing [`run_train.py`](https://github.com/huggingface/nanotron/blob/main/run_train.py):

```bash
python run_train.py \
  --config examples/config_tiny_llama.yaml \
  --pipeline_parallel_schedule 1f1b

```

## Performance Considerations

**Memory Requirements:** AFAB stores all activations for the complete pipeline, typically requiring **≥2× the per‑stage model size** in GPU memory. 1F1B maintains a constant memory footprint regardless of pipeline depth by immediately freeing activation buffers after backward computation.

**Throughput Optimization:** 1F1B generally achieves higher throughput on memory‑constrained systems because it reduces memory pressure, allowing increased micro‑batch sizes or deeper pipelines without out‑of‑memory errors.

**Debugging Workflow:** AFAB provides deterministic execution order (all forwards followed by all backwards), making it preferable for initial prototyping and numerical validation. Transition to 1F1B for production‑scale training runs where memory efficiency is critical.

## Summary

- **AFAB** (`schedule: "afab"`): Executes all forward passes before any backward passes; higher memory usage but deterministic and easier to debug.
- **1F1B** (`schedule: "1f1b"`): Interleaves forward and backward passes per micro‑batch; minimizes activation memory and scales to larger models.
- Configuration is controlled via `pipeline_parallel.schedule` in YAML files, `cfg.pipeline_parallel.schedule` programmatically, or the `--pipeline_parallel_schedule` CLI argument.

## Frequently Asked Questions

### What is the difference between AFAB and 1F1B in Nanotron?

AFAB runs all forward micro‑batches through the pipeline first, then executes all backward passes, keeping every activation in memory simultaneously. 1F1B immediately follows each forward pass with its corresponding backward pass, reducing peak memory usage by freeing activations as soon as gradients are computed.

### Where is the pipeline schedule defined in the Nanotron source code?

The schedule parameter is defined in [`nanotron/config/config.py`](https://github.com/huggingface/nanotron/blob/main/nanotron/config/config.py) within the `PipelineParallelConfig` dataclass as the `schedule` field, which accepts the string literals `"afab"` or `"1f1b"` according to the repository's pipeline parallel implementation.

### Can I switch between AFAB and 1F1B without modifying my model code?

Yes. The pipeline schedule is purely a runtime configuration parameter. You can switch strategies by changing the schedule value in your YAML configuration, passing the `--pipeline_parallel_schedule` CLI flag, or modifying the config object programmatically without altering model definitions or training loop logic.

### Which schedule should I use for large‑scale training?

Use **1F1B** for large models or when GPU memory is limited, as it maintains constant activation memory regardless of pipeline depth. Use **AFAB** when debugging distributed training issues or when you have abundant GPU memory and prefer the simplicity of deterministic execution ordering.