# Recommended Hardware Configurations for Training Different Model Sizes on Hugging Face Jobs: The Complete Guide

> Find optimal hardware for Hugging Face Jobs using our guide. Choose GPU flavors based on model size—from T4 for <1B models to A100 for >13B—to maximize cost efficiency and memory.

- Repository: [Hugging Face/skills](https://github.com/huggingface/skills)
- Tags: tutorial
- Published: 2026-03-08

---

**Select GPU flavors based on model parameter count using the VRAM rule of thumb—`t4-small` for tiny models (<1B), `a10g` series for small-to-large models (1B–13B), and multi-GPU `a10g-largex4` or `a100-large` for very-large models (>13B)—to optimize both cost efficiency and memory fit.**

The `huggingface/skills` repository provides definitive guidance for matching compute resources to model sizes through its detailed hardware documentation. According to the source code in [`skills/hugging-face-jobs/references/hardware_guide.md`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-jobs/references/hardware_guide.md) and [`SKILL.md`](https://github.com/huggingface/skills/blob/main/SKILL.md), selecting the proper hardware flavor ensures you avoid over-provisioning expensive GPUs while preventing out-of-memory errors during training.

## Hardware Selection Matrix by Model Size

The hardware guide implements a tiered recommendation system based on parameter count and training methodology. The following matrix maps model sizes to specific Hugging Face Jobs flavors:

| Model Size | Minimum VRAM | Recommended Flavor | Budget Tier | Use Case |
|------------|---------------|-------------------|-------------|----------|
| **Tiny (<1B)** | 2–4 GB | `t4-small` (16 GB) | <$5/hour | Testing, prototyping |
| **Small (1B–3B)** | 4–8 GB | `t4-medium` (16 GB) or `a10g-small` (24 GB) | $5–$20/hour | Fine-tuning small LLMs |
| **Medium (3B–7B)** | 8–16 GB | `a10g-small` (24 GB) or `a10g-large` (24 GB) | $20–$50/hour | Production LoRA training |
| **Large (7B–13B)** | 16–28 GB | `a10g-large` (24 GB) or `a100-large` (40 GB) | $50–$200/hour | Full-parameter or LoRA fine-tuning |
| **Very-Large (>13B)** | ≥28 GB | `a100-large` (40 GB) or `a10g-largex4` (96 GB) | $200+/hour | Multi-GPU distributed training |

## The VRAM Calculation Rule

As documented in [`references/hardware_guide.md`](https://github.com/huggingface/skills/blob/main/references/hardware_guide.md) (lines 131–140), memory requirements follow precise formulas based on training strategy:

- **Inference**: `Memory (GB) ≈ params_in_billions × 2–4`
- **Full fine-tuning**: `Memory (GB) ≈ params_in_billions × 20`
- **LoRA/QLoRA fine-tuning**: `Memory (GB) ≈ params_in_billions × 4`

For example, a 7B model requires approximately 28 GB for LoRA fine-tuning (7 × 4), making the 24 GB `a10g-large` suitable with gradient checkpointing, while full fine-tuning requires 140 GB, necessitating multi-GPU configurations.

## Flavor-Specific Recommendations

### Tiny Models (<1B Parameters)

For models under 1 billion parameters, the `t4-small` flavor provides 16 GB VRAM—four times the minimum requirement. This configuration handles small-scale experiments and tokenizer training efficiently.

```python
script = """

# /// script

# dependencies = ["transformers", "torch"]

# ///

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "facebook/opt-125m"  # 0.125B parameters

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id).to("cuda")

inputs = tokenizer("Hello, world!", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0]))
"""

hf_jobs("uv", {
    "script": script,
    "flavor": "t4-small",
    "timeout": "30m",
    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
})

```

### Small to Medium Models (1B–7B Parameters)

The `a10g-small` and `a10g-large` flavors both provide 24 GB VRAM, supporting LoRA fine-tuning for models up to 7B parameters. According to the hardware guide (lines 114–120), these A10G instances represent the optimal price-performance balance for production training jobs.

```python
script = """

# /// script

# dependencies = ["peft", "transformers", "datasets", "torch"]

# ///

from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments
from peft import LoraConfig, get_peft_model
from datasets import load_dataset
import torch

model_id = "meta-llama/Meta-Llama-3-8B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# LoRA reduces VRAM to ~4GB overhead

lora_config = LoraConfig(r=8, lora_alpha=32, target_modules=["q_proj", "v_proj"])
model = get_peft_model(model, lora_config)

dataset = load_dataset("json", data_files="train.json")["train"]

args = TrainingArguments(
    output_dir="/tmp/adapter",
    per_device_train_batch_size=2,
    num_train_epochs=1,
    learning_rate=2e-4,
    fp16=True
)

Trainer(model=model, args=args, train_dataset=dataset).train()
"""

hf_jobs("uv", {
    "script": script,
    "flavor": "a10g-large",
    "timeout": "8h",
    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
})

```

### Large and Very-Large Models (>7B Parameters)

For models exceeding 13 billion parameters, single-GPU configurations prove insufficient. The hardware guide recommends `a100-large` (40 GB) for single-GPU training or multi-GPU flavors like `a10g-largex4` (96 GB total) for distributed training via Hugging Face Accelerate.

```python
script = """

# /// script

# dependencies = ["accelerate", "transformers", "torch"]

# ///

from accelerate import Accelerator
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

accelerator = Accelerator()
model_id = "meta-llama/Meta-Llama-3-70B"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"  # Distributes across 4 GPUs

)

tokenizer = AutoTokenizer.from_pretrained(model_id)

# Training loop with accelerator

inputs = tokenizer("Training data", return_tensors="pt")
inputs = {k: v.to(accelerator.device) for k, v in inputs.items()}
outputs = model(**inputs, labels=inputs["input_ids"])
accelerator.backward(outputs.loss)
"""

hf_jobs("uv", {
    "script": script,
    "flavor": "a10g-largex4",
    "timeout": "24h",
    "secrets": {"HF_TOKEN": "$HF_TOKEN"}
})

```

## Compute Architecture Selection

The repository distinguishes between three compute categories in [`SKILL.md`](https://github.com/huggingface/skills/blob/main/SKILL.md):

- **CPU flavors** (`cpu-basic`, `cpu-upgrade`): Optimal for data preprocessing, statistical analysis, and tokenization workflows that don't require GPU acceleration
- **GPU flavors** (`t4-*`, `a10g-*`, `a100-*`, `l4-*`): Required for all deep learning training; select based on the VRAM calculation matrix above
- **TPU flavors** (`v5e-1x1`, `v5e-2x2`, `v5e-2x4`): Optimized for JAX/Flax pipelines requiring massive parallel computation

## Key Reference Files in the Repository

The hardware recommendations derive from specific source files within `huggingface/skills`:

- **[`skills/hugging-face-jobs/references/hardware_guide.md`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-jobs/references/hardware_guide.md)**: Contains the complete VRAM calculation formulas and flavor specifications (lines 101–140)
- **[`skills/hugging-face-jobs/SKILL.md`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-jobs/SKILL.md)**: Provides the high-level hardware selection matrix mapping workload types to recommended instances
- **[`skills/hugging-face-jobs/scripts/generate-responses.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-jobs/scripts/generate-responses.py)**: Demonstrates `a10g-large` usage for inference workloads
- **[`skills/hugging-face-jobs/scripts/cot-self-instruct.py`](https://github.com/huggingface/skills/blob/main/skills/hugging-face-jobs/scripts/cot-self-instruct.py)**: Implements multi-GPU `l4x4` configuration for synthetic data generation

## Summary

- **Calculate VRAM needs** using the parameter-based formulas: ×4 for LoRA, ×20 for full fine-tuning, ×2–4 for inference
- **Start with the smallest sufficient GPU** that meets your VRAM requirement, scaling up only when encountering OOM errors
- **Use `t4-small`** (16 GB) for models under 1B parameters and initial prototyping
- **Select `a10g-large`** (24 GB) for 3B–13B parameter models using LoRA techniques
- **Deploy multi-GPU flavors** (`a10g-largex4`, `a100-large`) for models exceeding 13B parameters or full fine-tuning scenarios
- **Set explicit timeouts** and configure Hub authentication to prevent data loss during long training runs

## Frequently Asked Questions

### What is the minimum VRAM required for fine-tuning a 7B parameter model?

Fine-tuning a 7B model requires approximately 28 GB VRAM for LoRA-based methods (7 × 4 GB) or 140 GB for full-parameter training (7 × 20 GB). The `a10g-large` flavor provides 24 GB VRAM, which accommodates 7B LoRA fine-tuning with gradient checkpointing enabled, while `a100-large` (40 GB) handles the workload without additional optimization techniques.

### How do I choose between T4, A10G, and A100 GPUs for training?

Select based on model size and budget constraints. **T4 GPUs** (`t4-small`, `t4-medium`) offer 16 GB VRAM suitable for models under 3B parameters at the lowest cost tier. **A10G GPUs** (`a10g-small`, `a10g-large`) provide 24 GB VRAM and represent the optimal price-performance ratio for 3B–13B models. **A100 GPUs** (`a100-large`) deliver 40 GB VRAM necessary for models exceeding 13B parameters or when implementing full fine-tuning on 7B+ models.

### Can I train large models on Hugging Face Jobs without multi-GPU setups?

Single GPU training is feasible for models up to approximately 13B parameters using memory-efficient techniques like LoRA, 4-bit quantization, and gradient checkpointing on `a100-large` (40 GB). However, models exceeding 13B parameters or requiring full fine-tuning necessitate multi-GPU flavors such as `a10g-largex2` (48 GB) or `a10g-largex4` (96 GB) to distribute layers across devices using Hugging Face Accelerate.

### What timeout should I set for different model training jobs?

Timeout configuration depends on dataset size and training steps. Tiny models (<1B) typically require 30 minutes to 2 hours. Small-to-medium models (1B–7B) with LoRA need 4–8 hours for single-epoch fine-tuning. Large models (7B–13B) require 12–24 hours, while very-large models (>13B) on multi-GPU setups should specify 24+ hour timeouts to accommodate checkpoint saving and validation loops.