# LlamaFactory | Yaowei Zheng | Knowledge Base | Instagit

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

GitHub Stars: 67.8k

Repository: https://github.com/hiyouga/LlamaFactory

---

## Articles

### [How to Check LlamaFactory Model Support Status: A Complete Guide to the Supported Models Registry](/hiyouga/LlamaFactory/how-can-i-check-the-status-of-llamafactorys-support-for-new-models)

Quickly check LlamaFactory model support status. Inspect the SUPPORTED_MODELS registry in constants.py for a full list of compatible architectures.

- Tags: how-to-guide
- Published: 2026-03-04

### [How LlamaFactory Integrates with Experiment Tracking Tools: A Complete Technical Guide](/hiyouga/LlamaFactory/how-does-llamafactory-integrate-with-experiment-tracking-tools)

Discover how LlamaFactory integrates with experiment tracking tools. Learn to route metrics and hyperparameters effortlessly for better ML model development with our technical guide.

- Tags: how-to-guide
- Published: 2026-03-04

### [Key Files for Configuring LlamaFactory: CLI, YAML, and Web UI Reference](/hiyouga/LlamaFactory/what-are-the-key-files-for-configuring-llamafactory)

Master LlamaFactory configuration with this guide. Discover key files for CLI, YAML, and Web UI setup, including parser.py and common.py. Optimize your LlamaFactory projects today.

- Tags: reference
- Published: 2026-03-04

### [How LlamaFactory Handles Large Datasets Using Streaming Mode: A Complete Technical Guide](/hiyouga/LlamaFactory/how-does-llamafactory-handle-large-datasets-using-streaming-mode)

Discover how LlamaFactory handles large datasets with streaming mode. Learn to train on massive corpora without loading everything into RAM. A complete technical guide.

- Tags: how-to-guide
- Published: 2026-03-04

### [Efficiency Techniques Implemented in LlamaFactory's Model Layer: A Complete Optimization Guide](/hiyouga/LlamaFactory/what-are-the-specific-efficiency-techniques-implemented-in-llamafactorys-model-layer)

Discover LlamaFactory's model layer efficiency techniques. Fine-tune 70B models on consumer hardware with gradient checkpointing, 4-bit quantization, GaLore, and more. Optimize GPU memory up to 75%.

- Tags: deep-dive
- Published: 2026-03-04

### [What Quantization Methods Are Supported by LlamaFactory? A Complete Guide to 9 Techniques](/hiyouga/LlamaFactory/what-quantization-methods-are-supported-by-llamafactory)

Explore LlamaFactory's comprehensive support for nine quantization methods like BNB GPTQ AWQ and more. Discover runtime and post-training techniques for efficient model deployment.

- Tags: how-to-guide
- Published: 2026-03-04

### [Does LlamaFactory Offer Zero-Code Training? A Complete Technical Guide](/hiyouga/LlamaFactory/does-llamafactory-offer-a-zero-code-training-option)

Discover if LlamaFactory offers zero-code training. This technical guide explains how to fine-tune 100+ LLMs using YAML and a simple CLI or web UI without writing Python code.

- Tags: how-to-guide
- Published: 2026-03-04

### [What is LlamaBoard and How Does It Work: The Complete Guide to LlamaFactory's Web UI](/hiyouga/LlamaFactory/what-is-llamaboard-and-how-does-it-work)

Discover LlamaBoard, the Gradio web UI for LlamaFactory. Easily configure, launch, and monitor LLM fine-tuning jobs via your browser. No Python code needed.

- Tags: how-to-guide
- Published: 2026-03-04

### [How to Use the OpenAI-Compatible API Provided by LlamaFactory](/hiyouga/LlamaFactory/how-do-i-use-the-openai-compatible-api-provided-by-llamafactory)

Learn how to use LlamaFactory's OpenAI-compatible API. Connect any model to standard OpenAI SDKs by changing the base URL and leverage powerful LLMs with ease.

- Tags: how-to-guide
- Published: 2026-03-04

### [LlamaFactory Inference Backends: The Complete Guide to Engine Selection](/hiyouga/LlamaFactory/which-inference-backends-does-llamafactory-support)

Explore LlamaFactory inference backends including HuggingFace Transformers vLLM SGLang and K-Transformers. Optimize your LLM deployment by selecting the best engine.

- Tags: how-to-guide
- Published: 2026-03-04

### [Inference Capabilities of LlamaFactory: A Complete Guide to vLLM, HuggingFace, and SGLang Backends](/hiyouga/LlamaFactory/what-are-the-inference-capabilities-of-llamafactory)

Explore LlamaFactory's inference capabilities with vLLM HuggingFace and SGLang backends. Deploy seamlessly across single-GPU multi-GPU and server environments.

- Tags: deep-dive
- Published: 2026-03-04

### [How to Enable Distributed Training with LlamaFactory: FSDP, DeepSpeed, and Multi-Node Setups](/hiyouga/LlamaFactory/how-can-i-enable-distributed-training-with-llamafactory)

Easily enable distributed training in LlamaFactory using FSDP, DeepSpeed, or multi-node setups. Configure your YAML and launch with torchrun for efficient model training.

- Tags: how-to-guide
- Published: 2026-03-04

### [Does LlamaFactory Support DeepSpeed for Training? Implementation Guide](/hiyouga/LlamaFactory/does-llamafactory-support-deepspeed-for-training)

Yes LlamaFactory supports DeepSpeed training via accelerate enabling ZeRO-2 and ZeRO-3 strategies with automatic initialization and checkpoint management.

- Tags: how-to-guide
- Published: 2026-03-04

### [What Training Workflows Are Available in LlamaFactory: The Complete Guide](/hiyouga/LlamaFactory/what-training-workflows-are-available-in-llamafactory)

Explore LlamaFactory's seven training workflows: SFT, RM, PT, PPO, KTO, DPO, and MCA. This guide details each modular entry point for efficient model fine-tuning.

- Tags: how-to-guide
- Published: 2026-03-04

### [How LlamaFactory Orchestrates the Training Process: From CLI to Distributed Workflows](/hiyouga/LlamaFactory/how-does-llamafactory-orchestrate-the-training-process)

Discover how LlamaFactory orchestrates training from CLI to distributed workflows. Learn its dispatch mechanism and stage-specific training paths for efficient LLM fine-tuning.

- Tags: internals
- Published: 2026-03-04

### [LlamaFactory Prompt Templates: Complete Guide to 80+ Built-In Chat Formats](/hiyouga/LlamaFactory/what-are-the-prompt-templates-supported-by-llamafactory)

Explore LlamaFactory prompt templates and discover 80+ built-in chat formats like Alpaca LLaMA-2 LLaMA-3 ChatML Qwen and Gemma. Easily use them via CLI Web UI or Python API.

- Tags: tutorial
- Published: 2026-03-04

### [How LlamaFactory Facilitates Data Processing and Formatting for Training: A Deep Dive into the Pipeline](/hiyouga/LlamaFactory/how-does-llamafactory-facilitate-data-processing-and-formatting-for-training)

Discover LlamaFactory's four-stage pipeline: Load, Convert, Process, and Collate. Streamline your data processing and formatting for efficient AI model training. Learn more.

- Tags: deep-dive
- Published: 2026-03-04

### [Supported Training Approaches in LlamaFactory: Complete Guide to LLM Fine-Tuning](/hiyouga/LlamaFactory/what-are-the-supported-training-approaches-in-llamafactory)

Explore LlamaFactory's eight supported LLM fine-tuning approaches including Pre-Training, SFT, PPO, DPO, KTO, ORPO and SimPO. Achieve efficient fine-tuning with LoRA and QLoRA.

- Tags: how-to-guide
- Published: 2026-03-04

### [How LlamaFactory Supports Multiple LLM Architectures: Auto Detection and Unified Training](/hiyouga/LlamaFactory/how-does-llamafactory-support-multiple-llm-architectures)

Discover how LlamaFactory auto detects and unifies training for diverse LLM architectures like causal LM, vision-language, and more via Hugging Face AutoModel in a single pipeline.

- Tags: deep-dive
- Published: 2026-03-04

### [Where to Find the Central Configuration Object for LlamaFactory: A Complete Guide](/hiyouga/LlamaFactory/where-can-i-find-the-central-configuration-object-for-llamafactory)

Discover the central configuration object for LlamaFactory in our complete guide. Learn where to find the get_args() tuple for efficient parameter parsing.

- Tags: how-to-guide
- Published: 2026-03-04

### [How LlamaFactory Handles Model Loading and Configuration: A Deep Dive into src/llamafactory/model/loader.py](/hiyouga/LlamaFactory/how-does-llamafactory-handle-model-loading-and-configuration)

Discover how LlamaFactory manages model loading and configuration. Explore the key functions in src/llamafactory/model/loader.py for seamless initialization and optimization.

- Tags: deep-dive
- Published: 2026-03-04

### [Core Components of the LlamaFactory Architecture: Modular Design for LLM Fine-Tuning](/hiyouga/LlamaFactory/what-are-the-core-components-of-the-llamafactory-architecture)

Discover the nine core components of LlamaFactory architecture enabling flexible LLM fine-tuning. Explore ModelEngine, DataEngine, and more for extensible model customization.

- Tags: architecture
- Published: 2026-03-04