# How to Perform SFT Training for Llama 3 Chinese Models: A Complete Guide

> Master SFT training for Llama 3 Chinese models with this guide. Prepare data, train with LoRA/QLoRA using LLaMA Factory, and create your custom Chinese chat model.

- Repository: [Xinlu Lai/llama3-chinese-chat](https://github.com/crazyboym/llama3-chinese-chat)
- Tags: tutorial
- Published: 2026-02-28

---

**Supervised Fine-Tuning (SFT) transforms the generic Llama 3 checkpoint into a Chinese-oriented chat model by preparing Chinese dialogue data, converting it to ShareGPT format, training with LoRA/QLoRA using frameworks like LLaMA-Factory, and merging the adapter weights back into a standalone checkpoint.**

The `crazyboym/llama3-chinese-chat` repository provides a complete toolchain to execute this pipeline. It includes data conversion scripts, integration points for popular training frameworks, and post-processing utilities to merge and quantize the resulting model. This guide walks through each stage using the exact file paths and commands found in the source code.

## Data Preparation Pipeline

Before training, raw Chinese dialogue data must be standardized into the ShareGPT format expected by modern SFT frameworks. The repository provides two conversion utilities to handle this transformation.

### Converting Raw Data to Firefly Format

Use [`tools/convert_raw_data_for_firefly.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_raw_data_for_firefly.py) to transform arbitrary JSONL files into the Firefly-compatible structure. This script acts as the first normalization step for custom datasets.

```bash
python tools/convert_raw_data_for_firefly.py \
    --input raw_dataset.jsonl \
    --output data/firefly_chinese.jsonl

```

### Converting to ShareGPT Format

Next, convert the Firefly data into ShareGPT format using [`tools/convert_firefly_data_to_sharegpt.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_firefly_data_to_sharegpt.py). This format is required by LLaMA-Factory, XTuner, and other PEFT-compatible trainers.

```bash
python tools/convert_firefly_data_to_sharegpt.py \
    --input data/firefly_chinese.jsonl \
    --output data/sharegpt_chinese.jsonl

```

The repository recommends several high-quality Chinese datasets already available in these formats, including Firefly-1.1M, MOSS-003 SFT data, CodeChat, and ShareGPT-Chinese-English-90k.

## Training Configuration and Framework Selection

The repository supports four community-maintained SFT backends. Each accepts the ShareGPT format prepared in the previous steps.

### Supported SFT Frameworks

- **Firefly** – https://github.com/yangjianxin1/Firefly
- **LLaMA-Factory** – https://github.com/hiyouga/LLaMA-Factory
- **unsloth** – https://github.com/unslothai/unsloth
- **XTuner (Llama3-XTuner-CN)** – https://github.com/SmartFlowAI/Llama3-XTuner-CN

### QLoRA Configuration Example

For memory-efficient training on a single A100, use QLoRA with 4-bit quantization. Save the following configuration to [`configs/sft_llama3_zh.yaml`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/configs/sft_llama3_zh.yaml):

```yaml
model_name_or_path: "meta-llama/Meta-Llama-3-8B"
template: llama3
dataset: "./data/sharegpt_chinese.jsonl"
finetuning_type: qlora
qlora:
  bits: 4
  lr: 2e-4
  epochs: 3
  batch_size: 8
  gradient_accumulation_steps: 4
  lora_r: 128
  lora_alpha: 256
  target_modules:
    - "q_proj"
    - "k_proj"
    - "v_proj"
    - "o_proj"
    - "gate_proj"
    - "up_proj"
    - "down_proj"
output_dir: "./outputs/llama3_zh_sft"
use_peft: true

```

## Executing the SFT Training

Launch the training job using LLaMA-Factory with the configuration file:

```bash
llamafactory train --config configs/sft_llama3_zh.yaml

```

Upon completion, the script saves a PEFT adapter to `./outputs/llama3_zh_sft/adapter_model`.

## Post-Training Processing

After training, you must merge the LoRA weights into the base model for standalone deployment or further quantize the result for edge devices.

### Merging LoRA Weights

Use [`tools/merge_weight.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/merge_weight.py) to combine the adapter with the base Llama 3 checkpoint:

```bash
python tools/merge_weight.py \
    --base_model meta-llama/Meta-Llama-3-8B \
    --adapter ./outputs/llama3_zh_sft/adapter_model \
    --output_dir ./merged/llama3_zh_sft

```

This produces a single `pytorch_model.bin` suitable for standard inference.

### Quantization to GGUF

For deployment in Ollama or LM Studio, convert the merged checkpoint to GGUF format using [`tools/convert_gguf.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_gguf.py):

```bash
python tools/convert_gguf.py \
    --input ./merged/llama3_zh_sft \
    --output ./gguf/llama3_zh_sft_q4_k_m.gguf \
    --quant_type q4_k_m

```

## Validation and Inference

Test the fine-tuned model using the provided inference script. The [`deploy/python/chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/deploy/python/chat_demo.py) file loads the merged checkpoint and applies the Llama 3 chat template defined in the repository.

```bash
python deploy/python/chat_demo.py \
    --model_path ./merged/llama3_zh_sft \
    --system_prompt "你是一个中文智者，请用中文、友好且详细的方式回答。"

```

You should observe fluent Chinese responses that reflect the conversational style of your SFT dataset.

## Summary

- **Data Preparation**: Use [`tools/convert_raw_data_for_firefly.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_raw_data_for_firefly.py) and [`tools/convert_firefly_data_to_sharegpt.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_firefly_data_to_sharegpt.py) to normalize raw Chinese dialogue into ShareGPT format.
- **Training**: Configure QLoRA with 4-bit quantization in a YAML file and launch via `llamafactory train` (or equivalent frameworks like XTuner or unsloth).
- **Merging**: Combine the PEFT adapter with the base Llama 3 weights using [`tools/merge_weight.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/merge_weight.py) to create a standalone checkpoint.
- **Quantization**: Convert the merged model to GGUF format with [`tools/convert_gguf.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_gguf.py) for edge deployment.
- **Inference**: Validate the result with [`deploy/python/chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/deploy/python/chat_demo.py), ensuring the model responds in Chinese according to the system prompt.

## Frequently Asked Questions

### What datasets are recommended for Chinese SFT training?

The repository points to several high-quality, publicly available Chinese datasets already formatted for training: Firefly-1.1M (sharegpt format), MOSS-003 SFT data, CodeChat, and ShareGPT-Chinese-English-90k. These cover general conversation, coding, and mixed Chinese-English dialogue scenarios.

### Which training framework should I use for QLoRA fine-tuning?

The repository supports four major frameworks: Firefly, LLaMA-Factory, unsloth, and XTuner. For most users, **LLaMA-Factory** is recommended due to its simple YAML configuration and built-in support for QLoRA with 4-bit quantization, requiring only a single A100 GPU for the 8B parameter model.

### How do I merge LoRA weights back into the base model?

After training completes, run `python tools/merge_weight.py` with three arguments: `--base_model` pointing to the original Meta-Llama-3-8B checkpoint, `--adapter` pointing to your `adapter_model` output directory, and `--output_dir` for the merged result. This produces a standalone `pytorch_model.bin` suitable for deployment without PEFT dependencies.

### Can I deploy the fine-tuned model on consumer hardware?

Yes. After merging the weights, use `python tools/convert_gguf.py` to quantize the model to formats like `q4_k_m`. This reduces the 8B model to approximately 4-5GB, enabling deployment on consumer GPUs via Ollama or LM Studio using the provided chat template.