How to Perform SFT Training for Llama 3 Chinese Models: A Complete Guide

Supervised Fine-Tuning (SFT) transforms the generic Llama 3 checkpoint into a Chinese-oriented chat model by preparing Chinese dialogue data, converting it to ShareGPT format, training with LoRA/QLoRA using frameworks like LLaMA-Factory, and merging the adapter weights back into a standalone checkpoint.

The crazyboym/llama3-chinese-chat repository provides a complete toolchain to execute this pipeline. It includes data conversion scripts, integration points for popular training frameworks, and post-processing utilities to merge and quantize the resulting model. This guide walks through each stage using the exact file paths and commands found in the source code.

Data Preparation Pipeline

Before training, raw Chinese dialogue data must be standardized into the ShareGPT format expected by modern SFT frameworks. The repository provides two conversion utilities to handle this transformation.

Converting Raw Data to Firefly Format

Use tools/convert_raw_data_for_firefly.py to transform arbitrary JSONL files into the Firefly-compatible structure. This script acts as the first normalization step for custom datasets.

python tools/convert_raw_data_for_firefly.py \
    --input raw_dataset.jsonl \
    --output data/firefly_chinese.jsonl

Converting to ShareGPT Format

Next, convert the Firefly data into ShareGPT format using tools/convert_firefly_data_to_sharegpt.py. This format is required by LLaMA-Factory, XTuner, and other PEFT-compatible trainers.

python tools/convert_firefly_data_to_sharegpt.py \
    --input data/firefly_chinese.jsonl \
    --output data/sharegpt_chinese.jsonl

The repository recommends several high-quality Chinese datasets already available in these formats, including Firefly-1.1M, MOSS-003 SFT data, CodeChat, and ShareGPT-Chinese-English-90k.

Training Configuration and Framework Selection

The repository supports four community-maintained SFT backends. Each accepts the ShareGPT format prepared in the previous steps.

Supported SFT Frameworks

• Firefly – https://github.com/yangjianxin1/Firefly
• LLaMA-Factory – https://github.com/hiyouga/LLaMA-Factory
• unsloth – https://github.com/unslothai/unsloth
• XTuner (Llama3-XTuner-CN) – https://github.com/SmartFlowAI/Llama3-XTuner-CN

QLoRA Configuration Example

For memory-efficient training on a single A100, use QLoRA with 4-bit quantization. Save the following configuration to configs/sft_llama3_zh.yaml:

model_name_or_path: "meta-llama/Meta-Llama-3-8B"
template: llama3
dataset: "./data/sharegpt_chinese.jsonl"
finetuning_type: qlora
qlora:
  bits: 4
  lr: 2e-4
  epochs: 3
  batch_size: 8
  gradient_accumulation_steps: 4
  lora_r: 128
  lora_alpha: 256
  target_modules:
    - "q_proj"
    - "k_proj"
    - "v_proj"
    - "o_proj"
    - "gate_proj"
    - "up_proj"
    - "down_proj"
output_dir: "./outputs/llama3_zh_sft"
use_peft: true

Executing the SFT Training

Launch the training job using LLaMA-Factory with the configuration file:

llamafactory train --config configs/sft_llama3_zh.yaml

Upon completion, the script saves a PEFT adapter to ./outputs/llama3_zh_sft/adapter_model.

Post-Training Processing

After training, you must merge the LoRA weights into the base model for standalone deployment or further quantize the result for edge devices.

Merging LoRA Weights

Use tools/merge_weight.py to combine the adapter with the base Llama 3 checkpoint:

python tools/merge_weight.py \
    --base_model meta-llama/Meta-Llama-3-8B \
    --adapter ./outputs/llama3_zh_sft/adapter_model \
    --output_dir ./merged/llama3_zh_sft

This produces a single pytorch_model.bin suitable for standard inference.

Quantization to GGUF

For deployment in Ollama or LM Studio, convert the merged checkpoint to GGUF format using tools/convert_gguf.py:

python tools/convert_gguf.py \
    --input ./merged/llama3_zh_sft \
    --output ./gguf/llama3_zh_sft_q4_k_m.gguf \
    --quant_type q4_k_m

Validation and Inference

Test the fine-tuned model using the provided inference script. The deploy/python/chat_demo.py file loads the merged checkpoint and applies the Llama 3 chat template defined in the repository.

python deploy/python/chat_demo.py \
    --model_path ./merged/llama3_zh_sft \
    --system_prompt "你是一个中文智者，请用中文、友好且详细的方式回答。"

You should observe fluent Chinese responses that reflect the conversational style of your SFT dataset.

Summary

• Data Preparation: Use tools/convert_raw_data_for_firefly.py and tools/convert_firefly_data_to_sharegpt.py to normalize raw Chinese dialogue into ShareGPT format.
• Training: Configure QLoRA with 4-bit quantization in a YAML file and launch via llamafactory train (or equivalent frameworks like XTuner or unsloth).
• Merging: Combine the PEFT adapter with the base Llama 3 weights using tools/merge_weight.py to create a standalone checkpoint.
• Quantization: Convert the merged model to GGUF format with tools/convert_gguf.py for edge deployment.
• Inference: Validate the result with deploy/python/chat_demo.py, ensuring the model responds in Chinese according to the system prompt.

Frequently Asked Questions

What datasets are recommended for Chinese SFT training?

The repository points to several high-quality, publicly available Chinese datasets already formatted for training: Firefly-1.1M (sharegpt format), MOSS-003 SFT data, CodeChat, and ShareGPT-Chinese-English-90k. These cover general conversation, coding, and mixed Chinese-English dialogue scenarios.

Which training framework should I use for QLoRA fine-tuning?

The repository supports four major frameworks: Firefly, LLaMA-Factory, unsloth, and XTuner. For most users, LLaMA-Factory is recommended due to its simple YAML configuration and built-in support for QLoRA with 4-bit quantization, requiring only a single A100 GPU for the 8B parameter model.

How do I merge LoRA weights back into the base model?

After training completes, run python tools/merge_weight.py with three arguments: --base_model pointing to the original Meta-Llama-3-8B checkpoint, --adapter pointing to your adapter_model output directory, and --output_dir for the merged result. This produces a standalone pytorch_model.bin suitable for deployment without PEFT dependencies.

Can I deploy the fine-tuned model on consumer hardware?

Yes. After merging the weights, use python tools/convert_gguf.py to quantize the model to formats like q4_k_m. This reduces the 8B model to approximately 4-5GB, enabling deployment on consumer GPUs via Ollama or LM Studio using the provided chat template.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}