# Preparing Training Data for Llama 3 in ShareGPT Format: Step-by-Step Guide

> Easily prepare Llama 3 training data in ShareGPT format. This guide provides step-by-step instructions to convert JSONL to Firefly schema then to ShareGPT structure. Optimize your LLM training.

- Repository: [Xinlu Lai/llama3-chinese-chat](https://github.com/crazyboym/llama3-chinese-chat)
- Tags: how-to-guide
- Published: 2026-02-28

---

**To prepare training data for Llama 3 in ShareGPT format, convert raw JSONL datasets to the Firefly schema using [`tools/convert_raw_data_for_firefly.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_raw_data_for_firefly.py), then transform them to ShareGPT structure with [`tools/convert_firefly_data_to_sharegpt.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_firefly_data_to_sharegpt.py).**

The **crazyboym/llama3-chinese-chat** repository provides a complete data preparation pipeline for fine-tuning Llama 3 on Chinese conversational datasets. Preparing training data for Llama 3 in ShareGPT format requires transforming raw instruction-following datasets through a two-stage conversion process before they can be consumed by frameworks like LLaMA-Factory or Firefly.

## Repository Architecture for Data Preparation

The data preparation submodule follows a layered pipeline architecture designed to normalize diverse raw formats into the standard ShareGPT schema.

| Layer | Description | Key Files |
|-------|-------------|-----------|
| **Raw Data Ingestion** | Parses line-wise JSONL files containing fields like `instruction`, `input`, and `output`. | [`tools/convert_raw_data_for_firefly.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_raw_data_for_firefly.py) |
| **Schema Normalization** | Wraps entries into a single-element `conversation` list with `human` and `assistant` roles. | [`tools/convert_raw_data_for_firefly.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_raw_data_for_firefly.py) |
| **ShareGPT Conversion** | Maps normalized conversations to ShareGPT format using `from` and `value` keys. | [`tools/convert_firefly_data_to_sharegpt.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_firefly_data_to_sharegpt.py) |
| **Quality Utilities** | Provides sampling, token counting, and JSONL validation helpers. | [`tools/sample_data.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/sample_data.py), [`tools/count_data.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/count_data.py), [`tools/check_jsonl.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/check_jsonl.py) |

All conversion scripts reside in `/tools`, while inference demos and deployment utilities live under `/deploy`.

## Converting Raw Data to Firefly Schema

The first step transforms arbitrary instruction-format JSONL into the Firefly-style conversation structure. In [`tools/convert_raw_data_for_firefly.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_raw_data_for_firefly.py), the `convert_entry()` function builds human-assistant pairs from standard instruction fields.

```python

# tools/convert_raw_data_for_firefly.py

import json

def convert_entry(entry):
    # Combine instruction and input for the human turn

    return {
        "human": entry["instruction"] + entry["input"],
        "assistant": entry["output"]
    }

def convert_jsonl(input_file, output_file):
    with open(input_file, "r", encoding="utf-8") as f_in, \
         open(output_file, "w", encoding="utf-8") as f_out:
        for line in f_in:
            entry = json.loads(line)
            conv = {"conversation": [convert_entry(entry)]}
            json.dump(conv, f_out, ensure_ascii=False)
            f_out.write("\n")

```

Execute the conversion from the repository root:

```bash
python tools/convert_raw_data_for_firefly.py \
    --input ./m-a-p/COIG-CQIA/zhihu/zhihu_expansion.jsonl \
    --output zhihu_expansion_firefly.jsonl

```

This produces an intermediate file where each line contains a `conversation` list with `human` and `assistant` keys.

## Transforming Firefly to ShareGPT Format

The second step converts the Firefly schema to ShareGPT format using [`tools/convert_firefly_data_to_sharegpt.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_firefly_data_to_sharegpt.py). This script maps role names to ShareGPT identifiers (`human` → `human`, `assistant` → `gpt`) and restructures the data to use `from` and `value` fields.

```python

# tools/convert_firefly_data_to_sharegpt.py

import json

def convert_jsonl(input_file, output_file):
    with open(input_file, "r", encoding="utf-8") as fin, \
         open(output_file, "w", encoding="utf-8") as fout:
        for line in fin:
            data = json.loads(line.strip())
            convs = data["conversation"]
            new_convs = []
            for conv in convs:
                for role, txt in conv.items():
                    # Map roles to ShareGPT standard

                    new_role = "gpt" if role == "assistant" else "human"
                    new_convs.append({"from": new_role, "value": txt})
            fout.write(json.dumps({"conversations": new_convs},
                                 ensure_ascii=False) + "\n")

```

Run the transformation:

```bash
python tools/convert_firefly_data_to_sharegpt.py \
    --input zhihu_expansion_firefly.jsonl \
    --output zhihu_expansion_sharegpt.jsonl

```

The resulting ShareGPT records follow this schema:

```json
{
  "conversations": [
    {"from": "human", "value": "你好，请介绍一下中国古诗词。"},
    {"from": "gpt", "value": "中国古诗词历史悠久..."}
  ]
}

```

## Data Validation and Sampling Utilities

Before training, use the utility scripts to validate and inspect your dataset.

**Sampling a subset** with [`tools/sample_data.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/sample_data.py):

```python

# tools/sample_data.py

import random

def sample_jsonl(input_file, output_file, ratio=0.33):
    with open(input_file, "r", encoding="utf-8") as f:
        lines = f.readlines()
    sample = random.sample(lines, int(len(lines) * ratio))
    with open(output_file, "w", encoding="utf-8") as f:
        f.writelines(sample)

```

**Counting token statistics** with [`tools/count_data.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/count_data.py):

```python

# tools/count_data.py

def count_jsonl(input_file):
    max_len = min_len = total = cnt = 0
    with open(input_file, "r", encoding="utf-8") as f:
        for line in f:
            length = len(line.strip().split())
            max_len = max(max_len, length)
            min_len = min(min_len, length) if cnt else length
            total += length
            cnt += 1
    print(f"max: {max_len}, min: {min_len}, avg: {total / cnt:.2f}, lines: {cnt}")

```

## Integrating with Fine-Tuning Frameworks

Once your data is in ShareGPT format, it is compatible with major fine-tuning frameworks. Both **LLaMA-Factory** and **Firefly** accept JSONL files where each line contains a `conversations` array with `from` and `value` keys.

Example LLaMA-Factory training command:

```bash
llama_factory train \
    --model_name_or_path meta-llama/Meta-Llama-3-8B \
    --train_file zhihu_expansion_sharegpt.jsonl \
    --output_dir output/llama3_zhihu \
    --lora_r 128 --lora_alpha 256 \
    --epochs 3

```

## Inference and Deployment

After fine-tuning, test your model using the provided inference demos. The repository includes [`deploy/python/chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/deploy/python/chat_demo.py) for CLI interaction and [`deploy/streamlit/web_llama3_chat.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/deploy/streamlit/web_llama3_chat.py) for web-based testing.

Run the CLI demo:

```bash
python deploy/python/chat_demo.py \
    --model_name_or_path shareAI/llama3-Chinese-chat-8b \
    --load_in_4bit False

```

For 4-bit quantized inference, set `--load_in_4bit True` and configure the `BitsAndBytesConfig` in the script.

Launch the Streamlit web interface:

```bash
streamlit run deploy/streamlit/web_llama3_chat.py \
    --theme.base="dark" \
    /path/to/llama3-chinese-chat-8b

```

## Best Practices for ShareGPT Data Preparation

- **Validate JSONL structure** before training by running `python -m json.tool` on sample lines or using [`tools/check_jsonl.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/check_jsonl.py) to catch malformed entries early.
- **Maintain flat conversation lists** — ShareGPT expects a linear sequence of turns. Nested conversation objects will cause training failures in most frameworks.
- **Sample small subsets first** using [`tools/sample_data.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/sample_data.py) to verify data loading before committing to full training runs.
- **Use 4-bit quantization** for inference on limited GPU memory by enabling `load_in_4bit=True` in [`chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/chat_demo.py).
- **Preserve schema consistency** by version-controlling any modifications to the conversion scripts in `tools/`.

## Summary

- **Two-stage conversion** is required: raw JSONL → Firefly schema ([`tools/convert_raw_data_for_firefly.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_raw_data_for_firefly.py)) → ShareGPT format ([`tools/convert_firefly_data_to_sharegpt.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_firefly_data_to_sharegpt.py)).
- **ShareGPT schema** requires `conversations` arrays containing objects with `from` (role) and `value` (text) fields.
- **Utility scripts** ([`sample_data.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/sample_data.py), [`count_data.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/count_data.py)) help validate dataset size and distribution before training.
- **Framework compatibility** extends to LLaMA-Factory, Firefly, and SWIFT once data is in ShareGPT format.
- **Inference tools** include a Python CLI demo and Streamlit web interface under the `/deploy` directory.

## Frequently Asked Questions

### What is the ShareGPT format for Llama 3 training?

The ShareGPT format structures conversations as a JSON object with a `conversations` key containing an array of turns. Each turn has a `from` field identifying the speaker (`human` or `gpt`) and a `value` field containing the message text. According to the `crazyboym/llama3-chinese-chat` source code, this schema is produced by [`tools/convert_firefly_data_to_sharegpt.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_firefly_data_to_sharegpt.py) and is compatible with major fine-tuning frameworks like LLaMA-Factory.

### Can I convert existing instruction datasets to ShareGPT format?

Yes. If your dataset uses standard instruction fields (`instruction`, `input`, `output`), use [`tools/convert_raw_data_for_firefly.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_raw_data_for_firefly.py) to create an intermediate Firefly schema, then run [`tools/convert_firefly_data_to_sharegpt.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/convert_firefly_data_to_sharegpt.py) to produce the final ShareGPT file. This pipeline handles the role mapping and structural normalization automatically.

### How do I verify my ShareGPT file is correctly formatted?

Use [`tools/check_jsonl.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/tools/check_jsonl.py) to validate JSON structure, or run `python tools/count_data.py your_file.jsonl` to check line counts and token statistics. Additionally, inspect a few samples manually to ensure the `conversations` array contains alternating `human` and `gpt` entries with valid `from` and `value` keys.

### Which fine-tuning frameworks support this ShareGPT format?

The ShareGPT format produced by this repository is compatible with **LLaMA-Factory**, **Firefly**, and **SWIFT**. These frameworks expect the `conversations` array structure and automatically handle the `from`/`value` schema during tokenization and training.