# Handling Multi-Turn Conversations with Llama3-Chinese: A Complete Implementation Guide

> Master multi-turn conversations with Llama3-Chinese. Follow our guide to implement chat templates for seamless interaction and enhanced AI dialogue.

- Repository: [Xinlu Lai/llama3-chinese-chat](https://github.com/crazyboym/llama3-chinese-chat)
- Tags: how-to-guide
- Published: 2026-02-28

---

**Handle multi-turn conversations in Llama3-Chinese by concatenating system instructions, historical exchanges, and the current query into the official Llama 3 chat template, then feeding the resulting token IDs to the model's generation loop.**

The crazyboym/llama3-chinese-chat repository demonstrates context-aware dialogue through two production-ready implementations. This guide examines the deterministic prompt-building pipeline used to maintain conversation state, manage context windows, and generate coherent responses across multiple turns.

## Understanding the Llama3 Chat Template

The repository implements the official **Llama 3 chat format**, which structures every interaction with special control tokens. The template separates system instructions, user inputs, and assistant responses using explicit delimiters:

```text
<|begin_of_text|><<SYS>>
{system_prompt}
<</SYS>>

<|user|>
{user_message}<|eot_id|>
<|assistant|>
{assistant_message}<|end_of_text|>

```

This format is registered in [`deploy/python/chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/deploy/python/chat_demo.py) within the `template_dict['llama3']` entry. The **system prompt** establishes the assistant's persona, while historical turns accumulate between the special tokens to maintain context.

## Building Multi-Turn Prompts

The repository provides two distinct approaches for assembling prompts from conversation history: a Streamlit web interface and a pure Python CLI.

### Streamlit Implementation (`combine_history`)

In [`deploy/web_streamlit_for_v1.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/deploy/web_streamlit_for_v1.py), the `combine_history()` function constructs the full prompt by iterating over session state:

```python
def combine_history(prompt):
    messages = st.session_state.messages
    total_prompt = ''
    for message in messages:
        cur_content = message['content']
        if message['role'] == 'user':
            cur_prompt = user_prompt.format(user=cur_content)
        elif message['role'] == 'robot':
            cur_prompt = robot_prompt.format(robot=cur_content)
        total_prompt += cur_prompt
    system = system_prompt.format(content="你是一个超级人工智能，拥有全人类的智慧汇合……")
    total_prompt = system + total_prompt + cur_query_prompt.format(user=prompt)
    return total_prompt

```

This function retrieves stored messages from `st.session_state.messages`, formats each according to its role, prepends a fixed **system instruction**, and appends the current user query.

### CLI Implementation (`build_prompt`)

The command-line version in [`deploy/python/chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/deploy/python/chat_demo.py) works directly with token IDs rather than strings:

```python
def build_prompt(tokenizer, template, query, history, system=None):
    # system prompt handling

    if system_format is not None and system is not None:
        system_text = system_format.format(content=system)
        input_ids = tokenizer.encode(system_text, add_special_tokens=False)

    # append historical rounds

    for item in history:
        role, message = item['role'], item['message']
        if role == 'user':
            message = user_format.format(content=message, stop_token=tokenizer.eos_token)
        else:
            message = assistant_format.format(content=message, stop_token=tokenizer.eos_token)
        input_ids += tokenizer.encode(message, add_special_tokens=False)

    # current query

    # (the caller already appended the query to `history` before invoking)

    return torch.tensor([input_ids], dtype=torch.long)

```

Unlike the Streamlit version, `build_prompt()` returns a PyTorch tensor ready for GPU inference, encoding the system text, historical turns, and current query into a contiguous sequence of token IDs.

## Managing Conversation History and Context Windows

Both implementations maintain **conversation history** as a flat list of dictionaries tracking roles and content. The standard loop pattern appears in [`chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/chat_demo.py):

```python

# Initialize empty history

history = []

while True:
    query = input('# User：').strip()

    
    # Build prompt including past turns

    input_ids = build_prompt(tokenizer, template, query,
                            copy.deepcopy(history), system=None).to(model.device)
    
    # Generate response

    outputs = model.generate(input_ids,
                             max_new_tokens=512,
                             do_sample=True,
                             top_p=0.9,
                             temperature=0.7,
                             repetition_penalty=1.1,
                             eos_token_id=stop_id)
    
    # Decode and clean response

    response = tokenizer.decode(outputs[0][len(input_ids[0]):])
    response = response.strip().replace(template.stop_word, '')
    
    # Store both sides of the turn

    history.append({'role': 'user', 'message': query})
    history.append({'role': 'assistant', 'message': response})
    
    # Prune old turns to manage context window

    if len(history) > 12:    # 12 entries = 6 rounds

        history = history[-12:]
    
    print(f"# Llama3-Chinese：{response}")

```

**History pruning** occurs after every turn, retaining only the last 12 entries (6 complete exchanges) to prevent exceeding the model's **context window**. The `max_new_tokens` parameter caps generation length, while `top_p`, `temperature`, and `repetition_penalty` control output diversity.

## Running the Chat Demos

### Command-Line Interface

Run the minimal CLI demo locally:

```bash
python deploy/python/chat_demo.py

```

The script automatically loads the model, initializes the template from `template_dict['llama3']`, and enters the interactive loop using `build_prompt()` for prompt assembly.

### Streamlit Web Interface

Launch the browser-based UI:

```bash
streamlit run deploy/web_streamlit_for_v1.py /path/to/model --theme.base="dark"

```

The web interface stores messages in `st.session_state` and triggers `combine_history()` for each new user input, streaming responses through `generate_interactive()`.

### Customizing the System Prompt

Modify the assistant's behavior by overriding the default system text:

```python

# In chat_demo.py, after loading the template:

template = template_dict['llama3']
template.system = "You are a bilingual scholar fluent in Chinese and English."

```

In the Streamlit implementation, edit the `system_prompt` variable inside `combine_history()` to inject domain-specific instructions without modifying the core template dictionary.

## Summary

- **Multi-turn conversations** rely on concatenating system instructions, historical exchanges, and current queries into the Llama 3 chat format using special delimiter tokens.
- The repository provides two reference implementations: `combine_history()` in [`deploy/web_streamlit_for_v1.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/deploy/web_streamlit_for_v1.py) for web UIs and `build_prompt()` in [`deploy/python/chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/deploy/python/chat_demo.py) for CLI applications.
- Conversation history is stored as a flat list of role-content dictionaries, with automatic pruning after 6 rounds (12 entries) to manage context window constraints.
- Both approaches use the same underlying template structure but differ in output format—string concatenation versus token ID tensors.

## Frequently Asked Questions

### How does the repository handle multi-turn conversation history?

The repository stores each turn as a dictionary with `role` and `content` (or `message`) keys in a flat list. Before generation, it iterates through this history to build a single prompt string or token ID sequence that includes all previous exchanges, allowing the model to maintain context across interactions.

### What is the default conversation history limit?

By default, the implementations retain the last 6 complete exchanges (12 entries in the history list). When the list exceeds 12 items, older turns are pruned using `history = history[-12:]` to prevent context window overflow while maintaining recent conversational context.

### Can I modify the system prompt for specialized tasks?

Yes. In [`chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/chat_demo.py), access `template_dict['llama3']` and modify the `system` attribute before the chat loop begins. In the Streamlit demo, edit the `system_prompt` format string inside `combine_history()` to inject custom persona instructions or domain knowledge.

### How do the Streamlit and CLI implementations differ?

The Streamlit version in [`web_streamlit_for_v1.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/web_streamlit_for_v1.py) uses `combine_history()` to build string-based prompts suitable for the Gradio-style interface, while the CLI version in [`chat_demo.py`](https://github.com/crazyboym/llama3-chinese-chat/blob/main/chat_demo.py) uses `build_prompt()` to directly produce PyTorch tensors of token IDs. Both use identical history structures and pruning logic but target different deployment environments.