Handling Multi-Turn Conversations with Llama3-Chinese: A Complete Implementation Guide

Handle multi-turn conversations in Llama3-Chinese by concatenating system instructions, historical exchanges, and the current query into the official Llama 3 chat template, then feeding the resulting token IDs to the model's generation loop.

The crazyboym/llama3-chinese-chat repository demonstrates context-aware dialogue through two production-ready implementations. This guide examines the deterministic prompt-building pipeline used to maintain conversation state, manage context windows, and generate coherent responses across multiple turns.

Understanding the Llama3 Chat Template

The repository implements the official Llama 3 chat format, which structures every interaction with special control tokens. The template separates system instructions, user inputs, and assistant responses using explicit delimiters:

<|begin_of_text|><<SYS>>
{system_prompt}
<</SYS>>

<|user|>
{user_message}<|eot_id|>
<|assistant|>
{assistant_message}<|end_of_text|>

This format is registered in deploy/python/chat_demo.py within the template_dict['llama3'] entry. The system prompt establishes the assistant's persona, while historical turns accumulate between the special tokens to maintain context.

Building Multi-Turn Prompts

The repository provides two distinct approaches for assembling prompts from conversation history: a Streamlit web interface and a pure Python CLI.

Streamlit Implementation (combine_history)

In deploy/web_streamlit_for_v1.py, the combine_history() function constructs the full prompt by iterating over session state:

def combine_history(prompt):
    messages = st.session_state.messages
    total_prompt = ''
    for message in messages:
        cur_content = message['content']
        if message['role'] == 'user':
            cur_prompt = user_prompt.format(user=cur_content)
        elif message['role'] == 'robot':
            cur_prompt = robot_prompt.format(robot=cur_content)
        total_prompt += cur_prompt
    system = system_prompt.format(content="你是一个超级人工智能，拥有全人类的智慧汇合……")
    total_prompt = system + total_prompt + cur_query_prompt.format(user=prompt)
    return total_prompt

This function retrieves stored messages from st.session_state.messages, formats each according to its role, prepends a fixed system instruction, and appends the current user query.

CLI Implementation (build_prompt)

The command-line version in deploy/python/chat_demo.py works directly with token IDs rather than strings:

def build_prompt(tokenizer, template, query, history, system=None):
    # system prompt handling

    if system_format is not None and system is not None:
        system_text = system_format.format(content=system)
        input_ids = tokenizer.encode(system_text, add_special_tokens=False)

    # append historical rounds

    for item in history:
        role, message = item['role'], item['message']
        if role == 'user':
            message = user_format.format(content=message, stop_token=tokenizer.eos_token)
        else:
            message = assistant_format.format(content=message, stop_token=tokenizer.eos_token)
        input_ids += tokenizer.encode(message, add_special_tokens=False)

    # current query

    # (the caller already appended the query to `history` before invoking)

    return torch.tensor([input_ids], dtype=torch.long)

Unlike the Streamlit version, build_prompt() returns a PyTorch tensor ready for GPU inference, encoding the system text, historical turns, and current query into a contiguous sequence of token IDs.

Managing Conversation History and Context Windows

Both implementations maintain conversation history as a flat list of dictionaries tracking roles and content. The standard loop pattern appears in chat_demo.py:

# Initialize empty history

history = []

while True:
    query = input('# User：').strip()

    
    # Build prompt including past turns

    input_ids = build_prompt(tokenizer, template, query,
                            copy.deepcopy(history), system=None).to(model.device)
    
    # Generate response

    outputs = model.generate(input_ids,
                             max_new_tokens=512,
                             do_sample=True,
                             top_p=0.9,
                             temperature=0.7,
                             repetition_penalty=1.1,
                             eos_token_id=stop_id)
    
    # Decode and clean response

    response = tokenizer.decode(outputs[0][len(input_ids[0]):])
    response = response.strip().replace(template.stop_word, '')
    
    # Store both sides of the turn

    history.append({'role': 'user', 'message': query})
    history.append({'role': 'assistant', 'message': response})
    
    # Prune old turns to manage context window

    if len(history) > 12:    # 12 entries = 6 rounds

        history = history[-12:]
    
    print(f"# Llama3-Chinese：{response}")

History pruning occurs after every turn, retaining only the last 12 entries (6 complete exchanges) to prevent exceeding the model's context window. The max_new_tokens parameter caps generation length, while top_p, temperature, and repetition_penalty control output diversity.

Running the Chat Demos

Command-Line Interface

Run the minimal CLI demo locally:

python deploy/python/chat_demo.py

The script automatically loads the model, initializes the template from template_dict['llama3'], and enters the interactive loop using build_prompt() for prompt assembly.

Streamlit Web Interface

Launch the browser-based UI:

streamlit run deploy/web_streamlit_for_v1.py /path/to/model --theme.base="dark"

The web interface stores messages in st.session_state and triggers combine_history() for each new user input, streaming responses through generate_interactive().

Customizing the System Prompt

Modify the assistant's behavior by overriding the default system text:

# In chat_demo.py, after loading the template:

template = template_dict['llama3']
template.system = "You are a bilingual scholar fluent in Chinese and English."

In the Streamlit implementation, edit the system_prompt variable inside combine_history() to inject domain-specific instructions without modifying the core template dictionary.

Summary

• Multi-turn conversations rely on concatenating system instructions, historical exchanges, and current queries into the Llama 3 chat format using special delimiter tokens.
• The repository provides two reference implementations: combine_history() in deploy/web_streamlit_for_v1.py for web UIs and build_prompt() in deploy/python/chat_demo.py for CLI applications.
• Conversation history is stored as a flat list of role-content dictionaries, with automatic pruning after 6 rounds (12 entries) to manage context window constraints.
• Both approaches use the same underlying template structure but differ in output format—string concatenation versus token ID tensors.

Frequently Asked Questions

How does the repository handle multi-turn conversation history?

The repository stores each turn as a dictionary with role and content (or message) keys in a flat list. Before generation, it iterates through this history to build a single prompt string or token ID sequence that includes all previous exchanges, allowing the model to maintain context across interactions.

What is the default conversation history limit?

By default, the implementations retain the last 6 complete exchanges (12 entries in the history list). When the list exceeds 12 items, older turns are pruned using history = history[-12:] to prevent context window overflow while maintaining recent conversational context.

Can I modify the system prompt for specialized tasks?

Yes. In chat_demo.py, access template_dict['llama3'] and modify the system attribute before the chat loop begins. In the Streamlit demo, edit the system_prompt format string inside combine_history() to inject custom persona instructions or domain knowledge.

How do the Streamlit and CLI implementations differ?

The Streamlit version in web_streamlit_for_v1.py uses combine_history() to build string-based prompts suitable for the Gradio-style interface, while the CLI version in chat_demo.py uses build_prompt() to directly produce PyTorch tensors of token IDs. Both use identical history structures and pruning logic but target different deployment environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}