# How to Properly Format Prompts for Pretrained vs Chat-Finetuned Llama 2 Models

> Learn to format prompts for Llama 2 models. Understand input requirements for pretrained text completion and chat-finetuned message dictionaries to optimize your AI interactions.

- Repository: [Meta Llama/llama](https://github.com/meta-llama/llama)
- Tags: best-practices
- Published: 2026-03-05

---

**Pretrained Llama 2 models accept raw text strings via the `text_completion` method, while chat-finetuned variants require structured message dictionaries with `role` and `content` keys processed through `chat_completion`.**

The `meta-llama/llama` repository implements distinct inference pathways for base and conversational model variants. Understanding how to properly format prompts for pretrained versus chat-finetuned Llama 2 models ensures correct tokenization and generation behavior according to the source implementation.

## Architectural Differences in Prompt Handling

The `Llama` class defined in [`llama/model.py`](https://github.com/meta-llama/llama/blob/main/llama/model.py) exposes two primary generation methods that handle input formatting differently based on the model's training regime.

### Pretrained Base Models

Pretrained Llama 2 checkpoints are causal language models trained on raw text corpora. The `text_completion` method expects a `List[str]` containing raw prompt strings. According to the implementation in [`llama/generation.py`](https://github.com/meta-llama/llama/blob/main/llama/generation.py), these strings are tokenized using the `Tokenizer` class from [`llama/tokenizer.py`](https://github.com/meta-llama/llama/blob/main/llama/tokenizer.py), which automatically appends the **EOS** token before feeding the sequence into the transformer.

### Chat-Finetuned Models

Chat-finetuned Llama 2 models undergo additional fine-tuning on conversational data formatted with special role tokens. The `chat_completion` method requires a `List[Dict[str, str]]` where each dictionary contains:

- `role`: One of `"system"`, `"user"`, or `"assistant"`
- `content`: The message text

As implemented in [`llama/generation.py`](https://github.com/meta-llama/llama/blob/main/llama/generation.py), this method constructs the token sequence by injecting special tokens such as `<|system|>`, `<|user|>`, `<|assistant|>`, and `<|end|>` between messages before generation.

## Code Examples for Each Model Type

### Pretrained Text Completion

When working with base models, pass raw text prompts directly to `text_completion`. This pattern from [`example_text_completion.py`](https://github.com/meta-llama/llama/blob/main/example_text_completion.py) demonstrates the implementation:

```python
from llama import Llama

generator = Llama.build(
    ckpt_dir="path/to/pretrained_ckpt",
    tokenizer_path="path/to/tokenizer.model",
    max_seq_len=128,
    max_batch_size=4,
)

prompts = [
    "I believe the meaning of life is",
    "Simply put, the theory of relativity states that ",
]

results = generator.text_completion(
    prompts,
    max_gen_len=64,
    temperature=0.6,
    top_p=0.9,
)

```

The method returns a list of dictionaries containing the generated text under the `generation` key.

### Chat Completion Format

For conversational models, structure your input as a message list. This example from [`example_chat_completion.py`](https://github.com/meta-llama/llama/blob/main/example_chat_completion.py) shows the proper format:

```python
from llama import Llama

chatbot = Llama.build(
    ckpt_dir="path/to/llama-2-7b-chat",
    tokenizer_path="path/to/tokenizer.model",
    max_seq_len=256,
    max_batch_size=2,
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of France?"},
]

result = chatbot.chat_completion(
    messages,
    max_gen_len=80,
    temperature=0.7,
    top_p=0.95,
)

```

The `chat_completion` method automatically handles conversion from message dictionaries to the underlying token sequence required by chat-finetuned checkpoints.

## Key Implementation Files

Several files in the `meta-llama/llama` repository define the prompt formatting logic:

- [`llama/model.py`](https://github.com/meta-llama/llama/blob/main/llama/model.py): Contains the `Llama` class with `text_completion` and `chat_completion` method signatures
- [`llama/generation.py`](https://github.com/meta-llama/llama/blob/main/llama/generation.py): Implements the generation loops and message-to-token conversion for chat models
- [`llama/tokenizer.py`](https://github.com/meta-llama/llama/blob/main/llama/tokenizer.py): Handles tokenization including special token insertion for chat formats
- [`example_text_completion.py`](https://github.com/meta-llama/llama/blob/main/example_text_completion.py): Demonstrates raw prompt formatting for pretrained models
- [`example_chat_completion.py`](https://github.com/meta-llama/llama/blob/main/example_chat_completion.py): Shows structured message formatting for chat-finetuned models

## Summary

- **Pretrained models** require raw text strings passed to `text_completion`, processed as continuous token streams with automatic EOS handling
- **Chat-finetuned models** require `List[Dict]` inputs with `role` and `content` keys passed to `chat_completion`, which injects special role tokens like `<|system|>` and `<|end|>`
- Both model types use the same `Llama.build` constructor but expect different input formats through their respective generation methods
- The tokenization and special token handling occurs automatically in [`llama/generation.py`](https://github.com/meta-llama/llama/blob/main/llama/generation.py) and [`llama/tokenizer.py`](https://github.com/meta-llama/llama/blob/main/llama/tokenizer.py)

## Frequently Asked Questions

### Can I use raw text prompts with chat-finetuned Llama 2 models?

While technically possible to tokenize raw strings, chat-finetuned models expect the specific role-based token structure provided by `chat_completion`. Using raw text bypasses the special token formatting that conditions the model for conversational behavior, resulting in suboptimal outputs.

### What roles are supported in the chat completion format?

The `chat_completion` method supports three roles as defined in the chat training data: `"system"` for setting behavioral context, `"user"` for human queries, and `"assistant"` for model responses. These roles map to special tokens `<|system|>`, `<|user|>`, and `<|assistant|>` in the underlying implementation within [`llama/generation.py`](https://github.com/meta-llama/llama/blob/main/llama/generation.py).

### How does the tokenizer handle different prompt formats?

The `Tokenizer` class in [`llama/tokenizer.py`](https://github.com/meta-llama/llama/blob/main/llama/tokenizer.py) processes raw text directly for base models, while [`llama/generation.py`](https://github.com/meta-llama/llama/blob/main/llama/generation.py) pre-processes chat messages by concatenating role tokens and content before tokenization. This ensures chat models receive the exact token pattern seen during fine-tuning.

### Do I need to manually add EOS tokens to prompts?

No. The generation infrastructure automatically appends the EOS token to prompts in `text_completion` and handles the `<|end|>` token sequence for chat messages within the `chat_completion` method. Manual addition of these tokens is unnecessary and may cause generation issues.