How to Implement Custom Chat Templates for Conversation Mode in llama.cpp

You implement custom chat templates in llama.cpp by supplying a Jinja-formatted string to common_chat_templates_init(), then rendering prompts with common_chat_templates_apply() or common_chat_format_single().

Llama.cpp uses Jinja-style chat templates to convert structured conversation history into the raw text prompt that the model processes. When running in conversation mode (-cnv or --chat-template), the library parses your custom template and applies it to every turn. This guide shows you how to create, initialize, and deploy custom templates using the exact APIs found in the llama.cpp source code.

Understanding Jinja-Style Chat Templates in llama.cpp

A chat template is a Jinja2-compatible string that iterates over a messages array and emits formatted text. The template has access to several variables:

• messages – Array of objects with role and content fields
• bos_token and eos_token – Model-specific boundary tokens
• tools – Optional tool definitions for function-calling models
• add_generation_prompt – Boolean flag to append the assistant prefix

The default built-in template is CHATML_TEMPLATE_SRC, defined in common/chat.cpp around line 889. It formats conversations with <|im_start|> and <|im_end|> tokens.