How to Use GBNF Grammar Files in llama.cpp for Constrained Generation

llama.cpp supports constrained generation via GBNF (GGML BNF) grammar files using the --grammar-file CLI flag or JSON API field, parsing them into a finite-state automaton that filters token sampling to guarantee syntactically valid outputs.

GBNF grammar files allow you to restrict large language model outputs to specific formats like JSON, code snippets, or custom token sequences. In the ggml-org/llama.cpp repository, these .gbnf files define formal constraints that the sampling engine enforces at runtime according to the specifications in grammars/README.md and the parser implementation in common/peg-parser.cpp.

Understanding the GBNF Format

GBNF extends classic Backus-Naur Form with regex-style quantifiers and direct token matching. The format is lexer-free, operating directly on Unicode code-points rather than requiring a separate tokenization stage.

Key syntax features include:

• Rule definitions: root ::= expression
• Quantifiers: * (zero or more), + (one or more), ? (optional), {m,n} (repetition range)
• Character classes: [a-z], [^"\n]
• Token ID matches: <[123]> forces specific vocabulary tokens by ID
• String literals: "exact match" or case-insensitive variants

The complete syntax reference lives in grammars/README.md in the repository root.

Loading Grammar Files via llama-cli

Pass GBNF constraints to the command-line interface using either --grammar-file for external files or --grammar for inline strings.

Basic JSON Constrained Generation

llama.cpp ships with pre-built grammars in the grammars/ directory. To enforce JSON output:

llama-cli -m model.gguf -n 256 \
    --grammar-file grammars/json.gbnf \
    -p 'Request: schedule a call at 8pm; Command:'

The model will emit only valid JSON objects matching the grammar schema.

Custom Vocabulary Constraints

Create a file emoji.gbnf to restrict output to specific Unicode characters:

root ::= ( "😀" | "🦊" | "🚀" )+

Execute with:

llama-cli -m model.gguf -n 128 \
    --grammar-file emoji.gbnf \
    -p "Give me three happy emojis:"

Token-ID Hard Matching

For advanced use cases, force specific token sequences using vocabulary IDs:

root ::= <[1000]> <[1001]> .*

Assuming token 1000 corresponds to and 1001 to, this grammar forces the model to open with a thinking block before generating content.

Using Grammar Files with llama-server

The HTTP server accepts grammars via the JSON API. Pass either a file path (when server has filesystem access) or the raw GBNF string in the grammar field.

curl -X POST http://localhost:8080/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "model.gguf",
        "prompt": "Write a valid JSON list of colors:",
        "max_tokens": 64,
        "grammar": "root ::= \"[\" ( \"\\\"red\\\"\" | \"\\\"green\\\"\" | \"\\\"blue\\\"\" ) ( \",\" ( \"\\\"red\\\"\" | \"\\\"green\\\"\" | \"\\\"blue\\\"\" ) )* \"]\""
      }'

The server forwards this string to the same compilation pipeline used by the CLI tools defined in common/arg.cpp.

Internal Implementation and Parsing Pipeline

When llama.cpp processes a grammar file, it executes a multi-stage pipeline implemented in common/peg-parser.cpp.

PEG Parsing and GBNF Generation

The input grammar first passes through a PEG (Parsing Expression Grammar) parser. The to_gbnf lambda function (lines 1248–1399 in common/peg-parser.cpp) traverses the PEG abstract syntax tree and converts it into a normalized GBNF string representation.

Reachable Rule Collection

The system optimizes grammar size through collect_reachable_rules. If the --lazy flag is set, only rules reachable from specified trigger_rules are emitted; otherwise, the compiler emits the complete grammar rooted at root_. This lazy evaluation prevents unused rules from bloating the finite-state automaton.

Automaton Compilation

The resulting GBNF string compiles into a finite-state automaton that drives token sampling:

• Terminals become literal token match constraints
• Character classes become per-character validation checks
• Token-IDs (<[123]>) become hard constraints against specific vocabulary indices during sampling

During generation, the sampler only proposes tokens that keep the partially generated output on a valid path through this automaton.

Converting JSON Schema to GBNF

For complex data structures, manually writing grammars is error-prone. The repository includes examples/json_schema_to_grammar.py, which automatically converts JSON Schema definitions into valid GBNF files. This script handles nested objects, arrays, enums, and string patterns, generating optimized grammars that avoid common performance pitfalls.

Performance Considerations and Best Practices

Certain patterns can cause exponential backtracking during parsing. Specifically, chained optional repetitions like x? x? … should be avoided.

Inefficient pattern:

item ::= word? word? word? word?

Efficient alternative:

item ::= word{0,4}

Prefer explicit repetition bounds {0,N} over multiple optional operators. The "Efficient optional repetitions" section in grammars/README.md provides detailed patterns for high-performance grammars.

Debugging Grammar Files

Validate custom grammars against test strings using the test harness in tests/test-gbnf-validator.cpp. This utility verifies that your grammar accepts intended strings and rejects malformed input without requiring a full model inference run.

Summary

• GBNF (GGML BNF) extends standard BNF with regex quantifiers and token-ID matching for lexer-free parsing in llama.cpp
• Use --grammar-file <path> with llama-cli or the grammar JSON field with llama-server to constrain outputs
• The common/peg-parser.cpp module converts PEG syntax to GBNF via the to_gbnf lambda and optimizes via collect_reachable_rules
• Grammar rules compile into finite-state automata that filter token sampling at runtime
• Convert JSON Schemas to GBNF automatically using examples/json_schema_to_grammar.py
• Avoid x? x? chains to prevent exponential backtracking; use {0,N} quantifiers instead

Frequently Asked Questions

What is the difference between --grammar and --grammar-file?

The --grammar flag accepts a raw GBNF string directly on the command line, suitable for short grammars or shell scripts. The --grammar-file flag reads from a .gbnf file path, which is preferable for complex grammars with multiple rules or when reusing the same constraints across different prompts.

Can I use GBNF grammar files with any GGUF model?

Yes, GBNF constraints are applied at the sampling level and are model-agnostic regarding architecture. However, token-ID matches like <[1000]> are vocabulary-specific and will only work correctly if the referenced token IDs exist in the specific model's vocabulary.

How do I debug a grammar that is not generating valid output?

First, verify your grammar syntax using tests/test-gbnf-validator.cpp to ensure it accepts your expected strings. Check that all rules are reachable from the root rule (or your specified trigger rule if using lazy mode). For complex structures, start with the pre-built grammars/json.gbnf and modify incrementally.

Does using a grammar file slow down token generation?

Grammar constraints add minimal overhead to the sampling loop. The finite-state automaton operates in constant time relative to the grammar complexity during generation. However, certain poorly constructed patterns (like chained optional repetitions) can increase memory usage and parsing time during the initial compilation phase.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}