# Grammar-Based Output Constraints in llama.cpp: A Complete Guide to Structured JSON Generation

> Learn to generate structured JSON with llama.cpp using grammar-based output constraints. Ensure valid JSON output without post-processing by compiling formal grammars into state machines.

- Repository: [ggml/llama.cpp](https://github.com/ggml-org/llama.cpp)
- Tags: how-to-guide
- Published: 2026-02-27

---

**Grammar-based output constraints in llama.cpp use a GBNF (Generalized Backus-Naur Form) parser to compile formal grammars into deterministic state machines, filtering candidate tokens during sampling to guarantee syntactically valid JSON output without post-processing.**

The llama.cpp repository ships with a built-in **grammar engine** that constrains token generation to user-defined formal languages, eliminating the need for external validation when producing structured outputs. By implementing **grammar-based output constraints**, developers can force large language models to emit well-formed JSON, custom domain-specific languages, or any context-free structure with complete syntactic guarantees. This mechanism compiles GBNF specifications into deterministic automata that the sampler consults on every token candidate, rejecting any token that would violate the grammar.

## How Grammar-Based Constraints Work in llama.cpp

### The GBNF Parser and State Machine

The core grammar engine lives in [`src/llama-grammar.h`](https://github.com/ggml-org/llama.cpp/blob/main/src/llama-grammar.h) and [`src/llama-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/src/llama-grammar.cpp). When you provide a grammar, the `llama_grammar_parser::parse` function tokenizes the GBNF text and builds a map of symbol IDs alongside production rules represented as `llama_grammar_element` structures.

The parser produces a C-style array of rules via `c_rules()` and identifies a root symbol ID. The `llama_grammar_init_impl` function then allocates the runtime `llama_grammar` structure, storing the compiled deterministic automaton that tracks valid parse states throughout generation.

### Sampling Integration

Grammar validation integrates directly into the sampling pipeline through [`common/sampling.h`](https://github.com/ggml-org/llama.cpp/blob/main/common/sampling.h) and [`common/sampling.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/common/sampling.cpp). During each generation step, the sampler constructs a list of candidate token IDs (`common_sampler_candidate`). When `accept_grammar` is set to `true` (the default for completions, disabled for prompt processing as seen in [`tools/completion/completion.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/tools/completion/completion.cpp) lines 713-738), the system calls `llama_grammar_accept_token(grammar, stack, token_id)` for each candidate.

Only tokens that maintain a valid grammar stack state are retained. Because this filtering occurs before softmax sampling, the model physically cannot emit malformed JSON or violate structural constraints, providing guaranteed syntactic correctness without post-hoc validation.

### Lazy Grammar and Triggers

For bounded outputs, llama.cpp supports **lazy grammar mode** via optional trigger conditions defined in [`common/sampling.h`](https://github.com/ggml-org/llama.cpp/blob/main/common/sampling.h) (`common_grammar_trigger` structure). When `grammar_lazy` is enabled, the engine stops sampling immediately upon satisfying any specified trigger rule.

The server API exposes this through `"grammar_lazy"` and `"grammar_triggers"` fields, parsed in [`tools/server/server-task.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/tools/server/server-task.cpp) (lines 316-403) and validated in [`tools/server/server-common.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/tools/server/server-common.cpp) (lines 918-1094). This feature proves essential for generating single JSON objects or specific structural fragments without unnecessary continuation tokens.

## Implementing Grammar-Based Output Constraints: 4 Methods

### Method 1: CLI with GBNF File

Define your grammar using GBNF syntax. For a simple JSON object:

```text
root ::= '{' members '}'
members ::= pair (',' pair)*
pair ::= string ':' value
value ::= string | number | object | array | "true" | "false" | "null"
object ::= '{' members? '}'
array ::= '[' elements? ']'
elements ::= value (',' value)*
string ::= '"' (/[^\"]+/) '"'
number ::= /-?[0-9]+(\.[0-9]+)?([eE][+-]?[0-9]+)?/

```

Save this as `json_object.gbnf`, then run:

```bash
./build/bin/main -m models/7B/ggml-model-q4_0.bin \
    -p "Generate a user profile in JSON:" \
    --grammar json_object.gbnf \
    -n 200

```

The `--grammar` flag instructs the parser in [`src/llama-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/src/llama-grammar.cpp) to compile the file and enforce constraints during sampling.

### Method 2: CLI with JSON Schema

Write a standard JSON Schema:

```json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "name":   { "type": "string" },
    "age":    { "type": "integer" },
    "email":  { "type": "string", "format": "email" }
  },
  "required": ["name", "age"]
}

```

Save as [`user_schema.json`](https://github.com/ggml-org/llama.cpp/blob/main/user_schema.json) and invoke:

```bash
./build/bin/main -m models/7B/ggml-model-q4_0.bin \
    -p "Create a JSON user record:" \
    --json-schema user_schema.json \
    -n 200

```

Internally, the repository calls `json_schema_to_grammar()` (defined in [`common/json-schema-to-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/common/json-schema-to-grammar.cpp) line 321) to produce the equivalent GBNF before feeding it to the sampler.

### Method 3: Server HTTP API

For production deployments using the llama.cpp server, send grammar constraints via JSON payload:

```bash
curl -X POST http://localhost:8080/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "llama-7b",
        "prompt": "Give me a JSON invoice:",
        "max_tokens": 256,
        "grammar": "root ::= '\"' '\"' { members } '\"' '\"' ...",
        "grammar_lazy": true,
        "grammar_triggers": [{ "type": 0, "word": "}" }]
      }'

```

The request body fields `"grammar"`, `"grammar_lazy"` and `"grammar_triggers"` are parsed in [`tools/server/server-task.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/tools/server/server-task.cpp) (lines 316-403) and validated in [`tools/server/server-common.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/tools/server/server-common.cpp) (lines 918-1094) before passing the compiled grammar to the sampling engine.

### Method 4: Python Wrapper

The `llama-cpp-python` binding exposes the same functionality:

```python
import llama_cpp

model = llama_cpp.Llama(model_path="models/7B/ggml-model-q4_0.bin")
grammar = open("json_object.gbnf").read()

output = model(
    prompt="Return a product catalog as JSON:",
    max_tokens=300,
    grammar=grammar,
    grammar_lazy=True,
    grammar_triggers=[{"type": 0, "word": "}"}]
)

print(output["choices"][0]["text"])

```

The Python binding forwards the `grammar` argument to `llama_cpp_set_grammar` (see the wrapper in [`python/llama_cpp/__init__.py`](https://github.com/ggml-org/llama.cpp/blob/main/python/llama_cpp/__init__.py)), which ultimately invokes the same C++ path as the CLI and server.

## Key Source Files and Architecture Reference

| File | Role |
|------|------|
| [`src/llama-grammar.h`](https://github.com/ggml-org/llama.cpp/blob/main/src/llama-grammar.h) / [`src/llama-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/src/llama-grammar.cpp) | Core GBNF parser, grammar compiler, and runtime state machine (`llama_grammar_parser::parse`, `llama_grammar_init_impl`). |
| [`common/json-schema-to-grammar.h`](https://github.com/ggml-org/llama.cpp/blob/main/common/json-schema-to-grammar.h) / [`common/json-schema-to-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/common/json-schema-to-grammar.cpp) | Converts JSON Schema documents to GBNF (`json_schema_to_grammar()` at line 321). |
| [`common/sampling.h`](https://github.com/ggml-org/llama.cpp/blob/main/common/sampling.h) / [`common/sampling.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/common/sampling.cpp) | Integrates grammar validation into the token sampler (`common_sampler_accept`, `llama_grammar_accept_token`). |
| [`tools/server/server-task.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/tools/server/server-task.cpp) | Parses HTTP API grammar fields (lines 316-403). |
| [`tools/server/server-common.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/tools/server/server-common.cpp) | Validates grammar requests (lines 918-1094). |
| [`tests/test-llama-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/tests/test-llama-grammar.cpp) | Unit tests for grammar parsing and stack handling. |
| [`tests/test-json-schema-to-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/tests/test-json-schema-to-grammar.cpp) | Tests for JSON Schema conversion pipeline. |

## Summary

- **Grammar-based output constraints** compile GBNF specifications into deterministic state machines that filter tokens during sampling, guaranteeing syntactically valid JSON.
- The **GBNF parser** ([`src/llama-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/src/llama-grammar.cpp)) and **JSON-Schema converter** ([`common/json-schema-to-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/common/json-schema-to-grammar.cpp)) provide flexible input methods for defining constraints.
- **Integration points** include the CLI (`--grammar`, `--json-schema`), HTTP server (`"grammar"`, `"grammar_lazy"`, `"grammar_triggers"`), and Python bindings (`grammar=` parameter).
- **Lazy grammar mode** with triggers allows early termination when specific structural boundaries are reached, optimizing token usage for bounded outputs.

## Frequently Asked Questions

### What is GBNF and why does llama.cpp use it?

GBNF (Generalized Backus-Naur Form) is a formal notation for describing context-free grammars. llama.cpp uses GBNF because it compiles efficiently into a deterministic state machine that can validate token sequences in real-time during sampling. The parser in [`src/llama-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/src/llama-grammar.cpp) converts GBNF rules into `llama_grammar_element` structures that the sampler queries for every candidate token, ensuring only syntactically valid sequences are emitted.

### Can I use standard JSON Schema directly without converting to GBNF?

Yes. The repository includes a built-in converter in [`common/json-schema-to-grammar.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/common/json-schema-to-grammar.cpp) that translates standard JSON Schema documents into equivalent GBNF automatically. When using the CLI with `--json-schema` or the server API with a schema reference, the system invokes `json_schema_to_grammar()` (line 321) to perform the conversion before compilation, allowing you to work with familiar schema definitions while still benefiting from grammar-constrained sampling.

### How does grammar-based sampling affect generation speed?

Grammar-based sampling adds minimal overhead because the state machine evaluation happens in C++ during the candidate filtering phase. The `llama_grammar_accept_token` function (called from `common_sampler_accept` in [`common/sampling.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/common/sampling.cpp)) performs a fast stack-based validation against pre-compiled rules. While complex grammars with many recursive rules may introduce slight latency, the deterministic filtering typically adds less than 5-10% overhead compared to unconstrained generation, making it suitable for production APIs.

### What happens if the model cannot satisfy the grammar constraints?

If the model reaches a state where no valid continuation tokens exist according to the grammar rules, the sampler will return an empty candidate list, effectively stopping generation. In [`common/sampling.cpp`](https://github.com/ggml-org/llama.cpp/blob/main/common/sampling.cpp), when `llama_grammar_accept_token` rejects all candidates for a given position, the generation halts rather than emitting invalid tokens. For lazy grammar mode with triggers, the system stops early when triggers are satisfied, but if the model fails to reach a valid parse state before hitting token limits, the output may be incomplete though still syntactically valid up to the point of termination.