# How to Customize Chat Template Kwargs Per Model in oMLX: Complete Guide

> Learn to customize chat template kwargs per model in oMLX. Control Jinja parameters, merge settings, and enforce policies with forced_ct_kwargs for robust chat management in your Jundot/omlx projects.

- Repository: [Jun Kim/omlx](https://github.com/jundot/omlx)
- Tags: how-to-guide
- Published: 2026-05-11

---

**oMLX allows you to define model-specific Jinja template parameters through `chat_template_kwargs` in `ModelSettings`, merge them with request-level overrides, and protect sensitive keys using `forced_ct_kwargs` to enforce server-side policies.**

oMLX provides a flexible mechanism for customizing how prompts are formatted before being sent to the model. By configuring **chat template kwargs** per model, operators can set default behaviors—such as enabling reasoning modes or setting custom tokens—while still allowing selective client overrides.

## Understanding Chat Template Kwargs Storage

In oMLX, per-model chat template configurations are stored in the `ModelSettings` dataclass defined in [`omlx/model_settings.py`](https://github.com/jundot/omlx/blob/main/omlx/model_settings.py). This class contains two critical fields for template customization:

- **`chat_template_kwargs`**: A dictionary of default key-value pairs passed to the Jinja template when rendering prompts
- **`forced_ct_kwargs`**: A list of keys that cannot be overridden by client requests, ensuring policy compliance

The `ModelSettingsManager` class in the same file persists these settings to a [`model_settings.json`](https://github.com/jundot/omlx/blob/main/model_settings.json) file, making configurations durable across server restarts.

## Configuring Model-Level Chat Template Kwargs

### Programmatic Setup

You can define template kwargs directly in Python before starting the server. This approach is ideal for initial configuration or automated deployments:

```python
from pathlib import Path
from omlx.model_settings import ModelSettings, ModelSettingsManager

# Initialize the settings manager (typically done once at server startup)

mgr = ModelSettingsManager(base_path=Path("/var/omlx"))

# Create settings for a specific model

settings = ModelSettings(
    chat_template_kwargs={
        "enable_thinking": True,
        "reasoning_effort": "high",
        "custom_system_prompt": "You are a helpful assistant."
    },
    forced_ct_kwargs=["reasoning_effort"]  # Clients cannot override this key

)

mgr.set_settings("gemma4", settings)

```

When persisted, these settings are stored in [`model_settings.json`](https://github.com/jundot/omlx/blob/main/model_settings.json) and loaded automatically when the model is requested.

### Via Admin API

For runtime adjustments, oMLX exposes an HTTP endpoint in [`omlx/admin/routes.py`](https://github.com/jundot/omlx/blob/main/omlx/admin/routes.py) that allows you to update template kwargs without restarting the server:

```bash
curl -X PATCH "http://localhost:8000/admin/settings/gemma4" \
     -H "Content-Type: application/json" \
     -d '{
           "chat_template_kwargs": {
               "enable_thinking": false,
               "reasoning_effort": "medium"
           },
           "forced_ct_kwargs": ["reasoning_effort"]
         }'

```

The admin handler updates the in-memory `ModelSettings` object and persists the change to disk immediately.

## Request-Level Overrides and Merge Logic

When a client sends a chat completion request, oMLX merges the model-level defaults with request-specific overrides. The merge logic, implemented in [`omlx/server.py`](https://github.com/jundot/omlx/blob/main/omlx/server.py), follows a strict hierarchy:

1. **Base layer**: Model-level `chat_template_kwargs` from `ModelSettings`
2. **Policy layer**: The `enable_thinking` boolean from model settings (if not None) overrides any kwargs value
3. **Client layer**: Request-level `chat_template_kwargs` are applied, except for keys listed in `forced_ct_kwargs`

```python

# Simplified merge logic from omlx/server.py (lines 2082-2095)

ms = settings_manager.get_settings(model_id)
merged_ct_kwargs = {}
forced_keys = set(ms.forced_ct_kwargs or [])

if ms.chat_template_kwargs:
    merged_ct_kwargs.update(ms.chat_template_kwargs)  # Model defaults

if ms.enable_thinking is not None:
    merged_ct_kwargs["enable_thinking"] = ms.enable_thinking  # Policy override

if request.chat_template_kwargs:
    for k, v in request.chat_template_kwargs.items():
        if k not in forced_keys:
            merged_ct_kwargs[k] = v  # Client override (respecting forced keys)

```

The final `merged_ct_kwargs` dictionary is then passed to the engine in [`omlx/engine/batched.py`](https://github.com/jundot/omlx/blob/main/omlx/engine/batched.py) (and similar files like [`vlm.py`](https://github.com/jundot/omlx/blob/main/vlm.py) and [`dflash.py`](https://github.com/jundot/omlx/blob/main/dflash.py)) when calling `tokenizer.apply_chat_template`.

### Example Request with Overrides

```json
POST /v1/chat/completions
{
  "model": "gemma4",
  "messages": [{ "role": "user", "content": "Explain quantum tunneling." }],
  "chat_template_kwargs": {
    "reasoning_effort": "low",
    "custom_note": "demo"
  }
}

```

**Result**: Because `reasoning_effort` is listed in `forced_ct_kwargs`, the request's `"low"` value is ignored, retaining the model's `"high"` setting. However, `custom_note` is successfully merged into the final kwargs.

## Protecting Sensitive Template Parameters

The **`forced_ct_kwargs`** field serves as a security mechanism to prevent clients from modifying critical template behavior. When a key appears in this list, the server guarantees that only the model-level value from `chat_template_kwargs` (or the `enable_thinking` toggle) will be used, regardless of what the client requests.

This is particularly useful for:
- Enforcing specific reasoning effort levels across all interactions
- Locking system prompts or persona definitions
- Maintaining compliance with usage policies that require specific template formatting

## Using Kwargs in Jinja Templates

To consume these parameters, your model's `chat_template.jinja` file should reference the variables passed through `chat_template_kwargs`:

```jinja
{% if enable_thinking %}
<start_thinking>
{% endif %}

{{ messages[0].content }}

{% if enable_thinking %}
</end_thinking>
{{ reasoning_effort | default('low') }}
{% endif %}

{{ custom_note | default('') }}

```

When the engine renders this template, it receives the merged dictionary containing `enable_thinking`, `reasoning_effort`, and any client-provided overrides, allowing dynamic prompt construction based on per-model configuration.

## Summary

- **Store defaults** in `ModelSettings.chat_template_kwargs` via [`omlx/model_settings.py`](https://github.com/jundot/omlx/blob/main/omlx/model_settings.py) for persistent per-model behavior
- **Force immutability** by adding sensitive keys to `forced_ct_kwargs` to prevent client overrides
- **Merge hierarchy** follows: model defaults → `enable_thinking` policy → filtered request kwargs (excluding forced keys)
- **Access anywhere** in your Jinja templates by referencing the kwargs as variables, enabling dynamic prompt formatting
- **Update dynamically** through the Admin API in [`omlx/admin/routes.py`](https://github.com/jundot/omlx/blob/main/omlx/admin/routes.py) or programmatically via `ModelSettingsManager`

## Frequently Asked Questions

### How do I prevent clients from changing specific chat template parameters?

Add the parameter name to the `forced_ct_kwargs` list in your `ModelSettings`. Any key listed there will be ignored when processing client requests, ensuring the model-level value remains in effect. This is implemented in the merge logic within [`omlx/server.py`](https://github.com/jundot/omlx/blob/main/omlx/server.py) where request kwargs are filtered against the forced keys set.

### Can I set different chat template kwargs for different models on the same server?

Yes. The `ModelSettingsManager` stores a separate `ModelSettings` object for each model ID in [`model_settings.json`](https://github.com/jundot/omlx/blob/main/model_settings.json). When you call `mgr.set_settings("model_name", settings)`, the configuration is scoped specifically to that model identifier, allowing completely independent template configurations per model.

### What happens if `enable_thinking` is set in both `chat_template_kwargs` and as a standalone field?

The standalone `enable_thinking` boolean in `ModelSettings` takes precedence. According to the merge logic in [`omlx/server.py`](https://github.com/jundot/omlx/blob/main/omlx/server.py), after loading the base `chat_template_kwargs`, the code explicitly checks `if ms.enable_thinking is not None:` and overwrites the dictionary value, ensuring the top-level toggle always controls the final parameter sent to the template.

### Where are the merged kwargs actually used in the inference pipeline?

After merging in [`omlx/server.py`](https://github.com/jundot/omlx/blob/main/omlx/server.py), the final dictionary is passed to `engine.count_chat_tokens()` and the generation methods in [`omlx/engine/batched.py`](https://github.com/jundot/omlx/blob/main/omlx/engine/batched.py). The engine then passes these kwargs directly to `tokenizer.apply_chat_template` when formatting messages for the model, making them available as variables within the Jinja template context.