# How to Configure Recommended Sampling Parameters per Model in Forge

> Easily configure recommended sampling parameters per model in Forge. Learn how to apply official Hugging Face sampling defaults with recommended_sampling=True for optimal results.

- Repository: [Antoine/forge](https://github.com/antoinezambelli/forge)
- Tags: how-to-guide
- Published: 2026-05-22

---

**Forge maintains a centralized dictionary called `MODEL_SAMPLING_DEFAULTS` in [`src/forge/clients/sampling_defaults.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/sampling_defaults.py) that maps model identifiers to their official Hugging Face sampling recommendations, which you can apply by setting `recommended_sampling=True` in any client constructor.**

The `antoinezambelli/forge` repository simplifies LLM inference by encoding model-specific sampling parameters directly into its client libraries. When you configure recommended sampling parameters per model, you ensure your applications run with the exact temperature, top-p, and penalty values specified by model authors, eliminating guesswork and improving output quality.

## Where Sampling Defaults Are Stored

The canonical source of truth lives in **[[`src/forge/clients/sampling_defaults.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/sampling_defaults.py)](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/sampling_defaults.py)** as the **`MODEL_SAMPLING_DEFAULTS`** dictionary. This map contains Hugging Face-recommended values for temperature, top-p, top-k, min-p, repeat-penalty, presence-penalty, and any extra `chat_template_kwargs`.

Two helper functions expose this data:

- **`get_sampling_defaults(model: str)`** — Returns a fresh copy of the parameter dictionary for the supplied model, or an empty dict if the model is unknown. This pure lookup function performs no logging and raises no errors, making it ideal for manual inspection or merging.
- **`apply_sampling_defaults(model: str, *, strict: bool)`** — Implements the policy layer used by client constructors. When `strict=True`, it returns defaults for known models or raises `UnsupportedModelError` (defined in [`src/forge/errors.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/errors.py)) for unknown ones. When `strict=False`, it logs an INFO message once per process-model pair and returns an empty dict.

## How to Configure Recommended Sampling Parameters per Model

All primary model backends—**Ollama**, **Llamafile**, and **Anthropic**—accept a boolean `recommended_sampling` parameter during construction. This parameter defaults to **`False`**, meaning clients use backend-native defaults unless explicitly instructed otherwise.

### Strict Mode: Enforce Official Defaults

Set `recommended_sampling=True` to strictly apply the mapped values. If the model is not present in `MODEL_SAMPLING_DEFAULTS`, the client raises `UnsupportedModelError` immediately, preventing silent fallback to sub-optimal settings.

```python
from forge.clients.ollama import OllamaClient

client = OllamaClient(
    model="qwen3:8b-q4_K_M",
    recommended_sampling=True,  # Strict mode: applies temperature=0.6, top_p=0.95, etc.

)

```

Under the hood, the Ollama client invokes `apply_sampling_defaults(model, strict=True)`, ensuring you receive the exact parameter set recorded from the model's Hugging Face card.

### Opt-Out Mode: Use Backend Defaults

Set `recommended_sampling=False` (or omit the parameter) to bypass the map entirely. The client will proceed with the backend's own defaults, typically vendor-specific values like `temperature=0.7` and `top_p=1.0`.

```python
from forge.clients.llamafile import LlamafileClient

client = LlamafileClient(
    model="gemma-4-31B-it-Q4_K_M",
    recommended_sampling=False,  # Uses Llamafile's native defaults

)

```

## Understanding Model Naming Conventions

Each entry in `MODEL_SAMPLING_DEFAULTS` is keyed by every identity form that a client might receive, allowing vendor-specific overrides without breaking aliases:

- **Ollama-style strings** (e.g., `"qwen3:8b-q4_K_M"`)
- **GGUF stems** for Llamafile binaries (e.g., `"Qwen3-8B-Q4_K_M"`)
- **Llamafile stems** (same as above but referencing `.llamafile` binaries)

All three forms map to independent rows, so you can fine-tune sampling for a specific quantization or serving method without affecting other variants.

## How to Manually Inspect and Customize Defaults

Use **`get_sampling_defaults`** to examine values before construction or to merge recommendations with custom overrides.

### Inspecting Defaults Programmatically

```python
from forge.clients.sampling_defaults import get_sampling_defaults

model = "mistral-nemo:12b-instruct-2407-q4_K_M"
defaults = get_sampling_defaults(model)

print(f"Default params for {model}: {defaults}")

# Output: {'temperature': 0.3}

```

### Overriding Specific Parameters

Retrieve the defaults, modify individual values, and pass the dictionary directly to the client constructor:

```python
from forge.clients.sampling_defaults import get_sampling_defaults
from forge.clients.ollama import OllamaClient

model = "qwen3:8b-q4_K_M"
defaults = get_sampling_defaults(model)
defaults["temperature"] = 0.5  # Custom temperature tweak

client = OllamaClient(
    model=model,
    recommended_sampling=False,  # Disable strict policy to allow injection

    **defaults                   # Unpack modified parameters

)

```

## Adding New Models to the Sampling Map

To extend support for new models, update `MODEL_SAMPLING_DEFAULTS` with provenance-tracked entries:

1. Locate the official model card on Hugging Face.
2. Extract the recommended sampling values (temperature, top-p, etc.).
3. Add a row to `MODEL_SAMPLING_DEFAULTS` with an inline comment pointing to the card URL.
4. Run the test suite to verify the entry is reachable.

```python

# In src/forge/clients/sampling_defaults.py

MODEL_SAMPLING_DEFAULTS.update({
    "mynewmodel:1b-q4_K_M": {
        "temperature": 0.8,
        "top_p": 0.9,
        "top_k": 40,
    },  # https://huggingface.co/username/mynewmodel

})

```

After editing, validate the lookup logic:

```bash
pytest tests/unit/test_sampling_defaults.py::test_get_sampling_defaults

```

**Important:** Never add entries without verifying the exact values against the live model card; the comment URL serves as a permanent provenance record.

## Summary

- **Centralized storage**: All recommended sampling parameters live in [`src/forge/clients/sampling_defaults.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/sampling_defaults.py) within `MODEL_SAMPLING_DEFAULTS`.
- **Strict enforcement**: Set `recommended_sampling=True` in Ollama, Llamafile, or Anthropic clients to apply Hugging Face-sourced defaults and raise `UnsupportedModelError` for unknown models.
- **Flexible lookup**: Use `get_sampling_defaults()` to inspect values manually or merge them with custom settings.
- **Naming coverage**: The map supports Ollama strings, GGUF stems, and Llamafile stems as independent keys.
- **Provenance required**: Add new models with inline URL comments pointing to their Hugging Face cards, then run `pytest` to verify.

## Frequently Asked Questions

### What happens if I enable recommended_sampling for an unsupported model?

If you set `recommended_sampling=True` and the model is not present in `MODEL_SAMPLING_DEFAULTS`, the `apply_sampling_defaults` function raises `UnsupportedModelError` from [`src/forge/errors.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/errors.py). This strict behavior prevents unintentional use of generic backend defaults on models that require specific sampling configurations.

### Can I override specific parameters while keeping the recommended defaults?

Yes. Call `get_sampling_defaults(model)` to retrieve a mutable dictionary of the official values, modify the specific keys you want to change (such as `temperature`), then pass the dictionary to the client constructor using the unpacking operator `**`. You must set `recommended_sampling=False` when using this approach to avoid conflicts with the strict policy layer.

### Where does Forge source its recommended sampling values?

All values originate from the official Hugging Face model cards. Each entry in `MODEL_SAMPLING_DEFAULTS` includes an inline comment linking directly to the source URL. Forge encodes parameters such as temperature, top-p, top-k, min-p, repeat-penalty, and presence-penalty exactly as specified by the model authors.

### Which clients support the recommended_sampling parameter?

The `recommended_sampling` parameter is implemented across all three primary backends: **Ollama** ([`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py)), **Llamafile** ([`src/forge/clients/llamafile.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/llamafile.py)), and **Anthropic** clients. Each imports `apply_sampling_defaults` and invokes it during construction, using the same `strict` semantics to determine whether to enforce the map or fall back to backend defaults.