How to Configure Recommended Sampling Parameters per Model in Forge

Forge maintains a centralized dictionary called MODEL_SAMPLING_DEFAULTS in src/forge/clients/sampling_defaults.py that maps model identifiers to their official Hugging Face sampling recommendations, which you can apply by setting recommended_sampling=True in any client constructor.

The antoinezambelli/forge repository simplifies LLM inference by encoding model-specific sampling parameters directly into its client libraries. When you configure recommended sampling parameters per model, you ensure your applications run with the exact temperature, top-p, and penalty values specified by model authors, eliminating guesswork and improving output quality.

Where Sampling Defaults Are Stored

The canonical source of truth lives in [src/forge/clients/sampling_defaults.py](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/sampling_defaults.py) as the MODEL_SAMPLING_DEFAULTS dictionary. This map contains Hugging Face-recommended values for temperature, top-p, top-k, min-p, repeat-penalty, presence-penalty, and any extra chat_template_kwargs.

Two helper functions expose this data:

• get_sampling_defaults(model: str) — Returns a fresh copy of the parameter dictionary for the supplied model, or an empty dict if the model is unknown. This pure lookup function performs no logging and raises no errors, making it ideal for manual inspection or merging.
• apply_sampling_defaults(model: str, *, strict: bool) — Implements the policy layer used by client constructors. When strict=True, it returns defaults for known models or raises UnsupportedModelError (defined in src/forge/errors.py) for unknown ones. When strict=False, it logs an INFO message once per process-model pair and returns an empty dict.

How to Configure Recommended Sampling Parameters per Model

All primary model backends—Ollama, Llamafile, and Anthropic—accept a boolean recommended_sampling parameter during construction. This parameter defaults to False, meaning clients use backend-native defaults unless explicitly instructed otherwise.

Strict Mode: Enforce Official Defaults

Set recommended_sampling=True to strictly apply the mapped values. If the model is not present in MODEL_SAMPLING_DEFAULTS, the client raises UnsupportedModelError immediately, preventing silent fallback to sub-optimal settings.

from forge.clients.ollama import OllamaClient

client = OllamaClient(
    model="qwen3:8b-q4_K_M",
    recommended_sampling=True,  # Strict mode: applies temperature=0.6, top_p=0.95, etc.

)

Under the hood, the Ollama client invokes apply_sampling_defaults(model, strict=True), ensuring you receive the exact parameter set recorded from the model's Hugging Face card.

Opt-Out Mode: Use Backend Defaults

Set recommended_sampling=False (or omit the parameter) to bypass the map entirely. The client will proceed with the backend's own defaults, typically vendor-specific values like temperature=0.7 and top_p=1.0.

from forge.clients.llamafile import LlamafileClient

client = LlamafileClient(
    model="gemma-4-31B-it-Q4_K_M",
    recommended_sampling=False,  # Uses Llamafile's native defaults

)

Understanding Model Naming Conventions

Each entry in MODEL_SAMPLING_DEFAULTS is keyed by every identity form that a client might receive, allowing vendor-specific overrides without breaking aliases:

• Ollama-style strings (e.g., "qwen3:8b-q4_K_M")
• GGUF stems for Llamafile binaries (e.g., "Qwen3-8B-Q4_K_M")
• Llamafile stems (same as above but referencing .llamafile binaries)

All three forms map to independent rows, so you can fine-tune sampling for a specific quantization or serving method without affecting other variants.

How to Manually Inspect and Customize Defaults

Use get_sampling_defaults to examine values before construction or to merge recommendations with custom overrides.

Inspecting Defaults Programmatically

from forge.clients.sampling_defaults import get_sampling_defaults

model = "mistral-nemo:12b-instruct-2407-q4_K_M"
defaults = get_sampling_defaults(model)

print(f"Default params for {model}: {defaults}")

# Output: {'temperature': 0.3}

Overriding Specific Parameters

Retrieve the defaults, modify individual values, and pass the dictionary directly to the client constructor:

from forge.clients.sampling_defaults import get_sampling_defaults
from forge.clients.ollama import OllamaClient

model = "qwen3:8b-q4_K_M"
defaults = get_sampling_defaults(model)
defaults["temperature"] = 0.5  # Custom temperature tweak

client = OllamaClient(
    model=model,
    recommended_sampling=False,  # Disable strict policy to allow injection

    **defaults                   # Unpack modified parameters

)

Adding New Models to the Sampling Map

To extend support for new models, update MODEL_SAMPLING_DEFAULTS with provenance-tracked entries:

1. Locate the official model card on Hugging Face.
2. Extract the recommended sampling values (temperature, top-p, etc.).
3. Add a row to MODEL_SAMPLING_DEFAULTS with an inline comment pointing to the card URL.
4. Run the test suite to verify the entry is reachable.

# In src/forge/clients/sampling_defaults.py

MODEL_SAMPLING_DEFAULTS.update({
    "mynewmodel:1b-q4_K_M": {
        "temperature": 0.8,
        "top_p": 0.9,
        "top_k": 40,
    },  # https://huggingface.co/username/mynewmodel

})

After editing, validate the lookup logic:

pytest tests/unit/test_sampling_defaults.py::test_get_sampling_defaults

Important: Never add entries without verifying the exact values against the live model card; the comment URL serves as a permanent provenance record.

Summary

• Centralized storage: All recommended sampling parameters live in src/forge/clients/sampling_defaults.py within MODEL_SAMPLING_DEFAULTS.
• Strict enforcement: Set recommended_sampling=True in Ollama, Llamafile, or Anthropic clients to apply Hugging Face-sourced defaults and raise UnsupportedModelError for unknown models.
• Flexible lookup: Use get_sampling_defaults() to inspect values manually or merge them with custom settings.
• Naming coverage: The map supports Ollama strings, GGUF stems, and Llamafile stems as independent keys.
• Provenance required: Add new models with inline URL comments pointing to their Hugging Face cards, then run pytest to verify.

Frequently Asked Questions

What happens if I enable recommended_sampling for an unsupported model?

If you set recommended_sampling=True and the model is not present in MODEL_SAMPLING_DEFAULTS, the apply_sampling_defaults function raises UnsupportedModelError from src/forge/errors.py. This strict behavior prevents unintentional use of generic backend defaults on models that require specific sampling configurations.

Can I override specific parameters while keeping the recommended defaults?

Yes. Call get_sampling_defaults(model) to retrieve a mutable dictionary of the official values, modify the specific keys you want to change (such as temperature), then pass the dictionary to the client constructor using the unpacking operator **. You must set recommended_sampling=False when using this approach to avoid conflicts with the strict policy layer.

Where does Forge source its recommended sampling values?

All values originate from the official Hugging Face model cards. Each entry in MODEL_SAMPLING_DEFAULTS includes an inline comment linking directly to the source URL. Forge encodes parameters such as temperature, top-p, top-k, min-p, repeat-penalty, and presence-penalty exactly as specified by the model authors.

Which clients support the recommended_sampling parameter?

The recommended_sampling parameter is implemented across all three primary backends: Ollama (src/forge/clients/ollama.py), Llamafile (src/forge/clients/llamafile.py), and Anthropic clients. Each imports apply_sampling_defaults and invokes it during construction, using the same strict semantics to determine whether to enforce the map or fall back to backend defaults.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}