LLM Temperature and Top_p Parameters in Hiring-Agent: Configuration and Usage

The interviewstreet/hiring-agent repository defines temperature and top_p in prompt.py’s MODEL_PARAMETERS dictionary, injecting these values into provider configs via models.py to control generation randomness and nucleus sampling across Ollama and Gemini models.

The interviewstreet/hiring-agent codebase centralizes LLM generation behavior through deterministic configuration of temperature and top_p parameters. These settings, stored in prompt.py and applied through models.py, govern output creativity and token selection probability for every supported model from gemma3:4b to gemini-2.0-flash.

Configuration Architecture for Temperature and Top_p

The MODEL_PARAMETERS Registry in prompt.py

The source of truth for all model-specific settings resides in prompt.py, where the MODEL_PARAMETERS dictionary maps each supported model to its generation parameters. According to the interviewstreet/hiring-agent source code, this centralized design allows per-model tuning without modifying calling code.

MODEL_PARAMETERS = {
    # Ollama models

    "qwen3:1.7b": {"temperature": 0.0, "top_p": 0.9},
    "gemma3:1b": {"temperature": 0.0, "top_p": 0.9},
    "qwen3:4b": {"temperature": 0.1, "top_p": 0.4},
    "gemma3:4b": {"temperature": 0.1, "top_p": 0.9},
    # Google Gemini models

    "gemini-2.0-flash": {"temperature": 0.1, "top_p": 0.9},
    ...
}

Default Model Initialization

The repository sets gemma3:4b as the DEFAULT_MODEL with temperature=0.1 and top_p=0.9. Environment variables override this selection at runtime, automatically pulling the associated parameters from MODEL_PARAMETERS.

Parameter Injection into LLM Operations

Generation Config Construction in models.py

The models.py file constructs the generation_config dictionary that providers consume. As implemented in interviewstreet/hiring-agent, lines 342-345 conditionally inject parameters:

if "temperature" in options:
    generation_config["temperature"] = options["temperature"]
if "top_p" in options:
    generation_config["top_p"] = options["top_p"]

Provider-Specific Application

Temperature values range from 0.0 (fully deterministic) to 1.0 (maximum creativity). Top_p (nucleus sampling) restricts token selection to the smallest set whose cumulative probability exceeds the threshold, typically set between 0.0 and 1.0. The repository supports both Ollama local models and Google Gemini APIs through this unified configuration layer.

Model-Specific Parameter Reference

The Hiring-Agent repository optimizes each model differently:

• qwen3:1.7b and gemma3:1b: Set to temperature=0.0 and top_p=0.9 for deterministic, focused outputs.
• qwen3:4b: Uses temperature=0.1 with a restrictive top_p=0.4 for balanced creativity.
• gemma3:4b (Default): Configured with temperature=0.1 and top_p=0.9 as a general-purpose setting.
• gemini-2.0-flash: Matches the default gemma settings at 0.1 and 0.9.

Implementing Temperature and Top_p in Code

Retrieving Default Parameters

Access the current model settings by importing from prompt.py:

# Example 1 – Retrieve parameters for the default model

from prompt import MODEL_PARAMETERS, DEFAULT_MODEL

params = MODEL_PARAMETERS.get(DEFAULT_MODEL, {})
print(f"Default model {DEFAULT_MODEL} → temperature={params['temperature']}, top_p={params['top_p']}")

# Output: Default model gemma3:4b → temperature=0.1, top_p=0.9

Passing Parameters to LLM Providers

When calling the provider interface, pass the looked-up options directly:

# Example 2 – Use a custom model with its own settings

from prompt import MODEL_PARAMETERS
from models import ModelProvider, LLMProvider

def generate_text(provider: LLMProvider, model_name: str, messages: list):
    # Look up model‑specific parameters

    opts = MODEL_PARAMETERS.get(model_name, {"temperature": 0.5, "top_p": 0.9})
    # Provider‑agnostic chat call

    response = provider.chat(
        model=model_name,
        messages=messages,
        options=opts,
    )
    return response["content"]

# Assuming `ollama` implements LLMProvider

# generate_text(ollama, "qwen3:4b", [{"role": "user", "content": "Explain recursion"}])

Runtime Configuration Overrides

Switch models via environment variables before importing:

# Example 3 – Overriding defaults at runtime

import os
os.environ["DEFAULT_MODEL"] = "qwen3:4b"   # Switch default model

os.environ["LLM_PROVIDER"] = "ollama"      # Ensure Ollama is used

from prompt import DEFAULT_MODEL, MODEL_PARAMETERS
print(MODEL_PARAMETERS[DEFAULT_MODEL])

# → {'temperature': 0.1, 'top_p': 0.4}

Custom Parameters in Consumer Modules

While most modules like pdf.py and github.py rely on defaults, evaluator.py demonstrates runtime customization by passing temperature=0.5 for specific resume evaluation tasks, overriding the global configuration for that operation.

Summary

• Temperature and top_p are centralized in prompt.py’s MODEL_PARAMETERS dictionary.
• Default configuration uses gemma3:4b with temperature=0.1 and top_p=0.9.
• models.py injects these values into generation_config at lines 342-345.
• Environment variables DEFAULT_MODEL and LLM_PROVIDER control runtime model selection.
• Consumer modules can override defaults by passing custom option dictionaries.

Frequently Asked Questions

What do temperature and top_p control in Hiring-Agent?

Temperature scales the probability distribution of next tokens, where values near 0.0 produce deterministic outputs and 1.0 increases randomness. Top_p implements nucleus sampling, restricting the model to consider only tokens comprising the top p probability mass (e.g., 0.9 includes the most likely tokens until their cumulative probability reaches 90%).

How do I change the default temperature for all operations?

Modify the MODEL_PARAMETERS dictionary in prompt.py for your target model, or override at runtime by passing a custom options dictionary when calling provider.chat() in modules like evaluator.py.

Why does qwen3:4b use top_p=0.4 while other models use 0.9?

The repository configures qwen3:4b with a lower top_p threshold to constrain token diversity for this specific model architecture, producing more focused outputs compared to the broader sampling allowed for gemma3 or Gemini variants.

Are these parameters supported for both Ollama and Gemini providers?

Yes. The models.py abstraction layer translates the temperature and top_p values from MODEL_PARAMETERS into provider-specific API calls, ensuring consistent behavior across local Ollama instances and Google Gemini endpoints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}