How to Override Hypster Parameter Values at Runtime Using the `values` Dictionary

You can override any parameter defined in a Hypster configuration function by passing a values dictionary to instantiate(), which supports flat keys, dotted-path notation, and nested structures.

The Hypster configuration framework allows you to inject runtime values without modifying your configuration source code. By leveraging the values parameter in the instantiate function, you can override defaults defined in your configuration functions across namespaces. This article explains the mechanics of runtime parameter overriding based on the gilad-rubin/hypster source code.

The Core Override Mechanism

Runtime parameter overriding centers on three components in the Hypster codebase:

• instantiate in src/hypster/core.py (lines 17‑65): The entry point that receives the values dictionary, creates an HP instance, and executes the configuration function.
• HP._get_value_for_param in src/hypster/hp.py (lines 68‑96): Handles lookup logic for parameter values using exact keys, dotted-path keys, or nested dictionary structures.
• HP.nest in src/hypster/hp.py (lines 35‑90): Enables sub-configurations to be overridden with separate dictionaries that merge automatically with parent values.

When you call instantiate(config_func, values={...}), the supplied dictionary is stored inside the newly created HP object. Every time a hyper-parameter method like hp.int, hp.select, or hp.multi_float is invoked, the HP object executes _get_value_for_param to resolve the final value.

How Parameter Lookup Works

The HP class implements a hierarchical lookup strategy in _get_value_for_param:

1. Exact match: If the requested parameter name (e.g., "batch_size") exists as a top-level key in values, that value is used immediately.
2. Qualified match: For parameters inside namespaces, the method constructs a full dotted path using the current namespace_stack. It then checks for a key matching that path (e.g., "model.lr").
3. Nested dict fallback: The dictionary is processed through unflatten_dict (located in src/hypster/utils.py), allowing nested structures like {"model": {"lr": 0.1}} to satisfy lookups for "model.lr".

If none of these strategies locate a value, the parameter falls back to its declared default.

Practical Override Patterns

Flat Dictionary with Dotted Keys

You can override deeply nested parameters using dot notation without restructuring the configuration:

from hypster import hp, instantiate

def model_cfg(hp: hp.HP):
    batch = hp.int(32, name="batch_size")
    lr = hp.float(0.01, name="lr", min=0.0)
    family = hp.select(["logistic", "rf", "lr"], name="family", default="logistic")
    
    if family == "rf":
        n_estim = hp.int(100, name="n_estimators")
        max_d = hp.float(10.0, name="max_depth")
        return {"family": family, "rf": {"n_estimators": n_estim, "max_depth": max_d}}
    return {"family": family, "lr": {"lr": lr}}

# Override using flat dotted keys

result = instantiate(
    model_cfg,
    values={"batch_size": 64, "family": "rf", "rf.n_estimators": 200}
)

# Result: {'family': 'rf', 'rf': {'n_estimators': 200, 'max_depth': 10.0}}

Nested Dictionary Structures

Alternatively, supply nested dictionaries that mirror your configuration hierarchy:

result2 = instantiate(
    model_cfg,
    values={
        "batch_size": 64, 
        "family": "rf", 
        "rf": {"n_estimators": 250, "max_depth": 12}
    }
)

# Result: {'family': 'rf', 'rf': {'n_estimators': 250, 'max_depth': 12}}

Overriding Sub‑Configurations with hp.nest

When using hp.nest to compose configurations, overrides work across namespace boundaries:

def child_cfg(hp: hp.HP):
    depth = hp.int(5, name="depth")
    return {"depth": depth}

def parent_cfg(hp: hp.HP):
    tree_cfg = hp.nest(child_cfg, name="tree")
    return {"tree": tree_cfg}

# Override child's parameter through parent values

parent_res = instantiate(
    parent_cfg,
    values={"tree": {"depth": 9}}  # Equivalent to {"tree.depth": 9}

)

# Result: {'tree': {'depth': 9}}

Handling Unknown Parameters

By default, Hypster warns about keys in values that never get accessed during configuration execution. You can control this behavior with the on_unknown parameter:

• on_unknown="warn" (default): Emits a warning for unused keys.
• on_unknown="raise": Raises an exception if any key goes unclaimed.
• on_unknown="ignore": Silently ignores unused keys.

def strict_cfg(hp: hp.HP):
    rate = hp.float(0.1, name="rate", min=0.0, max=1.0, strict=True)
    return {"rate": rate}

# This raises because 2.0 exceeds the max bound and is unrecognized

instantiate(strict_cfg, values={"rate": 2.0}, on_unknown="raise")

Complete Working Examples

Below are three self‑contained patterns demonstrating different override strategies.

Example 1: Simple Top‑Level Overrides

from hypster import hp, instantiate

def cfg(hp: hp.HP):
    lr = hp.float(0.01, name="lr", min=0.0)
    bs = hp.int(32, name="batch_size", min=1)
    return {"lr": lr, "batch_size": bs}

params = {"lr": 0.05, "batch_size": 128}
print(instantiate(cfg, values=params))

# → {'lr': 0.05, 'batch_size': 128}

Example 2: Dotted‑Key Overrides for Nested Configs

def child(hp: hp.HP):
    depth = hp.int(3, name="depth")
    return {"depth": depth}

def parent(hp: hp.HP):
    tree_cfg = hp.nest(child, name="tree")
    return {"tree": tree_cfg}

print(instantiate(parent, values={"tree.depth": 7}))

# → {'tree': {'depth': 7}}

Example 3: Strict Validation with Runtime Overrides

def cfg_strict(hp: hp.HP):
    # strict=True forbids values outside declared bounds

    rate = hp.float(0.1, name="rate", min=0.0, max=1.0, strict=True)
    return {"rate": rate}

# This will raise a ValueError for out-of-bounds override

try:
    instantiate(cfg_strict, values={"rate": 2.0})
except ValueError as e:
    print(f"Validation failed: {e}")

Summary

• Pass a values dictionary to instantiate() to override any default parameter at runtime.
• Use flat dotted keys (e.g., "rf.n_estimators") or nested dictionaries (e.g., {"rf": {"n_estimators": 200}}) interchangeably.
• The HP._get_value_for_param method in src/hypster/hp.py handles lookup priority: exact match → dotted path → unflattened nested dict.
• Use hp.nest to create composable sub‑configurations that maintain namespace isolation while remaining overridable.
• Control validation strictness with on_unknown and parameter-specific strict flags.

Frequently Asked Questions

Can I mix dotted keys and nested dictionaries in the same values dictionary?

Yes. Hypster processes the values dictionary uniformly, so {"a.b": 1} and {"a": {"b": 1}} are resolved identically during parameter lookup. You can combine both styles within the same call to instantiate() according to readability preferences.

What happens if I override a parameter that doesn't exist in the configuration?

By default, Hypster emits a warning via on_unknown="warn". If you set on_unknown="raise", the system throws an exception listing the unrecognized keys. Setting on_unknown="ignore" suppresses all feedback for unused keys, which is useful when passing large configuration dictionaries where only a subset of parameters apply.

How do I override parameters in deeply nested sub‑configurations?

Use the hp.nest method to instantiate child configurations within parent namespaces. Then reference the child's parameters using the parent's namespace prefix—either as dotted keys like "tree.submodule.param" or as nested dictionaries under the namespace key. The namespace_stack in the HP class automatically tracks these hierarchical relationships.

Does the strict=True flag affect runtime overrides?

Yes. When a parameter is defined with strict=True, Hypster validates that the overridden value satisfies all constraints (min, max, allowed values) before accepting it. If the runtime value violates these constraints, instantiate() raises a ValueError regardless of whether on_unknown is set to warn or ignore.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}