How Hypster's Instantiate Function Executes Configuration Functions

Hypster's instantiate function transforms user-defined configuration functions into concrete configurations by validating function signatures, injecting an HP object to resolve and track parameter access, and enforcing strict post-execution validation with configurable unknown-parameter handling policies.

The instantiate function serves as the primary entry point in gilad-rubin/hypster, converting declarative Python functions into executable configuration objects. By intercepting parameter calls through a specialized HP instance, the system ensures type safety and catches misconfigurations early. This article breaks down the complete execution pipeline based on the actual implementation in the source repository.

Phase 1: Pre-Execution Validation and Setup

Validating the Function Signature

Before executing any configuration logic, instantiate ensures the user function adheres to the expected contract. The helper validate_config_func_signature in src/hypster/utils.py (lines 85-115) inspects the function signature to verify that the first parameter is named hp and optionally typed as HP. This validation prevents runtime errors by catching malformed configuration functions before they enter the execution pipeline.

Initializing the HP Object and Value Mapping

Once validation passes, instantiate prepares the execution context in src/hypster/core.py (lines 52-57). The function accepts a values dictionary containing flat, dotted, or nested parameter overrides, storing it unchanged. It then creates an HP instance initialized with these values and an empty called_params set that will track which parameters the configuration function actually accesses during execution.

Phase 2: Configuration Execution and Parameter Tracking

Executing the User Function

With the HP object prepared, instantiate invokes the configuration function in src/hypster/core.py (lines 58-62). The function receives the HP instance as its first argument, followed by any additional *args or **kwargs passed by the caller. All return values from the user's function are passed directly back to the caller without modification.

Tracking Parameter Access via the HP Instance

During execution, every call to hp.int(), hp.float(), hp.select(), or other parameter methods appends the fully-qualified parameter name to the called_params set within the HP object. This tracking mechanism enables the system to distinguish between parameters that were actually referenced by the configuration logic versus those that were merely provided in the input values dictionary but never consumed.

Phase 3: Post-Execution Validation and Error Handling

Detecting Unknown or Unreachable Parameters

After the configuration function returns, instantiate triggers _handle_unknown_parameters in src/hypster/core.py (lines 73-107). This routine compares the set of parameters that were accessed (hp.called_params) against the keys supplied in the values dictionary. Any parameters present in values but absent from called_params are flagged as unknown or unreachable, preventing silent misconfigurations from propagating downstream.

Handling the on_unknown Policy

The instantiate function accepts an on_unknown parameter that determines how to handle detected mismatches, as implemented in _handle_unknown_parameters:

• "warn" (default): Emits a UserWarning listing the unknown parameters but continues execution.
• "raise": Raises a ValueError with a detailed message including typo suggestions generated by the similarity matching logic in src/hypster/utils.py.
• "ignore": Silently discards the mismatch without alerting the user.

Error Propagation and API Cleanup

Any HPCallError exceptions raised during parameter validation are caught and re-wrapped as standard ValueError instances in src/hypster/core.py (lines 68-71). This transformation provides a cleaner public API by hiding internal error types while preserving the original diagnostic information for debugging.

Practical Implementation Examples

Basic Usage with Default Values

When no values dictionary is provided, instantiate uses the default values specified in the configuration function:

from hypster import instantiate, hp

def model_cfg(hp):
    # Integer hyper-parameter with bounds

    n_estimators = hp.int(100, name="n_estimators", min=10, max=500)
    # Float hyper-parameter

    lr = hp.float(0.01, name="lr", min=0.0, max=1.0)
    return {"n_estimators": n_estimators, "lr": lr}

# No values supplied → defaults are used

cfg = instantiate(model_cfg)

# cfg == {'n_estimators': 100, 'lr': 0.01}

Source: tests/test_basic_parameters.py (lines 18-22)

Overriding Configuration Values

Pass a values dictionary to override specific parameters while keeping others at their defaults:

cfg = instantiate(
    model_cfg,
    values={"n_estimators": 200, "lr": 0.05}
)

# cfg == {'n_estimators': 200, 'lr': 0.05}

Source: tests/test_basic_parameters.py (lines 32-34)

Detecting Unknown Parameters (Default Warning)

By default, instantiate warns when you supply parameters that the configuration function does not access:

cfg = instantiate(
    model_cfg,
    values={"n_estimators": 200, "unknown_param": 42}
)

# Emits a UserWarning:

#   Unknown or unreachable parameters:

#     - 'unknown_param': Unknown parameter

Source: tests/test_error_handling.py (lines 21-24)

Raising on Unknown Parameters with Typo Detection

Set on_unknown="raise" to treat unknown parameters as hard errors. The system suggests similar valid parameter names if it detects potential typos:

cfg = instantiate(
    model_cfg,
    values={"n_estimators": 200, "unkn": 42},
    on_unknown="raise"
)  # → ValueError with suggestion if a typo is detected

Source: tests/test_error_handling.py (lines 49-52)

Nested Configuration Functions

Use hp.nest to compose configurations hierarchically. The called_params tracking works across the entire tree:

def child_cfg(hp):
    x = hp.int(0, name="x")
    return {"x": x}

def parent_cfg(hp):
    # nest the child under the prefix "child"

    hp.nest(child_cfg, name="child")
    # parent can also define its own parameters

    flag = hp.bool(True, name="flag")
    return {"child": {"x": hp.int(5, name="x")}, "flag": flag}
    
cfg = instantiate(parent_cfg, values={"child.x": 15, "flag": False})

# cfg == {'child': {'x': 15}, 'flag': False}

Source: tests/test_nesting.py (lines 20-27)

Multi-Value and Selection Parameters

Handle complex parameter types including lists and categorical selections:

def multi_cfg(hp):
    layers = hp.multi_int([2, 4, 8], name="layers")
    optimizer = hp.select(["sgd", "adam"], name="optimizer", default="adam")
    return {"layers": layers, "optimizer": optimizer}

cfg = instantiate(
    multi_cfg,
    values={"layers": [3, 5, 9], "optimizer": "sgd"}
)

# cfg == {'layers': [3, 5, 9], 'optimizer': 'sgd'}

Source: tests/test_multi_parameters.py (lines 16-22)

Summary

• Signature Validation: instantiate verifies that configuration functions accept an hp parameter before execution, preventing interface mismatches.
• Value Resolution: The HP class in src/hypster/core.py resolves flat, dotted, or nested value dictionaries while tracking every parameter access in the called_params set.
• Safety Mechanisms: Post-execution validation in _handle_unknown_parameters catches unused or misspelled parameters, with configurable policies for warning, raising, or ignoring mismatches.
• Nested Support: The parameter tracking system seamlessly handles nested configurations via hp.nest, ensuring unknown parameter detection works across the entire configuration tree.
• Clean API: Internal HPCallError exceptions are re-wrapped as ValueError to provide a standardized public interface while maintaining detailed diagnostic information.

Frequently Asked Questions

What happens if I pass a parameter that the configuration function doesn't use?

By default, instantiate emits a UserWarning listing the unknown or unreachable parameters while continuing execution. This behavior prevents silent misconfigurations where you might believe a parameter is being applied when it actually has no effect. You can change this behavior using the on_unknown parameter.

How does Hypster suggest corrections for misspelled parameter names?

When on_unknown is set to "raise", the error handling logic in src/hypster/core.py utilizes similarity matching utilities from src/hypster/utils.py to analyze the unknown parameter names against valid options. If a close match is detected, the resulting ValueError includes suggestions like "Did you mean 'n_estimators'?" to help developers quickly identify typos.

Can instantiate handle nested configuration functions?

Yes. The hp.nest method allows you to compose configuration functions hierarchically. When nesting occurs, the same called_params tracking set is shared across parent and child configurations, ensuring that the unknown parameter detection in _handle_unknown_parameters validates the entire configuration tree as a unified namespace.

What types of value dictionaries does instantiate support?

The values parameter accepts dictionaries in three formats: flat (e.g., {"child.x": 15}), dotted (with dot notation for nesting), or nested (e.g., {"child": {"x": 15}}). The HP object normalizes these representations internally, allowing you to override parameters using whichever structure is most convenient for your use case.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}