Creating Reusable Configuration Templates or Components with Hypster

Hypster enables reusable configuration templates by expressing configuration logic as plain Python functions that compose hierarchically via the hp.nest() method, allowing parameter overrides through dotted keys or nested dictionaries.

Creating reusable configuration templates with Hypster allows you to build modular, maintainable configuration systems for machine learning experiments and application stacks. The gilad-rubin/hypster library treats configuration as plain Python functions, making components composable through a simple nesting API that supports parameter overriding and conditional logic.

Understanding the Hypster Architecture

The architecture centers on the instantiate entry point and the HP class, both defined in src/hypster/core.py and src/hypster/hp.py respectively. According to the source code, instantiate validates function signatures, creates an HP instance, executes the configuration function, and validates that all provided parameters were consumed.

The HP Class and nest Method

The HP class in src/hypster/hp.py provides the public API for declaring parameters (int, float, text, bool, select, multi_*) and the critical nest method for composition. When you call hp.nest(child, name="sub"), the system:

1. Builds a prefix namespace (e.g., "sub")
2. Extracts values starting with "sub." or nested under the key "sub" (lines 82‑89)
3. Executes the child function with a scoped HP instance
4. Records accessed parameters with their full path (e.g., "sub.param") in the parent's called_params set (lines 97‑104)
5. Returns the child result as the value for key sub

Validation and Error Handling

Validators such as IntValidator, FloatValidator, and SelectValidator reside in src/hypster/hp_calls.py and raise HPCallError exceptions with full parameter paths (e.g., "model.optimizer.lr"). Utility helpers in src/hypster/utils.py handle signature validation and dictionary flattening operations.

Building Reusable Configuration Components

A reusable component in Hypster is simply a Python function accepting hp: HP and returning a configuration value, typically a dictionary.

Basic Component Structure

Define standalone configuration functions that declare their parameters using the HP API:

# component.py

from hypster import HP

def database_cfg(hp: HP) -> dict:
    host = hp.text("localhost", name="host")
    port = hp.int(5432, name="port")
    return {"host": host, "port": port}

Composing Components with nest

Use hp.nest() to embed child configurations within parent configurations, as implemented in src/hypster/hp.py:

# app_cfg.py

from hypster import HP, instantiate
from component import database_cfg

def app_cfg(hp: HP) -> dict:
    db = hp.nest(database_cfg, name="db")
    debug = hp.bool(False, name="debug")
    return {"db": db, "debug": debug}

Execute the configuration hierarchy:

result = instantiate(app_cfg)

# → {'db': {'host': 'localhost', 'port': 5432}, 'debug': False}

Overriding Nested Values

The nest method supports parameter overrides via dotted notation or nested dictionaries. According to the source code in lines 74‑77 of hp.py, nested dictionary values take precedence over dotted keys:

result = instantiate(
    app_cfg,
    values={
        "db.host": "db.example.com",          # dotted key

        "db": {"port": 6543}                  # nested dict takes precedence

    },
)

# → {'db': {'host': 'db.example.com', 'port': 6543}, 'debug': False}

Passing Arguments to Child Configurations

The nest method accepts args and kwargs to pass static values to child configuration functions:

def multiplier_cfg(hp: HP, factor: int, offset: int = 0) -> int:
    base = hp.int(5, name="base")
    return base * factor + offset

def parent_cfg(hp: HP) -> dict:
    a = hp.nest(multiplier_cfg, name="a", args=(2,))
    b = hp.nest(multiplier_cfg, name="b", args=(3,), kwargs={"offset": 10})
    return {"a": a, "b": b}

instantiate(parent_cfg)

# → {'a': 10, 'b': 25}

Conditional Nesting

Because configuration functions execute as regular Python code, you can implement conditional logic that selects different components at runtime:

def config(hp: HP) -> dict:
    model_type = hp.select(["a", "b"], name="model_type", default="a")
    if model_type == "a":
        model = hp.nest(model_a, name="model")
    else:
        model = hp.nest(model_b, name="model")
    return {"model": model}

Only the selected branch executes, and its parameters become reachable, enabling conditional configuration without validation errors for unused branches.

Summary

• Configuration as code: Hypster expresses configuration logic as plain Python functions in src/hypster/core.py and src/hypster/hp.py, providing type-safe parameter declarations.
• Hierarchical composition: The hp.nest() method enables reusable configuration templates by executing child functions in scoped namespaces and merging parameter tracking back to the parent.
• Flexible overrides: Parameters in nested components can be overridden via dotted keys ("db.host") or nested dictionaries, with nested dict values taking precedence.
• Dynamic behavior: Standard Python control flow (if/else) enables conditional component selection, while args and kwargs support passing static configuration to child templates.

Frequently Asked Questions

How does Hypster handle unknown parameters in nested components?

The instantiate function in src/hypster/core.py tracks all accessed parameters through the HP instance's called_params set. After execution, it validates that every key in the input values dictionary was consumed. If a parameter targets an unused conditional branch, it remains uncalled and triggers a validation error unless the branch is selected.

Can I nest components multiple levels deep?

Yes. The nest method creates a new HP instance with an updated prefix for each level, as shown in lines 82‑89 of src/hypster/hp.py. Each level tracks its full parameter path (e.g., "parent.child.param"), and overrides propagate through the hierarchy using the same dotted-key or nested-dict mechanisms.

What happens when both dotted keys and nested dictionaries define the same parameter?

According to the implementation in src/hypster/hp.py (lines 74‑77), nested dictionary values take precedence over dotted keys when both are present. If you pass {"db.port": 5432, "db": {"port": 6543}}, the nested dict value 6543 wins.

Is it possible to share a component instance across multiple parents?

Yes, because configuration functions are stateless and receive a fresh HP instance during each nest call. You can import and reuse the same function (like database_cfg) across multiple parent configurations, and each parent receives an independent execution context with isolated parameter tracking.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}