How to Define Nested Configurations Using Hypster's Pythonic API

Define nested configurations in Hypster by calling hp.nest(child_func, name="identifier") inside a parent configuration function, passing child-specific values via dotted keys like "identifier.param" or nested dictionaries where the explicit nested dict takes precedence over dotted notation.

Hypster, the lightweight configuration framework from gilad-rubin/hypster, enables composition of complex parameter trees through pure Python functions. By leveraging the HP.nest method implemented in src/hypster/core.py, you can embed child configurations within parent scopes while maintaining isolated namespaces and automatic parameter tracking. This approach eliminates YAML boilerplate and lets you define hierarchical machine learning pipelines or application stacks using standard Python control flow.

How HP.nest Orchestrates Nested Configurations

The nesting mechanism centers on the HP.nest method defined in [core.py lines 35‑95](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/core.py#L35). When you invoke hp.nest(child, name="child"), Hypster executes a four-phase orchestration:

1. Namespace validation. The method checks the nested_scopes registry (see [core.py lines 55‑57](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/core.py#L55)) to prevent duplicate prefix definitions and ensure consistent addressing.
2. Value extraction and merging. Hypster collects all values prefixed with "child." from the parent’s inputs. It then uses utils.unflatten_dict ([utils.py lines 34‑45](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/utils.py#L34)) to transform dotted notation into hierarchical dictionaries. If you provide an explicit nested dict under values["child"], utils.merge_nested_dicts ([utils.py lines 47‑82](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/utils.py#L47)) merges the two sources, with the explicit nested dict overriding dotted keys.
3. Child HP instantiation. A fresh HP instance receives the extracted values and an extended namespace_stack (via _get_full_param_path, [core.py lines 58‑62](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/core.py#L58)), ensuring parameters inside the child see fully-qualified paths like child.x.
4. Parameter tracking. After the child function executes, HP.nest merges the child’s called_params back into the parent’s tracking dictionary, converting relative names to absolute paths (e.g., x becomes child.x) as shown in [core.py lines 97‑104](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/core.py#L97).

This architecture guarantees that each nested scope maintains its own parameter namespace while remaining addressable from the parent configuration.

Basic Nesting Pattern

At minimum, a nested configuration requires a child function that accepts an hp: HP argument and a parent that calls hp.nest() with a unique name:

from hypster import HP, instantiate

def child(hp: HP):
    x = hp.int(10, name="x")
    return {"x": x}

def parent(hp: HP):
    child_cfg = hp.nest(child, name="child")
    y = hp.int(20, name="y")
    return {"child": child_cfg, "y": y}

result = instantiate(parent)

# → {'child': {'x': 10}, 'y': 20}

Source: Implementation validated in [tests/test_nesting.py lines 8‑21](https://github.com/gilad-rubin/hypster/blob/master/tests/test_nesting.py#L8).

Passing Arguments to Nested Functions

You can inject additional Python arguments into child configurations using args and kwargs. Hypster validates the child function signature via utils.validate_config_func_signature ([utils.py lines 85‑115](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/utils.py#L85)), ensuring the hp: HP parameter remains first:

def child(hp: HP, multiplier: int, offset: int = 0):
    base = hp.int(5, name="base")
    return base * multiplier + offset

def parent(hp: HP):
    a = hp.nest(child, name="calc1", args=(2,))
    b = hp.nest(child, name="calc2", args=(3,), kwargs={"offset": 10})
    return {"calc1": a, "calc2": b}

result = instantiate(parent)

# → {'calc1': 10, 'calc2': 25}

Source: Example extracted from [tests/test_nesting.py lines 28‑42](https://github.com/gilad-rubin/hypster/blob/master/tests/test_nesting.py#L28).

Conditional Nesting and Control Flow

Because Hypster configurations are pure Python, you can nest different child configurations based on runtime selections:

def model_a(hp: HP):
    return {"type": "A", "param": hp.int(1, name="param")}

def model_b(hp: HP):
    return {"type": "B", "param": hp.float(2.0, name="param")}

def config(hp: HP):
    model_type = hp.select(["a", "b"], name="model_type", default="a")
    if model_type == "a":
        model = hp.nest(model_a, name="model")
    else:
        model = hp.nest(model_b, name="model")
    return {"model": model}

# Default behavior

instantiate(config)  

# → {'model': {'type': 'A', 'param': 1}}

# Override via dotted keys

instantiate(config, values={"model_type": "b", "model.param": 3.0})

# → {'model': {'type': 'B', 'param': 3.0}}

Source: Logic demonstrated in [tests/test_nesting.py lines 44‑68](https://github.com/gilad-rubin/hypster/blob/master/tests/test_nesting.py#L44).

Value Precedence: Dotted Keys vs. Nested Dictionaries

When overriding nested parameters, you can use either dotted notation ("child.x") or an explicit nested dict ({"child": {"x": …}}). According to utils.merge_nested_dicts ([utils.py lines 47‑82](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/utils.py#L47)), the explicit nested dictionary takes precedence over dotted keys when both define the same parameter:

def child(hp: HP):
    x = hp.int(10, name="x")
    y = hp.int(20, name="y")
    return {"x": x, "y": y}

def parent(hp: HP):
    return hp.nest(child, name="child")

result = instantiate(
    parent,
    values={
        "child.x": 100,          # dotted notation

        "child": {"x": 200},     # explicit nested dict

    },
)

# → {'x': 200, 'y': 20}   # nested dict overrides dotted key

Source: Precedence rules tested in [tests/test_nesting.py lines 70‑89](https://github.com/gilad-rubin/hypster/blob/master/tests/test_nesting.py#L70).

Summary

• Use hp.nest(func, name="id") to embed child configurations; the child must accept hp: HP as its first argument.
• Address child parameters via fully-qualified dotted keys ("child.param") or nested dictionaries, with explicit dicts overriding dotted notation.
• Pass additional arguments to child functions using args and kwargs after the name parameter.
• Leverage Python control flow (if/else, loops) to conditionally select which nested configuration to instantiate.
• Track parameter origins through HP.called_params and HP.nested_scopes, which prevent duplicate definitions and prefix collisions.

Frequently Asked Questions

Can I nest configurations multiple levels deep?

Yes. Hypster’s namespace_stack mechanism in HP._get_full_param_path ([core.py lines 58‑62](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/core.py#L58)) supports arbitrary nesting depths. Each level appends its name to the stack, producing paths like grandparent.parent.child.param. The same rules for value extraction and merging apply recursively at each layer.

What happens if I define the same parameter in both dotted notation and a nested dict?

The explicit nested dictionary wins. The utils.merge_nested_dicts function ([utils.py lines 47‑82](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/utils.py#L47)) merges the two input sources, emitting a warning when the same key appears in both forms but consistently giving precedence to the hierarchical structure over the flattened dotted keys.

How does Hypster validate that child functions accept the HP argument?

Before invoking a nested function, Hypster calls utils.validate_config_func_signature ([utils.py lines 85‑115](https://github.com/gilad-rubin/hypster/blob/master/src/hypster/utils.py#L85)) to inspect the function signature. If the first parameter is not named hp or lacks the HP type annotation, Hypster raises a ValueError immediately, preventing runtime failures inside the configuration tree.

Can I pass additional Python arguments to nested configuration functions?

Yes. The HP.nest method accepts optional args (tuple) and kwargs (dict) parameters that are forwarded to the child function after the hp argument. This allows you to inject constants, file paths, or environment-specific flags into reusable configuration components without polluting the parameter space tracked by Hypster.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}