# How Hypster's `namespace_stack` Manages Nested Configurations for Hierarchical Hyperparameters

> Discover how Hypster's namespace_stack simplifies nested configurations. Automatically manage hierarchical hyperparameters with dot-notation paths effortlessly.

- Repository: [Gilad Rubin/hypster](https://github.com/gilad-rubin/hypster)
- Tags: internals
- Published: 2026-03-02

---

**The `namespace_stack` is an internal list maintained by Hypster's `HP` class that automatically tracks nesting depth and constructs dot-notation parameter paths like `model.layer.hidden_units` without requiring manual string concatenation.**

Hypster is an open-source Python library that enables hierarchical hyperparameter definitions through nested configuration functions. At the heart of this capability lies the `namespace_stack`, a data structure that maintains the current path context and ensures parameters receive fully qualified names as configuration functions call other configuration functions.

## What Is the `namespace_stack` in Hypster?

The `namespace_stack` is a **list of string prefixes** maintained by the `HP` class that represents the current nesting depth within a configuration hierarchy. As users invoke the `nest` method to embed child configuration functions, the stack accumulates namespace identifiers, enabling the library to construct dot-notation paths such as `layer.hidden_units` or `model.optimizer.learning_rate`.

This mechanism allows developers to write flat parameter definitions (`hp._int(name="hidden_units")`) inside nested functions while the library automatically resolves the appropriate fully-qualified name based on the current stack context.

## How `namespace_stack` Works Under the Hood

### Stack Initialization in the `HP` Class

Each `HP` instance initializes with an empty `namespace_stack` in the `__init__` method defined in [`src/hypster/hp.py`](https://github.com/gilad-rubin/hypster/blob/main/src/hypster/hp.py):

```python
self.namespace_stack: List[str] = []  # For nested name prefixes

```

This empty list indicates that the configuration starts at the root level with no namespace prefixes applied.

### Building Full Parameter Paths with `_get_full_param_path`

Every public helper method (`_int`, `_float`, `_select`, etc.) calls the internal `_get_full_param_path` method to resolve the complete dot-notation path. Located in [`src/hypster/hp.py`](https://github.com/gilad-rubin/hypster/blob/main/src/hypster/hp.py) at lines 58-63, this method concatenates the current stack with the parameter name:

```python
def _get_full_param_path(self, name: str) -> str:
    if self.namespace_stack:
        return ".".join(self.namespace_stack + [name])
    return name

```

When the stack contains `["model", "layer"]` and the method receives `name="hidden_units"`, it returns `"model.layer.hidden_units"`.

### Entering Nested Scopes via the `nest` Method

The `nest` method in [`src/hypster/hp.py`](https://github.com/gilad-rubin/hypster/blob/main/src/hypster/hp.py) (lines 82-86) creates child `HP` instances that inherit and extend the parent's `namespace_stack`:

```python
nested_hp = HP(nested_values)
nested_hp.namespace_stack = self.namespace_stack + [name]
nested_hp.called_params = self.called_params   # share tracking

```

This inheritance ensures that parameters defined within the nested function automatically receive the parent's path prefix plus the new namespace identifier.

### Parameter Propagation and Collision Detection

After nested execution completes, the `nest` method merges the child's `called_params` back into the parent, prefixing relative names with the full path (lines 96-104 in [`src/hypster/hp.py`](https://github.com/gilad-rubin/hypster/blob/main/src/hypster/hp.py)):

```python
for param in nested_params:
    if "." in param:
        self.called_params.add(param)
    else:
        full_nested_param = f"{full_path}.{param}"
        self.called_params.add(full_nested_param)

```

To prevent naming conflicts, the method validates that the proposed namespace prefix does not collide with existing parameters (lines 54-60):

```python
if name not in self.nested_scopes:
    for existing in self.called_params:
        if existing.startswith(full_path + ".") or existing == full_path:
            raise HPCallError(full_path, f"prefix '{name}' reserved")

```

This guard ensures that a namespace like `layer` cannot be used if a parameter `layer.units` already exists at the parent level.

## Practical Example: Nested Configuration Functions

Consider a neural network configuration where a top-level `model_cfg` nests a `layer_cfg` to organize hyperparameters hierarchically:

```python
from hypster import HP, instantiate

def model_cfg(hp: HP):
    # Top-level parameter → "learning_rate"

    hp._float(default=0.01, name="learning_rate")
    
    # Nested block for a sub-module

    hp.nest(layer_cfg, name="layer")

def layer_cfg(hp: HP):
    # Inside the "layer" namespace → "layer.hidden_units"

    hp._int(default=128, name="hidden_units")
    hp._select(options=[64, 128, 256], name="activation")

```

When instantiating with `instantiate(model_cfg, {"layer.hidden_units": 256})`, the `namespace_stack` ensures that:

- `hp._float` sees the name `learning_rate` and stores it as `"learning_rate"` (empty stack).
- Inside `layer_cfg`, `_get_full_param_path("hidden_units")` returns `"layer.hidden_units"` because `namespace_stack == ["layer"]`.

This automatic resolution eliminates manual string concatenation and prevents namespace pollution across nested configuration blocks.

## Summary

- The **`namespace_stack`** is a list maintained by the `HP` class that tracks the current nesting depth of configuration functions.
- It enables **automatic dot-notation path construction** (e.g., `model.layer.hidden_units`) via the `_get_full_param_path` method in [`src/hypster/hp.py`](https://github.com/gilad-rubin/hypster/blob/main/src/hypster/hp.py).
- The **`nest` method** creates child `HP` instances that inherit and extend the parent's stack, ensuring parameters receive fully qualified names.
- **Collision detection** prevents namespace conflicts by validating that new prefixes do not overlap with existing parameter paths.
- Users write flat parameter definitions while Hypster handles hierarchical name resolution internally.

## Frequently Asked Questions

### What happens when `namespace_stack` is empty?

When the `namespace_stack` is empty, parameters are registered at the **root level** with their bare names. The `_get_full_param_path` method in [`src/hypster/hp.py`](https://github.com/gilad-rubin/hypster/blob/main/src/hypster/hp.py) returns the parameter name unchanged when the stack contains no prefixes, effectively treating the current configuration function as the top-level namespace.

### How does Hypster prevent naming conflicts between nested configurations?

Hypster implements **collision detection** in the `nest` method (lines 54-60 of [`src/hypster/hp.py`](https://github.com/gilad-rubin/hypster/blob/main/src/hypster/hp.py)). Before creating a nested scope, it checks whether the proposed namespace prefix conflicts with existing parameters. If a parameter already exists that starts with the proposed path or matches it exactly, the library raises an `HPCallError` to prevent ambiguous or overlapping namespaces.

### Can I manually modify the `namespace_stack`?

While the `namespace_stack` is technically accessible as a public attribute of the `HP` class, **manual modification is not recommended**. The stack is designed to be managed automatically through the `nest` method, which handles proper inheritance, collision detection, and parameter propagation. Manual changes could break the internal path resolution logic and cause `HPCallError` exceptions or incorrect parameter naming.

### Where is the `namespace_stack` logic implemented in the source code?

The core `namespace_stack` logic resides in **[`src/hypster/hp.py`](https://github.com/gilad-rubin/hypster/blob/main/src/hypster/hp.py)**. Key implementations include:
- **Initialization** (lines 50-55): Empty list creation in `__init__`
- **Path building** (lines 58-63): `_get_full_param_path` method
- **Nesting** (lines 82-104): `nest` method handling stack inheritance and collision detection

Supporting utilities for dictionary flattening (used in nested value lookup) are located in [`src/hypster/utils.py`](https://github.com/gilad-rubin/hypster/blob/main/src/hypster/utils.py).