How to Integrate Hypster with Optuna for Hyperparameter Optimization

You integrate Hypster with Optuna by passing an Optuna trial object to the suggest_values function, which executes your HP configuration function through a proxy bridge and returns a flat dictionary of suggested hyperparameters ready for instantiation.

The gilad-rubin/hypster repository provides a declarative API for defining hyperparameter spaces using the HP object. When you integrate Hypster with Optuna for hyperparameter optimization, you leverage the optional hypster.hpo.optuna module to connect these configurations to Optuna's sampling algorithms without requiring Optuna as a mandatory production dependency.

How the Hypster-Optuna Integration Works

The integration centers on a proxy pattern that translates Hypster's configuration API into Optuna's suggest_* method calls. This design allows any object implementing Optuna's trial interface to drive the hyperparameter search, making the system both flexible and testable.

The Three Core Components

The integration consists of three specific components located in src/hypster/hpo/optuna.py and re-exported through src/hypster/integrations/optuna.py:

• _HPProxy — This internal class implements Hypster's int, float, and select methods by forwarding calls to the trial object's suggest_int, suggest_float, and suggest_categorical methods. It automatically handles namespace prefixing for nested configurations.

• suggest_values — This helper function runs your configuration function with a trial proxy and returns a flat dictionary mapping fully-qualified parameter names (e.g., "rf.n_estimators") to their suggested values.

• hypster.integrations.optuna — This public module provides a clean import path (from hypster.integrations.optuna import suggest_values) for users who want explicit integration access.

Optional Dependency Design

The src/hypster/hpo/optuna.py file implements a defensive import pattern that attempts import optuna as _optuna within a try/except block. If Optuna is unavailable, _optuna is set to None, but the rest of the integration code functions normally because it only interacts with the trial object interface rather than Optuna directly. This means your production deployments do not require Optuna unless you are actively running a hyperparameter optimization study.

Step-by-Step Integration Guide

Follow these four steps to connect your Hypster configurations to an Optuna study:

1. Define a configuration function using the HP object. Mark parameters as tunable by adding hpo_spec arguments with types like HpoInt, HpoFloat, or HpoCategorical.

2. Create an Optuna trial (or compatible mock). In a live study, Optuna provides this through study.ask() or your objective function parameter. For testing, implement an object with suggest_int, suggest_float, and suggest_categorical methods.

3. Call suggest_values with your trial and configuration function. This executes the configuration through the _HPProxy and returns a dictionary of suggested values.

4. Instantiate the model using hypster.instantiate with the suggested values dictionary. The configuration runs again, but the HP instance reads the supplied values instead of querying the trial.

Complete Code Example

First, define your hyperparameter configuration with HPO specifications:

from hypster import HP
from hypster.hpo.types import HpoInt, HpoFloat, HpoCategorical

def rf_cfg(hp: HP):
    n_estimators = hp.int(
        100,
        name="n_estimators",
        min=50,
        max=300,
        hpo_spec=HpoInt(step=50),  # Forwarded to Optuna's suggest_int

    )
    max_depth = hp.float(
        10.0,
        name="max_depth",
        min=2.0,
        max=30.0,
        hpo_spec=HpoFloat(step=0.5),
    )
    return {"n_estimators": n_estimators, "max_depth": max_depth}

def model_cfg(hp: HP):
    family = hp.select(
        ["rf", "lr"],
        name="family",
        hpo_spec=HpoCategorical(ordered=False),
    )
    if family == "rf":
        return hp.nest(rf_cfg, name="rf")
    # Additional branches omitted for brevity

Then integrate with Optuna in your optimization loop:

from optuna import Trial, create_study
from hypster import instantiate
from hypster.hpo.optuna import suggest_values

def objective(trial: Trial):
    # Get suggested values from Hypster configuration

    values = suggest_values(trial, config=model_cfg)
    
    # Instantiate concrete configuration

    config = instantiate(model_cfg, values=values)
    
    # Train and evaluate your model here

    # score = train_model(config)

    return 0.0  # Return your metric

study = create_study(direction="maximize")
study.optimize(objective, n_trials=100)

Testing Without Optuna

The repository includes a FakeTrial implementation in its test suite (located in tests/test_hpo_optuna_integration.py) that demonstrates how to test your integration without importing Optuna:

class FakeTrial:
    def __init__(self, choices=None):
        self.choices = choices or {}
        self.calls = []

    def suggest_int(self, name, low, high, step=None, log=False):
        self.calls.append({
            "fn": "int", "name": name, "low": low, 
            "high": high, "step": step, "log": log
        })
        return self.choices.get(name, low)

    def suggest_float(self, name, low, high, step=None, log=False):
        self.calls.append({
            "fn": "float", "name": name, "low": low,
            "high": high, "step": step, "log": log
        })
        return self.choices.get(name, low)

    def suggest_categorical(self, name, options):
        self.calls.append({
            "fn": "categorical", "name": name, 
            "options": list(options)
        })
        return self.choices.get(name, options[0])

You can use this pattern to provide fixed choices dictionaries for deterministic testing or to inspect which suggestion methods were called during configuration execution.

Key Implementation Details

Understanding these implementation specifics helps you debug integration issues and optimize your hyperparameter search strategy:

• Flat dictionary output — The suggest_values function (lines 39-51 in src/hypster/hpo/optuna.py) returns a flat mapping where nested parameters use dot notation (e.g., "rf.n_estimators"). This format matches what instantiate expects via its values parameter.

• Namespace handling — When using hp.nest(), the _HPProxy._full method automatically prepends the parent name to create unique parameter identifiers across branches, preventing name collisions in complex configuration trees.

• HPO spec forwarding — The proxy extracts step and scale parameters from HpoInt and HpoFloat objects and passes them directly to trial.suggest_int() and trial.suggest_float(), ensuring Optuna respects your specified sampling constraints.

• Override support via nest — You can pass a values dictionary to hp.nest() calls; these values take precedence over trial suggestions, allowing you to fix certain parameters while optimizing others.

• Source file locations — The core integration logic resides in src/hypster/hpo/optuna.py, the public API is exposed through src/hypster/integrations/optuna.py, and the main HP API including nest handling is defined in src/hypster/hp.py.

Summary

• Integration mechanism — The _HPProxy class bridges Hypster's HP API with Optuna's trial methods, forwarding int(), float(), and select() calls to suggest_int, suggest_float, and suggest_categorical.

• Primary entry point — Use from hypster.hpo.optuna import suggest_values to generate suggested hyperparameters from an Optuna trial and your configuration function.

• Optional dependency — Optuna is not required at runtime unless executing a study; the integration uses duck typing to accept any trial-like object.

• Output format — suggest_values returns a flat dictionary compatible with hypster.instantiate, using dot notation for nested parameters (e.g., "rf.max_depth").

• Testing capability — Implement a minimal FakeTrial class with the three suggest methods to unit test your configurations without installing Optuna.

Frequently Asked Questions

Do I need to install Optuna to use Hypster?

No. The Optuna integration is optional. The src/hypster/hpo/optuna.py module attempts to import Optuna but falls back to None if unavailable. You only need Optuna installed when you want to execute an actual hyperparameter optimization study using suggest_values with a real Optuna trial object.

How does suggest_values handle nested configurations?

The function returns a flat dictionary where keys use dot notation to represent nesting (e.g., "rf.n_estimators"). When you later call instantiate(model_cfg, values=values), the HP object uses this flat mapping to reconstruct the nested structure. The _HPProxy handles namespace prefixing automatically during the suggestion phase.

Can I fix some hyperparameters while optimizing others?

Yes. Pass a values dictionary to hp.nest() with fixed values for specific parameters. These overrides take precedence over trial suggestions, allowing you to freeze certain hyperparameters while still exploring the search space for others. This is useful for ablation studies or conditional optimization.

What HPO types are supported?

Hypster supports HpoInt (with step and log parameters), HpoFloat (with step, log, and scale options), and HpoCategorical (with ordered flags) through the hpo_spec argument in HP methods. These specifications are forwarded to Optuna's corresponding suggest_* methods via the proxy implementation in src/hypster/hpo/optuna.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}