How to Serialize and Save Hypster Configurations for Later Use

To serialize Hypster configurations, extract the values dictionary from the HP object, convert it to a flat format using flatten_dict from hypster.utils, and persist it to JSON or YAML for later reconstruction with unflatten_dict and instantiate().

To serialize and save Hypster configurations for later use in the gilad-rubin/hypster library, you leverage the fact that all parameter values are stored as plain Python dictionaries within the HP class. While Hypster does not expose a dedicated save() API, the conversion utilities in src/hypster/utils.py enable you to transform nested configuration dictionaries into flat, dotted-key representations that serialize cleanly to standard formats like JSON.

Understanding Configuration Storage in Hypster

Configurations in Hypster are defined via functions that accept an HP object and use methods like hp.int_(), hp.float_(), and hp.select() to declare parameters. When you call instantiate() from src/hypster/core.py, Hypster populates an HP instance—defined in src/hypster/hp.py—which stores the final resolved values in a standard Python dictionary accessible via the hp.values attribute. Because this data consists only of primitive types (strings, numbers, booleans, lists, and dictionaries), you can persist it using any standard serialization library once you handle the key structure correctly.

Serializing Configurations to JSON

The flatten_dict utility converts nested dictionaries (e.g., {"model": {"lr": 0.01}}) into flat, dotted-key dictionaries (e.g., {"model.lr": 0.01}). This format eliminates ambiguity when saving to JSON and ensures that Hypster's key-resolution logic works correctly when you reload the configuration.

Step 1: Instantiate and Capture Values

First, run your configuration with the desired overrides to establish the values you want to save:

from hypster import HP, instantiate
from hypster.utils import flatten_dict
import json

def my_config(hp: HP):
    hp.int_(default=10, name="model.n_estimators")
    hp.float_(default=0.1, name="model.learning_rate")
    hp.select(options=["rf", "lr"], default="rf", name="family")
    return {"model": "placeholder"}

# Run with specific overrides

values = {"family": "rf", "model.n_estimators": 200, "model.learning_rate": 0.05}
result = instantiate(my_config, values=values)

# Recreate HP to capture the resolved values (including defaults applied)

hp = HP(values)
filled_values = hp.values

Step 2: Flatten and Save

Convert the nested dictionary to a flat structure and write it to disk:

# Convert to dotted-key format

flat_values = flatten_dict(filled_values)

# Serialize to JSON

with open("my_config.json", "w", encoding="utf-8") as f:
    json.dump(flat_values, f, indent=2)

Loading and Reusing Saved Configurations

To restore a configuration, read the JSON file, optionally convert the flat dictionary back to a nested structure using unflatten_dict, and pass the result to instantiate():

from hypster import instantiate
from hypster.utils import unflatten_dict
import json

# Load the flat dictionary

with open("my_config.json", "r", encoding="utf-8") as f:
    loaded_flat = json.load(f)

# Convert back to nested (optional - instantiate accepts both formats)

loaded_nested = unflatten_dict(loaded_flat)

# Re-instantiate the configuration

result = instantiate(my_config, values=loaded_nested)
print("Re-instantiated result:", result)

Alternative Serialization Formats

Because flatten_dict returns a dictionary of primitive Python types, you can use any serialization format. Here is an example using YAML:

import yaml
from hypster.utils import flatten_dict, unflatten_dict

# Save as YAML

with open("my_config.yaml", "w", encoding="utf-8") as f:
    yaml.safe_dump(flatten_dict(values), f)

# Load from YAML

with open("my_config.yaml", "r", encoding="utf-8") as f:
    yaml_data = yaml.safe_load(f)

# Restore and instantiate

result = instantiate(my_config, values=unflatten_dict(yaml_data))

Capturing Final Resolved Values After Defaults

If you need to capture the exact dictionary that Hypster resolved—including default values applied to unspecified parameters—you can invoke the configuration function directly to populate the HP instance:

from hypster import HP

# Build HP with your overrides

hp = HP(values={"family": "rf", "model.n_estimators": 200})

# Execute the config function to populate hp.called_params and apply defaults

my_config(hp)

# hp.values now contains the final resolved state

final_values = hp.values

# Serialize as before

flat = flatten_dict(final_values)
with open("resolved_config.json", "w") as f:
    json.dump(flat, f, indent=2)

This approach ensures you persist the complete configuration state, not just the overrides you initially provided.

Summary

• No dedicated save API exists in Hypster; serialization relies on standard Python dictionaries and external libraries.
• flatten_dict and unflatten_dict in src/hypster/utils.py convert between nested and dotted-key formats, ensuring compatibility with JSON.
• hp.values in src/hypster/hp.py stores all configuration parameters as primitive Python types ready for serialization.
• Flat dictionaries are the recommended persistence format because they eliminate ambiguity in key resolution during reload.
• Any serialization format works (JSON, YAML, Pickle) because the underlying data is pure Python primitives.

Frequently Asked Questions

Can I use pickle instead of JSON to save Hypster configurations?

Yes, because hp.values contains only primitive Python types (integers, floats, strings, booleans, lists, and dictionaries), you can use pickle.dump() and pickle.load() directly on the flattened or nested dictionary. However, JSON or YAML are preferred for human-readable, version-control-friendly storage.

Should I save the flat dictionary or the nested one?

Save the flat dictionary produced by flatten_dict. The flat, dotted-key format (e.g., "model.lr": 0.01) ensures Hypster's key-resolution logic in src/hypster/hp.py handles overrides unambiguously when you reload. You can always reconstruct the nested structure later with unflatten_dict if your application requires it.

How do I ensure reproducibility when sharing configurations across environments?

To guarantee identical behavior, save the output of flatten_dict(hp.values) where hp is an HP instance that has executed your configuration function. This captures all default values that were applied, creating a complete snapshot independent of the code that generated it. Load this file with unflatten_dict before calling instantiate().

What happens if I add new parameters to my config after saving?

If you load a saved configuration that lacks newly added parameters, Hypster will apply the new parameters' default values as defined in the configuration function. The instantiate() function in src/hypster/core.py merges saved overrides with current defaults, ensuring backward compatibility while allowing configuration evolution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}