How to Use `HP.collect()` in Hypster to Gather Local Variables from Configurations

HP.collect() is a helper method in the Hypster library that automatically filters and returns local variables from configuration functions as a clean dictionary, eliminating manual dict construction while safely excluding internal objects and private names.

The HP.collect() method simplifies how you return values from Hypster configuration functions. Located in src/hypster/hp.py (lines 408–435) within the gilad-rubin/hypster repository, this utility captures locals() from your config function, applies intelligent filtering, and returns only the variables you actually want to expose.

How HP.collect() Works Under the Hood

When you call hp.collect(locals()) inside a configuration function, the method performs four distinct filtering operations:

• Default internal exclusions – Automatically removes hp, self, __builtins__, __name__, and __doc__ to prevent internal objects from leaking into your results.
• Private name protection – Strips any variable beginning with an underscore (_), ensuring temporary or private variables stay internal.
• Whitelist support – The optional include parameter accepts a list of names to exclusively retain, ignoring all other locals.
• Blacklist support – The optional exclude parameter lets you specify additional names to remove beyond the defaults.

The implementation returns a new dictionary containing only the filtered values, leaving the original locals() dictionary untouched.

Basic Usage Examples

Simple Variable Collection

The most common pattern involves calling collect() at the end of your configuration function to harvest all defined Hypster parameters:

from hypster import instantiate

def simple_cfg(hp):
    a = hp.int(10, name="a")
    b = hp.float(3.14, name="b")
    return hp.collect(locals())

config = instantiate(simple_cfg)
print(config)

# Output: {'a': 10, 'b': 3.14}

This pattern, demonstrated in tests/test_return_types.py, automatically excludes the hp object and any private variables while preserving your defined parameters.

Filtering Unwanted Locals with Exclude

When your configuration function contains helper functions or temporary variables, use the exclude parameter to remove them explicitly:

def cfg_with_helper(hp):
    learning_rate = hp.float(0.01, name="lr")
    _temp_calc = 42
    def helper_func(x): return x * 2
    
    return hp.collect(locals(), exclude=["_temp_calc", "helper_func"])

The resulting dictionary contains only {'learning_rate': 0.01} because _temp_calc (private) and helper_func (explicitly excluded) are filtered out.

Whitelisting Specific Variables with Include

For strict control over your output, the include parameter acts as a whitelist:

def selective_cfg(hp):
    a = hp.int(1, name="a")
    b = hp.int(2, name="b")
    c = hp.int(3, name="c")
    
    return hp.collect(locals(), include=["a", "c"])

This returns {'a': 1, 'c': 3}, deliberately omitting b despite it being present in the local scope.

Complete Workflow with Instantiate

The instantiate() function in src/hypster/core.py creates the HP instance, executes your config function, and returns whatever collect() produces. This integration allows you to override defaults before collection:

from hypster import instantiate

def training_config(hp):
    epochs = hp.int(10, name="epochs")
    batch_size = hp.int(32, name="batch_size")
    learning_rate = hp.float(0.001, name="lr")
    
    return hp.collect(locals())

# Override specific values before collection

final_config = instantiate(
    training_config, 
    values={"epochs": 50, "lr": 0.01}
)

print(final_config)

# Output: {'epochs': 50, 'batch_size': 32, 'learning_rate': 0.01}

Note that the dictionary keys match your variable names (epochs, batch_size, learning_rate), not the Hypster parameter names ("epochs", "batch_size", "lr"), because collect() captures the Python variable bindings directly.

Why Use HP.collect() Over Manual Returns?

HP.collect() offers three concrete advantages over manually constructing return dictionaries:

1. Eliminates boilerplate – You avoid repetitive dictionary construction like return {"a": a, "b": b} at the end of every config function.
2. Prevents accidental exposure – The automatic filtering of hp, helper functions, and private variables (_temp) protects against leaking implementation details.
3. Supports dynamic configuration – The include and exclude parameters let you reuse the same config function for different contexts, returning different subsets of variables without rewriting logic.

Summary

• HP.collect() lives in src/hypster/hp.py (lines 408–435) and processes locals() dictionaries from within Hypster config functions.
• Default safety filters automatically remove hp, self, __builtins__, __name__, __doc__, and any variable starting with _.
• Flexible filtering via include (whitelist) and exclude (blacklist) parameters allows precise control over returned data.
• Integration with instantiate() enables value overrides before collection, with results feeding directly into your application code.

Frequently Asked Questions

What variables does HP.collect() exclude by default?

By default, HP.collect() excludes the internal names hp, self, __builtins__, __name__, and __doc__, plus any variable beginning with an underscore (_). This ensures temporary calculations and private helper functions do not appear in your final configuration dictionary.

Can I use both include and exclude parameters together?

Yes, you can combine both parameters. When include is specified, only those named variables are considered, then exclude removes any additional items from that subset. Typically, you use one or the other, but the implementation supports both for complex filtering scenarios in src/hypster/hp.py.

Does HP.collect() modify the original locals() dictionary?

No, HP.collect() creates and returns a new dictionary containing the filtered variables. The original locals() dictionary passed into the function remains unchanged, preventing side effects in your configuration function scope.

How do I override values before they get collected?

Pass a values dictionary to instantiate() when calling your config function. According to the implementation in src/hypster/core.py, these values override the defaults defined in your configuration before HP.collect() processes the local scope, ensuring the collected dictionary contains your specified overrides.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}