# How to Manage LingBot-Map Configurations Using YAML Files: A Complete Guide

> Learn to manage LingBot-Map configurations with YAML files. Explore the ConfigManager class for auto-discovery and deep merging of dataset and method settings at runtime.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-06

---

**LingBot-Map configurations are managed through a hierarchical YAML file system coordinated by the `ConfigManager` class, which auto-discovers dataset and method configs from flat files and deep-merges them with base settings at runtime.**

The `Robbyant/lingbot-map` repository implements a lightweight, file-based configuration system built on YAML that enables flexible benchmark management without code changes. This architecture separates global defaults from dataset-specific and method-specific settings, allowing researchers to add new experiments by simply dropping new YAML files into designated directories. Understanding how **LingBot-Map configurations** are structured and parsed is essential for customizing evaluation pipelines and extending the framework.

## Configuration File Hierarchy in LingBot-Map

LingBot-Map uses a three-tier configuration system where settings cascade from global defaults to specific implementation details. All files are flat dictionaries parsed with `yaml.safe_load`, requiring no top-level keys.

### Base Configuration ([`configs/base.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/configs/base.yaml))

The foundation resides in [`benchmark/configs/base.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/configs/base.yaml). This file defines the **workspace path**, global evaluation defaults, and optional selection lists for datasets and methods. The `ConfigManager` loads this file first to establish the execution environment and determine which dataset and method configurations to activate.

### Dataset Configurations (`configs/datasets/*.yaml`)

Dataset configs are flat files stored in `configs/datasets/` where the filename stem serves as the config name. Each YAML specifies the dataset class, optional sampling parameters, and an `evaluation` block that can override global defaults. The system auto-discovers every `*.yaml` in this directory, parsing only those named in the base configuration's selection list.

### Method Configurations (`configs/methods/*.yaml`)

Method configurations follow the same flat-file pattern in `configs/methods/`. These files define the model class, optional conda environment via the `env` key, and method-specific evaluation overrides. Like datasets, these are auto-discovered and merged based on the base configuration selections.

## The ConfigManager Implementation

The `ConfigManager` class in [`benchmark/benchmark/core/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/core/config.py) orchestrates the loading and merging process. It provides a centralized interface for accessing configuration data while abstracting the file system operations. This class is typically instantiated in [`benchmark/run_worker.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/run_worker.py), which serves as the entry point for driving the benchmark pipeline.

### Auto-Discovery Mechanism

When initialized with a path to [`base.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/base.yaml), the `ConfigManager` automatically scans `configs/datasets/` and `configs/methods/` for YAML files. It matches discovered filenames against the `datasets` and `methods` lists defined in the base configuration, loading only the selected configurations into memory.

### Deep Merge Strategy

The private `_deep_merge` function implements a recursive dictionary merge that preserves nested structures while allowing later values to override earlier ones. The merge order follows: `base.yaml → dataset.evaluation → method.evaluation`. This guarantees that method-specific settings take precedence over dataset defaults, which in turn override global settings.

## Runtime Access to Merged Configurations

After construction, the `ConfigManager` exposes several methods for retrieving configuration data:

```python
from pathlib import Path
from lingbot_map.benchmark.core.config import ConfigManager

# Initialize with the central base config

cfg = ConfigManager(Path("benchmark/configs/base.yaml"))

# Retrieve workspace path

workspace = cfg.get_workspace()

# List activated datasets and methods

datasets = cfg.get_selected_dataset_names()
methods = cfg.get_selected_method_names()

# Access specific configurations

oxford_cfg = cfg.get_dataset_config("oxford")
lm_cfg = cfg.get_method_config("lingbot_map")

# Get merged evaluation config for a specific pair

eval_cfg = cfg.get_merged_evaluation_config(
    dataset_config_name="oxford",
    method_config_name="lingbot_map"
)

```

The `get_merged_evaluation_config()` method returns the final, deep-merged configuration containing all applicable settings for a specific dataset-method combination.

## Summary

- **LingBot-Map configurations** reside in three hierarchical tiers: base, dataset, and method YAML files located in `benchmark/configs/`.
- The `ConfigManager` class in [`benchmark/benchmark/core/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/core/config.py) auto-discovers configurations by scanning `configs/datasets/` and `configs/methods/` directories.
- Configuration inheritance follows a deep-merge strategy where [`base.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/base.yaml) provides defaults, overridden by dataset-specific `evaluation` blocks, then method-specific settings.
- The system uses flat YAML files without top-level keys, enabling new experiments through simple file additions rather than code modifications.
- Runtime access is provided through methods like `get_workspace()`, `get_dataset_config()`, and `get_merged_evaluation_config()`.

## Frequently Asked Questions

### Where is the ConfigManager class implemented in the LingBot-Map repository?

The `ConfigManager` class is implemented in [`benchmark/benchmark/core/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/core/config.py). This file contains the core logic for YAML parsing, auto-discovery of dataset and method configurations, and the `_deep_merge` helper function that handles recursive dictionary merging.

### How does LingBot-Map handle configuration overrides between base, dataset, and method settings?

The system uses a deep-merge strategy implemented in the `_deep_merge` function. When `get_merged_evaluation_config()` is called, it recursively merges dictionaries in the order: base defaults → dataset evaluation settings → method evaluation settings. Later values replace earlier ones while preserving nested structure, ensuring method-specific parameters take final precedence.

### Can I add new datasets or methods without modifying the source code?

Yes. The configuration system is designed for zero-code modifications. Simply create a new YAML file in `configs/datasets/` or `configs/methods/` using the flat dictionary structure (no top-level key), then add the filename stem to the `datasets` or `methods` list in [`configs/base.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/configs/base.yaml). The `ConfigManager` will auto-discover and load the new configuration on the next run.

### What is the difference between `get_dataset_config()` and `get_merged_evaluation_config()`?

The `get_dataset_config()` method returns the raw configuration dictionary for a specific dataset, including its class definition, parameters, and evaluation overrides. In contrast, `get_merged_evaluation_config()` returns the final, deep-merged evaluation configuration for a specific dataset-method pair, combining global defaults with both dataset-specific and method-specific evaluation settings.