# How the Three-Layer YAML Configuration System Works in Lingbot-Map

> Understand the three-layer YAML configuration system in Robbyant/lingbot-map. Learn how base configs merge with dataset and method overrides for flexible, modular settings.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-02

---

**The three-layer YAML configuration system in Robbyant/lingbot-map merges a base configuration with dataset-specific and method-specific overrides using a deep dictionary merge, allowing global defaults to cascade through modular, auto-discovered configuration files.**

The `lingbot-map` repository implements a hierarchical configuration architecture to manage complex benchmarking experiments without duplication. This three-layer YAML configuration system separates global defaults from specific dataset parameters and method-specific settings, enabling researchers to compose experiments by selecting combinations of pre-defined components. The system centers on the `ConfigManager` class in [`benchmark/core/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/core/config.py), which handles auto-discovery, loading, and merging of configuration layers.

## The Three Configuration Layers

The architecture divides settings across three distinct levels, each serving a specific purpose in the experiment lifecycle.

### Base Configuration Layer

The foundation resides in [`benchmark/configs/base.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/configs/base.yaml). This file establishes global defaults for workspace paths, logging levels, and evaluation criteria. It also contains the master selection lists under `datasets` and `methods` keys, determining which configurations the system discovers and runs during the benchmark phase.

### Dataset Configuration Layer

Individual dataset definitions live in `benchmark/configs/datasets/*.yaml`. Each file is flat (no top-level key) and named after the dataset it describes—for example, [`oxford.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/oxford.yaml) or [`tum.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/tum.yaml). These files contain dataset-specific parameters such as data paths, sampling strategies, and optional `evaluation` blocks that override base defaults. The filename stem serves as the configuration identifier throughout the system.

### Method Configuration Layer

Method-specific settings are stored in `benchmark/configs/methods/*.yaml`, following the same flat-file structure as datasets. Files like [`lingbot_map.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map.yaml) define algorithm parameters, optional conda environment specifications via the `env` field, and further `evaluation` overrides. This layer receives the highest precedence in the merge hierarchy, allowing methods to fine-tune evaluation parameters without modifying global or dataset configurations.

## How ConfigManager Merges the Layers

The `ConfigManager` class orchestrates the loading and merging process in [`benchmark/benchmark/core/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/core/config.py). During initialization (lines 84-99), it loads the base configuration and auto-discovers all YAML files in the datasets and methods directories.

Loading occurs via `_load_yaml` (lines 75-78), which uses `yaml.safe_load` to parse individual files:

```python
from pathlib import Path
from benchmark.core.config import ConfigManager

# Initialize manager with base config

cfg = ConfigManager(Path("benchmark/configs/base.yaml"))

# Access selected components

datasets = cfg.get_selected_dataset_names()   # ["oxford", "tum", ...]

methods = cfg.get_selected_method_names()     # ["lingbot_map", ...]

```

When computing effective configurations, the system applies `_deep_merge` (lines 19-35) through the public method `get_merged_evaluation_config` (lines 30-39). This implements a **deep dictionary merge**: nested dictionaries are combined recursively, with later values overwriting earlier ones in the sequence **base → dataset → method**.

```python

# Retrieve merged evaluation configuration

eval_cfg = cfg.get_merged_evaluation_config(
    dataset_config_name="oxford",
    method_config_name="lingbot_map"
)

```

The resulting `eval_cfg` contains base evaluation parameters, overridden by any `evaluation` block in [`oxford.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/oxford.yaml), then further overridden by any `evaluation` block in [`lingbot_map.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map.yaml).

## Configuration Hierarchy and Precedence

The precedence rules follow a simple but strict hierarchy:

1. **Base values** provide the fallback defaults
2. **Dataset configurations** override base values for specific data handling
3. **Method configurations** override both previous layers for algorithm-specific needs

This deep merge behavior ensures that partial overrides are possible. A dataset can change only the evaluation metrics it affects, while a method can adjust logging or output directories without redeclaring unrelated base parameters. The `env` field in method configs remains separate from the merge and is accessed directly via `get_method_config()`.

## Code Examples in Practice

The benchmark pipeline in [`benchmark/run.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/run.py) demonstrates practical consumption of the three-layer system. The run phase retrieves method configurations and merged evaluation settings for each dataset-method combination:

```python

# Inside benchmark/run.py workflow

method_cfg = config_manager.get_method_config(method_name)
merged_eval = config_manager.get_method_config(method_name)

# Direct access to specific layer configurations

dataset_params = cfg.get_dataset_config("oxford")["params"]
method_env = cfg.get_method_config("lingbot_map").get("env")  # conda environment name

```

Each configuration retrieval method returns a dictionary containing:
- **Dataset configs**: `dataset_class`, `params`, `sampling`, optional `evaluation`
- **Method configs**: `method_class`, `params`, `env`, optional `evaluation`

## Summary

- The three-layer YAML configuration system separates concerns across global defaults, dataset specifics, and method specifics
- `ConfigManager` in [`benchmark/core/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/core/config.py) auto-discovers configurations and performs deep merges in **base → dataset → method** order
- Dataset files in `benchmark/configs/datasets/*.yaml` override base settings without duplication
- Method files in `benchmark/configs/methods/*.yaml` receive final precedence and can specify conda environments via the `env` field
- The merge algorithm is recursive, ensuring nested dictionaries combine properly rather than replacing each other wholesale

## Frequently Asked Questions

### What happens if the same key exists in multiple configuration layers?

The deep merge algorithm resolves conflicts by giving precedence to later layers. Values from [`base.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/base.yaml) are overwritten by dataset configurations, which are then overwritten by method configurations. For nested dictionaries, the merge is recursive, meaning only conflicting leaf values are replaced, preserving the structure of non-conflicting siblings.

### How does the system discover available dataset and method configurations?

`ConfigManager` auto-discovers configurations by scanning the directories specified in the base config. During initialization (lines 86-99 in [`benchmark/core/config.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/core/config.py)), it iterates through `benchmark/configs/datasets/` and `benchmark/configs/methods/`, loading every YAML file found and using the filename stem as the configuration identifier.

### Can I specify different conda environments for different methods?

Yes. Each method configuration file can include an optional `env` field specifying the conda environment name. This field is accessed directly through `get_method_config()` and is not part of the merged evaluation configuration, allowing different algorithms to run in isolated environments without affecting the evaluation logic.

### Where is the merged configuration actually used in the benchmark pipeline?

The merged evaluation configuration is consumed in [`benchmark/run.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/run.py), where the pipeline iterates through selected dataset-method combinations. For each pair, it calls `get_merged_evaluation_config()` to produce the final parameter set passed to the evaluation routines, ensuring each experiment runs with the correct combination of dataset-specific and method-specific parameters.