# How to Configure Relevance Scoring in Headroom: BM2SScorer vs EmbeddingScorer

> Learn how to configure relevance scoring in Headroom. Compare BM25Scorer and EmbeddingScorer for optimal search results and understand automatic scorer selection.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: how-to-guide
- Published: 2026-06-18

---

**Headroom delegates relevance decisions to objects implementing the `RelevanceScorer` protocol defined in [`headroom/relevance/base.py`](https://github.com/chopratejas/headroom/blob/main/headroom/relevance/base.py), automatically selecting `EmbeddingScorer` for semantic matching when `fastembed` is present or falling back to the lexical `BM2SScorer` for zero-dependency environments.**

The `chopratejas/headroom` library uses a pluggable relevance scoring system to determine which document sections best match a query before compression. Configuring relevance scoring allows you to choose between fast lexical matching (BM25-style) and dense semantic embeddings, depending on your runtime constraints and accuracy requirements. You can explicitly instantiate either scorer or rely on the automatic `HybridScorer` detection mechanism provided in [`headroom/relevance/hybrid.py`](https://github.com/chopratejas/headroom/blob/main/headroom/relevance/hybrid.py).

## Understanding Headroom's Relevance Scoring Architecture

Headroom's compression pipeline relies on a scorer to rank document fragments before trimming or rewriting them. The architecture centers on a protocol-based design that enables swappable implementations.

### The RelevanceScorer Protocol

All scorers inherit from the abstract base class defined in [`headroom/relevance/base.py`](https://github.com/chopratejas/headroom/blob/main/headroom/relevance/base.py). This protocol standardizes how relevance scores are calculated across different algorithms, ensuring the `Headroom` client can interact with any scorer implementation uniformly.

### Available Scorer Implementations

Headroom ships with two concrete implementations:

- **`EmbeddingScorer`** – A semantic scorer that generates dense embeddings using the `fastembed` library and the **BAAI/bge-small-en-v1.5** model (approximately 30MB). It captures synonyms and paraphrases through vector similarity.
- **`BM2SScorer`** – A lexical scorer implementing BM25-style term weighting. It operates using pure Python standard libraries with no external model downloads, making it ideal for constrained environments.

## Automatic Scorer Selection with HybridScorer

When you instantiate `Headroom` without specifying a scorer, the library uses `HybridScorer` to automatically detect available dependencies.

As implemented in [`headroom/relevance/embedding.py`](https://github.com/chopratejas/headroom/blob/main/headroom/relevance/embedding.py), the `EmbeddingScorer.is_available()` method attempts to import `fastembed` and returns `True` only if the package is present. `HybridScorer` checks this condition and defaults to `EmbeddingScorer` when available, otherwise falling back to `BM2SScorer`.

```python
from headroom.relevance import HybridScorer

# Automatically selects EmbeddingScorer if fastembed is installed,

# otherwise uses BM2SScorer

scorer = HybridScorer()

```

## Explicit Configuration Methods

For production deployments requiring deterministic behavior, instantiate your preferred scorer explicitly and pass it to the `Headroom` client.

### Installing Optional Dependencies

To enable semantic scoring, install the relevance extras:

```bash
pip install "headroom[relevance]"

```

This command pulls in `fastembed` and its ONNX runtime dependencies.

### Instantiating Specific Scorers

Force a specific scorer implementation regardless of installed packages:

```python
from headroom import Headroom
from headroom.relevance import EmbeddingScorer, BM2SScorer

# Force semantic scoring with embeddings

headroom_semantic = Headroom(relevance_scorer=EmbeddingScorer())

# Force lexical BM25-style scoring

headroom_lexical = Headroom(relevance_scorer=BM2SScorer())

```

## Tuning Scorer Parameters

Both scorers expose constructor arguments for fine-tuning behavior.

### EmbeddingScorer Configuration

Override the default model or caching behavior:

```python
from headroom.relevance import EmbeddingScorer

# Use a custom embedding model

scorer = EmbeddingScorer(
    model_name="BAAI/bge-base-en-v1.5",
    cache_model=True
)

```

### BM2SScorer Hyperparameters

Adjust the BM25 algorithm parameters:

```python
from headroom.relevance import BM2SScorer

# Tune k1 and b parameters for specific document collections

scorer = BM2SScorer(k1=1.5, b=0.8)

```

## Practical Implementation Examples

### Semantic Scoring with EmbeddingScorer

Score individual fragments using dense vector similarity:

```python
from headroom.relevance import EmbeddingScorer

scorer = EmbeddingScorer()
result = scorer.score(
    '{"status": "failed", "error": "connection refused"}',
    "show me the errors"
)

print(result.score)   # 0.73 (example)

print(result.reason)  # "Embedding: semantic similarity 0.73"

```

### Batch Processing

Process multiple document fragments efficiently:

```python
items = [
    '{"status": "failed"}',
    '{"status": "success"}',
    '{"error": "timeout"}',
]

scorer = EmbeddingScorer()
batch = scorer.score_batch(items, "what errors happened?")

for r in batch:
    print(r.score, r.reason)

```

### Lexical Scoring with BM2SScorer

Use term-frequency based matching without external dependencies:

```python
from headroom.relevance import BM2SScorer

scorer = BM2SScorer(k1=1.2, b=0.75)
result = scorer.score(
    '{"status": "failed", "error": "connection refused"}',
    "show me the errors"
)

print(result.score)   # 0.45 (example, based on term overlap)

```

### Runtime Selection with HybridScorer

Verify which scorer was selected at runtime:

```python
from headroom.relevance import HybridScorer

scorer = HybridScorer()
print(type(scorer))  # <class 'headroom.relevance.embedding.EmbeddingScorer'>

```

## Summary

- Headroom uses the `RelevanceScorer` protocol defined in [`headroom/relevance/base.py`](https://github.com/chopratejas/headroom/blob/main/headroom/relevance/base.py) to standardize relevance calculations.
- **`EmbeddingScorer`** requires `pip install "headroom[relevance]"` and provides semantic matching via the BAAI/bge-small-en-v1.5 model.
- **`BM2SScorer`** provides fast, dependency-free lexical matching suitable for resource-constrained environments.
- **`HybridScorer`** automatically selects `EmbeddingScorer` when `fastembed` is available, falling back to `BM2SScorer` otherwise.
- Pass explicit scorer instances to the `Headroom` constructor via the `relevance_scorer` parameter for deterministic behavior.

## Frequently Asked Questions

### How do I force Headroom to use lexical scoring even if fastembed is installed?

Instantiate `BM2SScorer` explicitly and pass it to the `Headroom` constructor:

```python
from headroom import Headroom
from headroom.relevance import BM2SScorer

headroom = Headroom(relevance_scorer=BM2SScorer())

```

This bypasses the `HybridScorer` automatic selection logic and guarantees the use of BM25-style term matching regardless of optional dependencies.

### Can I use a different embedding model with EmbeddingScorer?

Yes. The `EmbeddingScorer` constructor accepts a `model_name` parameter to specify any model supported by `fastembed`. For example:

```python
scorer = EmbeddingScorer(model_name="BAAI/bge-base-en-v1.5")

```

The default model is `BAAI/bge-small-en-v1.5`, which balances accuracy and size at approximately 30MB.

### What are the performance trade-offs between BM2SScorer and EmbeddingScorer?

**BM2SScorer** operates entirely in Python with no model loading overhead, making it faster for short queries and suitable for environments without GPU or significant RAM. **EmbeddingScorer** incurs a one-time model download (approximately 30MB) and higher memory usage, but provides superior accuracy for semantic queries involving synonyms, paraphrases, or conceptually related terms that lack lexical overlap.

### Where is the automatic scorer selection logic implemented?

The automatic fallback mechanism resides in [`headroom/relevance/hybrid.py`](https://github.com/chopratejas/headroom/blob/main/headroom/relevance/hybrid.py). This module implements `HybridScorer`, which checks `EmbeddingScorer.is_available()` (defined in [`headroom/relevance/embedding.py`](https://github.com/chopratejas/headroom/blob/main/headroom/relevance/embedding.py)) to determine whether the semantic dependencies are importable before defaulting to the pure-Python `BM2SScorer`.