# What Is Lineage in WhichLLM’s Ranking System? Recency-Aware Scoring for Model Families

> Discover how WhichLLM's lineage system uses recency-aware scoring to fairly rank model families, avoiding penalties for older generations and highlighting new releases.

- Repository: [andy/whichllm](https://github.com/Andyyyy64/whichllm)
- Tags: deep-dive
- Published: 2026-06-09

---

**Lineage in WhichLLM’s ranking system is a family-aware generational ordering that applies a decaying multiplicative factor to frozen benchmark scores, ensuring newer model generations are not unfairly penalized by missing current data.**

WhichLLM uses **lineage** to solve a specific leaderboard problem: when only outdated benchmark snapshots exist, a new model release might have no current scores to compete with older, heavily-tested versions. This mechanism injects recency awareness directly into the scoring pipeline by analyzing model family generations. Understanding lineage is critical for interpreting WhichLLM rankings and contributing new family mappings.

## How Lineage Works in WhichLLM’s Ranking System

The lineage heuristic operates across four distinct stages inside the WhichLLM codebase. Each stage transforms static model metadata into an adjustable recency score that prevents stale benchmarks from dominating the leaderboard.

### Static Model Family Tables

The lineage mapping lives in [`src/whichllm/data/lineage.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/data/lineage.py). This file defines `MODEL_LINEAGE_VERSIONS`, a dictionary that assigns generation indices to known model families such as *qwen*, *llama*, and *deepseek*. Each entry contains regex patterns paired with an integer index, where a higher index represents a newer generation.

### Regex Pre-compilation in the Ranker

At import time, [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py) compiles these patterns into a fast-lookup table stored in `_LINEAGE_REGEX`. Lines 28–33 iterate over every family and compile each pattern:

```python
_LINEAGE_REGEX = {
    family: [(re.compile(pat), idx) for pat, idx in entries]
    for family, entries in MODEL_LINEAGE_VERSIONS.items()
}

```

This pre-compilation ensures that family detection adds minimal overhead during the ranking process.

### Recency Factor Calculation

When a model has only **frozen** scores—for example, from the Open LLM Leaderboard v2—the `_lineage_recency_factor()` function in [`src/whichllm/models/benchmark.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/benchmark.py) computes a multiplier. According to lines 86–102, the logic measures how many generations separate a model from the newest entry in its family:

```python
gens_old = max(0, max_idx - idx)          # generations older than newest

factor = max(0.55, 1.0 - 0.12 * gens_old) # each older gen ≈‑12%

```

The resulting factor is **1.0** for the newest generation and decays linearly to a hard floor of **0.55** for the oldest tracked releases.

### Application to Frozen Benchmark Scores

The `_apply_lineage_recency_demotion` function (lines 108–128 of the same file) applies this factor exclusively to frozen-score entries during the benchmark merging stage:

```python
factor = _lineage_recency_factor(k)
out[k] = round(v * factor, 1)

```

By multiplying frozen values before they are combined with current scores, WhichLLM prevents a stale high score from outranking a newer model that has not yet appeared in frozen leaderboards. As documented in [`docs/scoring.md`](https://github.com/Andyyyy64/whichllm/blob/main/docs/scoring.md) under the "Generation lineage" section, this design gives newer generations a small bonus and older generations a small penalty.

## Practical Code Examples for WhichLLM Lineage

You can interact with the lineage system directly using the private helper functions exposed in the WhichLLM Python package.

### Querying the Lineage Factor for a Model ID

To see the exact recency multiplier assigned to a specific model:

```python
from whichllm.models.benchmark import _lineage_recency_factor

model_id = "meta-llama-3.2-70b"
factor = _lineage_recency_factor(model_id)
print(f"Lineage factor for {model_id!r}: {factor}")

# → 0.88   (two generations older than the newest llama entry)

```

### Adding a New Model Family

Contributors can extend lineage support by editing [`src/whichllm/data/lineage.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/data/lineage.py) and appending to `MODEL_LINEAGE_VERSIONS`:

```python

# src/whichllm/data/lineage.py – add a new family

MODEL_LINEAGE_VERSIONS["mynewmodel"] = [
    (r"mynewmodel-v5", 5),   # newest generation

    (r"mynewmodel-v4", 4),
    (r"mynewmodel-v3", 3),
]

```

The ranker will automatically pick up the new family on the next import because `_LINEAGE_REGEX` is built at import time.

### Inspecting the Compiled Regex Table

For debugging or auditing purposes, inspect the compiled patterns that the ranker uses:

```python
from whichllm.engine.ranker import _LINEAGE_REGEX

for fam, patterns in _LINEAGE_REGEX.items():
    print(f"{fam}: {[p.pattern for p, _ in patterns]}")

```

### Demonstrating Lineage Demotion on Frozen Scores

This example shows how an older generation’s frozen score is demoted while a newer generation remains unchanged:

```python
from whichllm.models.benchmark import _apply_lineage_recency_demotion

combined = {"llama-3.1-8b": 85.0, "llama-2-7b": 90.0}
frozen   = {"llama-3.1-8b": 85.0, "llama-2-7b": 90.0}
current  = {}                     # no current evidence

adjusted = _apply_lineage_recency_demotion(combined, frozen, current)
print(adjusted)

# {'llama-3.1-8b': 85.0, 'llama-2-7b': 77.0}

# ^ older generation (llama‑2) received a ~0.12*1 = 0.88 factor → 90×0.88≈77

```

## Summary

- **Lineage** is a recency heuristic that prevents stale frozen benchmarks from distorting WhichLLM rankings.
- The static family tables are maintained in [`src/whichllm/data/lineage.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/data/lineage.py) via `MODEL_LINEAGE_VERSIONS`.
- [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py) pre-compiles regexes at import time for efficient model detection.
- [`src/whichllm/models/benchmark.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/benchmark.py) calculates the recency factor via `_lineage_recency_factor()` and applies it through `_apply_lineage_recency_demotion`.
- The factor ranges from **1.0** for the newest generation down to a minimum of **0.55** for the oldest tracked generation.

## Frequently Asked Questions

### What does lineage mean in WhichLLM?

In WhichLLM, lineage refers to a **family-aware generational ordering** that tracks how old a model release is relative to the newest version in its family. It is used solely to adjust frozen benchmark scores so that newer hardware or architecture improvements are reflected fairly in the ranking pipeline.

### How does WhichLLM calculate the lineage recency factor?

The WhichLLM source code calculates the factor in [`src/whichllm/models/benchmark.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/models/benchmark.py) by finding a model’s generation index and subtracting it from the family’s maximum index. The formula `factor = max(0.55, 1.0 - 0.12 * gens_old)` applies approximately a **12% penalty per older generation**, capped at a floor of 0.55.

### Which models are affected by lineage adjustments?

Only models backed exclusively by **frozen benchmark data** receive lineage adjustments. If current, live benchmark evidence exists for a model, WhichLLM does not apply the recency demotion because the score already reflects up-to-date performance.

### How can I add a new model family to WhichLLM’s lineage system?

You can extend `MODEL_LINEAGE_VERSIONS` inside [`src/whichllm/data/lineage.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/data/lineage.py) with regex patterns and ascending generation indices. Because [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py) rebuilds `_LINEAGE_REGEX` at import time, the ranker automatically recognizes the new family on the next application startup without additional wiring.