# How detect_repeat_token Identifies Generation Failures in Chandra

> Learn how Chandra inspects model outputs for repeating token sequences using detect_repeat_token to identify and prevent generation failures.

- Repository: [Datalab/chandra](https://github.com/datalab-to/chandra)
- Tags: internals
- Published: 2026-03-27

---

**The `detect_repeat_token` utility detects potential generation failures by scanning the tail of model outputs for excessive consecutive repetitions of any token sequence, using a dynamic threshold that scales inversely with sequence length.**

Chandra's generation pipeline integrates with vLLM to produce text outputs, but large language models can occasionally enter repetitive loops that stall meaningful generation. Located in [`chandra/model/util.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/util.py) (lines 68-99), the `detect_repeat_token` function provides the core detection mechanism that identifies these failure patterns by analyzing the raw token string for abnormal repetition at the tail end of the output.

## How Repetition Signals Generation Failure

When a language model gets stuck in a generation loop, it emits the same token or short token sequence repeatedly instead of progressing toward a coherent completion. This behavior indicates a **soft generation failure**—the model has lost context or confidence and is recycling safe, repetitive patterns. Chandra treats these loops as recoverable errors, triggering automatic retries with adjusted sampling parameters to break the cycle.

## The detect_repeat_token Algorithm

The detection logic implements a sliding-window analysis that examines candidate sequences at the end of the generated text. The function accepts `predicted_tokens` as a single concatenated string and optionally trims trailing characters via the `cut_from_end` parameter to ignore partially formed tokens.

### Sliding-Window Sequence Extraction

For every possible sequence length `seq_len` ranging from 1 to `window_size // 2` (default 250), the function extracts a candidate sequence from the string's tail:

```python
candidate_seq = predicted_tokens[-seq_len:]

```

Starting just before this candidate, the function walks backwards through the string, comparing each block of size `seq_len` against `candidate_seq`. The loop counts consecutive matches until encountering a mismatch, yielding a `repeat_count` for that specific sequence length.

### Dynamic Repeat Thresholds

Rather than using a fixed repeat limit, the function calculates a length-dependent threshold that allows short sequences more repetitions while flagging longer sequences sooner:

```python
max_repeats = int(base_max_repeats * (1 + scaling_factor / seq_len))

```

Short patterns naturally occur more frequently in valid text, so they require higher repeat counts to trigger a failure signal. Longer repetitive patterns indicate more severe looping and are flagged after fewer consecutive appearances. If `repeat_count` exceeds `max_repeats` for any sequence length, the function immediately returns `True`; otherwise, it completes the scan and returns `False`.

## Integration with Chandra's vLLM Wrapper

The generation wrapper in [`chandra/model/vllm.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/vllm.py) (lines 104-112) integrates `detect_repeat_token` into its retry logic through the `_should_retry` helper. This wrapper calls the detector twice to maximize accuracy:

```python
has_repeat = detect_repeat_token(result.raw) or (
    len(result.raw) > 50 and detect_repeat_token(result.raw, cut_from_end=50)
)

```

The first call checks the full raw output for obvious loops. The second call—conditional on output length exceeding 50 characters—trims the final 50 characters via `cut_from_end=50` to avoid false positives from partially formed tokens at the generation boundary (lines 5-8). When `has_repeat` evaluates to `True` and the retry budget remains available, the pipeline logs the detection and regenerates the prompt with a slightly higher temperature to perturb the model's sampling distribution.

## Practical Implementation Examples

### Detecting Loops in Raw Output

You can use `detect_repeat_token` directly to validate model outputs before further processing:

```python
from chandra.model.util import detect_repeat_token

# Simulated stuck generation repeating "hello "

output = "The answer is: hello hello hello hello hello "

if detect_repeat_token(output):
    print("Repeat loop detected – consider retrying.")
else:
    print("No repeat loop.")

```

This returns `True` because the short token sequence exceeds the dynamic repeat threshold for its length.

### Automatic Retry Logic

When using Chandra's batch generation interface, the repeat detection operates transparently behind the `generate_batch` function:

```python
from chandra.model.vllm import generate_batch

results = generate_batch(
    batch=["Explain quantum computing", "Define machine learning"],
    temperature=0.6,
    top_p=0.9,
    max_retries=3,
    max_failure_retries=2,
)

for r in results:
    if r.error:
        print("Generation failed after retries.")
    elif r.was_retried_for_repeat:
        print("Succeeded after recovering from repeat loop.")
    else:
        print("Generated:", r.raw)

```

The wrapper automatically increments temperature and re-invokes the model when `detect_repeat_token` identifies a loop, up to the specified `max_retries` limit.

## Summary

- **`detect_repeat_token`** in [`chandra/model/util.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/util.py) scans the tail of generated text for consecutive repetitions of any sequence length between 1 and 250 characters.
- **Dynamic thresholds** scale inversely with sequence length (`max_repeats = int(base_max_repeats * (1 + scaling_factor / seq_len))`), allowing short patterns more occurrences than long ones.
- **Sliding-window analysis** extracts candidate sequences from the string's end and counts backward matches to determine repeat density.
- **Dual-pass detection** in [`chandra/model/vllm.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/vllm.py) checks both full outputs and trimmed versions (minus 50 characters) to eliminate edge-case false positives.
- **Automatic recovery** triggers temperature-adjusted retries when loops are detected, treating repetition as a soft failure recoverable through sampling perturbation.

## Frequently Asked Questions

### What triggers detect_repeat_token to return True?

The function returns `True` when any sequence at the tail of the output repeats consecutively beyond its calculated `max_repeats` threshold. This threshold varies by sequence length—shorter sequences require more repeats to trigger failure than longer ones. For example, a 5-character sequence repeating 20 times might pass, while a 50-character sequence repeating 5 times would fail.

### Why does the function use a dynamic threshold instead of a fixed count?

Valid natural language contains legitimate short repetitions (words like "the" or "and"), so short sequences need higher repeat allowances to avoid false positives. Longer repetitive patterns almost always indicate model degeneration. The inverse scaling formula mathematically encodes this linguistic reality, tightening scrutiny as sequence length increases.

### How does Chandra recover from detected repeat loops?

When `detect_repeat_token` identifies a loop, the vLLM wrapper in [`chandra/model/vllm.py`](https://github.com/datalab-to/chandra/blob/main/chandra/model/vllm.py) treats this as a retry-eligible condition. If retries remain in the budget, the pipeline increases the sampling temperature and regenerates the completion. This perturbation typically breaks the model out of its local minimum, allowing diverse token selection on subsequent attempts.

### What is the performance impact of this detection?

The algorithm operates in O(n×m) time where n is the `window_size` (default 250) and m is the string length being checked, though early exit on threshold breach prevents worst-case scenarios. Since detection runs only once per generation attempt and uses simple string slicing rather than tokenization, the overhead remains negligible compared to the GPU inference time of the vLLM backend.