# How to Tune RIME Grammar Configuration: `collocation_max_length` and `collocation_penalty` Deep Dive

> Master RIME grammar configuration. Tune collocation_max_length and collocation_penalty to optimize multi-character phrase handling and scoring for better language model performance.

- Repository: [amzxyz/rime-lmdg](https://github.com/amzxyz/rime-lmdg)
- Tags: deep-dive
- Published: 2026-02-24

---

**The `collocation_max_length` (default 7) and `collocation_penalty` (default -10) parameters control how RIME handles multi-character phrases, determining maximum phrase length and the scoring penalty applied to low-frequency collocations.**

The RIME (Rime Input Method Engine) relies on statistical language modeling to rank input candidates. In the `amzxyz/rime-lmdg` repository, grammar configuration parameters defined in [`wanxiang/README.md`](https://github.com/amzxyz/rime-lmdg/blob/main/wanxiang/README.md) (lines 43-47) govern how the engine segments text into collocations and applies scoring penalties against dictionary data. Tuning these values directly impacts the balance between input precision and candidate recall.

## Understanding `collocation_max_length` and `collocation_min_length`

These parameters define the boundaries for what constitutes a valid collocation in the language model, interacting with entries in `wanxiang/pinyin_data/单字.dict.yaml` (single-character) and `wanxiang/pinyin_data/词组.dict.yaml` (multi-character phrase) files.

### `collocation_max_length`: Maximum Phrase Boundary

According to [`wanxiang/README.md`](https://github.com/amzxyz/rime-lmdg/blob/main/wanxiang/README.md), **`collocation_max_length`** defaults to **7** characters. This parameter sets the upper limit on how long a multi-character phrase can remain intact before the engine splits it into smaller units for scoring.

Raising this value allows longer fixed expressions—such as "中华人民共和国"—to remain whole during candidate ranking, improving recognition of specialized terminology. Lowering the value forces more aggressive segmentation, which can increase processing speed and reduce false positives when handling very long input strings.

### `collocation_min_length`: Minimum Sequence Threshold

The companion parameter **`collocation_min_length`** defaults to **2** characters. It establishes the shortest sequence that RIME will treat as a collocation rather than independent characters.

Decreasing this value to 1 would theoretically allow single characters, while keeping it at 2 ensures common bi-grams stay together. Raising it above 2 forces the engine to treat short pairs as separate entities, which helps when the dictionary contains ambiguous two-character entries that frequently combine incorrectly.

## Tuning `collocation_penalty` and Related Scoring Parameters

Three penalty parameters in the RIME grammar configuration adjust scoring based on collocation quality and context.

### `collocation_penalty`: Filtering Unlikely Phrases

**`collocation_penalty`** carries a default value of **-10**. This score modifier applies to candidates containing collocations that do not match high-frequency entries in the language model.

Making this penalty more negative (e.g., -15) aggressively discourages rare or unlikely word combinations, sharpening precision for standard vocabulary. Conversely, a milder penalty (e.g., -5) allows less common collocations to surface, which benefits specialized domains like medicine or law where technical phrases appear infrequently in general corpora.

### `non_collocation_penalty` and `weak_collocation_penalty`: Context Adjustments

When the input context suggests a collocation should exist but the candidate lacks one, **`non_collocation_penalty`** (default **-20**) applies. A strongly negative value here forces the engine to prefer candidates containing collocations, improving output fluency for continuous text. Reducing the magnitude (e.g., to -10) relaxes this bias, proving useful when typing acronyms or isolated characters where phrase continuity matters less.

For collocations with probabilities below a specific threshold, **`weak_collocation_penalty`** (default **-35**) adds an extra penalty layer. Increasing the absolute value filters out rare collocations aggressively, reducing noise in general typing scenarios. Lowering it (e.g., to -20) preserves domain-specific jargon that might otherwise be eliminated as statistically weak.

## How Collocation Parameters Interact During Scoring

The RIME engine applies these parameters in a specific sequence when evaluating input candidates against dictionary files:

1. **Base Language Model Scoring**: RIME first computes a probability score for each candidate based on the underlying dictionary data.

2. **Collocation Detection**: The engine checks whether candidates contain sequences falling between `collocation_min_length` and `collocation_max_length`.

3. **Penalty Application**:
   - If a collocation exists but shows low probability, `collocation_penalty` modifies the score.
   - If context indicates a collocation should be present but is missing, `non_collocation_penalty` applies.
   - If the collocation probability falls below the weak threshold, `weak_collocation_penalty` adds to the standard penalty.

This layered approach allows fine-grained control over the recall-precision trade-off, determining whether RIME prioritizes common phrases or preserves rare technical vocabulary.

## Implementing Custom Grammar Configuration in RIME

To apply these parameters, create a custom schema file that overrides the defaults documented in [`wanxiang/README.md`](https://github.com/amzxyz/rime-lmdg/blob/main/wanxiang/README.md). The following Python snippet generates a [`custom.schema.yaml`](https://github.com/amzxyz/rime-lmdg/blob/main/custom.schema.yaml) with tuned collocation settings:

```python
import yaml

# Base schema skeleton referencing amzxyz/rime-lmdg structure

schema = {
    "schema": {
        "schema_id": "custom",
        "name": "Custom Schema with Collocation Tuning",
        "version": "2024-02-24",
    },
    "engine": {
        "segmentor": {
            "type": "simple",
        },
        # Collocation tuning section

        "collocation": {
            "max_length": 8,          # collocation_max_length

            "min_length": 2,          # collocation_min_length

            "penalty": -12,           # collocation_penalty

            "non_penalty": -18,       # non_collocation_penalty

            "weak_penalty": -30,      # weak_collocation_penalty

        },
    },
}

# Write to file

with open("custom.schema.yaml", "w", encoding="utf-8") as f:
    yaml.safe_dump(schema, f, allow_unicode=True)

print("✅ custom.schema.yaml generated with collocation tuning.")

```

Place the generated file in your RIME user configuration directory (e.g., `~/.config/ibus/rime/`), then rebuild the schema using the deployment tool:

```bash

# Deploy the custom schema as referenced in wanxiang/wanxiang-tools.py workflow

rime_deployer --build custom.schema.yaml

```

## Summary

- **`collocation_max_length`** (default 7) controls the upper bound of multi-character phrases; increase it to preserve long technical terms, decrease it for faster segmentation.
- **`collocation_min_length`** (default 2) sets the minimum collocation size; adjust to control whether bi-grams stay united or split.
- **`collocation_penalty`** (default -10) penalizes low-frequency collocations; make more negative to improve precision, less negative to retain rare terms.
- **`non_collocation_penalty`** (default -20) applies when expected collocations are missing; tune to balance fluency versus isolated character input.
- **`weak_collocation_penalty`** (default -35) filters statistically weak phrases; increase magnitude to reduce noise, decrease to keep domain jargon.
- Configuration changes take effect in [`custom.schema.yaml`](https://github.com/amzxyz/rime-lmdg/blob/main/custom.schema.yaml) files deployed via `rime_deployer`, with defaults documented in [`wanxiang/README.md`](https://github.com/amzxyz/rime-lmdg/blob/main/wanxiang/README.md) (lines 43-47).

## Frequently Asked Questions

### What happens if I set `collocation_max_length` too high in RIME?

Setting `collocation_max_length` above 10-12 characters allows extremely long strings to remain intact as single candidates. While this helps with fixed expressions like long organization names or idioms, it can degrade input responsiveness and increase memory usage as the engine evaluates longer sequences against `wanxiang/pinyin_data/词组.dict.yaml` entries. The processing overhead grows with candidate length, potentially slowing generation during active typing.

### How does `collocation_penalty` differ from `weak_collocation_penalty`?

**`collocation_penalty`** applies to any collocation not matching high-frequency entries, while **`weak_collocation_penalty`** specifically targets collocations whose probabilities fall below an internal threshold. According to the `amzxyz/rime-lmdg` implementation, weak penalties stack atop standard penalties, creating a two-tier filtering system. Use `collocation_penalty` for general frequency filtering and `weak_collocation_penalty` to aggressively eliminate borderline rare phrases.

### Where are the default RIME grammar configuration values defined?

The default values for all five parameters reside in [`wanxiang/README.md`](https://github.com/amzxyz/rime-lmdg/blob/main/wanxiang/README.md) at lines 43-47 of the `amzxyz/rime-lmdg` repository. These defaults—`collocation_max_length: 7`, `collocation_min_length: 2`, `collocation_penalty: -10`, `non_collocation_penalty: -20`, and `weak_collocation_penalty: -35`—serve as the baseline for the Wanxiang language model distributed with the project.

### Can I tune these parameters for specific domains like medical terminology?

Yes. For specialized domains, decrease `weak_collocation_penalty` toward -20 (from the default -35) to prevent the engine from filtering rare technical phrases. Simultaneously, increase `collocation_max_length` to 8 or 9 to accommodate longer compound medical terms, and relax `collocation_penalty` to -5 or -8 to allow lower-frequency collocations to surface in candidates. Test changes using representative domain text and iterate based on candidate ranking performance.