# How rime固定或用户词典刷新为带声调编码.py Converts Auxiliary Codes in Rime Dictionaries

> Learn how rime固定或用户词典刷新为带声调编码.py converts auxiliary codes, preserving original suffixes while transforming phonetic roots to tone-marked pinyin. Optimize your Rime dictionary today.

- Repository: [amzxyz/rime-lmdg](https://github.com/amzxyz/rime-lmdg)
- Tags: how-to-guide
- Published: 2026-02-24

---

**The script preserves auxiliary code suffixes like `;sc` or `[xx]` by isolating the phonetic root using the `AUX_SEP_REGEX` pattern, converting only the root to tone-marked pinyin with `pypinyin`, then reattaching the original auxiliary suffix unchanged.**

The `rime固定或用户词典刷新为带声调编码.py` script in the [amzxyz/rime-lmdg](https://github.com/amzxyz/rime-lmdg) repository automates the conversion of flat pinyin into tone-marked encoding for Rime input method dictionaries. This utility processes both standard dictionary files and user database exports, ensuring that auxiliary codes—supplemental suffixes used for double pinyin or shape-based filtering—remain intact during the tone conversion process. Understanding how the script handles auxiliary code (辅助码) separation is essential for maintaining complex Rime schemas without breaking existing encoding schemes.

## How Auxiliary Code Detection Works

The script identifies auxiliary code boundaries using a dedicated regular expression defined at line 20 of `wanxiang/rime固定或用户词典刷新为带声调编码.py`:

```python
AUX_SEP_REGEX = r'[;\[]'

```

This pattern matches two specific separator characters that indicate the start of an auxiliary suffix:
- **Semicolon (`;`)** – Used for style codes like `;sc` (simplified Chinese preference) or `;um` (user-defined markers)
- **Opening bracket (`[`)** – Used for structural annotations like `[xx]` or `[shape]`

When processing dictionary lines, the script applies this regex to split each pinyin segment precisely at the boundary between the phonetic root and its auxiliary suffix.

## The Three-Step Conversion Process

The conversion logic resides primarily in the `tone_mark()` function (lines 60-66) and follows a strict segmentation workflow to ensure auxiliary codes pass through unmodified.

### Step 1: Isolating the Phonetic Root

For each pinyin segment, the script splits the string using `AUX_SEP_REGEX` to separate the tone-convertible portion from the auxiliary metadata:

```python
root = re.split(AUX_SEP_REGEX, seg)[0]  # Text before ; or [

suffix = seg[len(root):]                # Everything from ; or [ onwards

```

This extraction ensures that a segment like `bin;sc` decomposes into root `bin` and suffix `;sc`, while `zhang[xx]` becomes `zhang` and `[xx]`.

### Step 2: Applying Tone Marks with pypinyin

With the root isolated, the script calls the `pypinyin` library to inject tone diacritics:

```python
from pypinyin import pinyin, Style

# Inside tone_mark():

toned_root = pinyin(root, style=Style.TONE, heteronym=False, strict=False)[0][0]

```

The `Style.TONE` parameter instructs `pypinyin` to return the phonetic string with appropriate tone marks (e.g., transforming `bin` into `bīn` or `zhang` into `zhāng`), while auxiliary suffixes remain excluded from this linguistic processing.

### Step 3: Reconstructing the Segment

The final assembly concatenates the tone-marked root with the preserved suffix:

```python
new_seg = toned_root + suffix

```

This approach guarantees that `bin;sc` becomes `bīn;sc`—the tone mark appears only on the phonetic component, while the auxiliary code `;sc` maintains its original form and position.

## Dictionary Format Handlers

The script implements two distinct processing pipelines to accommodate different Rime dictionary structures, both preserving auxiliary codes using the same core logic.

### Processing Fixed Dictionary Lines with `normal_line()`

Located at lines 71-95 in `wanxiang/rime固定或用户词典刷新为带声调编码.py`, the `normal_line()` function handles standard [`.dict.yaml`](https://github.com/amzxyz/rime-lmdg/blob/main/.dict.yaml) files with three-column formatting (entry, pinyin, frequency). It iterates through space-separated pinyin segments in the second column:

```python
def normal_line(cols):
    # cols[1] contains space-separated pinyin segments

    segs = cols[1].split(' ')
    new_segs = [tone_mark(seg) for seg in segs]
    cols[1] = ' '.join(new_segs)
    return '\t'.join(cols)

```

Each segment passes through `tone_mark()`, ensuring that auxiliary codes attached to any syllable in multi-character words receive proper treatment.

### Processing User Database Lines with `userdb_line()`

The `userdb_line()` function (lines 98-116) processes userdb.txt exports where the first column contains the pinyin string and the second column contains the Chinese character entry:

```python
def userdb_line(cols):
    # cols[0] contains space-separated pinyin

    segs = cols[0].split(' ')
    new_segs = [tone_mark(seg) for seg in segs]
    cols[0] = ' '.join(new_segs)
    return '\t'.join(cols)

```

This handler applies identical auxiliary code preservation logic to user-specific vocabulary entries, maintaining consistency between fixed dictionaries and personal user data.

## Complete Code Examples

### Example 1: Fixed Dictionary with Semicolon Auxiliary Codes

**Input line** (tab-separated columns):

```

编码	bin;sc	100

```

**Processing breakdown**:

| Original Segment | Phonetic Root | Auxiliary Suffix | Tone-Added Root | Final Segment |
|------------------|---------------|-------------------|-----------------|---------------|
| `bin;sc`         | `bin`         | `;sc`             | `bīn`           | `bīn;sc`      |

**Output line**:

```

编码	bīn;sc	100

```

### Example 2: User Database with Bracket Auxiliary Codes

**Input line** (userdb format):

```

bin;sc ma[shape]	编码

```

**Processing flow**:
1. Split first column into segments: `['bin;sc', 'ma[shape]']`
2. Process `bin;sc` → `bīn;sc`
3. Process `ma[shape]` → `mǎ[shape]`
4. Rejoin with tab separator

**Output line**:

```

bīn;sc mǎ[shape]	编码

```

Both examples demonstrate that **only the phonetic root receives tonal annotation**, while auxiliary markers (`;sc`, `[shape]`) remain unmodified and properly positioned.

## Summary

- **Auxiliary code preservation** relies on `AUX_SEP_REGEX` (`r'[;\[]'`) to detect suffix boundaries at semicolons or opening brackets in `wanxiang/rime固定或用户词典刷新为带声调编码.py`.
- **Root isolation** occurs through regex splitting, separating tone-convertible phonetics from static metadata before processing.
- **Tone conversion** uses `pypinyin` with `Style.TONE` exclusively on the isolated root, ensuring linguistic accuracy without corrupting auxiliary schemes.
- **Format flexibility** is provided by `normal_line()` (lines 71-95) for standard dictionaries and `userdb_line()` (lines 98-116) for user database files, both implementing identical auxiliary code handling.
- **Suffix reconstruction** maintains the exact original auxiliary code characters, preventing data loss in complex Rime input schemas.

## Frequently Asked Questions

### What are auxiliary codes (辅助码) in Rime dictionaries?

Auxiliary codes are supplemental suffixes appended to pinyin segments to enable additional filtering or encoding schemes, such as double pinyin shortcuts or character shape components. Common formats include semicolon-separated codes (`;sc` for simplified Chinese preference) or bracket-enclosed annotations (`[xx]` for structural markers). These suffixes follow the phonetic spelling but precede any frequency or weight values in dictionary entries.

### Does the script modify the auxiliary code suffixes during conversion?

No, the script explicitly preserves auxiliary code suffixes unchanged. The `tone_mark()` function isolates the auxiliary portion using `AUX_SEP_REGEX`, processes only the phonetic root through `pypinyin`, then reattaches the original suffix via string concatenation. This ensures that `;sc`, `[xx]`, or any custom auxiliary markers remain exactly as they appeared in the source dictionary.

### How does the script handle multiple auxiliary codes in a single pinyin segment?

The script handles multiple auxiliary codes by treating everything from the first separator match as the preserved suffix. Since `AUX_SEP_REGEX` matches either `;` or `[`, a segment like `pin;sc[alt]` splits at the first character (`;`), resulting in root `pin` and suffix `;sc[alt]`. The entire suffix string passes through unchanged, maintaining complex compound auxiliary codes intact while only the leading phonetic root receives tone marks.

### Which dictionary formats does this script support?

The script supports two primary Rime dictionary formats implemented in `wanxiang/rime固定或用户词典刷新为带声调编码.py`: standard fixed dictionaries (three-column tab-separated format processed by `normal_line()` at lines 71-95) and user database exports (two-column format with pinyin first, processed by `userdb_line()` at lines 98-116). Both handlers iterate through space-separated pinyin segments and apply identical auxiliary code preservation logic, ensuring consistent tone conversion across dictionary types.