How the Reverse Lookup (反查) System Works in Wanxiang RIME

The reverse lookup (反查) system in Wanxiang RIME allows users to type a character followed by a trigger key (default `) and a filter string to search for characters matching specific phonetic or shape codes, using a multi-stage pipeline involving input parsing, database queries, and fuzzy matching algorithms.

The Wanxiang RIME engine is an open-source input method schema that implements sophisticated character lookup capabilities. Its reverse lookup (反查) feature enables users to find characters by their associated codes—such as pinyin or shape-based encodings—rather than by their direct input sequence. This system is governed by configuration files in YAML format and executed through Lua processors that handle parsing, caching, and matching operations.

Configuration and Schema Setup

The reverse lookup behavior is defined in wanxiang.schema.yaml within the wanxiang_lookup section. This configuration declares the trigger key, data sources, and processing options that activate the system.

A typical configuration block looks like this:

wanxiang_lookup:
  key: '`'                     # reverse-lookup trigger character

  enable_tone: true            # allow tone numbers (7-0) in filter

  data_source: [aux, db]       # consult comment codes first, then database

  lookup: [wanxiang_reverse]   # name of the ReverseLookup table

  tags: [abc]                  # processor tags that activate the feature

The key parameter defines the separator between the input text and the filter string. The data_source array specifies the lookup precedence: aux extracts codes from candidate comments, while db queries the reverse-lookup dictionary files like wanxiang_reverse.dict.yaml. The lookup field references ReverseLookup tables that map characters to their associated code strings.

Input Parsing and Segmentation

When a user types the trigger key, the engine invokes split_lookup_input in wanxiang/lua/super_lookup.lua (lines 30-53) to partition the raw input into components.

local function split_lookup_input(input, key, bypass_prefix)
    if not input or input == "" or not key or key == "" then return nil end
    local scan_from = 1
    if bypass_prefix and bypass_prefix ~= "" and input:sub(1, #bypass_prefix) == bypass_prefix then
        scan_from = #bypass_prefix + 1
    end
    local s_start, s_end = nil, nil
    local from = scan_from
    while true do
        local s, e = input:find(key, from, true)
        if not s then break end
        s_start, s_end = s, e
        from = s + 1
    end
    if not s_start then return nil end
    local code = input:sub(1, s_start - 1)
    local fuma = input:sub(s_end + 1)
    return code, fuma, s_start, s_end
end

This function returns two critical values: code (the text before the trigger that will be committed) and fuma (the filter string after the trigger used for matching). It also respects a bypass prefix (such as ;; for user dictionary creation), ensuring the trigger is only recognized after the prefix is consumed.

Building Reverse Lookup Groups

The core lookup logic resides in build_reverse_group within wanxiang/lua/super_lookup.lua. For each character in a candidate, the engine queries the ReverseLookup tables and applies schema-defined projections to generate code variants.

local function build_reverse_group(main_projection, xlit_projection, db_table, text)
    local group_main, seen_main = {}, {}
    local group_xlit, seen_xlit = {}, {}
    for _, db in ipairs(db_table) do
        local code = db:lookup(text)
        if code and #code > 0 then
            for part in code:gmatch('%S+') do
                local main_variants, xlit_variants = expand_code_variant(main_projection, xlit_projection, part)
                for _, v in ipairs(main_variants) do
                    if not seen_main[v] then seen_main[v]=true; group_main[#group_main+1]=v end
                end
                for _, v in ipairs(xlit_variants) do
                    if not seen_xlit[v] then seen_xlit[v]=true; group_xlit[#group_xlit+1]=v end
                end
            end
        end
    end
    return group_main, group_xlit
end

This process generates two collections: group_main containing primary codes and group_xlit containing transliteration variants. The function utilizes expand_code_variant to apply the main and xlit projection rules defined in the schema's algebra section, ensuring all possible code representations are available for matching.

Matching Algorithms: Prefix vs. Fuzzy Recursive

The system employs different matching strategies based on query complexity.

Single-Character Matching

For single-character queries, the engine uses group_match (lines 74-80), which performs simple prefix matching:

local function group_match(group, fuma)
    if not group then return false end
    for i = 1, #group do 
        if string.sub(group[i], 1, #fuma) == fuma then return true end 
    end
    return false
end

Multi-Character Fuzzy Matching

For phrases containing multiple characters, the engine falls back to match_fuzzy_recursive (lines 82-118). This depth-first algorithm allows the filter string to skip characters within code groups while maintaining sequence order:

local function match_fuzzy_recursive(codes_sequence, idx, input_str, input_idx, memo, is_phrase_mode)
    -- implementation details
    for _, code in ipairs(codes) do
        if is_phrase_mode and #code > 3 then skip = true end
        -- fuzzy matching logic
        while i_curr <= i_limit and c_curr <= c_limit do
            if input_str:byte(i_curr) == code:byte(c_curr) then i_curr = i_curr + 1 end
            c_curr = c_curr + 1
        end
        if match_fuzzy_recursive(codes_sequence, idx + 1, input_str, i_curr, memo, is_phrase_mode) then
            result = true; break
        end
    end
end

In phrase mode, the matcher skips codes longer than three characters to prevent over-matching across long phrases.

Tone Filtering and Auxiliary Data Integration

When enable_tone is active, the system processes digits 7-0 as tone indicators. The f.func implementation (lines 70-600) extracts these digits from the fuma string:

for i = 1, #fuma do
    local char = fuma:sub(i,i)
    if char == "7" or char == "8" or char == "9" or char == "0" then
        table.insert(tone_filter_seq, char)
    else
        clean_fuma = clean_fuma .. char
    end
end

If the reverse-lookup database lacks tone information for a specific character, the engine can "borrow" the tone from auxiliary codes derived from candidate comments. This enables queries like `ni5 to match characters with neutral tone (tone 5) even when the primary database stores toneless codes.

Practical Usage Examples

Simple Phonetic Lookup

To find the character 好 by its pinyin code:

Input: 好`ha

The parser returns code="好" and fuma="ha". The engine queries the reverse dictionary for 好, retrieves codes like ha0 and hao4, and matches them against the filter using prefix matching.

Tone-Specific Queries

To filter by specific tone numbers:

Input: 好`hao4

The system extracts tone_filter_seq={"4"} and clean_fuma="hao". It validates that the candidate's codes include the tone digit 4, checking auxiliary comment codes if the primary database entry lacks tonal data.

Bypass Prefix Handling

When using user-defined word creation prefixes:

Input: ;;ni`ni

The split_lookup_input function recognizes the bypass prefix ;;, scans for the trigger key after the prefix, and processes the reverse lookup normally while preserving the prefix functionality for dictionary entry.

Summary

• Configuration-driven: The wanxiang_lookup section in schema files defines triggers, data sources (aux and db), and tone handling options.
• Two-phase parsing: split_lookup_input separates commit text from filter strings, respecting bypass prefixes for advanced input modes.
• Cached database queries: The db_cache stores ReverseLookup results to minimize redundant disk access during candidate generation.
• Dual matching strategies: Single characters use fast prefix matching (group_match), while multi-character phrases employ recursive fuzzy matching (match_fuzzy_recursive).
• Tone awareness: Digits 7-0 are parsed as tone filters, with fallback to auxiliary comment data when primary database entries lack tonal information.
• Source files: Core logic resides in wanxiang/lua/super_lookup.lua, with configuration in wanxiang.schema.yaml and dictionary data in wanxiang_reverse.dict.yaml.

Frequently Asked Questions

How do I enable reverse lookup in my Wanxiang schema?

Add a wanxiang_lookup section to your wanxiang.schema.yaml file specifying the key (trigger character), data_source array (typically [aux, db]), and the lookup table names (such as [wanxiang_reverse]). Ensure your schema includes the processor tags referenced in the configuration.

What is the difference between aux and db data sources?

aux refers to codes extracted from candidate comments—the auxiliary information displayed alongside main candidates. db refers to the dedicated reverse-lookup dictionary files (like wanxiang_reverse.dict.yaml) accessed through the ReverseLookup class. The engine checks aux first if specified, then falls back to db if no match is found.

Why does my multi-character reverse lookup skip some codes?

The match_fuzzy_recursive function operates in phrase mode for multi-character inputs, which automatically skips codes longer than three characters to prevent incorrect partial matches. For single-character queries, the system uses strict prefix matching without this length restriction.

Can I use tone numbers in reverse lookup filters?

Yes, when enable_tone is set to true in the configuration. The system recognizes digits 7, 8, 9, and 0 as tone indicators (where 0 typically represents neutral tone). These digits are stripped from the filter string and matched against tone data from either the reverse-lookup database or auxiliary comment fields.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}