How Karukan’s IME State Machine Transitions Between Empty, Composing, and Conversion States

Karukan’s input method engine uses a deterministic state machine defined by the InputState enum that transitions from Empty to Composing on the first printable key, advances to Conversion when Space triggers candidate generation, and returns to Empty upon text commit or cancellation.

Karukan is a Rust-based Japanese input method engine that implements a lightweight, deterministic state machine to manage the lifecycle of user input. The core logic resides in the InputState enum defined in karukan-im/src/core/state.rs, which governs the three-phase transition from idle waiting to active composition and finally to candidate selection. Understanding these state transitions is essential for developers extending the engine or debugging input behavior.

The InputState Enum Structure

The state machine’s foundation is the InputState enum located in karukan-im/src/core/state.rs (lines 9-30). This enum defines three distinct variants that represent the engine’s current operational mode:

• Empty – The idle state where no input is present and the engine is waiting for the first keypress.
• Composing { preedit, romaji_buffer } – The user is actively building a pre-edit string, converting Romaji input to Hiragana, Katakana, or alphabet characters.
• Conversion { preedit, candidates } – The engine displays a candidate list and the user is selecting the final conversion for the composed text.

Empty to Composing: Initiating Input

When the engine is in the Empty state, InputMethodEngine::process_key in karukan-im/src/core/engine/mod.rs (lines 33-90) dispatches control to process_key_empty. Upon receiving the first printable key, the engine flushes any pending Romaji buffer and invokes set_composing_state() (lines 95-104) to transition the state.

The set_composing_state method constructs a Preedit from the current buffer and overwrites the engine’s state with InputState::Composing:

fn set_composing_state(&mut self) -> Preedit {
    let romaji_buffer = self.converters.romaji.buffer().to_string();
    let preedit = self.build_composing_preedit();
    self.state = InputState::Composing {
        preedit: preedit.clone(),
        romaji_buffer,
    };
    preedit
}

Any non-modifier key pressed while the engine is idle instantly creates a Composing state and begins tracking the Romaji buffer.

Composing to Conversion: Triggering Candidate Selection

While in the Composing state, most alphanumeric keys keep the engine in the same state, appending to the Romaji buffer. However, a conversion-trigger key—typically the Space key or Enter (defined in karukan-im/src/core/engine/input.rs, lines 180-215)—forces a transition to Conversion.

When detected, process_key_composing calls enter_conversion, which performs the following steps:

1. Flushes remaining Romaji to composed characters via flush_romaji_to_composed.
2. Runs the neural-kanji converter on the current buffer.
3. Constructs a CandidateList using build_candidate_list.
4. Replaces the engine state with InputState::Conversion { preedit, candidates }.

fn enter_conversion(&mut self) -> EngineResult {
    self.flush_romaji_to_composed();
    let preedit = self.build_conversion_preedit();
    let candidates = self.build_candidate_list(&preedit);
    self.state = InputState::Conversion { preedit, candidates };
    EngineResult::consumed().with_action(EngineAction::ShowCandidates)
}

Pressing Space after a sequence of Hiragana or Katakana characters displays the candidate window and transitions the engine into the Conversion state.

Conversion to Empty: Committing or Canceling

When the engine is in the Conversion state, process_key_conversion handles two primary exit paths:

• Candidate Selection: Interpreting Enter, Tab, or numeric keys extracts the chosen candidate’s text via commit_conversion, commits it to the editor, clears input_buf, and resets the state to Empty.
• Cancellation: Pressing Esc or aborting the conversion calls cancel_conversion, which discards the candidate list and falls back to Empty (or occasionally reverts to Composing if the user aborts early to preserve the pre-edit).

The commit_conversion method demonstrates the final transition:

fn commit_conversion(&mut self) -> EngineResult {
    let text = self.candidates().unwrap().selected_text().unwrap_or_default();
    self.input_buf.clear();
    self.state = InputState::Empty;
    EngineResult::consumed().with_action(EngineAction::Commit(text))
}

Every successful candidate selection or explicit cancellation ends the session and returns the engine to the idle Empty state.

Complete State Transition Example

The following Rust code demonstrates the full lifecycle of the Karukan state machine using the public API:

use karukan_im::InputMethodEngine;
use karukan_im::KeyEvent;
use karukan_im::Keysym;
use karukan_im::KeyModifiers;

// 1. Empty → Composing
let mut engine = InputMethodEngine::new();          // state = Empty
let a_key = KeyEvent::new(Keysym::KEY_A, true, KeyModifiers::default());
engine.process_key(&a_key);                        // state = Composing

// 2. Composing → Conversion (Space)
let space_key = KeyEvent::new(Keysym::KEY_SPACE, true, KeyModifiers::default());
engine.process_key(&space_key);                    // state = Conversion, candidates shown

// 3. Conversion → Empty (choose first candidate)
let enter_key = KeyEvent::new(Keysym::KEY_ENTER, true, KeyModifiers::default());
engine.process_key(&enter_key);                    // state = Empty, text committed

Summary

• InputState enum in karukan-im/src/core/state.rs defines the three phases: Empty, Composing, and Conversion.
• State dispatch occurs in InputMethodEngine::process_key in karukan-im/src/core/engine/mod.rs, which routes to specialized handlers for each state.
• Empty → Composing: Triggered by set_composing_state() after the first printable keypress.
• Composing → Conversion: Triggered by Space or Enter via enter_conversion() in karukan-im/src/core/engine/input.rs.
• Conversion → Empty: Achieved through commit_conversion() or cancel_conversion(), clearing the input buffer and returning to idle.

Frequently Asked Questions

What triggers the transition from Composing to Conversion?

Pressing the Space key (or Enter in some locales) triggers the enter_conversion() method, which flushes the Romaji buffer, runs the neural converter, and generates a candidate list, moving the state from Composing to Conversion.

Can I revert from Conversion back to Composing without committing?

Yes. Pressing Escape or aborting the conversion calls cancel_conversion(), which discards the candidate list and may revert to Composing to preserve the pre-edit text if the conversion was aborted before final selection.

Where is the state machine logic located in the repository?

The state definitions are in karukan-im/src/core/state.rs, while the transition logic is implemented in karukan-im/src/core/engine/mod.rs (dispatch and Empty/Composing handling) and karukan-im/src/core/engine/input.rs (conversion triggers and cancellation).

How does the engine handle the first keypress when Empty?

When in the Empty state, any printable key triggers process_key_empty, which immediately calls set_composing_state() to create a new Composing variant containing the initial Preedit and Romaji buffer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}