How keyboard.py Emulates Special Keys and Keyboard Layouts in the iOS Simulator Skill

The keyboard.py script emulates special keys by mapping human-readable names to iOS HID usage codes and invoking IDB commands, while delegating layout-specific character entry to the simulator's own input system.

The keyboard.py module in the conorluddy/ios-simulator-skill repository provides a Python wrapper around IDB (iOS Device Bridge) to programmatically control keyboard input in the iOS Simulator. It handles both special keys (like Return, Delete, and arrows) through deterministic HID code translation, and normal text entry by leveraging the simulator's active keyboard layout. This approach allows automation scripts to interact with iOS apps using either physical key emulation or layout-aware character input.

HID Usage Code Mapping for Special Keys

The foundation of special key emulation in keyboard.py relies on a static mapping of friendly names to hardware-level HID usage codes required by IDB.

The SPECIAL_KEYS Dictionary

At lines 80-92 in scripts/keyboard.py, the script defines a dictionary named SPECIAL_KEYS that translates human-readable key identifiers into the integer codes iOS expects for hardware key events. This mapping includes navigation keys, editing keys, and modifier keys:

• "return" maps to 40
• "delete" maps to 42
• Arrow keys (left, right, up, down) map to codes 79 through 82
• Modifier keys like "command", "option", and "shift" have their own distinct codes

This static table ensures that special key behavior remains consistent regardless of the simulator's current software keyboard layout or language settings.

Single Key Presses via press_key()

The press_key() function (lines 54-71) handles individual key events by looking up the requested key name in the SPECIAL_KEYS dictionary. The lookup is case-insensitive, and the function falls back to treating the input as a raw integer if the name is not found in the mapping.

Once resolved, the function constructs and executes an IDB command:

cmd = ["idb", "ui", "key", str(key_code)]

The script executes this via subprocess.run(), capturing output and raising exceptions on failure to ensure automation scripts fail fast when key events cannot be delivered.

Key Sequences and Combinations

Beyond single keystrokes, keyboard.py supports complex input patterns through sequence and combination methods that chain multiple HID codes together.

Sequential Key Events with press_key_sequence()

The press_key_sequence() function (lines 86-100) accepts a list of key names, resolves each to its corresponding HID code using the same SPECIAL_KEYS dictionary, and issues a single optimized IDB command:

idb ui key-sequence <code1> <code2> ...

This approach efficiently emulates ordered shortcuts like "Cmd + C" or navigation patterns like "Down arrow, then Return" without spawning multiple subprocess calls.

Modifier Key Combinations via press_key_combo()

Because IDB does not natively support simultaneous modifier presses, the press_key_combo() function (lines 66-73) translates modifier-plus-character patterns into key sequences that achieve the same functional result. For example, the combination ["command", "a"] becomes a sequence of the two respective HID codes sent in rapid succession, effectively selecting all text in the target application.

This workaround allows the script to support common shortcuts (Copy, Paste, Select All) while maintaining compatibility with IDB's command structure.

Keyboard Layout Handling

Unlike special keys, character entry follows the simulator's active layout. The script does not implement translation tables for different physical layouts (QWERTY vs. AZERTY vs. QWERTZ). Instead, it relies on IDB's ui text command for normal character input.

When you call type_text("Café"), the script passes the string directly to IDB, which respects the simulator's current input language and keyboard layout settings. This means the same script produces different hardware key events on a simulator set to French AZERTY versus one set to German QWERTZ, matching real-world user behavior.

Layout-specific behavior for special keys remains deterministic via the HID mapping table, while character production varies according to the iOS software keyboard state.

CLI and Programmatic Usage

You can invoke keyboard.py from the command line or import it as a module in automation scripts.

Command Line Examples

# Press a single special key (Enter)

python scripts/keyboard.py --key return --udid <SIM_UDID>

# Press Delete three times

python scripts/keyboard.py --key delete --count 3 --udid <SIM_UDID>

# Send a key sequence: Cmd+A (select all) then Delete (clear)

python scripts/keyboard.py --key-sequence command,a,delete --udid <SIM_UDID>

Python API Integration

from ios_simulator_skill.scripts.keyboard import KeyboardController

# Initialize with target simulator UDID

controller = KeyboardController(udid="ABC123DEF456")

# Type normal text (layout-dependent)

controller.type_text("Café")

# Press the left-arrow key twice

controller.press_key("left", count=2)

# Use a combo to copy (Cmd+C)

controller.press_key_combo(["command", "c"])

Summary

• keyboard.py wraps IDB commands to inject keyboard events into running iOS simulators.
• Special keys are emulated via a static SPECIAL_KEYS dictionary mapping names (like "return" or "delete") to iOS HID usage codes (lines 80-92).
• press_key() resolves names to codes and executes idb ui key commands (lines 54-71).
• Sequences and combinations use press_key_sequence() and press_key_combo() to chain multiple HID codes into single IDB invocations (lines 66-73, 86-100).
• Keyboard layouts are not hardcoded; normal text entry uses idb ui text, which respects the simulator's current input language and software keyboard layout.

Frequently Asked Questions

How does keyboard.py handle keys that aren't in the SPECIAL_KEYS dictionary?

If press_key() receives a key name not found in the SPECIAL_KEYS dictionary, it attempts to parse the input as a literal integer HID code. This fallback allows direct access to any HID usage code supported by IDB without requiring updates to the mapping table.

Can keyboard.py emulate hardware keyboard layouts like Dvorak or Colemak?

No, the script does not implement physical layout translation tables. For character entry, it delegates to IDB's ui text command, which interacts with the iOS software keyboard. The hardware layout perceived by the simulator depends on the host Mac's keyboard settings and how iOS interprets them, not on keyboard.py's configuration.

What is the difference between press_key_sequence and press_key_combo?

press_key_sequence() sends keys in a specific order with implicit delays suitable for navigation or multi-step shortcuts, while press_key_combo() specifically handles modifier-plus-character patterns (like Command+C) by translating them into sequences since IDB lacks native simultaneous key press support. Both methods ultimately invoke idb ui key-sequence, but press_key_combo() provides semantic clarity for modifier-based actions.

Why does the script use HID usage codes instead of ASCII or Unicode values?

iOS Device Bridge expects HID usage codes to emulate physical hardware key events at the system level. These codes represent physical key identities (like "the Return key" or "the Delete key") rather than character values, ensuring consistent behavior across different keyboard layouts and languages for special keys, while allowing the operating system to handle character translation for text entry.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}