How Fuzzy Text Matching Works in navigator.py: iOS Simulator Automation

Fuzzy text matching in navigator.py implements a lightweight case-insensitive substring search across concatenated accessibility labels and values, controlled by a boolean fuzzy flag in the find_element method.

The conorluddy/ios-simulator-skill repository provides Python utilities for automating iOS Simulator interactions, with navigator.py serving as the primary interface for locating UI elements. The fuzzy text matching mechanism enables flexible element discovery by searching for substring matches within accessibility metadata, contrasting with the stricter exact matching mode available through the same API.

The Fuzzy Matching Algorithm

The Navigator.find_element method in ios-simulator-skill/scripts/navigator.py implements fuzzy matching through a three-stage process that processes the iOS accessibility tree.

Element Extraction and Flattening

Before comparison begins, the accessibility hierarchy is flattened into a linear list of candidate elements. The _flatten_tree function (imported from common.py) recursively traverses the nested accessibility structure returned by the iOS Simulator, producing a flat list of Element objects containing label, value, and enabled properties.

The find_element method iterates this list, immediately discarding disabled elements and optionally filtering by element type or identifier before performing text comparisons.

Text Comparison Logic (Lines 84-95)

When the text parameter is provided, the method constructs a searchable string and selects comparison logic based on the fuzzy boolean flag:

# navigator.py – lines 84-95

if text:
    elem_text = (elem.label or "") + " " + (elem.value or "")
    if fuzzy:
        # fuzzy = case‑insensitive substring search

        if text.lower() not in elem_text.lower():
            continue
    elif text not in (elem.label, elem.value):
        # exact match (case‑sensitive)

        continue

Fuzzy mode concatenates the element's label and value fields with a space separator, then performs a case-insensitive substring check using Python's lower() method. If the normalized search text exists anywhere within the normalized element text, the element qualifies as a match.

Exact mode bypasses case conversion and substring logic, requiring the search text to match either the label or value field exactly, preserving case sensitivity.

CLI Argument Handling (Lines 305-307)

The fuzzy behavior is exposed through command-line arguments that set the matching mode when invoking the script:

# navigator.py – argument parsing (lines 305-307)

parser.add_argument("--find-text", help="Find element by text (fuzzy match)")
parser.add_argument("--find-exact", help="Find element by exact text")

When users specify --find-text, the script invokes find_element with fuzzy=True. The --find-exact argument triggers fuzzy=False, enforcing strict matching rules.

Practical Usage Examples

Fuzzy Matching via Command Line

To locate and tap the first button containing "login" regardless of case or surrounding text:

python scripts/navigator.py \
    --find-text "login" \
    --tap \
    --udid <simulator-udid>

This command searches all enabled elements for the substring "login" in a case-insensitive manner.

Exact Matching via Command Line

To find an element with the precise label "Login" (case-sensitive):

python scripts/navigator.py \
    --find-exact "Login" \
    --tap \
    --udid <simulator-udid>

This mode requires the accessibility label to match exactly, ignoring elements with values like "loginButton" or "LOGIN".

Programmatic API Access

Import the Navigator class directly for custom automation scripts:

from ios_simulator_skill.scripts.navigator import Navigator

nav = Navigator(udid="ABC123")

# Fuzzy search finds "Login", "login button", or "user login"

element = nav.find_element(text="login", fuzzy=True)

# Exact search matches only "Login"

element = nav.find_element(text="Login", fuzzy=False)

Integration with Accessibility Tree Processing

The fuzzy matching system relies on utilities defined in ios-simulator-skill/scripts/common.py. The flatten_tree and get_accessibility_tree functions retrieve and normalize the iOS Simulator's accessibility hierarchy into the Element objects that navigator.py searches.

This architecture separates tree traversal concerns from matching logic, allowing find_element to focus purely on text comparison while common.py handles the complexity of iOS Simulator communication and data structure normalization.

Summary

• Fuzzy matching in navigator.py uses simple case-insensitive substring comparison, not complex algorithms like Levenshtein distance.
• The search combines label and value fields into a single string: (elem.label or "") + " " + (elem.value or "").
• CLI control: --find-text enables fuzzy mode (default), while --find-exact forces case-sensitive exact matching.
• Disabled elements are automatically excluded from search results before text comparison occurs.
• The method is implemented in ios-simulator-skill/scripts/navigator.py at lines 84-95, with CLI parsing at lines 305-307.

Frequently Asked Questions

What algorithm does navigator.py use for fuzzy matching?

The implementation uses a lightweight case-insensitive substring search. It converts both the search text and the concatenated element label/value to lowercase using Python's lower() method, then checks for substring containment with the in operator. This differs from advanced fuzzy matching algorithms like Levenshtein distance or trigram matching.

How do I perform a case-sensitive search with Navigator?

Use the --find-exact CLI argument or pass fuzzy=False to the find_element method. This mode requires the search text to match either the element's label or value field exactly, preserving original casing and disallowing partial matches.

What fields are searched during fuzzy matching?

The algorithm searches a concatenated string containing the element's accessibility label and value properties. These fields are joined with a space separator: (elem.label or "") + " " + (elem.value or ""). The search examines both fields simultaneously rather than individually.

Where does navigator.py get the element list for matching?

The element list originates from ios-simulator-skill/scripts/common.py, specifically through the flatten_tree and get_accessibility_tree functions. These utilities query the iOS Simulator's accessibility API and normalize the hierarchical tree structure into a flat list of Element objects that find_element iterates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}