iOS Simulator Parsed Accessibility Tree Representation: Complete Data Field Schema

The parsed accessibility tree representation retrieved via IDB contains eight core data fields: type, AXLabel, AXValue, AXUniqueId, frame, traits, enabled, and children, with type, frame, and children always present while others are optional.

The conorluddy/ios-simulator-skill repository provides Python utilities for interacting with iOS Simulator accessibility data. Understanding the parsed accessibility tree representation is essential for building automation scripts, accessibility audits, and UI navigation tools that interact with iOS applications through the Simulator.

Core Data Fields in the Accessibility Tree

The accessibility tree is fetched from IDB using get_accessibility_tree() in ios-simulator-skill/scripts/common/idb_utils.py. Each UI element in the returned JSON structure exposes a consistent schema with both mandatory and optional attributes.

Always Present Fields

type — The UI element's class identifier (e.g., Window, Button, TextField, NavigationBar). This field determines the element's role in the interface hierarchy according to the docstring at lines 35-40 of idb_utils.py.

frame — Geometry data containing the element's position and dimensions as { "x": int, "y": int, "width": int, "height": int }. For the root node, this defaults to the full screen size.

children — Array of nested child element objects enabling recursive tree traversal. Leaf nodes return an empty list [].

Optional Element Attributes

AXLabel — Human-readable accessibility label exposed to VoiceOver. May be empty string when not configured.

AXValue — Current value content (e.g., text entered in a field, switch state, or slider position).

AXUniqueId — Stable accessibilityIdentifier used for reliable element targeting across sessions. Critical for automation scripts that require consistent element references.

traits — List of accessibility traits describing behavior (e.g., button, selected, staticText, image).

enabled — Boolean flag indicating interaction capability. Defaults to True when omitted from the JSON response.

Tree Retrieval and Structure

The get_accessibility_tree() function in idb_utils.py handles the IDB communication and returns the nested JSON structure. The root node typically represents the application window:

{
  "type": "Window",
  "AXLabel": "App Name",
  "frame": { "x": 0, "y": 0, "width": 390, "height": 844 },
  "children": [ … ]
}

Traversal utilities like flatten_tree(), count_elements(), and get_screen_size() in idb_utils.py (lines 71-115) consume these fields to provide flattened lists, element counts, and screen dimensions respectively.

Practical Code Examples

Fetching the Raw Tree Structure

from ios_simulator_skill.scripts.common.idb_utils import get_accessibility_tree

tree = get_accessibility_tree()
print("Root type:", tree["type"])
print("Label:", tree.get("AXLabel"))
print("Frame:", tree["frame"])
print("Children count:", len(tree.get("children", [])))

Traversing All Elements with Key Attributes

from ios_simulator_skill.scripts.common.idb_utils import flatten_tree

flat = flatten_tree(get_accessibility_tree())
for el in flat:
    print(
        f"{'  '*el['depth']}{el['type']}: "
        f"label={el.get('AXLabel')!r} "
        f"value={el.get('AXValue')!r} "
        f"id={el.get('AXUniqueId')!r} "
        f"enabled={el.get('enabled', True)}"
    )

Filtering Interactive Elements

from ios_simulator_skill.scripts.common.idb_utils import flatten_tree

INTERACTIVE = {"Button", "Link", "TextField", "SecureTextField", "Cell", "Switch"}
interactive = [
    e for e in flatten_tree(get_accessibility_tree())
    if e["type"] in INTERACTIVE and e.get("enabled", True)
]
print(f"Found {len(interactive)} interactive UI elements.")

Source Files Consuming the Tree Schema

According to the conorluddy/ios-simulator-skill source code, these files reference the parsed accessibility tree representation:

• scripts/common/idb_utils.py — Core utilities (get_accessibility_tree, flatten_tree, count_elements) that define and parse the schema (lines 35-40, 71-115).

• scripts/screen_mapper.py — Analyzes AXLabel, AXValue, AXUniqueId, enabled, and traits to generate screen summaries (lines 133-138).

• scripts/navigator.py — Uses the same field set for actionable UI navigation including find-tap operations and text entry (lines 124-130).

• scripts/accessibility_audit.py — Performs WCAG-style checks using AXLabel, traits, and enabled fields (lines 41-55).

Summary

• The parsed accessibility tree representation in conorluddy/ios-simulator-skill contains eight standardized fields: type, frame, children (always present), plus AXLabel, AXValue, AXUniqueId, traits, and enabled (optional).
• Root nodes represent application windows with full-screen frame dimensions.
• Stable targeting relies on AXUniqueId (accessibilityIdentifier), while human-readable identification uses AXLabel.
• The children array enables recursive tree traversal for complex interface analysis.
• Core parsing logic resides in idb_utils.py, with consumption patterns visible in screen_mapper.py, navigator.py, and accessibility_audit.py.

Frequently Asked Questions

What is the difference between AXLabel and AXValue in the accessibility tree?

AXLabel provides the static accessibility description announced by VoiceOver (e.g., "Submit Button"), while AXValue contains dynamic content state (e.g., the entered text "user@example.com" or a switch's "1" for on). Labels identify the element's purpose; values represent current data.

How do you reliably target specific elements using the parsed accessibility tree?

Use the AXUniqueId field, which maps to the iOS accessibilityIdentifier property. Unlike labels that may change with localization or values that change with content, AXUniqueId remains stable across app sessions and locales, making it the preferred target for automation scripts in navigator.py.

What coordinate system does the frame field use?

The frame object uses screen points relative to the application window's origin (0,0) with properties x, y, width, and height as integers. The root window frame matches the simulated device dimensions (e.g., 390×844 for iPhone 14), and child elements provide coordinates relative to this space.

How can you determine if a UI element supports interaction?

Check the enabled boolean (defaults to True if absent) and the type field against known interactive classes. As shown in accessibility_audit.py and navigation scripts, interactive types include Button, TextField, SecureTextField, Switch, Cell, and Link. The traits array may also contain interaction hints like button or allowsDirectInteraction.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}