# How the Element Dataclass Captures Spatial Information in iOS Simulator Automation

> Learn how the Element dataclass captures spatial data like frame and coordinates for iOS Simulator automation. Discover its center property for efficient tapping.

- Repository: [Conor/ios-simulator-skill](https://github.com/conorluddy/ios-simulator-skill)
- Tags: deep-dive
- Published: 2026-02-27

---

**The `Element` dataclass stores spatial data in a `frame` dictionary containing `x`, `y`, `width`, and `height` coordinates, and exposes a `center` property that calculates the midpoint for tapping interactions.**

The `Element` dataclass defined in `conorluddy/ios-simulator-skill` models UI elements extracted from the iOS accessibility tree, enabling precise geometric interactions with simulator interfaces. By encapsulating frame data and coordinate calculation logic, this component bridges the gap between accessibility node metadata and actionable screen coordinates for automation scripts.

## The Frame Attribute: Core Spatial Storage

Spatial information in the `Element` dataclass resides in the `frame` attribute, a `dict[str, float]` that captures the element's geometric footprint on the device screen. The dataclass definition in [`ios-simulator-skill/scripts/navigator.py`](https://github.com/conorluddy/ios-simulator-skill/blob/main/ios-simulator-skill/scripts/navigator.py) (lines 73-84) declares this structure explicitly:

```python
@dataclass
class Element:
    type: str
    label: str | None
    value: str | None
    identifier: str | None
    frame: dict[str, float]      # ← holds x, y, width, height

    traits: list[str]
    enabled: bool = True

```

When the `Navigator` class flattens the accessibility tree, it constructs `Element` instances by extracting the raw frame dictionary directly from each node's `frame` field:

```python
element = Element(
    type=node.get("type", "Unknown"),
    label=node.get("AXLabel"),
    value=node.get("AXValue"),
    identifier=node.get("AXUniqueId"),
    frame=node.get("frame", {}),          # ← populated from the tree

    traits=node.get("traits", []),
    enabled=node.get("enabled", True),
)

```

The frame dictionary contains four required floating-point values measured in points: **`x`** (horizontal origin), **`y`** (vertical origin), **`width`** (horizontal extent), and **`height`** (vertical extent).

## Converting Frames to Actionable Coordinates

To translate static frame data into interactive tap points, the `Element` dataclass provides a **`center`** property that computes the geometric midpoint of the rectangle. This property returns a `tuple[int, int]` representing integer screen coordinates suitable for `idb ui tap` commands:

```python
@property
def center(self) -> tuple[int, int]:
    """Calculate center point for tapping."""
    x = int(self.frame["x"] + self.frame["width"] / 2)
    y = int(self.frame["y"] + self.frame["height"] / 2)
    return (x, y)

```

The calculation performs half-width and half-height additions to the origin coordinates, then converts the results to integers to match the discrete pixel grid expected by the iOS simulator's input system.

## Practical Usage Examples

### Creating Elements Manually

You can instantiate `Element` objects directly when simulating accessibility data or building test fixtures:

```python
from ios_simulator_skill.scripts.navigator import Element

# Simulated frame returned by the accessibility tree

frame_info = {"x": 120.0, "y": 250.0, "width": 80.0, "height": 30.0}

button = Element(
    type="Button",
    label="Submit",
    value=None,
    identifier="submitBtn",
    frame=frame_info,
    traits=["clickable"],
)

print(button.center)          # → (160, 265)

print(button.description)     # → Button "Submit"

```

### Finding and Tapping Elements via Navigator

In production automation, the `Navigator` class handles element discovery and interaction:

```python
from ios_simulator_skill.scripts.navigator import Navigator

nav = Navigator(udid=None)               # auto-detects the booted simulator

element = nav.find_element(text="Login")  # returns an Element instance

if element:
    print(f"Found {element.description} at {element.frame}")
    print(f"Tap coordinates: {element.center}")
    nav.tap(element)                     # uses the calculated center

```

### Custom Coordinate Transformations

For scenarios requiring offset taps or coordinate adjustments, access the raw frame data to compute modified positions:

```python

# Offset the tap by 5 points horizontally from center

x, y = element.center
adjusted = (x + 5, y)

nav.tap_at(*adjusted)    # direct tap at the shifted position

```

## Summary

- The **`frame`** attribute in the `Element` dataclass stores spatial data as a dictionary with `x`, `y`, `width`, and `height` keys measured in points.
- The **`center`** property calculates the geometric midpoint by adding half-dimensions to origin coordinates and returning integer tuples.
- The **`Navigator`** class populates frame data directly from iOS accessibility tree nodes when constructing `Element` instances.
- All spatial calculations occur in [`ios-simulator-skill/scripts/navigator.py`](https://github.com/conorluddy/ios-simulator-skill/blob/main/ios-simulator-skill/scripts/navigator.py), which serves as the primary interface for coordinate-based simulator interactions.

## Frequently Asked Questions

### What format does the frame dictionary use?

The frame dictionary uses string keys mapped to floating-point values: `{"x": float, "y": float, "width": float, "height": float}`. All measurements represent points on the device screen coordinate system, with the origin (0,0) located at the top-left corner.

### How does the center property calculate coordinates?

The `center` property computes the horizontal midpoint by adding half the width to the x-coordinate, and the vertical midpoint by adding half the height to the y-coordinate. It then converts both results to integers using Python's `int()` constructor to ensure compatibility with discrete pixel-based input systems.

### Can I access the raw frame data directly?

Yes, the `frame` attribute is a public dictionary accessible as `element.frame`. You can read individual values directly (e.g., `element.frame["x"]`) or use the entire dictionary for custom geometric calculations beyond the standard `center` property.

### Where is the Element dataclass defined?

The `Element` dataclass is defined in [`ios-simulator-skill/scripts/navigator.py`](https://github.com/conorluddy/ios-simulator-skill/blob/main/ios-simulator-skill/scripts/navigator.py) at lines 73-84 according to the source code. This file also contains the `Navigator` class responsible for building `Element` instances from accessibility tree nodes and executing tap commands based on the calculated coordinates.