What Data Points Does test_recorder.py Capture for iOS Simulator Test Debugging?

The TestRecorder class in test_recorder.py captures 11 distinct data points per test step—including sequential identifiers, timestamps, UI element counts, accessibility hierarchies, screenshot references, and assertion results—to build a comprehensive debugging trail for iOS simulator automation.

The conorluddy/ios-simulator-skill repository provides a robust debugging framework through its TestRecorder implementation. Located at ios-simulator-skill/scripts/test_recorder.py, this module systematically records every interaction during iOS simulator test execution, storing granular details that enable precise post-failure analysis. Understanding exactly what data points test_recorder.py captures helps developers reconstruct UI states and timing conditions when tests fail.

Per-Step Data Captured by TestRecorder

The step() method aggregates detailed information into a step_data dictionary (appended to self.steps at line 250) each time a test action is recorded.

Step Identification and Timing

Every recorded step receives three fundamental tracking identifiers:

• Step number: A sequential integer (self.current_step) assigned at line 222 that maintains execution order.
• Description: Human-readable text explaining the action, captured at line 223.
• Timestamp: Precise seconds elapsed since recorder initialization (line 224), enabling performance analysis and temporal debugging.

UI State Documentation

To reconstruct the visual context of failures, the recorder captures structural UI data:

• Element count: The total number of UI elements detected in the accessibility tree (line 225), sourced via count_elements from common/__init__.py.
• Accessibility snapshot file name: The JSON filename containing the full accessibility hierarchy (line 226), generated by get_accessibility_tree in the common utilities.

Screenshot Metadata

Visual debugging data varies based on storage configuration:

• Screenshot mode: Indicates whether images are saved as "file" or embedded as Base64 "inline" (line 227).
• Screenshot size: Token-optimization level specifying resolution (full, half, quarter, or thumb) set at line 228.
• Screenshot reference: For file mode (lines 232-235), stores the PNG path and filename; for inline mode (lines 236-241), stores Base64-encoded image data and pixel dimensions.

Validation and Assertion Tracking

When tests include explicit checks:

• Assertion: The validation text describing the expected condition (line 243).
• Assertion result: Boolean pass/fail flag (True/False) recorded at line 244, indicating whether the check succeeded.

Custom Metadata

Developers can inject context via arbitrary key-value pairs:

• Metadata: Optional dictionary supplied by test authors (lines 247-248) for environment flags, user IDs, or other debugging context.

Global Test-Level Data Aggregation

When generate_report() executes, it compiles per-step data into comprehensive outputs. The method creates:

• Markdown report (report.md): Human-readable documentation written at lines 202-236.
• Metadata JSON (metadata.json): Machine-parseable file containing test name, total duration, overall timestamp, and the complete steps array (lines 238-250).

Practical Implementation Example

from ios_simulator_skill.scripts.test_recorder import TestRecorder

# Initialize recorder (UDID auto-detected via device_utils.py if omitted)

recorder = TestRecorder(
    test_name="Authentication Flow",
    output_dir="test_artifacts",
    inline=False,            # File-based screenshot storage

    screenshot_size="half",  # Token-efficient resolution

    app_name="MyApp"
)

# Record procedural steps

recorder.step(
    description="Launch application",
    screen_name="LaunchScreen",
    metadata={"build": "v2.1.0", "env": "staging"}
)

recorder.step(
    description="Verify login button visible",
    screen_name="LoginScreen",
    assertion="Login button exists"
)

# Generate debugging artifacts

paths = recorder.generate_report()
print(f"Debug report: {paths['markdown_path']}")
print(f"Raw data: {paths['metadata_path']}")

The resulting metadata.json structure follows this pattern:

{
  "test_name": "Authentication Flow",
  "duration": 8.5,
  "timestamp": "2026-02-27T14:32:10.123456",
  "steps": [
    {
      "number": 1,
      "description": "Launch application",
      "timestamp": 1.2,
      "element_count": 42,
      "accessibility": "001-launch-application.json",
      "screenshot_mode": "file",
      "screenshot_size": "half",
      "screenshot": "screenshots/001-launch-application.png",
      "screenshot_name": "001-launch-application.png",
      "metadata": { "build": "v2.1.0", "env": "staging" }
    }
  ]
}

Summary

• test_recorder.py captures 11 distinct data points per step through the TestRecorder class, including sequential numbering, timestamps, UI element counts, and accessibility snapshots from common/__init__.py.
• Screenshot metadata differentiates between file-based storage and inline Base64 encoding, with configurable resolution levels for token optimization.
• Assertion results and custom metadata enable developers to track validation states and inject environmental context into debug traces.
• Global test data aggregates all steps into metadata.json and report.md via generate_report(), creating a complete debugging pipeline.

Frequently Asked Questions

How does test_recorder.py handle screenshot storage for debugging?

The recorder supports two modes controlled by the inline parameter. When inline=False, screenshots save as PNG files to disk with paths recorded in the step data (lines 232-235). When inline=True, images encode as Base64 strings stored directly in the JSON metadata (lines 236-241), eliminating external file dependencies while increasing document size.

What accessibility information does TestRecorder capture?

Each step records the total count of UI elements via count_elements (line 225) and the filename of a JSON accessibility tree snapshot (line 226). This data is generated through utilities in common/__init__.py, allowing developers to reconstruct exact UI hierarchies and element relationships during post-mortem debugging.

Can developers add custom debugging context to test steps?

Yes. The step() method accepts an optional metadata parameter (lines 247-248) that accepts arbitrary key-value dictionaries. Test authors commonly use this for environment identifiers, build versions, or user credentials that aid in reproducing specific failure conditions across different test runs.

How does the recorder track test assertions?

When recording a step containing an assertion parameter, test_recorder.py stores the assertion text (line 243) alongside a boolean assertion_result flag (line 244). This creates an explicit pass/fail record within the step data, enabling automated analysis of which validation checks failed during test execution without parsing log files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}