How FluidVoice Uses macOS Accessibility APIs to Type Text into Other Applications

FluidVoice captures the focused UI element using system-wide AX APIs, then inserts text via a cascading pipeline of CGEvent Unicode injection, Accessibility attribute manipulation, and clipboard fallback, all orchestrated in TypingService.swift.

FluidVoice is an open-source macOS voice-to-text application from altic-dev/FluidVoice that inserts transcribed speech into any foreground application. Understanding how FluidVoice uses accessibility APIs to type into other applications requires examining the sophisticated hierarchy-traversal and attribute-manipulation techniques implemented in its Swift source code. The system prioritizes speed through low-level CGEvent injection while maintaining reliability via semantic Accessibility API fallbacks.

The Text Injection Architecture

The core typing logic resides in Sources/Fluid/Services/TypingService.swift, which implements a six-stage pipeline. This architecture first attempts high-performance synthetic key events, then escalates through progressively more robust Accessibility-based methods until the text successfully appears in the target application.

Capturing the Focused UI Element

The process begins with captureSystemFocusedPID, a static method defined in TypingService.swift (lines 31-39). This function creates a system-wide accessibility element using AXUIElementCreateSystemWide() and queries the kAXFocusedUIElementAttribute to identify which UI element currently holds keyboard focus.

Once identified, the service extracts the owning process identifier (PID) from the accessibility element. This PID becomes crucial for targeting subsequent CGEvent injections specifically at the intended application, preventing text from being routed to the wrong window.

The Multi-Strategy Insertion Pipeline

The main entry point insertTextInstantly (lines 40-108) implements a decision tree based on the user-configurable SettingsStore.TextInsertionMode. The method attempts insertion strategies in order of preference:

1. Direct CGEvent Unicode insertion targeting the focused PID via insertTextBulkInstant
2. Accessibility-based insertion into the focused element
3. HID-only CGEvent insertion for applications that reject standard event types
4. Clipboard paste (global or PID-targeted) via insertTextViaClipboard
5. Character-by-character fallback using individual key events

This tiered approach ensures compatibility across diverse application types, from native Cocoa apps to Electron-based interfaces that may swallow synthetic events.

Locating Text Elements via Accessibility

When CGEvent insertion fails, FluidVoice falls back to the Accessibility API to locate a valid text input element. The service employs three hierarchical discovery strategies implemented in TypingService.swift:

Strategy 1: Focused Element – getFocusedTextElement (lines 834-846) attempts to retrieve the currently focused accessibility element directly.

Strategy 2: UI Hierarchy Walk – findTextElementInFrontmostApp (lines 854-862) recursively traverses the accessibility tree of the active application, calling findTextElementRecursively (lines 864-888) to locate elements with the kAXTextFieldRole or kAXTextAreaRole.

Strategy 3: Keyboard-Focused Element – findKeyboardFocusedElement (lines 891-904) searches for elements specifically marked as keyboard-focused within the application's accessibility hierarchy.

Executing AX-Based Text Insertion

Once an AXUIElement candidate is located, the method tryAllTextInsertionMethods (lines 911-942) executes four distinct Accessibility-based write operations in sequence:

• Insert at current cursor using kAXSelectedTextRangeAttribute combined with kAXValueAttribute
• Direct value replacement by setting kAXValueAttribute directly
• Selection replacement via kAXSelectedTextAttribute
• Insertion point targeting using insertTextAtInsertionPoint

Cursor-Based Insertion

The most reliable Accessibility method involves cursor manipulation via insertTextAtCursorUsingSelectedRange (lines 1162-1188). This function reads the current text value using kAXValueAttribute, retrieves the current selection range via kAXSelectedTextRangeAttribute, constructs a new string with the inserted text at the correct position, writes the updated value back, and finally advances the caret position to reflect the insertion.

This approach maintains undo history and preserves existing text formatting, unlike direct value replacement which may overwrite entire fields.

Verification and Fallback Mechanisms

After each insertion attempt, FluidVoice verifies success through waitForFocusedTextVerification (lines 3100-3135). This method re-reads the target field's value, checks for caret movement, or times out to confirm the text actually appeared.

If all Accessibility strategies fail, the system falls back to insertTextViaClipboard, which temporarily replaces the system clipboard, sends a Cmd+V keystroke event, and restores the original clipboard contents after a short delay. The final fallback sends character-by-character CGEvent key events in a loop at the bottom of insertTextInstantly.

Practical Code Examples

Typing into the Currently Focused Application

import Fluid

let typing = TypingService()
typing.typeTextInstantly("Hello, world!")

The typeTextInstantly method automatically captures the focused PID, selects the optimal injection strategy based on current settings, and restores focus after completion.

Retrieving the Focused Process Identifier

if let pid = TypingService.captureSystemFocusedPID() {
    print("Focused element belongs to PID:", pid)
}

This extracts the process identifier from the system-wide focused accessibility element for targeted event injection.

Manual Accessibility Path Implementation

let service = TypingService()

// Find a text element using multiple strategies
guard let element = service.getFocusedTextElement()
      ?? service.findTextElementInFrontmostApp()
      ?? service.findKeyboardFocusedElement() else {
    fatalError("No text field found")
}

// Insert via cursor-based technique
let success = service.insertTextAtCursorUsingSelectedRange(
    element, 
    "Inserted via AX"
)
print("Insertion succeeded:", success)

This mirrors the internal fallback chain while providing explicit control over element selection.

Clipboard Fallback Usage

let service = TypingService()
let clipboardSuccess = service.insertTextViaClipboard("Clipboard paste text")
print("Clipboard paste succeeded:", clipboardSuccess)

This method handles the temporary clipboard swap and restoration automatically when Accessibility APIs fail.

Key Source Files

• Sources/Fluid/Services/TypingService.swift – Implements the complete text injection pipeline, from PID capture through Accessibility insertion and fallback strategies.
• Sources/Fluid/Persistence/SettingsStore.swift – Stores the TextInsertionMode preference (reliablePaste, directUnicode, etc.) that determines which injection path is attempted first.
• Sources/Fluid/Views/NotchContentViews.swift – Demonstrates focus restoration via TypingService.activateApp after the voice input overlay dismisses.
• Sources/Fluid/Views/BottomOverlayView.swift – Contains the user-facing entry point that triggers typeTextInstantly when transcription completes.

Summary

• FluidVoice uses AXUIElementCreateSystemWide in captureSystemFocusedPID to identify the target application via accessibility APIs.
• The service attempts CGEvent Unicode injection first for speed, then falls back to Accessibility attribute manipulation via kAXValueAttribute and kAXSelectedTextRangeAttribute.
• Three discovery strategies locate text elements: direct focus, recursive hierarchy traversal, and keyboard-focus search.
• Cursor-based insertion via insertTextAtCursorUsingSelectedRange provides the most reliable AX-based text entry by manipulating selection ranges.
• Verification logic in waitForFocusedTextVerification ensures text actually appears, with automatic fallback to clipboard paste or character-by-character input.

Frequently Asked Questions

What happens if an application blocks Accessibility APIs?

If an application restricts Accessibility access, FluidVoice falls back to CGEvent Unicode injection targeting the specific PID, or ultimately to clipboard paste via insertTextViaClipboard. The service sends a Cmd+V keystroke after temporarily replacing the clipboard content, then restores the original clipboard state, bypassing the need for AX permissions entirely.

How does FluidVoice handle caret positioning when inserting text?

The insertTextAtCursorUsingSelectedRange method reads the current kAXSelectedTextRangeAttribute to determine the insertion point, writes the new string by combining existing text with the insertion, and updates the selection range to position the caret after the inserted text. This preserves the user's cursor location and maintains undo stack integrity within the target application.

Why does FluidVoice prefer CGEvent over Accessibility APIs initially?

CGEvent Unicode injection via insertTextBulkInstant offers significantly lower latency and higher throughput than Accessibility API calls, making it ideal for rapid dictation. However, some applications (particularly those using non-standard input handling like certain Electron apps) may reject these synthetic events, necessitating the Accessibility fallback chain for universal compatibility.

Which Accessibility attributes does FluidVoice manipulate to insert text?

The service primarily uses kAXValueAttribute to read and write text content, kAXSelectedTextRangeAttribute to determine cursor position and selection bounds, and kAXSelectedTextAttribute to replace selected text directly. These attributes provide a semantic interface to the text field's content independent of the underlying rendering technology.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}