# MCP Servers for OS-Level UI Automation: 7 Tools for Desktop Control

> Discover 7 MCP servers for OS-level UI automation. Control mouse keyboard windows and inspect accessibility trees on Linux Windows and macOS.

- Repository: [Frank Fiegel/awesome-mcp-servers](https://github.com/punkpeye/awesome-mcp-servers)
- Tags: tutorial
- Published: 2026-08-31

---

**Seven MCP servers in the awesome-mcp-servers repository enable genuine OS-level UI automation across Linux, Windows, and macOS, exposing tools for screenshots, mouse/keyboard injection, window management, and accessibility tree inspection.**

The punkpeye/awesome-mcp-servers repository maintains a curated OS Automation category that catalogs Model Context Protocol (MCP) servers capable of controlling desktop operating systems without application-specific integrations. According to the source code analysis of [`README.md`](https://github.com/punkpeye/awesome-mcp-servers/blob/main/README.md) (lines 2863–2877), these servers translate MCP tool calls into native OS actions, allowing AI agents to observe and manipulate graphical user interfaces directly.

## Linux UI Automation MCP Servers

### GNOME Desktop Control (sbuysse/gnome-desktop-mcp)

The **sbuysse/gnome-desktop-mcp** server provides comprehensive automation for GNOME desktop environments versions 45–49. Located in the OS Automation section at line 2867, this server exposes tools for **screenshots**, **window management**, **mouse and keyboard injection**, **clipboard** operations, **workspace** manipulation, and **notification** handling.

### Hybrid Computer-Use Runtime (dimpagk92/cellar)

**dimpagk92/cellar** operates on macOS (Apple Silicon) and Linux through a hybrid "computer-use" architecture that merges accessibility tree data, Chrome DevTools Protocol, and computer vision. As documented at line 2868, this server exposes four high-level MCP tools: `see`, `act`, `think`, and `perceive`, enabling both UI observation and interaction at the OS level.

## Windows UI Automation MCP Servers

### Native Windows UI Automation (Harusame64/desktop-touch-mcp)

**Harusame64/desktop-touch-mcp** targets native Windows UI using the UI Automation (UIA) framework and Chrome DevTools Protocol (CDP). Listed at lines 2869–2870, this server provides **entity-based clicking**, keyboard control, mouse manipulation, clipboard access, terminal control, and UI notifications.

### Remote Windows Control (glasswarp/mcp-server)

The **glasswarp/mcp-server** enables automation of any Windows PC—local or remote over LAN—without requiring source code changes to target applications. Documented at line 2870, its capabilities include **UI observation** via screenshots and UIA, clicking, typing, dragging, scrolling, and application launching.

### WPF Application Automation (faze79/WPFVisualTreeMcp)

For .NET ecosystems, **faze79/WPFVisualTreeMcp** specializes in running WPF desktop applications. According to line 2874, this server can inspect the **visual tree**, read and write properties, capture screenshots with pop-up handling, perform clicks, select items, input text, and execute keyboard shortcuts.

## macOS UI Automation MCP Servers

### Window Management & OCR (ostapondo/Plonk)

**ostapondo/Plonk** provides macOS-specific window manager and menu-bar utilities. Detailed at line 2876, this server exposes tools to list monitors and windows, arrange layouts, snap windows, save workspaces, keep the system awake, measure screen dimensions, capture window-specific screenshots, annotate images, and perform **on-device OCR**—all callable through standard MCP tools.

## Cross-Platform UI Automation

### Universal GUI Automation (fixed-width/glass)

**fixed-width/glass** offers the broadest platform support, running on Linux, macOS, Windows, Android, and iOS without requiring modifications to target applications. As noted at line 2875, this server provides tools to launch applications, capture screenshots, read accessibility trees, inject mouse/keyboard gestures, tail logs, and perform frame diffs.

## Implementing UI Automation with MCP Tools

MCP clients discover automation capabilities through the standard `tools/list` method, then invoke specific actions using the `tools/call` method. The exact tool names and schemas vary by server, but the JSON-RPC pattern remains consistent.

### Capturing Screenshots

To capture the current screen on a GNOME desktop:

```json
{
  "method": "tools/call",
  "params": {
    "name": "screenshot",
    "arguments": {}
  }
}

```

*Result:* Returns a base64-encoded PNG of the current display.

### Injecting Mouse Events

To click a specific UI element on Windows using element selectors:

```json
{
  "method": "tools/call",
  "params": {
    "name": "click",
    "arguments": {
      "selector": "window.title='Calculator' && element.id='Button5'"
    }
  }
}

```

*Result:* Returns a success flag and any UI state changes.

### Listing Active Windows

To enumerate visible windows using the cross-platform glass server:

```json
{
  "method": "tools/call",
  "params": {
    "name": "list_windows",
    "arguments": {}
  }
}

```

*Result:* Returns a JSON array of objects containing `id`, `title`, and `bounds` for each window.

### Performing OCR on UI Elements

To extract text from a specific screen region on macOS:

```json
{
  "method": "tools/call",
  "params": {
    "name": "ocr_region",
    "arguments": {
      "window_name": "Notes",
      "region": {"x": 10, "y": 20, "width": 300, "height": 120}
    }
  }
}

```

*Result:* Returns extracted text strings with confidence scores.

## Server Configuration and Manifests

Each OS automation server defines its concrete tool signatures in an [`mcp_manifest.json`](https://github.com/punkpeye/awesome-mcp-servers/blob/main/mcp_manifest.json) file located in the repository root. These manifests specify `input_schema` and `output_schema` definitions that MCP clients use to validate requests. The [`README.md`](https://github.com/punkpeye/awesome-mcp-servers/blob/main/README.md) file (lines 2863–2877) serves as the authoritative index, mapping platform capabilities to specific GitHub repositories.

## Summary

- **Seven specialized MCP servers** provide OS-level UI automation according to the awesome-mcp-servers repository.
- **Platform coverage** includes Linux (GNOME), Windows (native UIA, WPF, remote), macOS (window management, OCR), and cross-platform implementations.
- **Core capabilities** span screenshots, mouse/keyboard injection, window management, accessibility tree inspection, and clipboard operations.
- **Implementation pattern** follows standard MCP protocol: clients call `tools/list` to discover capabilities, then `tools/call` with JSON arguments to execute actions.
- **Source reference** lines 2863–2877 in [`README.md`](https://github.com/punkpeye/awesome-mcp-servers/blob/main/README.md) contain the complete OS Automation category listing.

## Frequently Asked Questions

### Which MCP servers support cross-platform UI automation?

**fixed-width/glass** is the only server in the OS Automation category that supports Linux, macOS, Windows, Android, and iOS simultaneously. According to line 2875, it requires no source code changes to target applications and provides universal tools for screenshot capture, accessibility tree reading, and input injection across all supported platforms.

### How do MCP servers inject keyboard and mouse events on Windows?

Windows-specific servers use different native APIs. **Harusame64/desktop-touch-mcp** utilizes the **UI Automation (UIA) framework** and Chrome DevTools Protocol for entity-based interaction (line 2869), while **glasswarp/mcp-server** implements direct input injection for clicking, typing, dragging, and scrolling operations (line 2870). Both expose these capabilities as standard MCP tools callable by any compatible client.

### Can these servers automate applications without accessible APIs?

Yes. **dimpagk92/cellar** (line 2868) specifically addresses this by merging accessibility tree data with **computer vision** capabilities, allowing it to interact with applications lacking proper accessibility implementations. Similarly, **fixed-width/glass** performs frame differencing and visual analysis to automate arbitrary GUI apps regardless of their accessibility support.

### What is the difference between gnome-desktop-mcp and Plonk?

While both provide OS-level automation, **sbuysse/gnome-desktop-mcp** targets **Linux GNOME** environments (versions 45–49) with deep desktop integration including workspace and notification management (line 2867). **ostapondo/Plonk** focuses exclusively on **macOS**, offering specialized window tiling, workspace saving, and on-device OCR capabilities not available in the GNOME implementation (line 2876).