How HID Keyboard and Mouse Simulation Works in scrcpy: UHID vs AOA Explained

scrcpy simulates physical USB HID devices by translating SDL input events into standard HID reports and transmitting them via either the UHID virtual device interface (Linux/Android 8+) or the Android Open Accessory (AOA) USB protocol.

HID keyboard and mouse simulation in scrcpy allows your computer's input devices to appear as physical USB peripherals to the Android device, bypassing the standard Android API injection layer. According to the Genymobile/scrcpy source code, this capability is implemented through two independent transport stacks that convert SDL events into binary HID reports and deliver them at the USB level.

Architecture Overview

scrcpy implements HID simulation through two distinct transport mechanisms:

• UHID (User-space HID): Creates virtual HID devices on the Android device using kernel drivers, communicating via the scrcpy control socket.
• AOA (Android Open Accessory): Registers the host as a USB accessory and pushes HID reports directly over USB control transfers using libusb.

Both stacks share the same HID report generation logic but use different transport implementations.

The UHID Stack (Linux and Android 8+)

The UHID stack creates virtual HID devices on the Android side by sending control messages from the scrcpy client. This method requires Linux or Android 8.0 (API 26) and above.

Device Registration and HID Descriptors

When initializing a UHID keyboard, sc_keyboard_uhid_init() in app/src/uhid/keyboard_uhid.c generates a HID report descriptor using sc_hid_keyboard_generate_open() and sends it via a SC_CONTROL_MSG_TYPE_UHID_CREATE control message.

// sc_keyboard_uhid_init() – lines 43-55
sc_hid_keyboard_generate_open(&hid_open);
msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
msg.uhid_create.id = SC_HID_ID_KEYBOARD;
msg.uhid_create.report_desc = hid_open.report_desc;
msg.uhid_create.report_desc_size = hid_open.report_desc_size;
sc_controller_push_msg(controller, &msg);

Input Report Generation and Transmission

Each SDL key event is converted to an HID input report by sc_hid_keyboard_generate_input_from_key(). The resulting report is wrapped in a SC_CONTROL_MSG_TYPE_UHID_INPUT message and pushed to the server via sc_keyboard_uhid_send_input().

// sc_key_processor_process_key() – lines 71-88
if (sc_hid_keyboard_generate_input_from_key(&kb->hid, &hid_input, event)) {
    sc_keyboard_uhid_send_input(kb, &hid_input);
}

// sc_keyboard_uhid_send_input() – lines 20-27
msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
msg.uhid_input.id = hid_input->hid_id;
memcpy(msg.uhid_input.data, hid_input->data, hid_input->size);
sc_controller_push_msg(kb->controller, &msg);

The mouse implementation follows an identical pattern using sc_mouse_uhid_init(), sc_hid_mouse_generate_input_from_motion(), and sc_mouse_uhid_send_input() defined in app/src/uhid/mouse_uhid.c.

Handling Output Reports (LED States)

The Android side can send LED state changes (NumLock, CapsLock, ScrollLock) via SC_CONTROL_MSG_TYPE_UHID_OUTPUT messages. These are received by sc_keyboard_uhid_process_hid_output() in app/src/uhid/keyboard_uhid.c, which updates the client's internal modifier state.

// sc_keyboard_uhid_process_hid_output() – lines 4-20
uint8_t hid_led = data[0];
kb->device_mod = sc_keyboard_uhid_to_sc_mod(hid_led);

The AOA Stack (Android Open Accessory Protocol)

The AOA stack uses the Android Open Accessory protocol version 2 to register HID interfaces directly over USB control transfers, requiring no scrcpy server process on the device.

HID Registration via USB Control Transfers

The client registers a HID interface using sc_aoa_register_hid() in app/src/usb/aoa_hid.c, which sends a USB control transfer to the Android accessory framework.

// sc_aoa_register_hid() – lines 97-108
request_type = LIBUSB_REQUEST_TYPE_VENDOR | LIBUSB_RECIPIENT_DEVICE | LIBUSB_ENDPOINT_OUT;
libusb_control_transfer(usb->handle, request_type, 51, accessory_id, 0, NULL, 0, 1000);

Following registration, the HID report descriptor is uploaded using sc_aoa_set_hid_report_desc().

Pushing Input Reports Over USB

Mouse motion events are processed by the sc_mouse_aoa object in app/src/usb/mouse_aoa.c. Each event is converted to an HID input report using sc_hid_mouse_generate_input_from_motion() and transmitted via sc_aoa_push_input().

// mouse_aoa.c – process_mouse_motion()
struct sc_hid_input hid_input;
sc_hid_mouse_generate_input_from_motion(&hid_input, event);
sc_aoa_push_input(mouse->aoa, &hid_input);

When the device detaches, the client issues sc_aoa_push_close() to clean up the HID interface. The keyboard AOA implementation in app/src/usb/keyboard_aoa.c follows identical registration and transmission patterns.

Control Message Plumbing

All UHID messages travel over the scrcpy control socket. The generic dispatcher in app/src/control_msg.c serializes and deserializes the SC_CONTROL_MSG_TYPE_UHID_* variants, forwarding them to the server side.

// control_msg.c – handling of UHID messages
case SC_CONTROL_MSG_TYPE_UHID_CREATE: { … }
case SC_CONTROL_MSG_TYPE_UHID_INPUT:  { … }
case SC_CONTROL_MSG_TYPE_UHID_DESTROY:{ … }

Practical Implementation Examples

The following examples demonstrate how to initialize HID devices and send input reports using scrcpy's internal APIs.

Initializing a UHID Mouse and Sending Motion

/* -------------------------------------------------
 *  Example: initialise a UHID mouse and send a motion
 * ------------------------------------------------- */
struct sc_controller *ctrl = …;           // already created by scrcpy client
struct sc_mouse_uhid mouse;

/* Initialise the UHID mouse – registers it on the device */
if (!sc_mouse_uhid_init(&mouse, ctrl)) {
    fprintf(stderr, "UHID mouse init failed\n");
    exit(1);
}

/* Build a motion event (e.g. from SDL) */
struct sc_mouse_motion_event ev = {
    .xrel = 10,          // move right 10 units
    .yrel = -5,          // move up 5 units
    .buttons_state = SC_MOUSE_BUTTON_LEFT,
};

/* Convert the motion to an HID report and push it */
struct sc_hid_input hid_input;
sc_hid_mouse_generate_input_from_motion(&hid_input, &ev);
sc_mouse_uhid_send_input(&mouse, &hid_input, "mouse motion");

Initializing an AOA Keyboard and Sending a Key Press

/* -------------------------------------------------
 *  Example: initialise an AOA keyboard and send 'A'
 * ------------------------------------------------- */
struct sc_aoa aoa;                     // already opened with USB accessory
struct sc_keyboard_aoa kb;

/* Register the HID interface (accessory_id = 1 for keyboard) */
sc_keyboard_aoa_init(&kb, &aoa);       // registers internally

/* Build a key‑down event for the 'A' scancode (0x04) */
struct sc_key_event key = {
    .scancode = SC_SCANCODE_A,
    .action   = SC_ACTION_DOWN,
    .mods_state = 0,
    .repeat   = false,
};

/* Generate the HID report and push it */
struct sc_hid_input hid_input;
if (sc_hid_keyboard_generate_input_from_key(&kb.hid, &hid_input, &key)) {
    sc_aoa_push_input(&aoa, &hid_input);
}

Key Source Files and References

File	Role	Link
app/src/hid/hid_mouse.c	Builds mouse input reports and descriptor	hid_mouse.c
app/src/hid/hid_keyboard.c	Builds keyboard input reports and descriptor	hid_keyboard.c
app/src/uhid/mouse_uhid.c	UHID mouse glue: init, send, control‑msg creation	mouse_uhid.c
app/src/uhid/keyboard_uhid.c	UHID keyboard glue, including output handling	keyboard_uhid.c
app/src/usb/mouse_aoa.c	AOA mouse implementation (register, push)	mouse_aoa.c
app/src/usb/keyboard_aoa.c	AOA keyboard implementation	keyboard_aoa.c
app/src/usb/aoa_hid.c	Low‑level AOA HID registration & descriptor upload	aoa_hid.c
app/src/control_msg.c	Serialization of the SC_CONTROL_MSG_TYPE_UHID_* messages	control_msg.c
doc/mouse.md	User‑facing description of mouse modes	mouse.md
doc/keyboard.md	User‑facing description of keyboard modes	keyboard.md

Summary

• scrcpy implements HID keyboard and mouse simulation through two transport stacks: UHID (virtual device via control socket) and AOA (direct USB accessory protocol).
• The UHID stack requires Android 8+ and uses SC_CONTROL_MSG_TYPE_UHID_CREATE and SC_CONTROL_MSG_TYPE_UHID_INPUT messages to register devices and send reports via sc_controller_push_msg().
• The AOA stack uses libusb control transfers via sc_aoa_register_hid() and sc_aoa_push_input(), requiring no server process on the device.
• Input events are converted to standard HID reports using functions in app/src/hid/hid_keyboard.c and app/src/hid/hid_mouse.c.
• Output reports (LED states) are handled by sc_keyboard_uhid_process_hid_output() in app/src/uhid/keyboard_uhid.c for UHID keyboards.

Frequently Asked Questions

What is the difference between UHID and AOA in scrcpy?

UHID creates a virtual HID device on the Android device using the kernel's Userspace HID driver, sending reports through the scrcpy control socket via SC_CONTROL_MSG_TYPE_UHID_INPUT messages. It requires Android 8.0 or higher. AOA uses the Android Open Accessory protocol to register the computer as a USB accessory, pushing HID reports directly over USB control transfers using sc_aoa_push_input() without requiring a server on the device.

Does HID simulation work on all Android devices?

No. UHID requires Android 8.0 (API 26) or higher and a kernel compiled with UHID support. AOA requires the device to support the Android Open Accessory protocol (most devices running Android 4.1+ support this, though some manufacturers disable it). Additionally, AOA requires a direct USB connection and appropriate permissions on the host computer to open the USB device.

How does scrcpy handle keyboard LED indicators like Caps Lock?

When using the UHID stack, the Android device can send output reports back to the host indicating LED states (NumLock, CapsLock, ScrollLock). These arrive as SC_CONTROL_MSG_TYPE_UHID_OUTPUT messages and are processed by sc_keyboard_uhid_process_hid_output() in app/src/uhid/keyboard_uhid.c, which updates the client's internal modifier state. The AOA stack does not support receiving LED status updates from the device.

Can I use HID simulation without the scrcpy server?

Yes, but only with the AOA stack. The AOA protocol communicates directly over USB using libusb control transfers via sc_aoa_register_hid() and sc_aoa_push_input(), bypassing the need for the scrcpy server binary running on the Android device. The UHID stack requires the server because it uses the scrcpy control socket to send SC_CONTROL_MSG_TYPE_UHID_* messages to the device.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}