# How Screenshot Retry Logic Works in UI-TARS Desktop: Configuration and Customization Guide

> Learn how UI-TARS Desktop's screenshot retry logic handles failures with async-retry. Customize retry attempts and callbacks using the RetryConfig interface for robust UI automation.

- Repository: [Bytedance Inc./UI-TARS-desktop](https://github.com/bytedance/UI-TARS-desktop)
- Tags: how-to-guide
- Published: 2026-05-10

---

**UI-TARS Desktop wraps the operator's `screenshot()` method with `async-retry` to automatically handle transient failures, allowing developers to customize retry attempts and callbacks via the `RetryConfig` interface in `GUIAgentConfig`.**

UI-TARS Desktop captures a screenshot before every model inference to understand the current UI state. The screenshot retry logic, implemented in the `GUIAgent` class within the bytedance/UI-TARS-desktop repository, uses the `async-retry` library to make the agent resilient against temporary glitches or flaky ADB connections.

## How Screenshot Retry Logic Works in UI-TARS Desktop

The core retry mechanism resides in [`packages/ui-tars/sdk/src/GUIAgent.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/sdk/src/GUIAgent.ts). Before each model inference, the agent invokes the operator's screenshot method through an `asyncRetry` wrapper:

```ts
// packages/ui-tars/sdk/src/GUIAgent.ts (lines 85-89)
const snapshot = await asyncRetry(
  () => operator.screenshot(),
  {
    retries: retry?.screenshot?.maxRetries ?? 0,
    minTimeout: 5000,
    onRetry: retry?.screenshot?.onRetry,
  },
);

```

**Default behavior** sets `maxRetries` to `0` when no configuration is provided, meaning the screenshot is attempted only once. If this single attempt fails, the agent aborts and reports `ErrorStatusEnum.SCREENSHOT_RETRY_ERROR`. The retry logic specifically targets only the `operator.screenshot()` function—image processing, model invocation, and action execution remain outside this retry scope.

The `minTimeout` is fixed at **5000 milliseconds** (5 seconds) to prevent overwhelming the device with rapid successive attempts.

## Customizing Screenshot Retry Configuration

Developers inject custom retry policies through the `GUIAgentConfig` object passed to the `GUIAgent` constructor.

### Understanding the RetryConfig Interface

The retry configuration is defined in [`packages/ui-tars/sdk/src/types.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/sdk/src/types.ts):

```ts
export interface GUIAgentConfig<TOperator> {
  // ... other properties
  retry?: {
    model?: RetryConfig;
    screenshot?: RetryConfig;
    execute?: RetryConfig;
  };
}

export interface RetryConfig {
  maxRetries: number;           // Additional attempts after first failure
  onRetry?: (error: Error, attempt: number) => void;  // Failure callback
}

```

Setting `maxRetries: 3` results in **four total attempts** (initial plus three retries). The optional `onRetry` callback executes after each failure, enabling logging or adaptive backoff strategies.

### Implementing Custom Retry Policies

**Example: Enable three extra screenshot attempts with logging:**

```ts
import { GUIAgent } from '@ui-tars/sdk';
import { MyOperator } from './my-operator';

const agent = new GUIAgent({
  operator: new MyOperator(),
  model: myModel,
  retry: {
    screenshot: {
      maxRetries: 3,
      onRetry: (err, attempt) => {
        console.warn(
          `[GUIAgent] screenshot retry #${attempt} failed:`,
          err.message,
        );
      },
    },
  },
});

```

**Example: Configure retries across all stages:**

```ts
const agent = new GUIAgent({
  operator: new MyOperator(),
  model: myModel,
  retry: {
    model: { maxRetries: 2 },
    screenshot: { maxRetries: 3 },
    execute: { maxRetries: 1 },
  },
});

```

## Error Handling and Loop Interaction

When screenshot retries are exhausted, the agent increments `snapshotErrCnt` and pauses for one second (`await sleep(1000)`) before attempting the next cycle. This continues until `MAX_SNAPSHOT_ERR_CNT` (defined in [`packages/ui-tars/shared/constants.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/shared/constants.ts)) is reached, at which point the agent terminates with `ErrorStatusEnum.SCREENSHOT_RETRY_ERROR`.

If the screenshot succeeds within the allotted retries, the captured image is stored in the `conversations` array and emitted via the `onData` callback, allowing the model inference loop to proceed normally.

## Summary

- **Location**: The retry logic lives in [`packages/ui-tars/sdk/src/GUIAgent.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/sdk/src/GUIAgent.ts), wrapping `operator.screenshot()` with `async-retry`.
- **Defaults**: Zero retries by default (`maxRetries: 0`) with a fixed 5-second minimum timeout between attempts.
- **Configuration**: Customize via `RetryConfig` in `GUIAgentConfig`, setting `maxRetries` and an optional `onRetry` callback.
- **Limits**: Persistent failures increment `snapshotErrCnt` until `MAX_SNAPSHOT_ERR_CNT` triggers agent termination.

## Frequently Asked Questions

### How do I completely disable screenshot retries?

Omit the `retry.screenshot` configuration or explicitly set `maxRetries: 0`. Since the default value is `0`, the agent attempts the screenshot exactly once and immediately fails on any error.

### Can I change the delay between retry attempts?

The `minTimeout` parameter is hardcoded to `5000` milliseconds in the `asyncRetry` call within [`GUIAgent.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/GUIAgent.ts). To modify this value, you must patch the source code or fork the repository, as the SDK deliberately maintains this fixed interval to prevent device hammering.

### What happens if all screenshot retries fail?

When all retry attempts exhaust, the agent increments the internal `snapshotErrCnt` counter, pauses for one second, and attempts to continue. If failures persist until reaching `MAX_SNAPSHOT_ERR_CNT` (typically defined in [`packages/ui-tars/shared/constants.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/shared/constants.ts)), the agent aborts with `ErrorStatusEnum.SCREENSHOT_RETRY_ERROR`.

### Does the screenshot retry logic apply to the model API calls?

No. The retry configuration in `GUIAgentConfig` supports separate policies for `model`, `screenshot`, and `execute` stages. The `screenshot` retry only wraps the operator's `screenshot()` method. Model inference retries require separate configuration under `retry.model`.