# UI-TARS SDK Retry Mechanism: How to Configure Retries for Screenshots, Models, and Actions

> Learn about the UI-TARS SDK retry mechanism and configure retries for screenshots, models, and actions. Master asynchronous retries for robust UI automation.

- Repository: [Bytedance Inc./UI-TARS-desktop](https://github.com/bytedance/UI-TARS-desktop)
- Tags: how-to-guide
- Published: 2026-05-10

---

**The UI-TARS SDK implements a configurable retry mechanism using the `async-retry` library (v1.3.3) for screenshot capture, model inference, and action execution, defaulting to zero retries but allowing granular per-operation configuration via the `RetryConfig` interface.**

The retry system in the `bytedance/UI-TARS-desktop` repository provides resilience for the three critical operations within the `GUIAgent` class. By default, the SDK performs no retries, but you can enable automatic re-attempts with custom callbacks and timeouts for specific failure scenarios.

## How the UI-TARS SDK Retry Mechanism Works

The retry logic is centralized in the `GUIAgent.run` method within [`packages/ui-tars/sdk/src/GUIAgent.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/sdk/src/GUIAgent.ts). The SDK wraps three distinct operations with the `async-retry` utility to handle transient failures in GUI automation workflows.

### Retry Operations and Default Behavior

The SDK applies retry logic to three specific phases of the agent execution cycle:

| Operation | Location in SDK | Default Max Retries | Min Timeout |
|-----------|----------------|---------------------|-------------|
| **Screenshot** | `GUIAgent.run` lines 85-89 | 0 | 5000ms |
| **Model invocation** | `GUIAgent.run` lines 96-100 | 0 | 30000ms |
| **Action execution** | `GUIAgent.run` lines 122-126 | 0 | 5000ms |

Each operation defaults to `maxRetries: 0`, meaning the SDK attempts each operation exactly once unless explicitly configured otherwise.

### Underlying Implementation with async-retry

The mechanism relies on the `async-retry` npm package declared in [`packages/ui-tars/sdk/package.json`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/sdk/package.json). Inside `GUIAgent.run`, each operation calls `asyncRetry` with operation-specific parameters:

```typescript
// Screenshot retry logic (GUIAgent.ts lines 85-89)
await asyncRetry(() => operator.screenshot(), {
  retries: retry?.screenshot?.maxRetries ?? 0,
  minTimeout: 5000,
  onRetry: retry?.screenshot?.onRetry,
});

// Model invocation retry logic (GUIAgent.ts lines 96-100)
await asyncRetry(async (bail) => { /* model call */ }, {
  retries: retry?.model?.maxRetries ?? 0,
  minTimeout: 30_000,
  onRetry: retry?.model?.onRetry,
});

// Action execution retry logic (GUIAgent.ts lines 122-126)
await asyncRetry(() => operator.execute({ /* action */ }), {
  retries: retry?.execute?.maxRetries ?? 0,
  minTimeout: 5000,
  onRetry: retry?.execute?.onRetry,
});

```

The `bail` function available in the model invocation wrapper allows immediate termination of retries for non-recoverable errors.

## Configuration Architecture

The retry system uses a strongly-typed configuration structure defined in [`packages/ui-tars/sdk/src/types.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/sdk/src/types.ts).

### RetryConfig Interface

The SDK exposes a `RetryConfig` interface at lines 81-84:

```typescript
interface RetryConfig {
  maxRetries?: number;
  onRetry?: (error: Error, attempt: number) => void | Promise<void>;
}

```

- **`maxRetries`**: Specifies the number of additional attempts beyond the initial call (e.g., `maxRetries: 2` allows 3 total attempts).
- **`onRetry`**: Optional callback invoked after each failed attempt, receiving the error and attempt number (1-indexed).

### GUIAgentConfig Integration

The top-level configuration accepts an optional `retry` field containing separate configs for each operation (lines 96-103):

```typescript
interface GUIAgentConfig {
  // ... other config options
  retry?: {
    model?: RetryConfig;
    screenshot?: RetryConfig;
    execute?: RetryConfig;
  };
}

```

This structure allows independent retry strategies for each phase of the automation loop.

## Configuring Retries in Practice

When instantiating `GUIAgent`, pass the `retry` configuration object to enable resilience for specific operations.

### Basic Configuration: Screenshot Retries

Enable retries only for screenshot operations when dealing with unstable screen capture:

```typescript
import { GUIAgent } from '@ui-tars/sdk';

const agent = new GUIAgent({
  operator: new MyOperator(),
  model: new MyModel(),
  retry: {
    screenshot: {
      maxRetries: 2,
      onRetry: (err, attempt) => {
        console.warn(`Screenshot failed, retry #${attempt}: ${err.message}`);
      },
    },
    // model and execute remain at default (no retries)
  },
});

await agent.run('Open the settings page');

```

This configuration attempts screenshots up to 3 times total (initial + 2 retries) with a 5000ms minimum timeout between attempts.

### Comprehensive Configuration: All Operations

Configure different retry limits for each phase of the agent loop:

```typescript
const agent = new GUIAgent({
  operator: new MyOperator(),
  model: new MyModel(),
  retry: {
    screenshot: { 
      maxRetries: 1  // 2 attempts total
    },
    model: {
      maxRetries: 3,  // 4 attempts total for LLM calls
      onRetry: (e, i) => console.info(`Model retry ${i}`, e),
    },
    execute: { 
      maxRetries: 2,  // 3 attempts total for action execution
      onRetry: (err) => console.error('Action failed:', err)
    },
  },
});

```

The model invocation uses a 30-second timeout suitable for API latency, while screenshot and execution use 5-second timeouts.

### Advanced: Custom onRetry Hooks with Exponential Backoff

Implement custom backoff strategies by returning a Promise from `onRetry`:

```typescript
function exponentialBackoff(error: Error, attempt: number) {
  const delay = Math.pow(2, attempt) * 1000; // 1s, 2s, 4s...
  console.warn(`Waiting ${delay}ms before retry #${attempt}: ${error.message}`);
  return new Promise((resolve) => setTimeout(resolve, delay));
}

const agent = new GUIAgent({
  operator: new MyOperator(),
  model: new MyModel(),
  retry: {
    model: {
      maxRetries: 3,
      onRetry: exponentialBackoff,
    },
  },
});

```

Because `async-retry` respects the resolution timing of the `onRetry` callback, this custom delay logic properly throttles retry attempts.

## Error Handling and Timeouts

When the retry limit exhausts, `async-retry` throws the final error, which the SDK catches and converts to a `GUIAgentError` via the `guiAgentErrorParser` utility. The error propagates through the agent's `onError` callback if configured.

Key timeout characteristics:
- **Screenshot retries**: 5000ms minimum between attempts
- **Model retries**: 30000ms minimum between attempts (accommodating LLM API latency)
- **Action retries**: 5000ms minimum between attempts

If omitted from the configuration, any operation defaults to immediate failure propagation without retry attempts.

## Summary

- The **UI-TARS SDK retry mechanism** uses `async-retry` to handle transient failures in screenshot, model, and execution operations.
- **Default behavior** is zero retries (`maxRetries: 0`), requiring explicit opt-in for resilience.
- Configure retries via the `RetryConfig` interface in `GUIAgentConfig`, supporting distinct strategies per operation type.
- Implement custom logic using the `onRetry` callback, which receives the error and attempt count and supports asynchronous delays.
- Source implementation resides in [`packages/ui-tars/sdk/src/GUIAgent.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/sdk/src/GUIAgent.ts) with type definitions in [`packages/ui-tars/sdk/src/types.ts`](https://github.com/bytedance/UI-TARS-desktop/blob/main/packages/ui-tars/sdk/src/types.ts).

## Frequently Asked Questions

### What is the default retry behavior in the UI-TARS SDK?

By default, the SDK performs **no retries**. The `maxRetries` parameter defaults to `0` for screenshot, model invocation, and action execution operations, meaning each operation executes exactly once and fails immediately on error. This conservative default prevents unnecessary latency in production environments.

### How do I configure different retry limits for screenshots versus model calls?

Pass separate `RetryConfig` objects within the `retry` field of `GUIAgentConfig`. The configuration supports distinct `maxRetries` values for `screenshot`, `model`, and `execute` properties. For example, set `retry.screenshot.maxRetries: 2` for image capture resilience while keeping `retry.model.maxRetries: 0` for strict single-shot LLM inference.

### Can I implement exponential backoff with the UI-TARS SDK retry mechanism?

Yes. The `onRetry` callback in `RetryConfig` supports async functions. Return a Promise that resolves after your desired delay to implement exponential backoff, jitter, or other custom timing strategies. The underlying `async-retry` library waits for the `onRetry` Promise to resolve before attempting the next retry.

### What happens when all retry attempts are exhausted?

When retry limits exhaust, the SDK throws the final error through the `guiAgentErrorParser` and propagates it via the `onError` callback of the `GUIAgent`. If unhandled, this terminates the current `agent.run()` execution. The error object contains the original failure details from the last attempt, allowing diagnostic logging or fallback logic in your application code.