How the Model Invoke Retry Mechanism Handles API Errors in UI-TARS Desktop

The UI-TARS desktop SDK wraps every model.invoke call with the async-retry library to automatically re-execute failed requests while distinguishing fatal user aborts from transient API errors.

The bytedance/UI-TARS-desktop repository provides aTypeScript SDK that powers GUI automation agents. Understanding how the model invoke retry mechanism manages API errors is essential for building fault-tolerant applications that gracefully handle network interruptions and service timeouts.

Retry Architecture Overview

The SDK implements resilience through the third-party async-retry package. In packages/ui-tars/sdk/src/GUIAgent.ts, the GUIAgent.run method encapsulates all model invocations inside an asyncRetry wrapper【1†L17】.

The retry behavior is governed by the retry.model configuration object defined in packages/ui-tars/sdk/src/types.ts【2†L81-L100】. This configuration allows developers to specify maximum attempt counts, timeout intervals, and custom callbacks for monitoring retry cycles.

Configuring Retry Behavior

Retry Parameters

The RetryConfig interface accepts three key properties:

• maxRetries: The total number of retry attempts allowed (default: 0)
• minTimeout: Base delay in milliseconds between attempts (default: 30000 for 30 seconds)
• onRetry: Optional callback invoked after each failed attempt with the error and attempt number

Configuration Example

Developers supply these values through the GUIAgentConfig:

import { GUIAgent, GUIAgentConfig } from '@ui-tars/sdk';

const config: GUIAgentConfig = {
  retry: {
    model: {
      maxRetries: 3,
      minTimeout: 1000 * 30, // 30 seconds
      onRetry: (err, attempt) => console.warn(`Attempt ${attempt} failed:`, err)
    }
  }
};

Error Handling Implementation in GUIAgent.ts

The retry logic resides in lines 262-298 of GUIAgent.ts【3†L262-L298】. When model.invoke(vlmParams) throws, the catch block executes a three-phase error handling protocol.

Abort Signal Detection

The mechanism first checks for user-initiated cancellations. If the error is an instance of APIUserAbortError or contains the string "aborted" in its message, the code invokes the bail function provided by async-retry:

if (error instanceof Error && (error?.name === 'APIUserAbortError' ||
    error?.message?.includes('aborted'))) {
  bail(error); // Immediately stops retrying
  return { prediction: '', parsedPredictions: [] };
}

Calling bail terminates the retry loop immediately, propagating the error upward without further attempts.

Structured Error Recording

For non-abort errors, the SDK records the failure using guiAgentErrorParser before allowing the retry:

Object.assign(data, {
  status: StatusEnum.ERROR,
  error: this.guiAgentErrorParser(
    ErrorStatusEnum.INVOKE_RETRY_ERROR,
    error as Error,
  ),
});

This transforms raw exceptions into consistent GUIAgentError objects attached to the agent's data payload.

The Retry Loop Mechanics

After recording the error, the code re-throws the exception to trigger async-retry's scheduling logic:

throw error;

The library then waits minTimeout milliseconds and re-invokes the async function, repeating until maxRetries is exhausted. If all attempts fail, the final error propagates out of the asyncRetry wrapper and halts the agent run.

Complete Implementation Pattern

Below is the完整 retry wrapper as implemented in the source:

const {
  prediction,
  parsedPredictions,
  costTime,
  costTokens,
  responseId,
} = await asyncRetry(
  async (bail) => {
    try {
      const result = await model.invoke(vlmParams);
      return result;
    } catch (error: unknown) {
      // Bail out on user aborts
      if (error instanceof Error && (error?.name === 'APIUserAbortError' ||
          error?.message?.includes('aborted'))) {
        bail(error);
        return { prediction: '', parsedPredictions: [] };
      }

      // Record structured error
      Object.assign(data, {
        status: StatusEnum.ERROR,
        error: this.guiAgentErrorParser(
          ErrorStatusEnum.INVOKE_RETRY_ERROR,
          error as Error,
        ),
      });

      // Trigger next retry
      throw error;
    }
  },
  {
    retries: retry?.model?.maxRetries ?? 0,
    minTimeout: 1000 * 30,
    onRetry: retry?.model?.onRetry,
  },
);

Summary

• Automatic retries are handled by the async-retry library wrapping model.invoke in GUIAgent.ts
• User aborts bypass the retry loop immediately via the bail function when APIUserAbortError is detected
• Error normalization occurs through guiAgentErrorParser, converting exceptions into structured GUIAgentError objects
• Configuration is controlled via GUIAgentConfig.retry.model, supporting maxRetries, minTimeout, and onRetry callbacks
• Transient failures are re-thrown to trigger the next attempt, while fatal aborts terminate the sequence

Frequently Asked Questions

How do I configure the maximum number of retries for model invocations?

Set the maxRetries property in the retry.model configuration object when initializing GUIAgent. The value represents the total retry attempts allowed beyond the initial failed call. If not specified, the default is 0, meaning no retries occur.

What happens when a user aborts the operation during a retry cycle?

The SDK detects abort signals by checking for APIUserAbortError instances or messages containing "aborted". When detected, the code calls bail(error), which instructs async-retry to stop immediately and propagate the error without attempting further retries.

How are API errors structured before being passed to the onRetry callback?

Raw exceptions are first processed through guiAgentErrorParser with the status INVOKE_RETRY_ERROR. The parser converts the error into a GUIAgentError object containing standardized fields, which is then attached to the agent's data payload and passed as the err parameter to the onRetry callback.

Can I customize the delay between retry attempts?

Yes, specify the minTimeout value in milliseconds within retry.model. The default configuration uses 30000 (30 seconds). This value determines the base delay async-retry waits before scheduling the next invocation attempt.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}