TUUI Cross-Vendor LLM API Orchestration Architecture: Inside the Electron-Based Multi-Provider System

TUUI implements a four-layer orchestration architecture inside its Electron app that abstracts vendor-specific LLM APIs through a JSON configuration layer, a secure preload bridge, Vue-Pinia state management, and a streaming completion composable.

TUUI is an open-source Electron application that unifies access to multiple large language model providers through a sophisticated cross-vendor LLM API orchestration system. This architecture eliminates vendor lock-in by decoupling provider-specific implementation details from the UI layer, allowing users to switch between OpenAI, Anthropic, and custom endpoints without modifying application code.

The Four-Layer Architecture of TUUI LLM Orchestration

TUUI’s cross-vendor LLM API orchestration system is organized into four distinct logical layers, each with specific responsibilities and clear interfaces.

Layer 1: Configuration Management and Type Safety

The foundation of the orchestration system resides in static configuration files and TypeScript interfaces that enforce vendor-specific contracts.

In src/types/llm.d.ts, the LlmConfig and ChatbotConfig interfaces define the complete schema for provider configuration:

// src/types/llm.d.ts
export interface LlmConfig {
  default: ChatbotConfig
  custom: ChatbotConfig[]
}

export interface ChatbotConfig {
  name: string
  apiKey: string
  apiCli: string
  icon: string
  url: string
  urlList: string[]
  path: string
  pathList: string[]
  model: string
  modelList: string[]
  authPrefix: string
  authPrefixList: string[]
  maxTokensValue?: string
  maxTokensPrefix: string
  maxTokensPrefixList: string[]
  temperature?: string
  topP?: string
  method: string
  contentType: string
  stream: boolean
  reasoningEffort?: number
  enableThinking?: number
  enableExtraBody: boolean
  extraBody: object
  authorization: boolean
  mcp: boolean
}

The main process loads and validates this configuration through loadLlmFile in src/main/mcp/config.ts. This function reads the JSON file located at Constants.ASSETS_PATH.llm, validates its structure, and surfaces parsing errors via desktop notifications.

// src/main/mcp/config.ts
export function loadLlmFile(path: string): LlmConfig {
  // Implementation reads and validates llm.json
  // Returns structured config or throws with notification
}

Layer 2: Secure Preload Bridge

Electron’s security model isolates the renderer process from Node.js APIs. TUUI bridges this gap through a carefully designed preload script that exposes only the necessary LLM configuration to the frontend.

In src/preload/index.ts, the application creates a cached configuration object and exposes it via contextBridge.exposeInMainWorld:

// src/preload/index.ts
const llm = {
  _currentAPI: {} as LlmConfig,
  get: () => llm._currentAPI,
  async init() {
    const llms = await ipcRenderer.invoke('list-llms')
    llm._currentAPI = llms
  }
}

contextBridge.exposeInMainWorld('llmApis', llm)

When the renderer process initializes via src/renderer/main.ts, it calls llm.init(), which triggers an IPC call to the main process. The configuration becomes available globally as window.llmApis, allowing Vue components to access vendor settings without direct filesystem access.

Layer 3: Renderer Store and Request Orchestration

The renderer process manages state and request logic through a combination of Pinia stores and Vue composables.

Chatbot Store

src/renderer/store/chatbot.ts provides reactive selectors that read from the preload bridge:

// src/renderer/store/chatbot.ts
const getDefaultLLM = () => window.llmApis?.get().default || CHATBOT_DEFAULTS
const getCustomLLMs = () => window.llmApis?.get().custom || []

These selectors decouple UI components from the global window object, enabling testable state management while maintaining reactivity.

Completion Composable

The heart of TUUI’s cross-vendor orchestration resides in src/renderer/composables/chatCompletions.ts. This composable handles five critical responsibilities:

Token Refresh Logic

For vendors requiring dynamic authentication, the composable implements checkTokenUpdate. This function decodes existing JWTs, checks expiration claims, and invokes getApiToken—a wrapper around ipcRenderer.invoke('msgGetApiToken')—to fetch fresh credentials from CLI-based authentication flows.

Request Construction

The composable merges static configuration (model, temperature, topP) with dynamic session data (conversation history, system prompts, tool definitions). It constructs headers based on the authorization boolean—injecting either Authorization: Bearer <token> or x-api-key: <key> depending on vendor requirements.

MCP Tool Integration

When chatbotConfig.mcp is enabled, the composable queries the agent store for tool definitions and appends them to the request payload under the appropriate schema for function calling.

Streaming Response Handling

The implementation uses the native fetch API with ReadableStream. A dedicated read helper continuously decodes chunks, parses newline-delimited JSON (NDJSON), and updates the Pinia message store in real-time, enabling word-by-word streaming display in the UI.

Error Handling

HTTP errors trigger user-friendly Snackbar notifications, while JSON parsing failures are logged to the console and reported through the notification system without crashing the stream.

Layer 4: IPC and Runtime Services

The main process exposes minimal surface area for LLM operations. In src/main/IPCs.ts, a single handler serves the configuration to the preload layer:

// src/main/IPCs.ts
ipcMain.handle('list-llms', () => {
  return loadLlmFile(Constants.ASSETS_PATH.llm)
})

Additional IPC channels support token retrieval (msgGetApiToken) and file operations, maintaining a thin, deterministic bridge between the sandboxed renderer and Node.js capabilities.

End-to-End Request Flow

Understanding the complete data flow clarifies how TUUI achieves seamless cross-vendor orchestration:

1. Application Startup – The preload script (src/preload/index.ts) invokes the 'list-llms' IPC channel.
2. Configuration Loading – The main process reads assets/config/llm.json via loadLlmFile and returns the structured LlmConfig.
3. State Initialization – The renderer caches the configuration in window.llmApis and the Pinia chatbotStore.
4. Chat Initiation – User input triggers createCompletion in src/renderer/composables/chatCompletions.ts.
5. Authentication Check – checkTokenUpdate validates JWT expiry and calls getApiToken if the apiCli field is configured.
6. Request Assembly – The composable constructs the HTTP payload using vendor-specific paths, models, and authentication headers.
7. Tool Integration – If mcp is enabled, tool definitions are appended to the request body.
8. Streaming Execution – A fetch call initiates the stream; the read helper parses NDJSON chunks and updates the UI in real-time.
9. Completion – The final assistant message is persisted to the session store, and the abort controller is cleaned up.

Practical Implementation Examples

Adding a New Vendor Configuration

To integrate a new LLM provider, modify assets/config/llm.json without touching application code:

{
  "default": {
    "name": "Anthropic-Claude-3",
    "apiKey": "",
    "apiCli": "claude auth token",
    "icon": "anthropic.svg",
    "url": "https://api.anthropic.com",
    "path": "/v1/messages",
    "model": "claude-3-opus-20240229",
    "method": "POST",
    "contentType": "application/json",
    "stream": true,
    "authorization": true,
    "authPrefix": "Bearer",
    "mcp": true,
    "enableExtraBody": false,
    "extraBody": {}
  },
  "custom": []
}

The new vendor appears immediately in the UI because window.llmApis reads this configuration at runtime.

Initiating a Chat Completion

Vue components interact with the orchestration layer through the Pinia store and completion composable:

import { createCompletion } from '@/renderer/composables/chatCompletions'
import { useSessionStore } from '@/renderer/store/session'

const session = useSessionStore().activeSession

async function sendMessage(userPrompt: string) {
  // Append user message to session history
  session.messages.push({ role: 'user', content: userPrompt })
  
  // Trigger the orchestration layer
  await createCompletion(session)
  // Assistant response streams into session.messages automatically
}

Handling Dynamic Token Refresh

For vendors requiring short-lived tokens, implement CLI-based refresh:

import { getApiToken } from '@/renderer/utils'

async function refreshVendorToken(vendorConfig: ChatbotConfig) {
  if (vendorConfig.apiCli) {
    const newToken = await getApiToken(vendorConfig.apiCli)
    // Update the store with fresh credentials
    chatbotStore.updateApiKey(vendorConfig.name, newToken)
    return newToken
  }
  return null
}

Summary

TUUI’s cross-vendor LLM API orchestration architecture delivers a secure, extensible abstraction layer through four coordinated components:

• Configuration Layer: Type-safe JSON schemas in src/types/llm.d.ts and file loading via loadLlmFile enable zero-code vendor additions
• Security Bridge: The preload script in src/preload/index.ts exposes LLM configs through contextBridge without compromising Electron sandbox security
• Orchestration Engine: The createCompletion composable in src/renderer/composables/chatCompletions.ts handles authentication, request building, MCP tool integration, and streaming response parsing
• IPC Runtime: Minimal main-process handlers in src/main/IPCs.ts serve configuration data while keeping the Node.js surface area small

This design pattern allows TUUI to support new LLM providers by editing a single JSON file, while maintaining type safety, secure process isolation, and real-time streaming capabilities across all vendors.

Frequently Asked Questions

How does TUUI handle authentication for different LLM vendors?

TUUI supports two authentication patterns through the authorization and apiCli fields in ChatbotConfig. For static API keys, the system injects headers based on the authPrefix field (e.g., Bearer or custom schemes) in src/renderer/composables/chatCompletions.ts. For dynamic tokens, the checkTokenUpdate function decodes JWT expiration claims and invokes getApiToken via IPC to execute the CLI command specified in apiCli, fetching fresh credentials without restarting the application.

Can I add a custom LLM provider without modifying TUUI's source code?

Yes. TUUI’s cross-vendor LLM API orchestration system is designed for zero-code vendor additions. By adding a new entry to assets/config/llm.json with the required ChatbotConfig fields—such as url, path, model, and authPrefix—the new provider automatically appears in the UI. The loadLlmFile function in src/main/mcp/config.ts validates the JSON at runtime, and the preload bridge exposes it to the renderer immediately.

How does TUUI manage streaming responses from different LLM APIs?

The createCompletion composable in src/renderer/composables/chatCompletions.ts implements a unified streaming handler using the native fetch API with ReadableStream. The read helper function continuously decodes chunks, parses newline-delimited JSON (NDJSON) responses, and updates the Pinia message store in real-time. This approach normalizes streaming differences between vendors—whether they use Server-Sent Events (SSE) or raw JSON streams—into a consistent reactive data flow that Vue components consume for word-by-word UI updates.

What is the role of MCP (Model Context Protocol) in TUUI's orchestration layer?

MCP support enables tool-calling capabilities across compatible LLM vendors. When chatbotConfig.mcp is set to true in the configuration, the createCompletion composable queries the agent store for available tool definitions and injects them into the request payload under the appropriate schema for function calling. This allows TUUI to leverage vendor-specific tool-use APIs—such as OpenAI's function calling or Anthropic's tool use—while maintaining a unified interface in the renderer store.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}