# How the Coco App Transcription Command Processes Audio Input and Returns Text

> Learn how the Coco App transcription command processes audio to text. Discover its efficient workflow from Base64 input to cleaned, tagged text output via Tauri and remote servers.

- Repository: [INFINI Labs/coco-app](https://github.com/infinilabs/coco-app)
- Tags: how-to-guide
- Published: 2026-03-04

---

**The `transcription` command in infinilabs/coco-app captures audio as Base64, routes it through Tauri's Rust backend to a remote Coco server, polls asynchronously for up to 30 seconds, and returns cleaned text with special markup tags stripped.**

The transcription command provides end-to-end speech-to-text functionality by bridging React frontend components with Rust backend logic. This implementation handles audio encoding, HTTP-based server communication, and resilient polling to deliver accurate transcripts. Understanding this flow reveals how the Coco App leverages Tauri to combine web UI flexibility with native backend performance.

## Step 1: Frontend Audio Capture and Base64 Encoding

### Recording and Blob Conversion

The process begins in the **AudioRecording** component located at [`src/components/AudioRecording/index.tsx`](https://github.com/infinilabs/coco-app/blob/main/src/components/AudioRecording/index.tsx). When a user stops recording, the component triggers a `record-end` event and converts the captured audio Blob into a Base64 string.

```typescript
// src/components/AudioRecording/index.tsx (lines 66-78)
const reader = new FileReader();
reader.readAsDataURL(audioBlob);
reader.onloadend = async () => {
  const base64Audio = reader.result as string;
  // base64Audio contains the data URL prefix + Base64 content
};

```

The Base64 string is then packaged into a JSON payload containing the `serverId` and `audioContent` fields required by the backend.

## Step 2: Command Invocation Through the Platform Adapter

The frontend invokes the transcription command via the **platformAdapter**, which abstracts Tauri-specific implementation details. The adapter wraps Tauri's `invoke` function to bridge TypeScript and Rust.

```typescript
// Frontend invocation pattern
const response: any = await platformAdapter.commands(
  "transcription",
  {
    serverId: currentService.id,
    audioContent: JSON.stringify({ content: base64Audio }),
  }
);

```

**Source files:**
- Platform abstraction: [`src/utils/platformAdapter.ts`](https://github.com/infinilabs/coco-app/blob/main/src/utils/platformAdapter.ts)
- Tauri bridge implementation: [`src/utils/tauriAdapter.ts`](https://github.com/infinilabs/coco-app/blob/main/src/utils/tauriAdapter.ts) (around line 37)

This call routes through the `commands` wrapper in [`tauriAdapter.ts`](https://github.com/infinilabs/coco-app/blob/main/tauriAdapter.ts), which ultimately invokes the registered Rust command using Tauri's `invoke` API.

## Step 3: Tauri Backend Command Registration

The Rust backend registers the transcription command in [`src-tauri/src/lib.rs`](https://github.com/infinilabs/coco-app/blob/main/src-tauri/src/lib.rs) at line 134:

```rust
// src-tauri/src/lib.rs
.invoke_handler(tauri::generate_handler![
    // ... other commands
    server::transcription::transcription,
])

```

When the frontend calls `"transcription"`, Tauri executes the `server::transcription::transcription` function defined in [`src-tauri/src/server/transcription.rs`](https://github.com/infinilabs/coco-app/blob/main/src-tauri/src/server/transcription.rs). This Rust module handles all HTTP communication with the remote Coco server.

## Step 4: Server Communication and Asynchronous Polling

### Initial Task Creation

The backend immediately posts the Base64 audio content to the server's transcription endpoint:

```rust
// src-tauri/src/server/transcription.rs (lines 18-25)
let init_response = HttpClient::post(
    &server_id,
    "/services/audio/transcription",
    None,
    Some(audio_content.into()),
).await?;

let task_id = init_response.task_id;

```

The server responds with a `task_id` that identifies the asynchronous transcription job.

### Polling for Results

The backend enters a polling loop (lines 58-88) that repeatedly queries the task status:

```rust
// Polling implementation excerpt
loop {
    let poll_response = HttpClient::get(
        &server_id,
        &format!("/services/audio/task/{}", transcription_task_id),
        None,
    ).await?;
    
    let body = get_response_body_text(poll_response).await?;
    let transcription_response: TranscriptionResponse = serde_json::from_str(&body)?;
    
    if !transcription_response.results.is_empty() {
        return Ok(transcription_response);
    }
    
    if start.elapsed() >= Duration::from_secs(30) {
        return Err("Transcription timeout".into());
    }
    
    tokio::time::sleep(Duration::from_millis(200)).await;
}

```

**Key characteristics:**
- **Polling interval**: 200 milliseconds between requests
- **Timeout threshold**: 30 seconds maximum wait time
- **Completion trigger**: Non-empty `results` array in the response

## Step 5: Response Processing and Text Extraction

The shared TypeScript types in [`src/types/commands.ts`](https://github.com/infinilabs/coco-app/blob/main/src/types/commands.ts) (lines 174-176) define the response structure:

```typescript
export interface TranscriptionResponse {
  text: string;
}

```

However, the raw response often contains nested arrays and formatting tags. The frontend extracts readable text by flattening the results structure and applying regex cleaning:

```typescript
// src/components/AudioRecording/index.tsx (lines 85-89)
const text = response?.results
  .flatMap((item: any) => item?.transcription?.transcripts)
  .map((item: any) => item?.text?.replace(/<\|[\/\w]+\|>/g, ""))
  .join(" ");

```

This processing:
1. Flattens nested `transcription.transcripts` arrays
2. Strips special markup tags matching the pattern `<|...|>`
3. Joins all transcript fragments into a single coherent string

## Error Handling and Resilience

Errors at any stage bubble up to the `invokeWithErrorHandler` utility in [`src/commands/servers.ts`](https://github.com/infinilabs/coco-app/blob/main/src/commands/servers.ts) (lines 37-101). This centralizes error logging via the `addError` function and presents user-friendly messages in the UI.

The `transcription` wrapper function (lines 364-368) in the same file provides a typed interface for components that need direct access without using the platform adapter:

```typescript
// src/commands/servers.ts
export const transcription = (payload: TranscriptionPayload) => {
  return invokeWithErrorHandler("transcription", payload);
};

```

## Code Implementation Examples

### Triggering Transcription from a Custom Component

```tsx
import platformAdapter from "@/utils/platformAdapter";

async function transcribeUserAudio(serverId: string, audioBlob: Blob) {
  // Convert to Base64
  const reader = new FileReader();
  reader.readAsDataURL(audioBlob);
  
  return new Promise((resolve) => {
    reader.onloadend = async () => {
      const base64Audio = reader.result as string;
      
      const payload = {
        serverId,
        audioContent: JSON.stringify({ content: base64Audio }),
      };
      
      const resp = await platformAdapter.commands("transcription", payload);
      
      // Extract and clean text
      const text = resp?.results
        .flatMap((r: any) => r?.transcription?.transcripts)
        .map((t: any) => t?.text?.replace(/<\|[\/\w]+\|>/g, ""))
        .join(" ");
        
      resolve(text);
    };
  });
}

```

### Direct TypeScript API Call

```typescript
import { transcription } from "@/commands/servers";

async function simpleTranscribe(serverId: string, base64Data: string) {
  const payload = {
    serverId,
    audioType: "audio/mpeg",
    audioContent: base64Data,
  };
  
  const { results } = await transcription(payload);
  
  return results
    .flatMap((r: any) => r?.transcription?.transcripts)
    .map((t: any) => t?.text?.replace(/<\|[\/\w]+\|>/g, ""))
    .join(" ");
}

```

### Rust Polling Logic (Simplified)

```rust
use std::time::{Duration, Instant};

async fn poll_transcription_result(
    server_id: &str, 
    task_id: &str
) -> Result<TranscriptionResponse, String> {
    let start = Instant::now();
    
    loop {
        let response = HttpClient::get(
            server_id,
            &format!("/services/audio/task/{}", task_id),
            None
        ).await.map_err(|e| e.to_string())?;
        
        let body = get_response_body_text(response).await?;
        let resp: TranscriptionResponse = serde_json::from_str(&body)
            .map_err(|e| e.to_string())?;
            
        if !resp.results.is_empty() {
            return Ok(resp);
        }
        
        if start.elapsed() >= Duration::from_secs(30) {
            return Err("Transcription timeout".into());
        }
        
        tokio::time::sleep(Duration::from_millis(200)).await;
    }
}

```

## Summary

- **Audio Input**: The `transcription` command accepts Base64-encoded audio content from the **AudioRecording** component in [`src/components/AudioRecording/index.tsx`](https://github.com/infinilabs/coco-app/blob/main/src/components/AudioRecording/index.tsx)
- **Command Routing**: Frontend calls route through `platformAdapter` to the Rust backend registered in [`src-tauri/src/lib.rs`](https://github.com/infinilabs/coco-app/blob/main/src-tauri/src/lib.rs)
- **Server Communication**: The Rust implementation in [`src-tauri/src/server/transcription.rs`](https://github.com/infinilabs/coco-app/blob/main/src-tauri/src/server/transcription.rs) POSTs audio to `/services/audio/transcription` and receives a task ID
- **Asynchronous Polling**: The backend polls `/services/audio/task/{task_id}` every 200ms with a 30-second timeout
- **Text Processing**: Results undergo flattening and regex cleaning (`/<\|[\/\w]+\|>/g`) to remove markup tags before display
- **Error Handling**: Centralized through `invokeWithErrorHandler` in [`src/commands/servers.ts`](https://github.com/infinilabs/coco-app/blob/main/src/commands/servers.ts) with user-facing error propagation

## Frequently Asked Questions

### What audio format does the transcription command expect?

The transcription command accepts audio encoded as **Base64 strings**. While the `audioType` parameter (e.g., `"audio/mpeg"`) can specify the MIME type, the backend primarily processes the Base64 content posted to the `/services/audio/transcription` endpoint. The AudioRecording component handles the Blob-to-Base64 conversion automatically.

### How long does the transcription command wait for server results?

The Rust backend implements a **30-second timeout** with **200-millisecond polling intervals**. After submitting audio to the server, the `transcription` function polls the task status endpoint until results are available or the timeout elapses, whichever occurs first.

### How does the app handle transcription errors?

Errors propagate through the `invokeWithErrorHandler` utility in [`src/commands/servers.ts`](https://github.com/infinilabs/coco-app/blob/main/src/commands/servers.ts). Network failures, server errors, or timeout conditions are caught, logged via the `addError` function, and presented to users through the application's error notification system. Components can wrap transcription calls in try-catch blocks to implement custom fallback behavior.

### Can I use the transcription command outside the AudioRecording component?

Yes. The transcription API is exposed through two interfaces: the **platformAdapter** (`platformAdapter.commands("transcription", payload)`) for generic use, and the direct **transcription** wrapper function exported from [`src/commands/servers.ts`](https://github.com/infinilabs/coco-app/blob/main/src/commands/servers.ts). Both accept a `TranscriptionPayload` containing `serverId` and `audioContent`, allowing any component or utility function to initiate speech-to-text processing.