How to Implement Voice Input and Transcription with CopilotChatAudioRecorder

CopilotKit provides a ready-to-use CopilotChatAudioRecorder component that handles microphone recording, visualizes audio input via canvas, and integrates with the runtime's transcription service to convert speech to text.

The CopilotKit repository ships a complete voice input solution centered around the CopilotChatAudioRecorder component. Located in packages/react-core/src/v2/components/chat/CopilotChatAudioRecorder.tsx, this recorder implements a finite-state machine for audio capture and connects seamlessly to the CopilotRuntime transcription service, allowing users to dictate messages instead of typing.

Setting Up the Voice Recorder Component

The CopilotChatAudioRecorder is designed to drop into your chat UI through the CopilotChatInput component. The recorder exposes a ref-based API (start(), stop(), state, dispose()) typed as AudioRecorderRef, which CopilotChatInput uses to coordinate recording actions.

Basic Implementation with CopilotChatInput

Pass the recorder component to the audioRecorder prop of CopilotChatInput:

import { CopilotChatInput, CopilotChatAudioRecorder } from "@copilotkit/react-core";

function Chat() {
  return (
    <CopilotChatInput
      audioRecorder={CopilotChatAudioRecorder}
      className="my-chat-input"
    />
  );
}

When the microphone button is clicked, CopilotChatInput calls audioRecorderRef.current.start() to begin recording. The recorder captures audio via the MediaRecorder API, renders a canvas visualization of the input levels, and returns a Blob when stop() is invoked. This blob is automatically forwarded to the runtime's /transcribe endpoint.

Understanding the State Machine

The recorder manages three distinct states:

• idle – Ready to record, showing the microphone icon
• recording – Actively capturing audio with canvas visualization
• processing – Uploading the audio blob to the transcription service

Transitions between these states are handled internally in CopilotChatAudioRecorder.tsx, with the current state exposed via the ref's state property.

Configuring the Runtime Transcription Service

For voice input to function, the backend must expose a transcription service. The runtime advertises availability via the audioFileTranscriptionEnabled flag on its /info endpoint, which is emitted by packages/runtime/src/v2/runtime/handlers/get-runtime-info.ts.

Enabling Transcription in CopilotRuntime

Attach a TranscriptionService implementation to your CopilotRuntime configuration:

import { CopilotRuntime } from "@copilotkit/runtime/v2";
import { TranscriptionServiceOpenAI } from "@copilotkit/voice";
import OpenAI from "openai";

const runtime = new CopilotRuntime({
  transcriptionService: new TranscriptionServiceOpenAI({
    openai: new OpenAI({ apiKey: process.env.OPENAI_API_KEY })
  })
});

When a transcription service is present, the runtime automatically sets audioFileTranscriptionEnabled to true, causing CopilotChatInput to display the microphone button.

Creating a Guarded Transcription Service

Production implementations should guard against missing API keys. The Strands demo in showcase/integrations/strands/src/app/api/copilotkit-voice/[[...slug]]/route.ts demonstrates this pattern:

import { TranscriptionService } from "@copilotkit/runtime/v2";
import { TranscriptionServiceOpenAI } from "@copilotkit/voice";

class GuardedOpenAI extends TranscriptionService {
  private delegate: TranscriptionServiceOpenAI | null;

  constructor() {
    super();
    const apiKey = process.env.OPENAI_API_KEY;
    this.delegate = apiKey 
      ? new TranscriptionServiceOpenAI({ openai: new OpenAI({ apiKey }) }) 
      : null;
  }

  async transcribeFile(opts) {
    if (!this.delegate) {
      throw new Error("OPENAI_API_KEY not configured");
    }
    return this.delegate.transcribeFile(opts);
  }
}

// Usage in runtime
const runtime = new CopilotRuntime({
  transcriptionService: new GuardedOpenAI()
});

This guard throws a clear error if the OpenAI key is missing, preventing runtime failures.

Customizing the Audio Recorder

You can wrap CopilotChatAudioRecorder to apply custom styling or behavior while preserving the ref API:

import { forwardRef } from "react";
import { CopilotChatAudioRecorder } from "@copilotkit/react-core";

const FancyRecorder = forwardRef((props, ref) => (
  <CopilotChatAudioRecorder
    {...props}
    className="border-dashed border-2 border-indigo-400 rounded-full"
    ref={ref}
  />
));

function Chat() {
  return <CopilotChatInput audioRecorder={FancyRecorder} />;
}

The test suite in packages/react-core/src/v2/components/chat/__tests__/CopilotChatInput.slots.e2e.test.tsx validates that custom components implementing the AudioRecorderRef interface are fully supported.

Understanding the End-to-End Data Flow

When a user initiates voice input:

1. Client-side capture – CopilotChatAudioRecorder uses the MediaRecorder API to capture audio and visualize levels on a canvas element
2. Blob generation – On stop, the recorder returns a Blob (typically audio/webm format) to CopilotChatInput
3. Transcription request – The chat component sends the blob to /api/copilotkit-voice/transcribe as a FormData file upload
4. Service processing – The runtime's TranscriptionService processes the audio (e.g., via OpenAI's Whisper API)
5. Message insertion – The returned transcript string is inserted into the chat as a user message

This flow is demonstrated in the Strands demo route, which wires the guarded transcription service to the CopilotKit runtime handler.

Summary

• CopilotChatAudioRecorder manages the recording lifecycle, canvas visualization, and MediaRecorder integration in packages/react-core/src/v2/components/chat/CopilotChatAudioRecorder.tsx
• CopilotChatInput accepts the recorder via the audioRecorder prop and coordinates start/stop actions through the AudioRecorderRef interface
• Runtime configuration requires attaching a TranscriptionService to CopilotRuntime, which automatically enables the microphone button via the audioFileTranscriptionEnabled flag
• Guarded implementations should validate API keys before delegating to services like TranscriptionServiceOpenAI
• Audio data flows from the client as a Blob to the runtime's /transcribe endpoint, returning text that populates the chat input

Frequently Asked Questions

What audio formats does CopilotChatAudioRecorder capture?

The component uses the browser's MediaRecorder API with default MIME types, typically producing audio/webm blobs. These are sent to the runtime's /transcribe endpoint as a file upload via FormData.

How does the runtime know to show the microphone button?

The runtime exposes audioFileTranscriptionEnabled in its /info response (handled in packages/runtime/src/v2/runtime/handlers/get-runtime-info.ts). This boolean is automatically set to true when a TranscriptionService is attached to the CopilotRuntime instance.

Can I use a custom transcription service instead of OpenAI?

Yes. Extend the abstract TranscriptionService class and implement the transcribeFile method to integrate with any speech-to-text provider (AWS Transcribe, Google Speech-to-Text, etc.). Pass your custom service to CopilotRuntime in the transcriptionService property.

Where should I configure the OpenAI API key for transcription?

Configure the API key in your runtime environment, not the client. Instantiate TranscriptionServiceOpenAI with an OpenAI client configured using process.env.OPENAI_API_KEY, wrapped in a guard class as demonstrated in the Strands demo to handle missing keys gracefully.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}