Main Features of moeru-ai/airi: A Modular AI-VTuber Framework Explained

AIRI is a modular, multi-platform AI-VTuber framework that combines a core engine with interchangeable UI layers and service plugins, enabling the same virtual character to run in browsers, desktop Electron apps, mobile Capacitor builds, and native GPU-accelerated environments.

The moeru-ai/airi repository delivers a comprehensive toolkit for building interactive AI virtual characters. By separating the core logic from presentation layers, AIRI allows developers to deploy consistent VTuber experiences across web, desktop, and mobile platforms while integrating with popular services like Discord, Telegram, Minecraft, and Factorio.

Core Capabilities: Brain, Ears, Mouth, and Body

AIRI organizes its functionality into four anatomical categories that mirror a virtual character's sensory and cognitive abilities.

Brain: Gaming, Chat, and Memory

The brain module handles high-level reasoning, external service integration, and persistent storage.

• Minecraft Integration: Uses Mineflayer to drive a Minecraft client, allowing the AI to play and interact within Minecraft worlds【/cache/repos/github.com/moeru-ai/airi/main/README.md#L115-L116】.
• Factorio Support (WIP): Proof-of-concept implementation via RCON for factory automation gameplay【/cache/repos/github.com/moeru-ai/airi/main/README.md#L116-L117】.
• Telegram Bot: Bi-directional chat integration for Telegram messaging【/cache/repos/github.com/moeru-ai/airi/main/README.md#L117】.
• Discord Voice Bot: Full voice-enabled Discord integration for server participation【/cache/repos/github.com/moeru-ai/airi/main/README.md#L118】.
• In-Browser Memory: Uses DuckDB-WASM (pglite) for local vector storage and retrieval, eliminating external database dependencies【/cache/repos/github.com/moeru-ai/airi/main/README.md#L120-L121】.
• Pure-WebGPU Inference: Planned client-side model execution for fully private, GPU-accelerated inference without server round-trips【/cache/repos/github.com/moeru-ai/airi/main/README.md#L122】.

Ears: Audio Input and Speech Recognition

The ears module captures and processes audio input across different environments.

• Browser Audio Capture: Direct microphone access via WebAudio API for web deployments【/cache/repos/github.com/moeru-ai/airi/main/README.md#L124】.
• Discord Voice Reception: Captures audio streams from Discord voice channels for server-side processing【/cache/repos/github.com/moeru-ai/airi/main/README.md#L125】.
• Client-Side Speech-to-Text: On-device STT processing to reduce latency and preserve privacy【/cache/repos/github.com/moeru-ai/airi/main/README.md#L126】.
• Voice Activity Detection (VAD): Real-time talking detection to trigger responses only during actual speech【/cache/repos/github.com/moeru-ai/airi/main/README.md#L127】.

Mouth: Text-to-Speech Output

AIRI integrates with ElevenLabs TTS for high-quality voice synthesis, delivering natural-sounding speech output for the virtual character【/cache/repos/github.com/moeru-ai/airi/main/README.md#L129】.

Body: Avatar Animation and Rendering

The body module controls visual representation through industry-standard VTuber formats.

• VRM Support: Full control of VRM avatars including bone animations, blend shapes, and real-time rendering【/cache/repos/github.com/moeru-ai/airi/main/README.md#L130-L136】.
• Live2D Support: Control and animation of Live2D models for 2D VTuber presentations【/cache/repos/github.com/moeru-ai/airi/main/README.md#L137-L141】.
• Autonomous Behaviors: Auto-blinking, look-at tracking, and idle eye movements for lifelike presence【/cache/repos/github.com/moeru-ai/airi/main/README.md#L138-L142】.

Modular Architecture and Project Structure

AIRI employs a layered architecture that separates concerns across the codebase, as documented in README.md lines 65-53【/cache/repos/github.com/moeru-ai/airi/main/README.md#L65-L53】 and detailed in AGENTS.md【/cache/repos/github.com/moeru-ai/airi/main/AGENTS.md】.

The architecture flows from core logic through UI primitives to platform-specific applications:

Core → STT, Memory, Server Runtime, xsAI, Factorio/Minecraft agents
UI primitives (ui, ui-transitions, loading-screens, fonts) → Stage-UI
Stage-UI + Core → Apps (stage-web, stage-tamagotchi, stage-pocket)
Server-SDK/Shared → Server-Runtime (provides channel for agents)

Key directories include:

• packages/stage-ui/src/ – UI components and composables powering web and desktop apps
• apps/stage-web/ – Browser-specific entry point and Vite configuration
• apps/stage-tamagotchi/ – Electron-based desktop application
• apps/stage-pocket/ – Capacitor-based mobile application
• packages/server-runtime/ – Server-side channel for agent communication
• packages/xsai/ – LLM provider abstraction layer
• packages/duckdb-wasm/ – In-browser database driver for the Memory feature

Extensible LLM Provider Stack with xsAI

AIRI includes xsAI, a built-in abstraction layer that standardizes communication with dozens of LLM APIs including OpenAI, Claude, Groq, and Mistral. The complete provider list appears under "Support of LLM API Providers" in the README【/cache/repos/github.com/moeru-ai/airi/main/README.md#L110-L130】.

Adding a new provider requires implementing the xsAI contract in packages/xsai/, allowing the core engine to remain agnostic about specific API implementations while supporting emerging models immediately.

Multi-Platform Deployment Options

AIRI supports four distinct deployment targets, all sharing the same core codebase with only the UI layer changing between platforms.

Stage Web (Browser)

Runs fully in the browser using WebGPU, WebAudio, and WebWorkers for client-side inference and rendering.

# Install dependencies

pnpm i

# Start the web UI (http://localhost:5173)

pnpm dev

This command originates from the README【/cache/repos/github.com/moeru-ai/airi/main/README.md#L152-L156】.

Stage Tamagotchi (Desktop)

An Electron-based desktop application with native CUDA/Metal acceleration for GPU-intensive tasks.

# Run the Tamagotchi (desktop) app

pnpm dev:tamagotchi

See README lines 164-168【/cache/repos/github.com/moeru-ai/airi/main/README.md#L164-L168】.

Stage Pocket (Mobile)

A Capacitor-based mobile application for iOS and Android deployment.

# Run on iOS simulator or device

pnpm dev:pocket:ios

Nix Package

For reproducible builds and deployments, AIRI provides a Nix flake:

nix run github:moeru-ai/airi

Quick Start and Configuration Examples

Enabling Discord Voice Bridge

To activate Discord voice channel integration, configure the desktop application settings:

// In the desktop settings UI → Settings → System → General
{
  "enableSecureWebSocket": true,
  "discordVoiceBridge": true
}

The README explains that Discord voice integration requires the desktop app running with elevated permissions and the secure WebSocket toggle enabled【/cache/repos/github.com/moeru-ai/airi/main/README.md#L88-L95】.

Adding a Custom LLM Provider

Extend AIRI's provider support by implementing the xsAI contract:

import { createProvider } from '@proj-airi/xsai';

export const myProvider = createProvider({
  name: 'MyCoolAPI',
  endpoint: 'https://api.mycool.ai/v1/chat/completions',
  headers: { Authorization: `Bearer ${process.env.MYCOOL_API_KEY}` },
  // xsAI expects the same schema as OpenAI ChatCompletion
});

This pattern follows the existing providers in packages/xsai-transformers, allowing immediate integration with the core engine.

Summary

• AIRI is a modular AI-VTuber framework separating core logic from UI layers, enabling deployment across web, desktop, and mobile platforms.
• The Brain integrates with Minecraft, Factorio, Telegram, and Discord while using DuckDB-WASM for local memory and planning WebGPU inference.
• Ears provide browser and Discord audio capture with client-side STT and VAD.
• Mouth uses ElevenLabs for high-quality TTS output.
• Body supports both VRM and Live2D avatars with autonomous behaviors like auto-blinking and look-at tracking.
• The xsAI abstraction layer supports dozens of LLM providers through a unified interface.
• Four deployment targets—Stage Web, Stage Tamagotchi, Stage Pocket, and Nix—share the same core codebase.

Frequently Asked Questions

What platforms does AIRI support?

AIRI supports four primary deployment targets: Stage Web for browser-based execution using WebGPU and WebAudio, Stage Tamagotchi for desktop environments via Electron with native GPU acceleration, Stage Pocket for mobile devices using Capacitor, and a Nix package for reproducible system-level deployments. All platforms share the identical core engine, differing only in their UI presentation layers.

How does AIRI handle AI model providers?

AIRI implements an abstraction layer called xsAI that standardizes communication across dozens of LLM APIs including OpenAI, Claude, Groq, and Mistral. Located in packages/xsai/, this layer allows developers to add new providers by implementing a simple contract that follows the OpenAI ChatCompletion schema, enabling the core engine to remain provider-agnostic while supporting emerging models immediately.

Can AIRI run entirely in the browser without external services?

Yes, AIRI is designed for fully client-side operation. The framework uses DuckDB-WASM (pglite) for in-browser vector storage and memory management, eliminating the need for external databases. Additionally, AIRI supports client-side speech-to-text (STT) and has planned support for Pure-WebGPU inference, which will enable complete local model execution without server round-trips, though some features like ElevenLabs TTS currently require external API calls.

What avatar formats does AIRI support?

AIRI supports both industry-standard VTuber formats: VRM for 3D avatars and Live2D for 2D models. The framework provides full control over VRM bone animations and blend shapes, while also supporting Live2D model control and animation. Both formats include autonomous behaviors such as auto-blinking, look-at tracking, and idle eye movements to create lifelike presence, implemented in the body control modules referenced in README.md lines 130-142.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}