# Structure of the Plugin System for Omi Apps: FastAPI Architecture Explained

> Discover the modular FastAPI plugin system structure for Omi Apps. Learn how third-party integrations register routers and deploy via Modal for seamless frontend discovery.

- Repository: [omi/omi](https://github.com/basedhardware/omi)
- Tags: architecture
- Published: 2026-02-26

---

**Omi Apps uses a modular FastAPI-based plugin architecture where third-party integrations live as independent Python packages under `plugins/`, register their own FastAPI routers in [`plugins/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/main.py), and deploy via Modal, while the frontend discovers them through both an approved backend endpoint and a community JSON catalog.**

The basedhardware/omi repository implements a four-layer plugin system that allows developers to extend the Omi AI wearable's capabilities without modifying core backend code. This architecture separates plugin logic into self-contained modules, exposes them through a unified FastAPI surface, and dynamically injects plugin context into LLM conversations. Understanding the structure of the plugin system for Omi Apps is essential for building custom integrations that interact with conversation data and external services.

## Plugin Package Structure

Each Omi plugin resides in its own sub-directory under the `plugins/` folder, following a consistent layout that separates concerns between router logic and data models.

The directory structure follows this pattern:

```

plugins/
  ├─ basic/
  │    └─ conversation_created.py   # Router for basic plugin events

  ├─ oauth/
  │    └─ conversation_created.py   # Router for OAuth-based plugins

  ├─ zapier/
  │    └─ conversation_created.py   # Router for Zapier integration

  ├─ omi-shopify-app/
  │    ├─ models.py                 # Pydantic models for Shopify

  │    └─ main.py                   # Core plugin logic

  └─ models.py                       # Shared types: PluginResult, Conversation

```

The shared [`plugins/models.py`](https://github.com/basedhardware/omi/blob/main/plugins/models.py) file defines the data structures that every plugin uses to communicate with the core system. According to the source code, this includes `PluginResult` which holds a plugin’s output and is imported by all plugin implementations to ensure type consistency across the ecosystem. `PluginResult` and `Conversation` types are defined here to standardize how plugins read conversation data and return processing results.

## FastAPI Router Registration

The central integration point for all plugins is [`plugins/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/main.py), which imports individual plugin routers and registers them with the main FastAPI application instance.

As implemented in [`plugins/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/main.py) (lines 70-88), the system wires each plugin's router into the central app:

```python

# plugins/main.py

app.include_router(basic_conversation_created_router.router)      # Basic plugin

app.include_router(oauth_conversation_created_router.router)      # OAuth plugin

app.include_router(zapier_conversation_created_router.router)     # Zapier plugin

app.include_router(multion_router.router)                         # External integration

# … (other routers)

```

This file serves as the entry point for the Modal-hosted service and acts as the single source of truth for which plugins are currently active. When adding a new plugin, developers must import their router module and call `app.include_router()` to expose the plugin's endpoints to the network.

## Containerized Deployment with Modal

The plugin service runs as a Modal application, utilizing serverless containers to handle HTTP requests without managing infrastructure. The deployment configuration in [`plugins/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/main.py) (lines 54-66) uses Modal-specific decorators to build the runtime environment.

The `@modal_app.function` decorator constructs a container image that pip-installs dependencies from [`requirements.txt`](https://github.com/basedhardware/omi/blob/main/requirements.txt) and mounts static template directories. The function then returns the FastAPI application via the `@asgi_app()` decorator, allowing Modal to serve the HTTP traffic:

```python
@modal_app.function(
    image=image,
    mounts=[modal.Mount.from_local_dir("./templates", remote_path="/templates")],
)
@modal.asgi_app()
def fastapi_app():
    return app

```

This architecture ensures that plugins run in isolated, reproducible environments while sharing the same network surface through the unified FastAPI app.

## Frontend Discovery Mechanisms

The web frontend discovers available plugins through two distinct channels: an official approved catalog and a community-maintained JSON file.

### Approved Apps Endpoint

The backend exposes `/v1/approved-apps` to serve curated plugins. The frontend consumes this through [`web/frontend/src/lib/api/apps.ts`](https://github.com/basedhardware/omi/blob/main/web/frontend/src/lib/api/apps.ts):

```typescript
const response = await fetch(`${envConfig.API_URL}/v1/approved-apps?...`);

```

TypeScript definitions in [`web/frontend/src/types/plugins/plugins.types.ts`](https://github.com/basedhardware/omi/blob/main/web/frontend/src/types/plugins/plugins.types.ts) (lines 2-14) mirror the backend schema, defining the `Plugin` interface with properties like `id`, `name`, `capabilities`, and `prompts` to ensure type safety across the API boundary.

### Community Plugin Catalog

For user-contributed plugins that don't require backend approval, the system reads from [`community-plugins.json`](https://github.com/basedhardware/omi/blob/main/community-plugins.json) at the repository root. The frontend imports this file via `getCommunityPlugins()` to display community apps alongside official ones, allowing any developer to publish a plugin by submitting a pull request to update the JSON file.

## LLM Integration and Conversation Context

When a user selects a plugin, the backend conversation engine stores the chosen `App` object in the state under `plugin_selected`. In [`backend/utils/llm/chat.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/llm/chat.py) (lines 426-440), the system injects plugin-specific instructions into the LLM prompt generator.

The LLM receives a `<plugin_instructions>` block containing the plugin’s personality, description, and custom prompts. This allows plugins to dynamically alter the AI's behavior during conversations. The retrieval engine in [`backend/utils/retrieval/graph.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/retrieval/graph.py) manages the state transitions to ensure the correct plugin context is active throughout the conversation lifecycle.

## Creating a Custom Omi Plugin

To build a new plugin, create a Python module with a FastAPI router and register it in the central application.

First, implement your plugin logic:

```python

# plugins/my-awesome-plugin/main.py

from fastapi import APIRouter, HTTPException
from pydantic import BaseModel

router = APIRouter(prefix="/my-awesome-plugin")

class Input(BaseModel):
    uid: str
    text: str

@router.post("/process")
async def process(req: Input):
    # Your plugin logic here

    return {"result": f"Handled {req.text} for {req.uid}"}

```

Then register the router in the plugin entry point:

```python

# plugins/main.py (excerpt)

from my_awesome_plugin import main as my_plugin_router

app.include_router(my_plugin_router.router)  # Makes endpoints live

```

To consume plugins in the frontend:

```typescript
import { getApprovedApps } from '@/src/lib/api/apps';

export async function loadPlugins() {
  const plugins = await getApprovedApps();   // returns Plugin[]
  return plugins.filter(p => p.capabilities.includes('chat'));
}

```

## Summary

- **Modular Structure**: Plugins live as independent packages under `plugins/`, each with their own routers and optional data models, while sharing common types through [`plugins/models.py`](https://github.com/basedhardware/omi/blob/main/plugins/models.py).
- **Router Aggregation**: [`plugins/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/main.py) serves as the integration point, importing and registering all plugin routers with the central FastAPI application.
- **Serverless Deployment**: The entire plugin surface deploys via Modal using `@modal_app.function` and `@asgi_app()` decorators, containerizing dependencies automatically.
- **Dual Discovery**: Frontend applications fetch official plugins from `/v1/approved-apps` and community plugins from [`community-plugins.json`](https://github.com/basedhardware/omi/blob/main/community-plugins.json), with TypeScript types ensuring API consistency.
- **LLM Context Injection**: Selected plugins modify conversation behavior through `<plugin_instructions>` blocks injected by [`backend/utils/llm/chat.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/llm/chat.py), allowing dynamic personality and capability changes.

## Frequently Asked Questions

### How do I register a new plugin in the Omi system?

To register a new plugin, create a subdirectory under `plugins/` containing a FastAPI router that defines your endpoints. Import this router in [`plugins/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/main.py) and add it using `app.include_router()`. The plugin automatically becomes available when the Modal service redeploys. You must also add your plugin to [`community-plugins.json`](https://github.com/basedhardware/omi/blob/main/community-plugins.json) or submit it for inclusion in the approved apps endpoint to make it discoverable by frontend clients.

### What is the difference between approved apps and community plugins?

**Approved apps** are vetted by the Omi team and served via the `/v1/approved-apps` backend endpoint, offering higher trust and potential backend integration privileges. **Community plugins** live in the [`community-plugins.json`](https://github.com/basedhardware/omi/blob/main/community-plugins.json) file at the repository root and are fetched directly by the frontend without backend moderation, allowing faster experimentation but with fewer guarantees about security or reliability.

### How do plugins influence the LLM's responses during conversations?

When a user activates a plugin, the conversation engine stores the selection in state under `plugin_selected`. According to the source code in [`backend/utils/llm/chat.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/llm/chat.py), the system injects a `<plugin_instructions>` XML block into the LLM prompt containing the plugin's description, personality settings, and custom prompts. This contextual injection modifies how the AI interprets and responds to user messages while the plugin remains active.

### What infrastructure hosts the Omi plugin system?

The plugin system runs on **Modal**, a serverless compute platform. The [`plugins/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/main.py) file defines a Modal app using the `@modal_app.function` decorator to build a container image with all Python dependencies, then exposes the FastAPI application through `@modal.asgi_app()`. This setup handles automatic scaling, container management, and HTTP request routing without requiring manual server configuration.