# How Omi Tool Integrations Work: A Deep Dive into Gmail, Calendar, and Google Contacts

> Discover how Omi tool integrations leverage OAuth 2.0 and LangChain to connect Gmail, Calendar, and Google Contacts for AI-powered chat functions. Explore the underlying technology.

- Repository: [omi/omi](https://github.com/basedhardware/omi)
- Tags: deep-dive
- Published: 2026-02-26

---

**Omi tool integrations convert external Google services into chat-aware functions that the AI can invoke during conversations by using OAuth 2.0 authentication, shared utility layers, and LangChain-compatible tool decorators.**

The Omi project (basedhardware/omi) implements a sophisticated tool system that allows its AI assistant to interact with Gmail, Google Calendar, and Google Contacts on behalf of users. This architecture separates authentication handling, API request management, and service-specific logic into distinct layers, enabling the LLM to trigger real-world actions through clean, high-level interfaces.

## OAuth 2.0 Authentication and Token Storage

Every Omi tool integration begins with a one-time authorization flow. When a user connects their Google account, the system handles the OAuth 2.0 exchange in [`plugins/omi-google-calendar-app/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/omi-google-calendar-app/main.py) (lines 27-45).

The resulting **access token** and **refresh token** are persisted in Firestore through the `database.users` module. This storage mechanism ensures that subsequent API calls can proceed without requiring the user to re-authenticate, while maintaining secure separation between user credentials and the application logic.

## Shared Google Utilities and Token Management

The [`backend/utils/retrieval/tools/google_utils.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/retrieval/tools/google_utils.py) file provides the foundational infrastructure for all Google service interactions. It exposes two critical functions:

- **`refresh_google_token()`** (lines 15-58) – Automatically exchanges a stored refresh token for a new access token when the current one expires, updating the Firestore record with the new credentials.
- **`google_api_request()`** – A thin wrapper around `requests.request()` that injects the `Bearer` authorization header, logs the outgoing call for debugging, and raises explicit exceptions for non-200 HTTP responses.

This utility layer ensures consistent error handling and authentication refresh logic across Gmail, Calendar, and Contacts integrations.

## Integration Base Layer for Access Validation

Before any tool executes a Google API call, it must validate that the user has authorized the service and that the access token is current. The [`backend/utils/retrieval/tools/integration_base.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/retrieval/tools/integration_base.py) module handles this through:

- **`prepare_access()`** (lines 84-107) – Extracts the `uid` from the LangChain `RunnableConfig`, queries the database for the requested integration (`google_calendar` or `gmail`), and returns a tuple containing the user ID, integration record, valid access token, and error message (if any).
- **`retry_on_auth()`** (lines 110-138) – A decorator that catches authentication-related failures, triggers `refresh_google_token()`, updates the stored integration, and retries the original API call exactly once.

This abstraction allows individual tool implementations to remain agnostic of OAuth mechanics, focusing solely on business logic.

## Service-Specific Tool Implementations

Each external service exposes functionality through Python functions decorated with LangChain's `@tool` decorator, making them discoverable and callable by the LLM during chat sessions.

### Gmail Tools

The [`backend/utils/retrieval/tools/gmail_tools.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/retrieval/tools/gmail_tools.py) module provides `get_gmail_messages_tool`, which fetches recent messages, parses the JSON response, and returns a human-readable markdown list. The implementation handles pagination and extracts key fields (subject, sender, snippet) to keep responses concise for chat contexts.

### Google Calendar Tools

The [`backend/utils/retrieval/tools/calendar_tools.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/retrieval/tools/calendar_tools.py) file contains a comprehensive suite of calendar operations:

- **`get_calendar_events_tool`** – Retrieves events within a specified date range, handling timezone conversion and formatting results as markdown.
- **`create_calendar_event_tool`** – Accepts natural language parameters (title, start/end times, attendees), resolves attendee names to email addresses, and creates the event via the Calendar API.
- **`update_calendar_event_tool`** – Modifies existing events by ID or search criteria.
- **`list_calendars`** – Exposed via the FastAPI manifest to show available calendars.

### Contact Resolution

To support natural language attendee names (e.g., "John Doe" instead of email addresses), the system implements contact resolution in [`calendar_tools.py`](https://github.com/basedhardware/omi/blob/main/calendar_tools.py):

- **`search_google_contacts()`** – Queries the Google People API for matching contacts.
- **`resolve_attendee_to_email()`** – Iterates through provided attendee strings, attempts to match names against the user's Google Contacts, and returns a validated list of email addresses for calendar invitations.

## FastAPI Endpoints and Tool Manifest

The Google Calendar plugin ([`plugins/omi-google-calendar-app/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/omi-google-calendar-app/main.py)) exposes HTTP routes that bridge the chat interface with the underlying tool implementations:

- **`/tools/list_events`**, **`/tools/create_event`**, **`/tools/update_event`**, **`/tools/delete_event`** – These endpoints receive JSON payloads containing the `uid` and parameters, then delegate to the corresponding `@tool` functions.

The plugin also serves an **Omi tools manifest** at [`/.well-known/omi-tools.json`](https://github.com/basedhardware/omi/blob/main//.well-known/omi-tools.json), which advertises available tools to the frontend, enabling dynamic discovery of capabilities without hardcoding integration logic.

## Complete Execution Flow Example

When a user asks "What's on my calendar this week?", the following sequence executes:

1. **Intent Recognition** – The LLM identifies the need for calendar data and selects `get_calendar_events_tool` based on the manifest.
2. **Access Preparation** – LangChain invokes the tool, which calls `prepare_access()` in [`integration_base.py`](https://github.com/basedhardware/omi/blob/main/integration_base.py) to extract the `uid` from `RunnableConfig` and retrieve valid Google Calendar credentials.
3. **Token Validation** – If the access token expired, `retry_on_auth()` catches the failure and triggers `refresh_google_token()` from [`google_utils.py`](https://github.com/basedhardware/omi/blob/main/google_utils.py), updating the Firestore record before retrying.
4. **API Execution** – `get_google_calendar_events()` constructs the `timeMin`/`timeMax` query parameters and invokes `google_api_request()` to fetch raw event data.
5. **Response Formatting** – The tool parses the JSON response and formats it as concise markdown (event titles, times, locations) suitable for chat display.
6. **Chat Integration** – The formatted result returns to the LLM, which presents the calendar summary to the user in natural language.

## Summary

- **OAuth Management**: Omi stores refresh tokens in Firestore and automatically renews access tokens via `refresh_google_token()` when they expire.
- **Unified Utilities**: The [`google_utils.py`](https://github.com/basedhardware/omi/blob/main/google_utils.py) module provides consistent HTTP handling and authentication headers across all Google services.
- **Access Abstraction**: [`integration_base.py`](https://github.com/basedhardware/omi/blob/main/integration_base.py) validates user connections and handles retry logic, keeping tool implementations focused on business logic.
- **LangChain Integration**: Service-specific tools in [`gmail_tools.py`](https://github.com/basedhardware/omi/blob/main/gmail_tools.py) and [`calendar_tools.py`](https://github.com/basedhardware/omi/blob/main/calendar_tools.py) use the `@tool` decorator to expose Gmail and Calendar operations as chat-aware functions.
- **Contact Resolution**: Natural language attendee names are automatically resolved to email addresses via the Google People API.
- **FastAPI Bridge**: The plugin exposes HTTP endpoints and a tools manifest, allowing the frontend to discover and invoke capabilities dynamically.

## Frequently Asked Questions

### How does Omi handle expired Google access tokens?

When a Google API call fails due to an expired token, the `retry_on_auth()` decorator in [`backend/utils/retrieval/tools/integration_base.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/retrieval/tools/integration_base.py) catches the authentication error. It automatically invokes `refresh_google_token()` from [`google_utils.py`](https://github.com/basedhardware/omi/blob/main/google_utils.py), which exchanges the stored refresh token for a new access token, updates the Firestore record, and retries the original API request exactly once.

### Can Omi create calendar events with attendee names instead of email addresses?

Yes. When creating events via `create_calendar_event_tool` in [`backend/utils/retrieval/tools/calendar_tools.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/retrieval/tools/calendar_tools.py), you can provide attendee names as strings (e.g., "Alex Johnson"). The tool calls `resolve_attendee_to_email()`, which queries the Google People API via `search_google_contacts()` to match names against the user's Google Contacts and automatically resolves them to valid email addresses before sending the invitation.

### What happens when a user asks about their email in the Omi chat interface?

When a user queries their Gmail, the LLM identifies the intent and invokes `get_gmail_messages_tool` from [`backend/utils/retrieval/tools/gmail_tools.py`](https://github.com/basedhardware/omi/blob/main/backend/utils/retrieval/tools/gmail_tools.py). The tool uses `prepare_access()` to validate the user's Gmail integration and retrieve a valid access token. It then fetches recent messages via the Gmail API, parses the JSON response to extract subjects, senders, and snippets, and returns a formatted markdown list that the LLM presents to the user.

### Where are the OAuth tokens stored in the Omi architecture?

OAuth tokens are stored in Firestore through the `database.users` module. When a user completes the Google OAuth flow in [`plugins/omi-google-calendar-app/main.py`](https://github.com/basedhardware/omi/blob/main/plugins/omi-google-calendar-app/main.py), the resulting access and refresh tokens are persisted in the user's document. Subsequent API calls retrieve these tokens via `prepare_access()` in [`integration_base.py`](https://github.com/basedhardware/omi/blob/main/integration_base.py), and refreshed tokens are updated back to Firestore by `refresh_google_token()` in [`google_utils.py`](https://github.com/basedhardware/omi/blob/main/google_utils.py).