How Omi Tool Integrations Work: A Deep Dive into Gmail, Calendar, and Google Contacts

Omi tool integrations convert external Google services into chat-aware functions that the AI can invoke during conversations by using OAuth 2.0 authentication, shared utility layers, and LangChain-compatible tool decorators.

The Omi project (basedhardware/omi) implements a sophisticated tool system that allows its AI assistant to interact with Gmail, Google Calendar, and Google Contacts on behalf of users. This architecture separates authentication handling, API request management, and service-specific logic into distinct layers, enabling the LLM to trigger real-world actions through clean, high-level interfaces.

OAuth 2.0 Authentication and Token Storage

Every Omi tool integration begins with a one-time authorization flow. When a user connects their Google account, the system handles the OAuth 2.0 exchange in plugins/omi-google-calendar-app/main.py (lines 27-45).

The resulting access token and refresh token are persisted in Firestore through the database.users module. This storage mechanism ensures that subsequent API calls can proceed without requiring the user to re-authenticate, while maintaining secure separation between user credentials and the application logic.

Shared Google Utilities and Token Management

The backend/utils/retrieval/tools/google_utils.py file provides the foundational infrastructure for all Google service interactions. It exposes two critical functions:

• refresh_google_token() (lines 15-58) – Automatically exchanges a stored refresh token for a new access token when the current one expires, updating the Firestore record with the new credentials.
• google_api_request() – A thin wrapper around requests.request() that injects the Bearer authorization header, logs the outgoing call for debugging, and raises explicit exceptions for non-200 HTTP responses.

This utility layer ensures consistent error handling and authentication refresh logic across Gmail, Calendar, and Contacts integrations.

Integration Base Layer for Access Validation

Before any tool executes a Google API call, it must validate that the user has authorized the service and that the access token is current. The backend/utils/retrieval/tools/integration_base.py module handles this through:

• prepare_access() (lines 84-107) – Extracts the uid from the LangChain RunnableConfig, queries the database for the requested integration (google_calendar or gmail), and returns a tuple containing the user ID, integration record, valid access token, and error message (if any).
• retry_on_auth() (lines 110-138) – A decorator that catches authentication-related failures, triggers refresh_google_token(), updates the stored integration, and retries the original API call exactly once.

This abstraction allows individual tool implementations to remain agnostic of OAuth mechanics, focusing solely on business logic.

Service-Specific Tool Implementations

Each external service exposes functionality through Python functions decorated with LangChain's @tool decorator, making them discoverable and callable by the LLM during chat sessions.

Gmail Tools

The backend/utils/retrieval/tools/gmail_tools.py module provides get_gmail_messages_tool, which fetches recent messages, parses the JSON response, and returns a human-readable markdown list. The implementation handles pagination and extracts key fields (subject, sender, snippet) to keep responses concise for chat contexts.

Google Calendar Tools

The backend/utils/retrieval/tools/calendar_tools.py file contains a comprehensive suite of calendar operations:

• get_calendar_events_tool – Retrieves events within a specified date range, handling timezone conversion and formatting results as markdown.
• create_calendar_event_tool – Accepts natural language parameters (title, start/end times, attendees), resolves attendee names to email addresses, and creates the event via the Calendar API.
• update_calendar_event_tool – Modifies existing events by ID or search criteria.
• list_calendars – Exposed via the FastAPI manifest to show available calendars.

Contact Resolution

To support natural language attendee names (e.g., "John Doe" instead of email addresses), the system implements contact resolution in calendar_tools.py:

• search_google_contacts() – Queries the Google People API for matching contacts.
• resolve_attendee_to_email() – Iterates through provided attendee strings, attempts to match names against the user's Google Contacts, and returns a validated list of email addresses for calendar invitations.

FastAPI Endpoints and Tool Manifest

The Google Calendar plugin (plugins/omi-google-calendar-app/main.py) exposes HTTP routes that bridge the chat interface with the underlying tool implementations:

• /tools/list_events, /tools/create_event, /tools/update_event, /tools/delete_event – These endpoints receive JSON payloads containing the uid and parameters, then delegate to the corresponding @tool functions.

The plugin also serves an Omi tools manifest at /.well-known/omi-tools.json, which advertises available tools to the frontend, enabling dynamic discovery of capabilities without hardcoding integration logic.

Complete Execution Flow Example

When a user asks "What's on my calendar this week?", the following sequence executes:

1. Intent Recognition – The LLM identifies the need for calendar data and selects get_calendar_events_tool based on the manifest.
2. Access Preparation – LangChain invokes the tool, which calls prepare_access() in integration_base.py to extract the uid from RunnableConfig and retrieve valid Google Calendar credentials.
3. Token Validation – If the access token expired, retry_on_auth() catches the failure and triggers refresh_google_token() from google_utils.py, updating the Firestore record before retrying.
4. API Execution – get_google_calendar_events() constructs the timeMin/timeMax query parameters and invokes google_api_request() to fetch raw event data.
5. Response Formatting – The tool parses the JSON response and formats it as concise markdown (event titles, times, locations) suitable for chat display.
6. Chat Integration – The formatted result returns to the LLM, which presents the calendar summary to the user in natural language.

Summary

• OAuth Management: Omi stores refresh tokens in Firestore and automatically renews access tokens via refresh_google_token() when they expire.
• Unified Utilities: The google_utils.py module provides consistent HTTP handling and authentication headers across all Google services.
• Access Abstraction: integration_base.py validates user connections and handles retry logic, keeping tool implementations focused on business logic.
• LangChain Integration: Service-specific tools in gmail_tools.py and calendar_tools.py use the @tool decorator to expose Gmail and Calendar operations as chat-aware functions.
• Contact Resolution: Natural language attendee names are automatically resolved to email addresses via the Google People API.
• FastAPI Bridge: The plugin exposes HTTP endpoints and a tools manifest, allowing the frontend to discover and invoke capabilities dynamically.

Frequently Asked Questions

How does Omi handle expired Google access tokens?

When a Google API call fails due to an expired token, the retry_on_auth() decorator in backend/utils/retrieval/tools/integration_base.py catches the authentication error. It automatically invokes refresh_google_token() from google_utils.py, which exchanges the stored refresh token for a new access token, updates the Firestore record, and retries the original API request exactly once.

Can Omi create calendar events with attendee names instead of email addresses?

Yes. When creating events via create_calendar_event_tool in backend/utils/retrieval/tools/calendar_tools.py, you can provide attendee names as strings (e.g., "Alex Johnson"). The tool calls resolve_attendee_to_email(), which queries the Google People API via search_google_contacts() to match names against the user's Google Contacts and automatically resolves them to valid email addresses before sending the invitation.

What happens when a user asks about their email in the Omi chat interface?

When a user queries their Gmail, the LLM identifies the intent and invokes get_gmail_messages_tool from backend/utils/retrieval/tools/gmail_tools.py. The tool uses prepare_access() to validate the user's Gmail integration and retrieve a valid access token. It then fetches recent messages via the Gmail API, parses the JSON response to extract subjects, senders, and snippets, and returns a formatted markdown list that the LLM presents to the user.

Where are the OAuth tokens stored in the Omi architecture?

OAuth tokens are stored in Firestore through the database.users module. When a user completes the Google OAuth flow in plugins/omi-google-calendar-app/main.py, the resulting access and refresh tokens are persisted in the user's document. Subsequent API calls retrieve these tokens via prepare_access() in integration_base.py, and refreshed tokens are updated back to Firestore by refresh_google_token() in google_utils.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}