# How the Macro Email Service Handles Multi-Account Gmail Synchronization

> Learn how the Macro email service synchronizes multiple Gmail accounts using OAuth, Google Watch, and Pub/Sub for real-time updates and efficient data fetching.

- Repository: [Macro/macro](https://github.com/macro-inc/macro)
- Tags: internals
- Published: 2026-08-14

---

**The Macro email service synchronizes multiple Gmail accounts per user through OAuth-linked accounts with unique `link_id` identifiers, Google Watch subscriptions for real-time notifications, and parallel Pub/Sub workers that fetch and store messages, contacts, and attachments.**

The Macro email service is designed from the ground up to support **multi-account Gmail synchronization**, allowing a single user to connect and sync many Gmail accounts simultaneously without performance degradation or interference between accounts. This article examines the architecture and implementation details directly from the `macro-inc/macro` source code.

## Account Linking and OAuth Token Management

Every synchronized Gmail account begins with a **link**—a persistent association between a Macro user and a specific Gmail account. The `enable_gmail_sync_for` helper in [`services/email_service/src/api/email/init.rs`](https://github.com/macro-inc/macro/blob/main/services/email_service/src/api/email/init.rs) creates this link and securely stores the OAuth refresh token:

```rust
// Conceptual flow from init.rs lines 389+
// enable_gmail_sync_for creates link record with:
// - user_id (Macro user)
// - gmail_address 
// - refresh_token (OAuth)
// - link_id (unique identifier for this account link)

```

The `link_id` becomes the central key that isolates all per-account operations. When a worker needs to interact with Gmail, it provides this identifier to retrieve the correct credentials.

### Centralized Token State

At service startup in [`services/email_service/src/main.rs`](https://github.com/macro-inc/macro/blob/main/services/email_service/src/main.rs) (around line 200), the system constructs **`GmailTokenState`**—a shared cache of **`GmailTokenProviderImpl`** instances keyed by `link_id`:

```rust
// From main.rs – GmailTokenState construction
let gmail_token_state = Arc::new(GmailTokenState::new(
    token_providers, // HashMap<link_id, GmailTokenProviderImpl>
));

```

This design avoids redundant token refresh operations. Multiple workers can reference the same underlying token provider for an account, and the provider handles OAuth refresh transparently when tokens expire.

## Google Watch Subscriptions for Real-Time Sync

The service does not poll Gmail. Instead, it leverages **Google's push notification API** through the `events.watch` endpoint. The **watch subscription lifecycle** is managed in [`services/email_service/src/pubsub/link_manager/process.rs`](https://github.com/macro-inc/macro/blob/main/services/email_service/src/pubsub/link_manager/process.rs) (around line 160 in the "Refresh operation"):

- Creates or renews a `watch` for each linked account
- Stores subscription details (expiration, resource ID) with the link record
- Automatically refreshes watches before they expire

When new mail arrives in any linked Gmail account, Google publishes a message to Macro's configured Pub/Sub topic. This fan-out mechanism enables near-real-time synchronization across hundreds or thousands of accounts without excessive API quota consumption.

## Parallel Inbox Synchronization Workers

The **`inbox_sync::worker`** in [`services/email_service/src/pubsub/inbox_sync/worker.rs`](https://github.com/macro-inc/macro/blob/main/services/email_service/src/pubsub/inbox_sync/worker.rs) forms the core synchronization engine. This worker:

1. Consumes watch notification messages from Pub/Sub
2. Extracts the `link_id` to identify which account changed
3. Retrieves a fresh token from `GmailTokenState`
4. Calls Gmail's `listMessages` or `listThreads` with appropriate filters
5. Upserts the fetched data into Macro's internal `email_db`

Critically, **workers run in parallel for every active link**. The architecture scales horizontally—additional worker instances can process the same topic, and each message carries the `link_id` ensuring correct account isolation.

## Contact Synchronization Pipeline

Contacts are handled separately from messages via **`sync_contacts`** in [`services/email_service/src/util/sync_contacts.rs`](https://github.com/macro-inc/macro/blob/main/services/email_service/src/util/sync_contacts.rs). This function implements **incremental synchronization** using per-account sync tokens:

```rust
// Conceptual flow from sync_contacts.rs
pub async fn sync_contacts(db: &Db, link_id: &str, token_state: &GmailTokenState) {
    // 1. Retrieve stored sync_token for this link_id from database
    let sync_token = db.get_contact_sync_token(link_id).await;
    
    // 2. Call Gmail People API with token (returns only changes since last sync)
    let response = people_api.list_connections()
        .sync_token(sync_token)
        .page_token(page_token)
        .await;
    
    // 3. Upsert contacts and store new sync_token for next iteration
    db.upsert_contacts(link_id, response.connections).await;
    db.update_contact_sync_token(link_id, response.next_sync_token).await;
}

```

If a sync token expires, the function implements fallback logic to perform a full resynchronization, then establishes a fresh incremental token.

## Attachment Streaming and Storage

When messages contain attachments, the **`upload_attachment`** helper in [`services/email_service/src/util/upload_attachment.rs`](https://github.com/macro-inc/macro/blob/main/services/email_service/src/util/upload_attachment.rs) handles retrieval and storage:

1. Fetches raw attachment data from Gmail using the per-account token
2. Streams bytes directly to Macro's S3-backed storage
3. Generates a presigned URL
4. Attaches the URL reference to the message record

This approach avoids storing binary data in the primary database while maintaining security through time-limited, revocable access URLs.

## Data Consistency Through Backfill and Cleanup

Two additional worker types ensure **eventual consistency** across all linked accounts:

| Worker | Location | Purpose |
|--------|----------|---------|
| **Backfill worker** | [`src/pubsub/backfill/worker.rs`](https://github.com/macro-inc/macro/blob/main/src/pubsub/backfill/worker.rs) | Periodically scans for missed threads or contacts; imports any gaps detected |
| **CRM cleanup worker** | [`src/pubsub/crm_cleanup/worker.rs`](https://github.com/macro-inc/macro/blob/main/src/pubsub/crm_cleanup/worker.rs) | Removes stale data when a Gmail account is disconnected |

Both operate per-`link_id`, maintaining isolation guarantees. The backfill worker is particularly important for recovering from missed Pub/Sub messages or temporary Gmail API outages.

## Summary

- **Account isolation** via unique `link_id` identifiers enables safe multi-account operation
- **`GmailTokenState`** centralizes OAuth management while keeping credentials scoped per account
- **Google Watch subscriptions** push notifications in real-time without polling overhead
- **Parallel workers** process inbox changes, contacts, and attachments simultaneously across all linked accounts
- **Periodic backfill** guarantees data completeness even when notifications fail

The Macro email service achieves scalable multi-account Gmail synchronization through careful separation of per-account state and efficient shared infrastructure for token management and event distribution.

## Frequently Asked Questions

### How does the service handle OAuth token expiration?

The `GmailTokenProviderImpl` caches tokens and automatically refreshes them using the stored refresh token when API calls return 401 errors. The `GmailTokenState` ensures all workers for a given `link_id` share the same provider instance, preventing race conditions during refresh.

### What happens if a Google Watch subscription expires?

The **link manager process** in [`src/pubsub/link_manager/process.rs`](https://github.com/macro-inc/macro/blob/main/src/pubsub/link_manager/process.rs) monitors subscription expiration times and proactively refreshes watches before they lapse. If a watch does expire, the backfill worker eventually catches any missed messages once the subscription is re-established.

### Can users sync more than one Gmail account simultaneously?

Yes. Every linked account receives its own `link_id`, watch subscription, and parallel worker allocation. The architecture imposes no hard limit on accounts per user; practical limits depend on Gmail API quota and infrastructure scaling.

### How are attachments secured after upload?

Attachments stream directly from Gmail to S3 without persisting on local filesystems. The generated **presigned URLs** include time-limited access tokens, and the Macro application validates user permissions before serving attachment content.