How the gogcli Gmail Watch Command Uses Pub/Sub: Implementation Guide

The gogcli Gmail watch command uses Google Cloud Pub/Sub to receive push notifications for new emails by starting a watch request that registers a Pub/Sub topic, then running an HTTP server to validate and process incoming push payloads that trigger Gmail History API calls.

The steipete/gogcli repository implements a complete Pub/Sub-based watch pipeline for Gmail notifications. This Go-based CLI tool manages the full lifecycle of Gmail push subscriptions, from initial registration to message processing and webhook delivery.

Starting a Gmail Watch with Pub/Sub

The gmail watch start command initiates the Pub/Sub subscription by constructing a gmail.WatchRequest and registering it with the Gmail API.

Building the Watch Request

In internal/cmd/gmail_watch_cmds.go, the requestGmailWatch function builds the request structure:

req := &gmail.WatchRequest{
    TopicName: topicName,  // e.g., "projects/myproj/topics/gmail-push"
    LabelIds:  labelIDs,    // Optional filter like ["INBOX", "Label_1"]
}

The function calls svc.Users.Watch("me", req).Do() to register the topic with Gmail. The API returns a gmail.WatchResponse containing the historyId and expiration time.

Persisting Watch State

The buildWatchState function transforms the response into a gmailWatchState structure that stores:

• Account email address
• Pub/Sub topic name
• Label filter list
• Expiration timestamp
• Renewal deadline
• Configured webhook URL ("hook")
• Current history ID

The state persists to $HOME/.config/gogcli/state/gmail-watch/<sanitized-account>.json via the Save method in internal/cmd/gmail_watch_state.go.

Receiving Pub/Sub Push Notifications

The gmail watch serve command starts an HTTP server that receives Pub/Sub push deliveries.

HTTP Server Configuration

The server listens on --bind and --port (default :8788) with a POST endpoint at --path (default /gmail-pubsub). In internal/cmd/gmail_watch_server.go, the ServeHTTP method handles incoming requests:

func (s *watchServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    // Validate authorization
    // Decode Pub/Sub payload
    // Process Gmail history
}

Validating Incoming Pushes

The authorize function implements two validation modes:

1. OIDC Token Verification: When --verify-oidc is set, the server validates the bearer token using Google's OIDC validator (idtoken.NewValidator) against the configured --oidc-audience and --oidc-email.

2. Shared Token: Otherwise, the server compares a static token from --shared-token using constant-time comparison to prevent timing attacks.

Decoding the Pub/Sub Payload

The decodeGmailPushPayload function unmarshals the base64-encoded data into a gmailPushPayload structure containing:

• historyId: The Gmail history checkpoint
• messageId: The specific message reference
• Email address: The target account

Pushes for different email addresses are silently ignored to prevent cross-account contamination.

Processing Gmail History Updates

Once authenticated and decoded, the handlePush function orchestrates Gmail data retrieval.

History ID Management

The server retrieves the starting history ID from persistent storage using store.StartHistoryID. If the push contains an older history ID than the stored value, the push is discarded as duplicate or out-of-order.

For valid pushes, the server calls the Gmail History.list API to retrieve all changes since the starting ID.

Fetching and Filtering Messages

The fetchMessages function in internal/cmd/gmail_watch_server.go processes each messageAdded entry:

1. Batch Fetch: Calls svc.Users.Messages.Get to retrieve full message metadata
2. Body Retrieval: If --include-body is set and a webhook is configured, fetches the text/plain body content up to --max-bytes
3. Label Filtering: Strips out labels listed in --exclude-labels (e.g., "SPAM", "TRASH") before building the payload

Webhook Delivery

If a webhook URL was supplied via --hook-url, the sendHook function POSTs a JSON gmailHookPayload containing:

• Message metadata (ID, thread ID, labels)
• Subject and sender information
• Body content (if enabled)
• History ID context

The request includes the --hook-token as a Bearer token if configured. The payload respects size limits set by --max-bytes.

Managing Watch Lifecycle

The CLI provides commands to maintain and terminate Pub/Sub subscriptions.

Renewing a Watch

The gmail watch renew command in internal/cmd/gmail_watch_cmds.go re-issues the watch request using the stored topic and label configuration from the state file. The --ttl flag adjusts the RenewAfterMs field in the state, scheduling the next renewal.

Stopping a Watch

The gmail watch stop command calls svc.Users.Stop("me") to deregister the Pub/Sub subscription with Gmail, then removes the local state file from ~/.config/gogcli/state/gmail-watch/.

Summary

• Start: The gmail watch start command registers a Pub/Sub topic with Gmail via requestGmailWatch and persists state to ~/.config/gogcli/state/gmail-watch/.
• Serve: The gmail watch serve command runs an HTTP server that validates OIDC or shared tokens, decodes Pub/Sub push payloads, and fetches Gmail history.
• Process: The server retrieves messages via the Gmail History API, filters by labels, optionally fetches body content, and delivers structured webhooks.
• Lifecycle: renew extends the watch expiration while stop deregisters the subscription and cleans up local state.

Frequently Asked Questions

How does gogcli authenticate incoming Pub/Sub push requests?

The gmail watch serve command supports two authentication methods implemented in internal/cmd/gmail_watch_server.go. By default, it validates a static shared token using constant-time comparison. When --verify-oidc is enabled, it uses Google's OIDC validator to verify the JWT bearer token against the configured audience and service account email, ensuring only legitimate Google Pub/Sub services can trigger the endpoint.

What happens when the Gmail history ID becomes stale during processing?

If the Gmail History API returns a "history ID stale" error during handlePush, the resyncHistory function in internal/cmd/gmail_watch_server.go automatically falls back to fetching the specific messageId directly using svc.Users.Messages.Get. This recovery mechanism ensures the system remains synchronized even when the history window expires or the stored checkpoint becomes invalid.

Can gogcli filter specific Gmail labels from webhook notifications?

Yes, the gmail watch serve command accepts an --exclude-labels flag that accepts comma-separated label names such as "SPAM" or "TRASH". During fetchMessages in internal/cmd/gmail_watch_server.go, the server strips these labels from the message metadata before constructing the webhook payload, effectively filtering out unwanted categories from downstream processing.

Where does gogcli store the Pub/Sub watch state between restarts?

The watch state persists as JSON files in ~/.config/gogcli/state/gmail-watch/<sanitized-account>.json, as defined in internal/config/paths.go and implemented in internal/cmd/gmail_watch_state.go. This gmailWatchState structure stores the Pub/Sub topic name, label filters, expiration timestamps, history IDs, and webhook configuration, enabling the CLI to resume operations and manage renewal cycles across process restarts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}