Openpilot Athena System: WebSocket Cloud Connectivity Protocol and Architecture

The Athena system provides a persistent, bidirectional WebSocket channel between openpilot devices and Comma.ai's cloud, using JSON-RPC messaging to enable remote commands, file uploads, SSH tunneling, and telemetry forwarding.

The Athena subsystem in the commaai/openpilot repository serves as the critical bridge connecting a vehicle's dongle to the cloud backend. It implements a secure, fault-tolerant communication protocol that supports everything from over-the-air log collection to remote debugging sessions without requiring direct network access. Understanding this architecture reveals how openpilot maintains real-time cloud connectivity despite intermittent cellular coverage.

Connection Lifecycle and Authentication

The Athena daemon establishes and maintains its cloud link through a rigorous registration and connection workflow defined in system/athena/registration.py and system/athena/athenad.py.

Device Registration and JWT Acquisition

Before any connection attempts, registration.register() authenticates the device with Comma.ai's backend. This function obtains a JWT token (register_token) and binds the unique dongle ID to the user's account. The dongle ID is then persisted in the Params key-value store for subsequent sessions.

WebSocket URI and Authentication

Once registered, athenad.py constructs the WebSocket endpoint at wss://athena.comma.ai/ws/v2/<dongle_id>. The connection authenticates by passing the JWT token via HTTP cookie:

ws_uri = f"{ATHENA_HOST}/ws/v2/{dongle_id}"
ws = create_connection(
    ws_uri,
    cookie="jwt=" + api.get_token(),
    enable_multithread=True,
    timeout=30.0
)

This pattern appears in athenad.py lines 28-33, ensuring encrypted transport with proper identity verification.

Keep-Alive and Reconnection Strategy

The daemon monitors connection health through WebSocket ping frames. On each received ping, it updates Params().put("LastAthenaPingTime", timestamp). If no ping arrives within RECONNECT_TIMEOUT_S (70 seconds), the connection is declared dead.

When disconnections occur, the system implements exponential backoff with jitter before retrying. The backoff(conn_retries) function calculates sleep intervals that grow with consecutive failures, preventing thundering herd problems against the cloud infrastructure.

JSON-RPC Messaging Architecture

All communication across the Athena WebSocket uses JSON-RPC 2.0 envelopes. The daemon maintains three distinct queues to prioritize traffic types, defined in athenad.py lines 26-42:

• recv_queue – Incoming messages from the cloud (text or binary frames)
• send_queue – High-priority outbound RPC responses and commands
• low_priority_send_queue – Non-critical telemetry, logs, and statistics

Dispatcher and Exposed Methods

The system uses a jsonrpc dispatcher to map incoming method names to Python callables decorated with @dispatcher.add_method. Key remote procedures exposed to the cloud include:

RPC Method	Functionality
getMessage	Pull a single message from a Cereal service (e.g., carState, gpsLocation)
uploadFileToUrl / uploadFilesToUrls	Queue files for HTTP(S) upload to signed URLs
listDataDirectory	Return filtered lists of local log files
listUploadQueue	Inspect pending upload queue status
cancelUpload	Remove a specific upload by ID
startLocalProxy	Open a local TCP port (typically SSH) and tunnel traffic through the WebSocket
getPublicKey, getSshAuthorizedKeys	Expose device identity for security verification
getNetworkType, getNetworkMetered	Report current connectivity status

These methods enable the cloud to query vehicle state and initiate operations remotely without direct shell access.

File Upload Mechanism

The Athena system handles large file transfers (qlogs, screenshots, etc.) through a sophisticated queue-based architecture that respects network conditions.

Upload Queue and Workers

When the cloud sends an uploadFileToUrl request, athenad.py creates UploadItem dataclass instances and places them into a PriorityQueue (lines 29-30). Four dedicated upload_handler threads, spawned within handle_long_poll, continuously process this queue.

Traffic Shaping and Retry Logic

Uploads use a custom HTTP session that sets DSCP/TOS bits to 0x20 (background traffic priority), ensuring file transfers do not interfere with critical driving communications. The UploadTOSAdapter class (lines 72-76) implements this network-layer optimization.

Failed uploads retry with exponential backoff up to MAX_RETRY_COUNT. Items exceeding MAX_AGE (31 days) are automatically purged from the queue to prevent stale data accumulation.

SSH Local Proxy for Remote Debugging

The startLocalProxy method (lines 84-115) enables secure remote debugging by tunneling TCP traffic through the WebSocket connection. When invoked, the daemon:

1. Creates a socket pair connecting to the requested local port (typically 22 for SSH)
2. Spawns ws_proxy_recv to forward data from the WebSocket to the local socket
3. Spawns ws_proxy_send to forward data from the local socket back to the WebSocket

This architecture allows Comma.ai technicians to establish SSH sessions with dongles behind NAT or firewalls, as all traffic routes through the authenticated WebSocket channel rather than requiring direct IP visibility.

Log and Statistics Forwarding

Athena continuously streams diagnostic data to the cloud through dedicated background threads:

• log_handler monitors Paths.swaglog_root() for unsent log files, wraps them in JSON-RPC forwardLogs calls, and places them on low_priority_send_queue (lines 98-130)
• stat_handler performs similar operations for files in Paths.stats_root(), using the storeStats method (lines 60-85)

Both handlers respect metered connection policies, deferring large transfers until unmetered WiFi is available when configured.

Practical Implementation Examples

Manually Opening an Athena WebSocket Connection

import json
from websocket import create_connection
from openpilot.common.api import Api

dongle_id = "YOUR_DONGLE_ID"
api = Api(dongle_id)

ws = create_connection(
    f"wss://athena.comma.ai/ws/v2/{dongle_id}",
    cookie="jwt=" + api.get_token(),
    enable_multithread=True,
    timeout=30
)

# Request GPS location data via JSON-RPC

request = {
    "jsonrpc": "2.0",
    "method": "getMessage",
    "params": {"service": "gpsLocation", "timeout": 1000},
    "id": 1
}
ws.send(json.dumps(request))

response = json.loads(ws.recv())
print(response)  # Contains latitude, longitude, and accuracy data

ws.close()

This implementation mirrors the connection logic in athenad.py lines 28-33 and demonstrates the JSON-RPC dispatcher interface.

Queueing a File Upload from the Vehicle

from openpilot.system.athena import athenad

# Upload a compressed log file to a pre-signed S3 URL

result = athenad.uploadFileToUrl(
    fn="2024-03-04--10-00-00--001.qlog.zst",
    url="https://s3.amazonaws.com/comma-logs/.../upload",
    headers={"Authorization": "Bearer ...", "Content-Type": "application/zstd"}
)

print(result)  # Output: {'enqueued': 1, 'items': [{'id': '...', 'fn': '...'}]}

The uploadFileToUrl function (lines 94-101) handles the queuing logic automatically, returning immediately while background workers manage the actual HTTP transfer.

Initiating an SSH Tunnel Through Athena

import threading
from openpilot.system.athena import athenad

# Start local proxy forwarding port 22 (SSH) through the cloud

proxy_result = athenad.startLocalProxy(
    global_end_event=threading.Event(),
    remote_ws_uri="wss://athena.comma.ai/ws/v2/YOUR_DONGLE_ID",
    local_port=22
)

print(proxy_result)  # Output: {'success': 1, 'port': 22, 'url': '...'}

This exposes the local SSH daemon to the cloud through the secure WebSocket tunnel, enabling remote shell access without port forwarding or VPN configuration.

Summary

• WebSocket (WSS) with JWT Authentication: Athena maintains persistent encrypted connections to wss://athena.comma.ai/ws/v2/<dongle_id>, authenticating via JWT cookies obtained through registration.py.
• JSON-RPC 2.0 Messaging: The system uses a three-tier queue architecture (receive, high-priority send, low-priority send) with a method dispatcher exposing vehicle control and query capabilities.
• Robust File Uploads: Four threaded workers process a priority queue with DSCP/TOS traffic shaping (0x20), exponential retry logic, and 31-day expiration windows.
• SSH Tunneling: The startLocalProxy method creates bidirectional socket bridges that tunnel local SSH traffic through the WebSocket, enabling remote debugging across NAT boundaries.
• Resilient Connectivity: Exponential backoff reconnection strategies and 70-second ping timeouts ensure the daemon survives intermittent cellular connectivity without manual intervention.

Frequently Asked Questions

What communication protocol does the Athena system use for cloud connectivity?

The Athena system uses WebSocket Secure (WSS) as its transport layer, establishing persistent connections to wss://athena.comma.ai/ws/v2/<dongle_id>. All messages are framed as JSON-RPC 2.0 objects, enabling structured request-response patterns and bidirectional command execution between the openpilot device and Comma.ai's cloud backend.

How does openpilot authenticate with the Athena backend?

Authentication occurs via JSON Web Tokens (JWT) obtained during device registration. The registration.py module handles initial device pairing and stores the dongle ID in Params. When athenad.py creates the WebSocket connection, it passes the token as an HTTP cookie (cookie="jwt=" + api.get_token()), allowing the cloud to verify device identity and user association for every connection attempt.

Can Athena upload files while driving or on metered connections?

Yes, but with traffic shaping policies. The upload workers in athenad.py set DSCP/TOS bits to 0x20 (background priority) to prevent file transfers from interfering with safety-critical communications. Additionally, the system checks getNetworkMetered() status and can defer large uploads until unmetered WiFi is available, though this behavior is configurable per the current network policy implementation.

How does the SSH tunnel feature work without direct network access?

The startLocalProxy method creates a local socket connection to the SSH daemon (port 22) and bridges it to the WebSocket through two dedicated threads. One thread forwards data from the WebSocket to the local socket, while the other moves data from the local socket back to the WebSocket. This effectively tunnels TCP traffic through the authenticated cloud connection, allowing external SSH access even when the device is behind cellular NAT or corporate firewalls.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}