OpenAI Dashboard Web Scraping Architecture in CodexBar: A 4-Layer Approach

CodexBar implements a four-layer architecture for OpenAI dashboard web scraping that isolates cookies per account, reuses off-screen WebViews, executes JavaScript extraction scripts, and parses results into typed Swift models.

CodexBar is an open-source macOS utility that retrieves usage data from the OpenAI dashboard without exposing user credentials. The project, hosted at steipete/CodexBar, employs a sophisticated OpenAI dashboard web scraping architecture that balances performance, security, and reliability through four cooperating layers.

1. Persistence Layer: Account-Scoped Cookie Storage

The architecture begins with strict isolation of web sessions. Each account email is normalized (lowercased) and converted into a deterministic UUID using SHA-256 hashing of the first 16 bytes. This UUID serves as the identifier for a persistent WKWebsiteDataStore created via WKWebsiteDataStore(forIdentifier:).

In Sources/CodexBarCore/OpenAIWeb/OpenAIDashboardWebsiteDataStore.swift, the store(forAccountEmail:) method manages a static cache (cachedStores) that ensures the same data store instance is reused across calls. This design allows OpenAIDashboardWebViewCache to treat the store’s identity as a cache key. Clearing a specific account’s session is handled by clearStore(forAccountEmail:scope:), which removes only that account’s ChatGPT/OpenAI cookies without affecting other accounts.

2. WebView Management: Off-Screen WebView Cache

To minimize resource overhead, CodexBar maintains a cache of reusable off-screen WKWebView instances. The OpenAIDashboardWebViewCache class (in Sources/CodexBarCore/OpenAIWeb/OpenAIDashboardWebViewCache.swift) manages these through an acquire() and release() pattern.

When acquire(store:usageURL:) is called, the cache checks for existing entries:

• Busy entries trigger the creation of a temporary fallback WebView.
• Idle entries are reused, optionally preserving the already-loaded dashboard page for a short hand-off window (preservedPageHandoffTimeout = 5 seconds).

After scraping completes, release() returns the WebView to the idle pool, clearing it to about:blank unless preserving the page for reuse. An idle-prune timer automatically removes entries inactive longer than idleTimeout (default 60 seconds). The WebView lives in a virtually invisible NSWindow (alpha ≈ 0.001) to keep WebKit timers active while avoiding UI artifacts.

3. JavaScript Scraper: In-Page Data Extraction

The actual data extraction happens inside the page via openAIDashboardScrapeScript (embedded in Sources/CodexBarCore/OpenAIWeb/OpenAIDashboardScrapeScript.swift). This script runs against the usage URL (https://chatgpt.com/codex/cloud/settings/analytics#usage) and performs several extraction tasks:

1. Detects login, workspace-picker, or Cloudflare challenge states.
2. Extracts the client-bootstrap JSON from <script id="client-bootstrap"> and Next.js data from __NEXT_DATA__.
3. Identifies the signed-in email and account plan via findPlan.
4. Walks the usage chart’s SVG path elements, extracting bar metadata through React internals (reactPropsOf, reactFiberOf) and reconstructing a daily breakdown JSON assigned to window.__codexbarUsageBreakdownJSON.
5. Locates the credits usage history table, scrolling it into view if virtualized to force hydration.
6. Detects the "add-more-credits" button and normalizes its URL.

The script returns a plain dictionary that OpenAIDashboardFetcher.scrape(webView:) converts into a ScrapeResult struct.

4. Swift Parsing: Model Transformation

The final layer transforms raw HTML/JSON payloads into typed Swift models. OpenAIDashboardParser (in Sources/CodexBarCore/OpenAIWeb/OpenAIDashboardParser.swift) provides pure-Swift parsing utilities:

• parseSignedInEmailFromClientBootstrap and parseAuthStatusFromClientBootstrap extract authentication details from embedded JSON.
• parseRateLimits, parseCreditsRemaining, and parseCodeReviewRemainingPercent use regular expressions to extract numeric values.
• parseCreditEvents converts credits table rows into typed CreditEvent structs.
• parsePlanFromHTML searches JSON payloads for keys containing "plan", "tier", or "subscription", normalizing values through CodexPlanFormatting.displayName.

The parser combines this data with official usage API responses (dashboardAPIData) to produce a final OpenAIDashboardSnapshot consumed by the rest of the application.

Practical Code Examples

Fetching the Latest Dashboard

Use OpenAIDashboardFetcher to orchestrate the entire scrape, from WebView acquisition to model parsing:

let fetcher = OpenAIDashboardFetcher()
let snapshot = try await fetcher.loadLatestDashboard(
    accountEmail: "alice@example.com",
    cacheScope: nil,                    // default cookie-header cache
    logger: { print("[OpenAI] \($0)") },
    debugDumpHTML: false,
    allowNavigationTimeoutRetry: true,
    timeout: 60)                        // seconds
print("Current plan: \(snapshot.accountPlan ?? "-")")
print("Credits remaining: \(snapshot.creditsRemaining ?? 0)")

This call internally creates or reuses a per-account WKWebsiteDataStore, acquires a cached WebView, runs the JavaScript scraper, merges API data, and returns a fully-typed snapshot.

Probing the Usage Page for CLI Tools

For lightweight checks (used by the codex status CLI), probe the page without performing a full scrape:

let fetcher = OpenAIDashboardFetcher()
let store = OpenAIDashboardWebsiteDataStore.store(forAccountEmail: "bob@example.com")
let probe = try await fetcher.probeUsagePage(
    websiteDataStore: store,
    logger: { print($0) },
    timeout: 30,
    preserveLoadedPageForReuse: true)

print("Signed-in email?  \(probe.signedInEmail ?? "none")")
print("Login required?   \(probe.loginRequired)")
print("Page URL:         \(probe.href ?? "-")")

The probeUsagePage method runs the scraper but stops early, returning only the minimal information needed to determine if a full dashboard load is required.

Clearing Stale Sessions

Force a fresh login flow by clearing specific account data:

await OpenAIDashboardFetcher.clearSessionData(
    accountEmail: "charlie@example.com",
    cacheScope: nil)          // clears only ChatGPT/OpenAI cookies for that account

After clearing, subsequent calls will trigger a fresh authentication flow or require new cookie imports.

Summary

• Account isolation is achieved through deterministic UUID-based WKWebsiteDataStore instances that persist cookies per email address.
• Resource efficiency comes from the OpenAIDashboardWebViewCache, which reuses off-screen WebViews with configurable idle timeouts and page preservation.
• Data extraction relies on an embedded JavaScript payload that navigates React internals and DOM structures to extract usage charts, credit tables, and subscription details.
• Type safety is maintained by OpenAIDashboardParser, which converts unstructured HTML into Swift models like OpenAIDashboardSnapshot and CreditEvent.

Frequently Asked Questions

How does CodexBar isolate cookies between multiple OpenAI accounts?

Each account email is normalized and hashed using SHA-256 to generate a unique identifier for a WKWebsiteDataStore. This creates separate persistent cookie stores per account, allowing users to switch between accounts without session leakage. The OpenAIDashboardWebsiteDataStore class manages this mapping and caches store instances for reuse.

Why does CodexBar use WebView caching instead of creating new instances for each scrape?

Creating WKWebView instances is resource-intensive. The OpenAIDashboardWebViewCache maintains a pool of off-screen WebViews with a 60-second idle timeout and 5-second page preservation window. This approach reduces memory pressure and improves scrape latency by reusing existing browser contexts while keeping the WebView hidden (alpha ≈ 0.001) to avoid UI interference.

What data can the JavaScript scraper extract from the OpenAI dashboard?

The openAIDashboardScrapeScript extracts the signed-in email, subscription plan, rate limits, credits remaining, daily usage breakdown from SVG chart metadata (via React fiber internals), complete credits history table rows, and the "add-more-credits" purchase URL. It also detects authentication states including login requirements and Cloudflare challenges.

How does CodexBar convert scraped HTML into Swift models?

The OpenAIDashboardParser uses regular expressions and JSON deserialization to parse HTML strings returned by the JavaScript scraper. It extracts specific fields like parseSignedInEmailFromClientBootstrap for authentication and parseCreditEvents for transaction history, ultimately assembling an OpenAIDashboardSnapshot that combines scraped data with official API responses.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}