How the Cost Usage Scanner Enables Local Token Tracking in CodexBar

The Cost Usage Scanner transforms raw JSON-Lines usage logs from OpenAI-compatible APIs into aggregated daily cost snapshots, enabling real-time local token tracking without external network calls.

CodexBar is a macOS menu-bar application that monitors AI API usage through local log files. The cost usage scanner for local token tracking continuously processes these files to calculate spending across multiple providers, storing results in a local cache for instant retrieval by both the menu-bar UI and command-line interface.

How the Cost Usage Scanner Works

The scanner operates as a six-stage pipeline that converts raw API logs into actionable cost data, implemented primarily in Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift.

Reading Local Usage Logs

The scanner walks the directory tree under ~/.codexbar/usage/ to discover JSON-Lines files written by each provider. It filters files based on enabled providers and streams them line-by-line to minimize memory overhead, ensuring efficient processing even as log files grow to gigabytes.

Parsing JSON Entries

Each log line contains a JSON object with fields such as time.created, model, completionTokens, promptTokens, and costUSD. The scanner uses CostUsageScanner+FastJSON.swift for the common OpenAI format, falling back to provider-specific parsers like CostUsageScanner+Claude.swift and CostUsageScanner+CodexFastJSON.swift for non-standard schemas.

Applying Provider-Specific Pricing

Raw token counts convert to dollar amounts via the CostUsagePricing table, which holds per-1,000-token rates for every known model. The scanner also applies priority heuristics to avoid double-counting when the same request appears in multiple logs. This pricing logic is implemented in CostUsagePricing.swift.

Aggregating into Daily Buckets

After parsing, entries transform into CostUsageDailyReport.Entry objects. The scanner groups these by calendar day (derived from UTC timestamps using Calendar.current) and sums token counts and USD costs. The resulting CostUsageTokenSnapshot contains an array of entries for the last N days, configurable via SettingsStore.costUsageHistoryDays. Model definitions live in CostUsageModels.swift.

Caching the Snapshot

Computed snapshots are written to disk via CostUsageCache.swift. Subsequent runs read the cache first and only scan new log entries, ensuring cheap operation even as historical data accumulates.

Exposing the Data

The CostUsageFetcher class loads snapshots (or forces fresh scans) and hands them to higher-level code. The UI layer in StatusItemController+CostMenuCard.swift and MenuCardView+Costs.swift, along with the CLI command CLICostCommand.swift, render the cost history chart and daily totals.

Edge Cases Handled by the Scanner

The scanner includes robust handling for real-world log inconsistencies to ensure accurate financial tracking.

Duplicate Log Detection

Some providers write both "fast-JSON" and "full" log files. The scanner identifies duplicates by matching providerID, requestID, and timestamps, then discards the lower-priority entry to prevent double-counting of the same API request.

Truncated Model Prefixes

OpenAI occasionally truncates model names (e.g., gpt-4 vs gpt-4-0125-preview). The CostUsageScanner+CodexTruncatedPrefix helper normalizes these names to ensure consistent pricing table lookups and correct rate application.

Partial Data Recovery

If a log line lacks the costUSD field (common in older logs), the scanner computes the value from token counts using the pricing table. If token counts are also missing, the entry is ignored entirely, keeping the snapshot free of invalid "nan" values.

Timezone-Agnostic Aggregation

All timestamps are stored as UTC epoch milliseconds. The scanner converts these to the user's local calendar day using Calendar.current, ensuring daily buckets align with the user's local time regardless of their timezone.

Implementation Examples

Running a Manual Scan from the CLI

Force a fresh scan and display the results:

codexbar cost --refresh

This invokes CLICostCommand, which creates a CostUsageFetcher with scannerOptions.forceRefresh = true. The scanner re-reads every usage log, recomputes the daily snapshot, and prints a formatted table:

Date       Tokens   Cost (USD)
2024-03-01  1,200K   $4.32
2024-03-02    950K   $3.41

Programmatic Access in Swift

Access cost data within the application:

let fetcher = CostUsageFetcher(scannerOptions: .default)
let snapshot = try await fetcher.loadTokenSnapshot()
// snapshot.daily is an array of CostUsageDailyReport.Entry
let today = snapshot.daily.first!
print("Today's cost: $\(today.costUSD, specifier: "%.2f")")

The call chain flows through CostUsageFetcher → CostUsageScanner.run → CostUsageCacheIO.load / save → CostUsageTokenSnapshot.

Adding Support for New Providers

To integrate a new provider that writes JSON-Lines files:

1. Add per-token rates to CostUsagePricing for the provider's models.
2. Create a CostUsageScanner+YourProvider.swift extension that maps the provider's JSON schema to CostUsageScanner.ParsedEntry.

The existing aggregation and caching logic processes the new entries automatically.

Key Source Files

• Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner.swift — Core driver that walks log files, parses entries, and builds the snapshot.
• Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift — Pricing tables and token-to-USD conversion helpers.
• Sources/CodexBarCore/Vendored/CostUsage/CostUsageCache.swift — On-disk cache read/write for daily snapshots.
• Sources/CodexBarCore/CostUsageFetcher.swift — Public façade used by UI and CLI to obtain snapshots.
• Sources/CodexBarCLI/CLICostCommand.swift — CLI entry point that triggers scans and prints results.
• Sources/CodexBar/StatusItemController+CostMenuCard.swift — UI code rendering the cost submenu in the menu bar.
• Sources/CodexBar/SettingsStore+TokenCost.swift — User settings for enabling tracking and configuring history length.
• Tests/CodexBarTests/UsageStoreCachedTokenHydrationTests.swift — Test suite verifying cache hydration and daily aggregation.

Summary

• The cost usage scanner for local token tracking processes JSON-Lines logs from AI providers to calculate daily spending totals without transmitting data externally.
• It uses provider-specific parsers (CostUsageScanner+FastJSON, CostUsageScanner+Claude) and a unified pricing table (CostUsagePricing) to normalize costs across different APIs.
• Results are cached locally (CostUsageCache) for instant UI retrieval and CLI access without re-scanning historical data.
• The scanner handles edge cases including duplicate entries, truncated model names, and missing cost fields through specialized helper classes.
• Configuration for retention days is available via SettingsStore.costUsageHistoryDays.

Frequently Asked Questions

How does the scanner handle different log formats from multiple providers?

The scanner uses a fast-path JSON parser for standard OpenAI formats via CostUsageScanner+FastJSON.swift, while provider-specific extensions like CostUsageScanner+Claude.swift and CostUsageScanner+CodexFastJSON.swift handle schema variations. Each extension maps its specific JSON structure to a common ParsedEntry type, allowing the aggregation logic to remain provider-agnostic.

What happens if a log entry is missing cost information?

When the costUSD field is absent, the scanner calculates the cost from completionTokens and promptTokens using the rates defined in CostUsagePricing.swift. If both the cost and token counts are missing, the entry is skipped entirely to prevent invalid data from corrupting the daily totals.

How does the scanner prevent duplicate charges from multiple log files?

The scanner identifies potential duplicates by matching providerID, requestID, and timestamps across different log files. When matches are found, it applies priority heuristics to keep only the authoritative entry, ensuring the same API request is never counted twice even if it appears in both a "fast-JSON" and a "full" log file.

Where is the cost data cached and how is it refreshed?

Computed snapshots are stored on disk via CostUsageCache.swift, typically under the app's local storage directory. By default, the scanner reads this cache and only processes new log entries since the last run. Users can force a complete refresh using the codexbar cost --refresh CLI command, which creates a CostUsageFetcher with forceRefresh enabled to bypass the cache and re-scan all historical logs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}