How to Configure the Analytics Dashboard to Track Costs and Performance in Local Deep Research

To configure the analytics dashboard for cost and performance tracking, modify the PRICING_TABLE dictionary in src/local_deep_research/metrics/pricing/cost_calculator.py to define per-model token rates, then access the dashboard at /cost-analytics/ where the api_cost_analytics endpoint aggregates data from the TokenUsage database model.

The local-deep-research repository by learningcircuit ships with a built-in Analytics Dashboard that visualizes LLM usage costs and system performance through Flask routes, a pricing engine, and a web interface. You can configure this dashboard to track expenses across different providers, monitor rate-limiting events, and analyze search activity by adjusting the pricing configuration and API parameters.

Understanding the Analytics Dashboard Architecture

The dashboard operates through four distinct layers that handle data collection, cost calculation, API serving, and visualization.

• Database Layer: The TokenUsage model in src/local_deep_research/database/models.py stores every LLM interaction, capturing model_name, provider, prompt_tokens, completion_tokens, research_id, and timestamp. The TokenCounter utility writes a row for each call.

• Pricing Engine: CostCalculator.calculate_cost_sync() in src/local_deep_research/metrics/pricing/cost_calculator.py references a Python dictionary of USD-per-1K-token rates to compute prompt_cost, completion_cost, and total_cost.

• Aggregation API: The api_cost_analytics function in src/local_deep_research/web/routes/metrics_routes.py (lines 1799-1839) queries TokenUsage rows, applies the cost calculator, and returns JSON summaries including overall totals, per-research costs, and the top-10 most expensive sessions.

• Frontend Layer: The cost_analytics.html template in src/local_deep_research/web/templates/pages/cost_analytics.html renders the UI and uses JavaScript to poll the /metrics/api/cost-analytics endpoint.

Configuring Cost Tracking

Setting Model Pricing in CostCalculator

The pricing table is a plain Python dictionary that requires no server restart when modified. Open src/local_deep_research/metrics/pricing/cost_calculator.py and locate the PRICING_TABLE dict to add or adjust model rates.

# src/local_deep_research/metrics/pricing/cost_calculator.py

PRICING_TABLE = {
    # model_name: (prompt_price_per_1k, completion_price_per_1k)

    "gpt-4": (0.03, 0.06),
    "claude-3-opus": (0.015, 0.075),
    "gemini-1.5-pro": (0.0005, 0.0015),
    # Local models incur zero cost

    "my-local-model": (0.0, 0.0),
}

The CostCalculator reads this table on every request, so updates take effect immediately.

Adjusting the Time Window for Reports

The api_cost_analytics endpoint accepts a period query parameter to filter historical data. Valid values are 7d, 30d (default), 90d, 365d, or all.

curl "http://localhost:5000/metrics/api/cost-analytics?period=90d"

The response includes aggregated totals and session breakdowns for the specified interval.

Enabling or Disabling UI Components

The dashboard template includes a visibility toggle via the showTemporarilyDisabled() function. If you encounter a "temporarily disabled" banner in cost_analytics.html (around line 12423), comment out that call to display live data while testing pricing configurations.

Configuring Performance Tracking

Performance monitoring encompasses rate-limiting events and search activity metrics.

Rate-Limiting Metrics

Rate-limiting data derives from the RateLimitAttempt and RateLimitEstimate tables. To customize displayed metrics, edit the api_rate_limiting_metrics function in src/local_deep_research/web/routes/metrics_routes.py (lines 68-84). The UI template rate_limiting.html renders this data using the same polling pattern as the cost analytics panel.

Search Activity Monitoring

The TokenCounter and SearchTracker utilities populate counters consumed by api_enhanced_metrics. To add custom performance_counters, modify src/local_deep_research/metrics/query_utils.py. These metrics automatically appear as time-series data in the dashboard's JavaScript (referenced in metrics.html around line 1949) without requiring frontend changes.

Accessing Cost Data Programmatically

You can retrieve raw cost analytics via HTTP requests for external reporting or automation.

cURL request for 30-day costs:

curl "http://localhost:5000/metrics/api/cost-analytics?period=30d"

Example JSON response:

{
  "status": "success",
  "period": "30d",
  "overview": {
    "total_cost": 12.34,
    "total_tokens": 45678,
    "prompt_tokens": 30000,
    "completion_tokens": 15678
  },
  "top_expensive_research": [
    {"research_id": "c3f9", "total_cost": 3.12}
  ],
  "research_count": 42
}

JavaScript integration:

(async () => {
  const period = '7d';
  const resp = await fetch(`/metrics/api/cost-analytics?period=${period}`);
  const data = await resp.json();
  
  document.getElementById('total-cost').textContent = 
    `$${data.overview.total_cost.toFixed(2)}`;
  document.getElementById('avg-research-cost').textContent = 
    `$${(data.overview.total_cost / data.research_count).toFixed(2)}`;
})();

Registering Custom Endpoints

If you extend the dashboard with custom analytics, register new routes in src/local_deep_research/web/routes/route_registry.py (line 247). Add URL rules to the metrics blueprint:

# src/local_deep_research/web/routes/route_registry.py

metrics_bp.add_url_rule("/api/custom-cost", view_func=custom_cost_view)

Ensure the path is also listed in src/local_deep_research/web/static/js/config/urls.js if you want first-party JavaScript to reference it.

Summary

• Modify PRICING_TABLE in cost_calculator.py to set per-model token costs; changes apply immediately without server restart.
• Query the api_cost_analytics endpoint with ?period= filters (7d, 30d, 90d, 365d, all) to retrieve aggregated cost data.
• Store usage data automatically via the TokenUsage model and TokenCounter utility on every LLM call.
• Monitor performance through rate-limiting tables (RateLimitAttempt) and search metrics aggregated in metrics_routes.py.
• Customize the UI by editing cost_analytics.html and ensuring routes are registered in route_registry.py.

Frequently Asked Questions

How do I add pricing for a custom local model?

Add an entry to the PRICING_TABLE dictionary in src/local_deep_research/metrics/pricing/cost_calculator.py with zero or custom rates (e.g., "my-local-llm": (0.0, 0.0)). Ensure your LLM calls record the exact model_name string in the TokenUsage table so the CostCalculator matches the usage to your pricing entry.

Where is token usage data physically stored?

Token usage is stored in the TokenUsage database model defined in src/local_deep_research/database/models.py. Every LLM request triggers the TokenCounter utility (in src/local_deep_research/metrics/token_counter.py) to write a row containing model_name, provider, token counts, research_id, and timestamp.

Can I track costs for specific research sessions only?

Yes. The api_cost_analytics endpoint returns a top_expensive_research array and per-research breakdowns. Each entry includes a research_id that correlates with the research_id field in the TokenUsage table. You can query the database directly or modify the aggregation logic in metrics_routes.py to filter by specific research IDs.

Do I need to restart the server after updating pricing?

No. The PRICING_TABLE is a standard Python dictionary read by CostCalculator.calculate_cost_sync() on every API request. Modifications to the pricing file take effect immediately for all subsequent cost calculations, though you should restart if you change the Python module structure or import paths.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}