How to Optimize OpenBB Query Performance Using Caching Mechanisms

OpenBB minimizes redundant network calls by integrating aiohttp_client_cache with per-provider SQLite backends, allowing you to optimize query performance through automatic HTTP response caching with configurable expiration times.

The OpenBB Platform (OpenBB-finance/OpenBB) implements a sophisticated data-provider layer designed to reduce latency and external API traffic. By leveraging SQLite-based caching mechanisms, the platform stores JSON responses locally, ensuring that repeated queries for financial data—whether equity searches, bond prices, or ETF metadata—return instantly from local storage rather than hitting remote servers.

Understanding OpenBB's Caching Architecture

OpenBB's caching system operates through a three-tier architecture: directory resolution, backend initialization, and session management. Each provider (TMX, SEC, ECONDB, etc.) maintains its own isolated cache to prevent data collisions and allow granular expiration policies.

Cache Directory Resolution

The foundation of OpenBB's caching mechanism resides in openbb_core/app/utils.py, which provides the get_user_cache_directory() function. This utility returns a platform-appropriate path (e.g., ~/.cache/openbb on Linux) where all provider-specific SQLite databases are stored.

from openbb_core.app.utils import get_user_cache_directory

cache_dir = get_user_cache_directory()

# Returns: /home/user/.cache/openbb

SQLite Backend Configuration

Each data provider creates a dedicated SQLite backend with specific expiration parameters. In openbb_platform/providers/tmx/openbb_tmx/utils/helpers.py, the platform instantiates SQLiteBackend from the aiohttp_client_cache library, specifying both the database path and the expire_after duration.

from aiohttp_client_cache import SQLiteBackend
from openbb_core.app.utils import get_user_cache_directory
from datetime import timedelta

backend = SQLiteBackend(
    f"{get_user_cache_directory()}/http/tmx_companies",
    expire_after=timedelta(days=1),
)

Implementing Query Caching in OpenBB

The caching layer operates transparently during query execution. When you invoke any OpenBB provider function, the system automatically checks for cached responses before initiating network requests.

Default Caching Behavior

By default, all OpenBB models expose a use_cache field set to True. When executing a query such as openbb.tmx.equity_search, the model passes this flag to helper functions like get_all_tmx_companies(), which wrap HTTP requests in a CachedSession.

import asyncio
from openbb_tmx.models.equity_search import EquitySearch

async def search_company(symbol: str):
    # use_cache=True is the default

    query = EquitySearch(symbol=symbol)
    result = await query.run()
    return result

asyncio.run(search_company("SHOP"))

The CachedSession automatically stores JSON responses in the provider's SQLite database, keyed by URL and parameters.

Bypassing Cache for Fresh Data

For real-time data requirements, set use_cache=False to force a fresh network request. This bypasses the SQLite backend entirely, ensuring you receive the latest available data from the external API.

import asyncio
from openbb_tmx.models.bond_prices import BondPrices

async def fetch_latest_bond():
    # Force network request, ignore cache

    query = BondPrices(use_cache=False)
    df = await query.run()
    return df

asyncio.run(fetch_latest_bond())

Custom Cache Sessions

Advanced users can manually configure CachedSession instances with custom expiration policies or storage locations. This approach leverages the same utilities used internally by OpenBB providers.

import asyncio
from aiohttp_client_cache import SQLiteBackend
from aiohttp_client_cache.session import CachedSession
from openbb_core.app.utils import get_user_cache_directory
from datetime import timedelta

async def custom_fetch(url: str):
    # Create custom 2-hour cache

    backend = SQLiteBackend(
        f"{get_user_cache_directory()}/http/custom",
        expire_after=timedelta(hours=2),
    )
    async with CachedSession(cache=backend) as session:
        resp = await session.get(url)
        return await resp.json()

asyncio.run(custom_fetch("https://api.example.com/data"))

Tuning Cache Expiration for Different Data Types

OpenBB implements time-to-live (TTL) policies tailored to data volatility. Static reference data receives longer cache durations than frequently changing market prices.

In openbb_platform/providers/tmx/openbb_tmx/utils/helpers.py, different endpoints utilize distinct expire_after values:

• Company lists: Cached for one day (timedelta(days=1)) due to stable corporate metadata
• ETF metadata: Cached for four hours (timedelta(hours=4)) to balance freshness with API rate limits
• Bond data: Cached for one day (timedelta(days=1)) given the slower update frequency of fixed-income reference data

The SEC provider (openbb_platform/providers/sec/openbb_sec/utils/helpers.py) mirrors this pattern, ensuring that regulatory filings and company metadata respect similar caching semantics while allowing real-time price queries to bypass storage when necessary.

Summary

• OpenBB optimizes query performance through SQLite-backed HTTP caching using aiohttp_client_cache, with each provider maintaining isolated storage in <user-cache-dir>/http/<provider>.
• The get_user_cache_directory() utility in openbb_core/app/utils.py ensures cross-platform cache location consistency.
• Default caching is enabled via use_cache=True in all provider models, automatically storing JSON responses and retrieving them for subsequent identical queries.
• Cache expiration is data-specific: company lists cache for 24 hours, ETF metadata for 4 hours, and bond data for 24 hours, balancing freshness with performance.
• Bypass caching by setting use_cache=False when real-time data is required, or create custom CachedSession instances for advanced expiration control.

Frequently Asked Questions

How do I clear the OpenBB cache manually?

Delete the SQLite database files located in your user cache directory under the http subdirectory. Run get_user_cache_directory() from openbb_core.app.utils to identify the exact path (e.g., ~/.cache/openbb/http/), then remove the specific provider databases or the entire http folder to reset all cached responses.

Can I change the default cache expiration time for a specific provider?

Yes, but it requires modifying the provider's helper file (e.g., openbb_platform/providers/tmx/openbb_tmx/utils/helpers.py) where the SQLiteBackend is instantiated. Adjust the expire_after parameter in the timedelta call for your specific data type. For persistent changes without modifying source code, create a custom CachedSession with your preferred TTL as shown in the advanced implementation example.

Why am I still hitting API rate limits despite caching being enabled?

Caching only reduces redundant identical requests; it does not eliminate the initial fetch or bypass provider-specific rate limits on unique endpoints. If your workflow involves querying many distinct symbols or parameters, each unique URL generates a separate cache entry and still requires an initial network request. To minimize rate limiting, increase cache expiration durations for stable data and batch your queries to maximize cache hit rates.

Does OpenBB cache work across Python sessions?

Yes. The SQLite backend persists cache entries to disk, meaning cached responses survive Python interpreter restarts and system reboots. As long as the SQLite database files in your user cache directory remain intact, subsequent Python sessions will retrieve stored responses without network requests until the expire_after timestamp passes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}