# How to Implement Custom Fetchers for Integrating New Data Providers in OpenBB

> Learn to integrate new data providers in OpenBB by subclassing the Fetcher class and defining Pydantic models. Easily extend OpenBB's data capabilities.

- Repository: [OpenBB/OpenBB](https://github.com/OpenBB-finance/OpenBB)
- Tags: how-to-guide
- Published: 2026-03-06

---

**You can integrate any external data source into OpenBB by subclassing the abstract `Fetcher` class located in [`openbb_platform/core/openbb_core/provider/abstract/fetcher.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/openbb_platform/core/openbb_core/provider/abstract/fetcher.py), defining Pydantic models for query parameters and responses, and registering your implementation in the provider's [`__init__.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/__init__.py) file to enable automatic discovery by the registry scanner.**

The OpenBB Platform uses a provider-based architecture that allows developers to seamlessly add new data sources through a standardized fetcher pattern. Understanding how to implement custom fetchers for integrating new data providers in OpenBB enables you to extend the platform's capabilities with proprietary or niche financial data APIs. This guide walks through the core abstractions, registry mechanisms, and concrete implementation steps based on the actual source code in the OpenBB-finance/OpenBB repository.

## Architecture Overview

OpenBB's data provider system relies on three core components that work together to standardize data ingestion across disparate sources.

### The Fetcher Abstract Base Class

At the heart of the system lies the `Fetcher` abstract base class defined in [`openbb_platform/core/openbb_core/provider/abstract/fetcher.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/openbb_platform/core/openbb_core/provider/abstract/fetcher.py). This class establishes a strict contract between the platform and data providers through generic type parameters `<Query, Response>`. Concrete implementations must override the asynchronous `_fetch_data(self, params: Query) -> Response` method, which contains the actual API call logic and data transformation. The public `fetch_data` method defined in the base class wraps this internal logic with error handling, caching checks, and validation, ensuring consistent behavior across all providers.

### Provider Registry and Auto-Discovery

The [`registry_map.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/registry_map.py) file in `openbb_platform/core/openbb_core/provider/` implements the auto-discovery mechanism that eliminates manual registration boilerplate. When OpenBB starts, the registry scans the `openbb_platform.providers.<provider_name>` namespace and automatically indexes any subclass of `Fetcher` it discovers. This registration links the fetcher to its provider identifier, making it available to the `QueryExecutor` and CLI without additional configuration.

### Query Execution Pipeline

The `QueryExecutor` class in [`openbb_platform/core/openbb_core/provider/query_executor.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/openbb_platform/core/openbb_core/provider/query_executor.py) orchestrates the data retrieval workflow. It instantiates the appropriate fetcher class based on the user's provider selection, validates input parameters against the Query model using Pydantic, and invokes the fetcher's `fetch_data` method. This pipeline handles retries, timeouts, and caching transparently, allowing fetcher implementations to focus solely on data extraction.

## Implementation Steps

Creating a functional custom fetcher requires four discrete steps that align with OpenBB's type-safe architecture.

### Step 1: Define Query and Response Models

Every fetcher requires two Pydantic models: a Query model that defines input parameters and a Response model that structures the returned data. These models live in your provider's `models/` directory and must inherit from `BaseModel`. The Query model drives CLI argument generation and API validation, while the Response model ensures type consistency across the platform.

### Step 2: Subclass the Fetcher

Create a new class that inherits from `Fetcher[YourQuery, YourResponse]` and implements the required interface. The generic type parameters bind your specific models to the fetcher, enabling static type checking throughout the execution pipeline. You may optionally define a `name` property to aid in debugging, though the class name serves as the default identifier.

### Step 3: Implement the Fetch Logic

Override the `_fetch_data` method with your API integration code. This method receives an instance of your Query model and must return an instance of your Response model. Use `async`/`await` syntax for non-blocking I/O operations, as the `QueryExecutor` expects an awaitable coroutine. Handle HTTP errors, rate limiting, and data normalization within this method, converting raw API responses into the structured format defined by your Response model.

### Step 4: Register Your Fetcher

Import your fetcher class in your provider package's [`__init__.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/__init__.py) file. The registry scanner imports all modules within `openbb_platform.providers.your_provider`, automatically detecting `Fetcher` subclasses and adding them to the available provider map. No explicit registration function call is required—simply ensuring the class loads into memory suffices.

## Complete Working Example

The following implementation demonstrates a custom fetcher for a hypothetical "MyData" provider supplying daily equity prices. This example follows the exact patterns used in production fetchers like `YFinanceEquityHistoricalFetcher` in [`openbb_platform/providers/yfinance/openbb_yfinance/models/equity_historical.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/openbb_platform/providers/yfinance/openbb_yfinance/models/equity_historical.py).

```python

# openbb_platform/providers/mydata/openbb_mydata/models/daily_price.py

from datetime import date
from typing import List

import httpx
import pandas as pd
from pydantic import BaseModel, Field

from openbb_core.provider.abstract.fetcher import Fetcher


class DailyPriceQuery(BaseModel):
    """Parameters for daily price data requests."""
    symbol: str = Field(..., description="Ticker symbol, e.g. AAPL")
    start_date: date = Field(..., description="Start of period (YYYY-MM-DD)")
    end_date: date = Field(..., description="End of period (YYYY-MM-DD)")


class DailyPriceResponse(BaseModel):
    """Structured response for daily OHLCV data."""
    date: List[date]
    open: List[float]
    high: List[float]
    low: List[float]
    close: List[float]
    volume: List[int]


class MyDataDailyPriceFetcher(Fetcher[DailyPriceQuery, DailyPriceResponse]):
    """Custom fetcher integrating the MyData API."""
    
    @property
    def name(self) -> str:
        return "mydata_daily_price"

    async def _fetch_data(self, params: DailyPriceQuery) -> DailyPriceResponse:
        """Fetch and transform data from MyData REST API."""
        url = (
            f"https://api.mydata.com/v1/prices?"
            f"symbol={params.symbol}"
            f"&start={params.start_date.isoformat()}"
            f"&end={params.end_date.isoformat()}"
        )
        
        async with httpx.AsyncClient() as client:
            response = await client.get(url, timeout=30.0)
            response.raise_for_status()
            raw_data = response.json()
        
        # Normalize JSON array into DataFrame for reliable column extraction

        df = pd.DataFrame(raw_data)
        
        return DailyPriceResponse(
            date=pd.to_datetime(df["date"]).dt.date.tolist(),
            open=df["open"].astype(float).tolist(),
            high=df["high"].astype(float).tolist(),
            low=df["low"].astype(float).tolist(),
            close=df["close"].astype(float).tolist(),
            volume=df["volume"].astype(int).tolist(),
        )

```

Register the fetcher in your provider's initialization file:

```python

# openbb_platform/providers/mydata/__init__.py

from openbb_mydata.models.daily_price import MyDataDailyPriceFetcher  # noqa: F401

```

After registration, the fetcher becomes accessible via the OpenBB CLI:

```bash
openbb mydata daily-price --symbol AAPL --start-date 2024-01-01 --end-date 2024-01-31

```

## Key Source Files Reference

Understanding the following files provides deep insight into the fetcher lifecycle and registration mechanics:

- **[`openbb_platform/core/openbb_core/provider/abstract/fetcher.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/openbb_platform/core/openbb_core/provider/abstract/fetcher.py)** – Defines the `Fetcher` abstract base class with the `fetch_data` entry point and `_fetch_data` abstract method requiring implementation.

- **[`openbb_platform/core/openbb_core/provider/registry_map.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/openbb_platform/core/openbb_core/provider/registry_map.py)** – Implements the auto-discovery scanner that maps provider namespaces to concrete fetcher classes at startup.

- **[`openbb_platform/core/openbb_core/provider/query_executor.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/openbb_platform/core/openbb_core/provider/query_executor.py)** – Orchestrates fetcher instantiation, parameter validation, and execution flow including caching and error handling layers.

- **[`openbb_platform/providers/yfinance/openbb_yfinance/models/equity_historical.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/openbb_platform/providers/yfinance/openbb_yfinance/models/equity_historical.py)** – Production reference implementation showing a complete fetcher with complex query parameters and data transformation logic.

## Summary

Implementing custom fetchers for integrating new data providers in OpenBB follows a standardized, type-safe pattern that minimizes boilerplate while maximizing flexibility. The architecture automatically handles validation, caching, and CLI integration once you provide the core data retrieval logic.

- **Subclass `Fetcher[Query, Response]`** from [`openbb_platform/core/openbb_core/provider/abstract/fetcher.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/openbb_platform/core/openbb_core/provider/abstract/fetcher.py) and implement the async `_fetch_data` method with your API-specific logic.
- **Define Pydantic models** for query parameters and response data to enable automatic validation, CLI argument generation, and OpenAPI schema creation.
- **Import your fetcher class** in the provider's [`__init__.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/__init__.py) to trigger auto-registration via [`registry_map.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/registry_map.py) without explicit function calls.
- **Reference existing implementations** in the YFinance provider for complex data transformation patterns and best practices for error handling.

## Frequently Asked Questions

### What is the difference between `fetch_data` and `_fetch_data` in the Fetcher class?

The public `fetch_data` method is defined in the abstract base class and should not be overridden. It orchestrates caching, validation, and error handling before and after calling your implementation. You must implement the protected `_fetch_data(self, params: Query) -> Response` method, which contains only the API-specific logic for retrieving and formatting data. This separation of concerns ensures consistent behavior across all providers while allowing flexibility in data source integration.

### How does OpenBB discover my custom fetcher without explicit registration?

OpenBB uses the [`registry_map.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/registry_map.py) scanner that imports all modules within `openbb_platform.providers.<your_provider>` at startup. Any class inheriting from `Fetcher` that loads into memory during this scan is automatically indexed and associated with its provider identifier. You only need to ensure your fetcher class is imported in your provider package's [`__init__.py`](https://github.com/OpenBB-finance/OpenBB/blob/main/__init__.py) file to trigger this discovery mechanism.

### Can I use synchronous HTTP libraries like `requests` instead of `httpx`?

While the `_fetch_data` method is defined as `async` in the abstract base class, you can wrap synchronous calls using `asyncio.get_event_loop().run_in_executor()` or similar patterns. However, using native async libraries like `httpx` or `aiohttp` is strongly recommended to prevent blocking the event loop and maintain performance when multiple fetchers run concurrently. The `QueryExecutor` expects an awaitable coroutine and will properly schedule async fetchers.

### What validation occurs on the Query and Response models?

OpenBB leverages Pydantic for strict type validation. The `QueryExecutor` validates user inputs against your Query model before calling `_fetch_data`, converting CLI arguments or API parameters to the correct Python types and raising validation errors for missing required fields or type mismatches. After your fetcher returns data, the Response model validates the structure, ensuring that all fields match their declared types and that the data conforms to the expected schema before passing it to the consumer.