# How the Data Source Manager Coordinates Multiple Data Sources for Different Markets in TradingAgents

> Learn how the TradingAgents data source manager coordinates multiple data sources. Discover dynamic selection, prioritization, and fallback strategies for diverse markets with centralized abstraction.

- Repository: [hsliuping/TradingAgents-CN](https://github.com/hsliuping/tradingagents-cn)
- Tags: how-to-guide
- Published: 2026-02-16

---

**The data source manager in TradingAgents uses a centralized abstraction layer with market-specific manager classes that dynamically select, prioritize, and fall back between multiple providers (Tushare, AKShare, yfinance, etc.) based on runtime configuration and availability.**

TradingAgents-CN implements a sophisticated data source manager to handle the complexity of multi-market financial data ingestion. Located in [`tradingagents/dataflows/data_source_manager.py`](https://github.com/hsliuping/TradingAgents-CN/blob/main/tradingagents/dataflows/data_source_manager.py), this component abstracts provider-specific implementations behind a unified interface, enabling seamless switching between Chinese A-share data sources and US equity providers without modifying downstream trading logic.

## Unified Data Source Registry

All data providers are identified by a single enum **`DataSourceCode`** defined in [`tradingagents/constants/data_sources.py`](https://github.com/hsliuping/TradingAgents-CN/blob/main/tradingagents/constants/data_sources.py) (lines 17-44). Each entry carries a canonical string identifier (e.g., `"tushare"`, `"yfinance"`), a human-readable name, and metadata specifying supported markets, API-key requirements, and rate limits.

This registry acts as the single source of truth for the entire system, ensuring that every module references providers through standardized constants rather than hard-coded strings.

## Market-Specific Manager Architecture

The system implements two concrete manager classes to handle regional market differences:

| Manager | Markets | Key Class Definition |
|---------|---------|----------------------|
| **ChinaDataSourceManager** | A-shares, Hong Kong | `class ChinaDataSource(Enum)` (lines 28-38) → `class DataSourceManager` (lines 57-84) |
| **USDataSourceManager** | US stocks | `class USDataSource(Enum)` (lines 41-51) → `class USDataSourceManager` (lines 217-226) |

Both managers inherit from a common base class but maintain separate provider enums (`ChinaDataSource` vs `USDataSource`) to enforce type safety across market boundaries.

## Provider Lifecycle and Fallback Coordination

The data source manager follows a strict lifecycle to ensure resilient data retrieval:

### Initialization and Availability Detection

During instantiation, the manager executes `_check_mongodb_enabled()` to determine if caching is available, followed by `_check_available_sources()` (lines 202-254 for China, lines 227-285 for US) to verify which provider libraries are installed and which API keys are configured. Providers failing import checks or lacking credentials are excluded from `self.available_sources`.

### Default Source Selection

The manager determines the default provider through `_get_default_source()`, which first checks environment variables (`DEFAULT_CHINA_DATA_SOURCE`, `DEFAULT_US_DATA_SOURCE`). If unset, it falls back to hard-coded priorities: **`AKSHARE`** for China markets and **`yfinance`** for US markets.

### Current Source Tracking and Switching

The active provider is stored in `self.current_source`. Callers retrieve it via `get_current_source()` (lines 538-549) or change it programmatically through `set_current_source()`, enabling dynamic provider switching without restarting the application.

### Priority-Based Fallback Mechanism

When a data request fails, the manager retrieves a downgrade list from `_get_data_source_priority_order()`. If the MongoDB `datasource_groupings` collection contains no configuration, the manager uses static default orders:
- **China**: `AKSHARE` → `TUSHARE` → `BAOSTOCK`
- **US**: `yfinance` → `Alpha Vantage` → `Finnhub`

The manager iterates through `self.available_sources` following this priority sequence until a successful response is obtained or all providers are exhausted.

## Runtime Configuration and Priority Overrides

The data source manager supports three configuration layers:

1. **Database Configuration**: The `system_configs` and `datasource_groupings` collections store enabled flags, API keys, and custom priority orders. The manager reads these in `_get_enabled_sources_from_db()` and `_get_datasource_configs_from_db()`.

2. **Environment Variables**: `DEFAULT_CHINA_DATA_SOURCE` and `DEFAULT_US_DATA_SOURCE` override compiled defaults without requiring database access.

3. **Package Detection**: Import errors during `_check_available_sources()` automatically exclude providers whose dependencies are missing, ensuring the system degrades gracefully when optional libraries are not installed.

## Practical Implementation Examples

### Basic Usage for China A-Shares

```python
from tradingagents.dataflows.data_source_manager import get_data_source_manager

# Obtain the singleton manager

dm = get_data_source_manager()

# Show the current source (e.g., AKSHARE)

print("Current source:", dm.get_current_source().value)

# Switch to Tushare explicitly (if enabled)

dm.set_current_source(dm.ChinaDataSource.TUSHARE)

# Fetch daily price data for Kweichow Moutai (600519)

report = dm.get_stock_data("600519", start_date="2024-01-01", end_date="2024-01-31")
print(report)

```

*Relevant implementation*: `get_current_source` and `set_current_source` (lines 538-549 in [`data_source_manager.py`](https://github.com/hsliuping/TradingAgents-CN/blob/main/data_source_manager.py)).

### US Market with Automatic Fallback

```python
from tradingagents.dataflows.data_source_manager import get_us_data_source_manager

us_dm = get_us_data_source_manager()

print("US default source:", us_dm.get_current_source().value)

# Attempt to retrieve price data; if yfinance fails it will fall back to Alpha Vantage, then Finnhub

price_report = us_dm.get_stock_data("AAPL", start_date="2024-01-01", end_date="2024-01-31")
print(price_report)

```

*Relevant implementation*: `USDataSourceManager.get_stock_data` uses the same fallback loop as its China counterpart (see the class body in lines 66-84 of the US manager section).

### Inspecting Available Sources

```python
dm = get_data_source_manager()
print("Available China sources:", [s.value for s in dm.available_sources])

us_dm = get_us_data_source_manager()
print("Available US sources:", [s.value for s in us_dm.available_sources])

```

*Relevant implementation*: `self.available_sources` is populated in `_check_available_sources` (China: lines 202-254; US: lines 227-285).

### Database-Driven Priority Override

If the `datasource_groupings` collection contains the following configuration:

```json
{
  "market_category_id": "us_stocks",
  "enabled": true,
  "data_source_name": "finnhub",
  "priority": 10
},
{
  "market_category_id": "us_stocks",
  "enabled": true,
  "data_source_name": "yfinance",
  "priority": 5
}

```

The manager reads this ordering in `_get_data_source_priority_order` (US section, lines 61-93) and will **try Finnhub first**, then fall back to yfinance.

## Summary

- **Centralized abstraction**: The `DataSourceManager` classes in [`tradingagents/dataflows/data_source_manager.py`](https://github.com/hsliuping/TradingAgents-CN/blob/main/tradingagents/dataflows/data_source_manager.py) provide a unified API that hides provider-specific implementations from trading logic.
- **Market segmentation**: Separate `ChinaDataSourceManager` and `USDataSourceManager` classes handle regional provider enums and default priorities.
- **Dynamic fallback**: The manager maintains an `available_sources` list and a priority order (configurable via MongoDB or static defaults) to automatically retry failed requests with alternative providers.
- **Runtime flexibility**: Environment variables and database configurations allow operators to change default sources and priority orders without code changes.
- **Singleton access**: Global helper functions `get_data_source_manager()` and `get_us_data_source_manager()` ensure consistent state across the application.

## Frequently Asked Questions

### How does the data source manager handle provider failures?

When a data request fails, the manager catches the exception and initiates a fallback sequence defined in `_get_data_source_priority_order()`. It iterates through the priority list (e.g., `AKSHARE` → `TUSHARE` → `BAOSTOCK` for China), skipping any providers not in `self.available_sources`, until a successful response is obtained or all options are exhausted.

### Can I add a new data provider without modifying the core manager logic?

Yes, but you must extend the appropriate enum (`ChinaDataSource` or `USDataSource`) in [`data_source_manager.py`](https://github.com/hsliuping/TradingAgents-CN/blob/main/data_source_manager.py), add the provider to the `DataSourceCode` registry in [`constants/data_sources.py`](https://github.com/hsliuping/TradingAgents-CN/blob/main/constants/data_sources.py), and implement the corresponding adapter method (e.g., `_get_newprovider_adapter()`) within the manager class. The existing fallback and initialization logic will automatically include the new provider if its dependencies are detected.

### What configuration takes precedence when setting the default data source?

The manager checks configurations in the following order of precedence: first, the `DEFAULT_CHINA_DATA_SOURCE` or `DEFAULT_US_DATA_SOURCE` environment variables; second, the priority configuration stored in the MongoDB `datasource_groupings` collection; third, the hard-coded defaults (`AKSHARE` for China, `yfinance` for US) defined in the `_get_default_source()` method.

### How does the manager optimize performance when multiple providers are available?

The manager optimizes performance through two mechanisms: **MongoDB caching**, which checks the cache before hitting any external API (highest priority), and **availability pre-filtering**, which maintains an `available_sources` list populated during initialization to avoid runtime import checks. The priority-based fallback ensures that the fastest or most reliable provider (as configured) is always attempted first, minimizing latency from failed requests.