# Where Missing Markets Are Stored in Poly Data: Complete File Location Guide

> Locate missing markets in Poly Data. Find and manage `missing_markets.csv` with our guide to Poly Data file locations and essential utility functions.

- Repository: [warproxxx/poly_data](https://github.com/warproxxx/poly_data)
- Tags: how-to-guide
- Published: 2026-04-21

---

**Missing markets in Poly Data are stored in a CSV file named `missing_markets.csv` in the repository root, with the `get_markets` and `update_missing_tokens` functions in [`poly_utils/utils.py`](https://github.com/warproxxx/poly_data/blob/main/poly_utils/utils.py) handling all read and write operations.**

If you're working with the [warproxxx/poly_data](https://github.com/warproxxx/poly_data) repository and need to locate or manage markets that couldn't be fetched during data collection, this guide covers the exact file paths, function signatures, and usage patterns you need.

## The Default Missing Markets File Location

Poly Data uses a simple, predictable file structure for handling incomplete market data. When the `update_missing_tokens` function discovers new token IDs that need to be fetched, it persists them to a dedicated CSV file.

### Primary Storage: `missing_markets.csv`

The default location where missing markets are stored is:

```

missing_markets.csv

```

This file resides in the working directory—typically the repository root—unless you specify an alternative path when calling the utility functions.

## Core Functions in [`poly_utils/utils.py`](https://github.com/warproxxx/poly_data/blob/main/poly_utils/utils.py)

All operations related to where missing markets are stored are handled by two functions in [`poly_utils/utils.py`](https://github.com/warproxxx/poly_data/blob/main/poly_utils/utils.py):

### `get_markets`: Loading Markets Including Missing Ones

The `get_markets` function automatically checks for and loads the missing markets file if it exists:

```python
def get_markets(main_file: str = "markets.csv",
                missing_file: str = "missing_markets.csv"):
    # …

    # Load missing markets file

    if os.path.exists(missing_file):
        missing_df = pl.scan_csv(missing_file, …).collect(streaming=True)
        dfs.append(missing_df)
        print(f"Loaded {len(missing_df)} markets from {missing_file}")

```

This design ensures that **missing markets are seamlessly integrated** with regular market data when loaded.

### `update_missing_tokens`: Writing Missing Markets to File

When new missing token IDs are discovered, `update_missing_tokens` fetches their data and writes (or overwrites) the missing markets file:

```python
def update_missing_tokens(missing_token_ids: List[str],
                          csv_filename: str = "missing_markets.csv"):
    # …

    # Save the dataframe to the CSV file

    df.write_csv(csv_filename)
    print(f"Saved missing markets to {csv_filename}")

```

## Practical Code Examples

### Loading All Markets Including Missing Ones

```python

# Example: Load all markets, including any missing ones

from poly_utils.utils import get_markets

# Returns a Polars DataFrame with both regular and missing markets

markets_df = get_markets()          # uses defaults: markets.csv + missing_markets.csv

print(markets_df.head())

```

### Updating the Missing Markets File

```python

# Example: Update the missing‑markets file after discovering new token IDs

from poly_utils.utils import update_missing_tokens

new_missing_ids = ["0x1234…", "0xabcd…"]
update_missing_tokens(new_missing_ids)   # writes to missing_markets.csv

```

## Customizing Where Missing Markets Are Stored

Both functions accept optional parameters to change where missing markets are stored:

| Parameter | Default | Purpose |
|-----------|---------|---------|
| `missing_file` in `get_markets` | `"missing_markets.csv"` | Path to read missing markets from |
| `csv_filename` in `update_missing_tokens` | `"missing_markets.csv"` | Path to write missing markets to |

Example with custom path:

```python
from poly_utils.utils import get_markets, update_missing_tokens

# Use a subdirectory for missing markets

markets_df = get_markets(missing_file="data/missing_markets.csv")

# Write to the same custom location

update_missing_tokens(new_ids, csv_filename="data/missing_markets.csv")

```

## Summary

- **Missing markets are stored in `missing_markets.csv`** by default, located in the working directory
- The **`get_markets`** function in [`poly_utils/utils.py`](https://github.com/warproxxx/poly_data/blob/main/poly_utils/utils.py) automatically loads this file when present
- The **`update_missing_tokens`** function writes new missing market data to the same file
- Both functions support **custom file paths** via the `missing_file` and `csv_filename` parameters
- The implementation uses **Polars** for efficient CSV reading and writing with streaming support

## Frequently Asked Questions

### What happens if `missing_markets.csv` doesn't exist?

If the file doesn't exist, `get_markets` simply returns the regular markets from `markets.csv` without error. The missing markets file is only loaded when `os.path.exists(missing_file)` returns `True`.

### Can I use a different filename for missing markets?

Yes. Pass your preferred filename to both functions: use `missing_file` in `get_markets()` and `csv_filename` in `update_missing_tokens()`. Both parameters accept relative or absolute paths.

### Does `update_missing_tokens` append or overwrite existing data?

The function **overwrites** the entire file. When you call `df.write_csv(csv_filename)`, it replaces any existing content. If you need to preserve existing missing markets, load them first with `get_markets` and merge before calling `update_missing_tokens`.