# poly_data | warproxxx | Knowledge Base | Instagit

Polymarket Data Retriever that fetches, processes, and structures Polymarket data including markets, order events and trades.

GitHub Stars: 1.4k

Repository: https://github.com/warproxxx/poly_data

---

## Articles

### [How to Load and Analyze Trade Data with Poly Data: A Complete Guide](/warproxxx/poly_data/load-analyze-trade-data-poly-data)

Learn to load and analyze trade data with Poly Data. This Python pipeline processes Polymarket events into normalized CSVs for quantitative analysis using Polars.

- Tags: how-to-guide
- Published: 2026-04-21

### [How Poly Data Retries on Network Failures: A Deep Dive into the Resilience Strategy](/warproxxx/poly_data/poly-data-retry-network-failures)

Discover how Poly Data ensures resilience against network failures. Learn about its multi-layered retry strategies for HTTP and GraphQL errors.

- Tags: deep-dive
- Published: 2026-04-21

### [Where Missing Markets Are Stored in Poly Data: Complete File Location Guide](/warproxxx/poly_data/missing-markets-storage-poly-data)

Locate missing markets in Poly Data. Find and manage `missing_markets.csv` with our guide to Poly Data file locations and essential utility functions.

- Tags: how-to-guide
- Published: 2026-04-21

### [How Poly Data Handles Missing Markets: A Complete Technical Guide](/warproxxx/poly_data/handle-missing-markets-poly-data)

Learn how Poly Data handles missing markets using a supplemental CSV and Python functions to automatically detect, fetch, and persist market data for comprehensive analysis.

- Tags: how-to-guide
- Published: 2026-04-21

### [How Price Is Calculated in Poly Data Trades: A Deep Dive into the Live-Trade Processor](/warproxxx/poly_data/calculate-price-poly-data-trades)

Discover how Poly Data calculates trade prices by dividing USDC-filled amounts by non-USDC token amounts. Learn the live-trade processor's logic for ratio direction.

- Tags: deep-dive
- Published: 2026-04-21

### [How Trade Direction Is Determined in Poly Data: A Code‑Level Analysis](/warproxxx/poly_data/trade-direction-determination-poly-data)

Discover how Poly Data determines trade direction. Understand which side holds USDC to identify buyers and sellers with this code-level analysis.

- Tags: internals
- Published: 2026-04-21

### [How Poly Data Identifies USDC in Trades: A Technical Deep Dive](/warproxxx/poly_data/identify-usdc-trades-poly-data)

Learn how Poly Data identifies USDC in trades by treating asset ID 0 as USDC and using Polars conditional expressions for accurate labeling.

- Tags: deep-dive
- Published: 2026-04-21

### [How Asset IDs Are Mapped to Markets in Poly Data](/warproxxx/poly_data/asset-id-mapping-poly-data-markets)

Discover how Poly Data maps asset IDs to markets using token identifiers in CSV files and a long lookup table for efficient market data retrieval. Learn the process now.

- Tags: internals
- Published: 2026-04-21

### [How Poly Data Achieves Idempotent Data Collection: CSV Offsets and Cursor State](/warproxxx/poly_data/poly-data-idempotency)

Discover how Poly Data ensures idempotent data collection using CSV offsets and cursor state. Learn about offset based resumption and JSON cursor files for reliable event streaming and deduplication.

- Tags: internals
- Published: 2026-04-21

### [How Poly Data Ensures Pipeline Resumability: A Technical Deep Dive](/warproxxx/poly_data/poly-data-pipeline-resumability)

Poly Data ensures pipeline resumability by detecting last written records using line counts, cursor files, and timestamps. Restart your process seamlessly from the exact point of failure.

- Tags: deep-dive
- Published: 2026-04-21

### [How Are Amounts Normalized in Poly Data Trades: A Complete Technical Guide](/warproxxx/poly_data/normalize-amounts-poly-data-trades)

Learn how Poly Data normalizes trade amounts by dividing raw on-chain integer values by 10⁶ in update_utils/process_live.py converting them to human-readable decimals.

- Tags: deep-dive
- Published: 2026-04-21

### [Fields in processed/trades.csv: Complete Schema Reference for Polymarket Trade Data](/warproxxx/poly_data/trades-csv-fields)

Understand the complete schema reference for processed/trades.csv in the poly_data repository. Explore market ID, addresses, trade flags, price, token amounts, and transaction hash.

- Tags: api-reference
- Published: 2026-04-21

### [How Poly Data Processes Raw Order Events Into Trades: A Complete Pipeline Breakdown](/warproxxx/poly_data/poly-data-process-order-events-trades)

Discover how Poly Data transforms raw order events into trade records with its Python Polars ETL pipeline. Understand the complete processing breakdown for analysis-ready data.

- Tags: internals
- Published: 2026-04-21

### [How to Resume Goldsky Data Scraping in Poly Data: Complete Guide](/warproxxx/poly_data/resume-goldsky-scraping-poly-data)

Easily resume Goldsky data scraping in Poly Data using the JSON cursor state file. Pick up where you left off without re-downloading historical data. Learn how to implement this essential feature.

- Tags: how-to-guide
- Published: 2026-04-21

### [What Data Is Stored in goldsky/orderFilled.csv: Complete Schema Reference](/warproxxx/poly_data/orderfilled-csv-fields)

Explore the goldsky/orderFilled.csv schema in warproxxx/poly_data. Understand on-chain trade data including timestamps, addresses, asset IDs, filled amounts, and transaction hashes for completed orders.

- Tags: api-reference
- Published: 2026-04-21

### [Sticky Timestamp Mechanism for Order Events in warproxxx/poly_data](/warproxxx/poly_data/sticky-timestamp-mechanism-order-events)

Understand the sticky timestamp mechanism in warproxxx/poly_data for lossless blockchain order event pagination. Prevent data gaps with this ID-based cursor solution.

- Tags: internals
- Published: 2026-04-21

### [How Poly Data Scrapes Order Events from Goldsky: Architecture and Implementation](/warproxxx/poly_data/poly-data-scrape-order-events-goldsky)

Discover how Poly Data scrapes order events from Goldsky using a GraphQL scraper with sticky cursor pagination for high-throughput blockchain data. Learn about its architecture and implementation.

- Tags: architecture
- Published: 2026-04-21

### [Goldsky Subgraph Integration in Poly Data: Resumable GraphQL Scraping Explained](/warproxxx/poly_data/goldsky-subgraph-integration-poly-data)

Explore the Goldsky subgraph integration in Poly Data. Learn how this resumable GraphQL scraper fetches orderFilled events, uses cursors, and stores data in CSVs for efficient, recoverable orderbook data.

- Tags: how-to-guide
- Published: 2026-04-21

### [How to Resume Market Data Collection in Poly Data: A Complete Guide](/warproxxx/poly_data/resume-market-data-collection-poly-data)

Resume market data collection in Poly Data effortlessly. Rerun the update_markets script to automatically continue fetching data from the last recorded offset.

- Tags: how-to-guide
- Published: 2026-04-21

### [How Poly Data Handles API Rate Limiting](/warproxxx/poly_data/poly-data-api-rate-limiting)

Discover how Poly Data tackles API rate limiting with automatic 429 response detection and a 10-second retry backoff for seamless Polymarket gamma-API data downloads.

- Tags: how-to-guide
- Published: 2026-04-21

### [What Data Is Included in markets.csv? Complete Schema Guide for warproxxx/poly_data](/warproxxx/poly_data/markets-csv-fields)

Explore the markets.csv schema in warproxxx/poly_data. Discover 13 columns of market metadata including IDs, outcomes, volumes, and temporal data for comprehensive insights.

- Tags: schema-guide
- Published: 2026-04-21

### [How Does Poly Data Fetch Market Data from Polymarket?](/warproxxx/poly_data/poly-data-fetch-market-data)

Discover how Poly Data fetches market data from Polymarket using Python routines for full crawls and targeted retrieval. Store and analyze normalized data efficiently.

- Tags: how-to-guide
- Published: 2026-04-21

### [Stages of the Poly Data Pipeline: A 3-Step ETL Guide](/warproxxx/poly_data/poly-data-pipeline-stages)

Explore the 3 stages of the Poly Data pipeline: collect market data, scrape order events, and process trades. Understand this ETL guide for efficient data analysis.

- Tags: how-to-guide
- Published: 2026-04-21

### [How to Run the Poly Data Pipeline: A Complete ETL Guide for Polymarket Data](/warproxxx/poly_data/how-to-run-poly-data-pipeline)

Run the Poly Data pipeline effortlessly. Execute python update_all.py to complete the Polymarket ETL workflow, extracting market data and on-chain events for analysis.

- Tags: how-to-guide
- Published: 2026-04-21