Prediction Market Data Downloads: CSV, Parquet and API Exports

Match the format to the workload

Fields not to lose

• Venue, event, market and outcome identifiers in their native and normalized forms.
• Exchange/source time and receive/observation time as separate fields.
• Bid prices, bid sizes, ask prices and ask sizes at matching array positions.
• Capture method, interval or event sequence needed to interpret cadence.
• Settlement outcome and reference values stored without leaking into earlier rows.
• A stable null or missing convention that never silently becomes zero.

Partition for the questions you actually ask

Large depth archives should be partitioned by high-selectivity fields such as venue and date, then by a stable market or asset key where it matches the workload. A million tiny files can be as costly as one unpartitioned export, so measure the query engine rather than copying a generic lake layout.

Keep a manifest beside bulk exports. It should record schema version, generation time, requested filters, row or observation counts and any incomplete partitions. That manifest turns an opaque download into a reproducible research input.

Buying daily Parquet files from DepthFeed

Choose an available product and UTC date range in the public bulk catalogue. Create a free account and accept the data licence to receive an exact quote. Pay by hosted card checkout or existing credits. Bulk file purchases are independent of API subscriptions; an API plan does not include these file orders.

Polymarket book files retain token identifiers and bid/ask arrays. Kalshi files retain ticker identity and venue-native YES/NO ladders. Polymarket US is a separate slug-based dataset. A product file can contain multiple markets, and a first/last date does not establish uninterrupted capture or every-market coverage.

The free Parquet, JSON/schema and Python notebook samples contain eight fabricated rows for compatibility checks. They are not captured market history or evidence of archive coverage. Purchased archive versions can have a different schema, so inspect the actual files before processing.

Individual signed download links expire after one hour. Paid orders allow fresh links for 30 days; retain the files locally during that window. The current catalogue, quote and data licence govern availability, pricing and access conditions.

Two valid order-book file models

A nested snapshot model stores one observation per row with arrays for bid prices, bid sizes, ask prices and ask sizes. It is compact, preserves book boundaries and maps naturally to Parquet list columns or JSON. The consumer must validate that each price array matches its size array and that the arrays are correctly ordered.

A normalized levels model stores one row per observation, side and price level. It is convenient for SQL aggregation but multiplies row counts and requires a stable observation identifier to reconstruct a complete book. Neither model is universally better; the manifest should state which one is used and how deletions, empty books and partial observations are represented.

A derived CSV schema for a small extract

The table below is an illustrative research schema, not the schema promised for purchased files. DepthFeed sells the bulk archive as Parquet. You can create a CSV locally with DuckDB or Python, but must document how nested arrays are encoded or expanded. Keep Kalshi native YES/NO fields unless you explicitly document a valid transformation; do not relabel them as Polymarket bid/ask arrays.

Parquet layout for analytical workloads

For large historical-depth studies, Parquet usually reduces storage and scan cost because timestamps, identifiers and numeric arrays compress efficiently and query engines can skip unrelated partitions. Partition first by fields that consistently narrow reads, commonly venue and date, then test whether asset or market family improves the real workload.

Avoid encoding every market into its own tiny file. Small-file overhead can dominate query planning and object-store requests. Compact files to a measured target, keep row groups large enough for efficient scans, and publish checksums so a research run can prove exactly which objects it consumed.

Download quality-control checklist

• Compare manifest row counts with the sum of file or partition counts.
• Verify checksums before parsing and preserve the original manifest with the research output.
• Check timestamp monotonicity within each market without assuming all venues share one cadence.
• Reject mismatched price-size arrays, out-of-range prices and unexplained crossed books.
• Quantify duplicate observations, empty books, gaps and markets with no settlement mapping.
• Run a sample depth-walk and reconcile its VWAP against an independent implementation.
• Pin the schema version and record every transformation applied after download.

When to use a download, REST or WebSocket

Use a bulk download when the study repeatedly scans a large fixed historical range or must run close to the compute engine. Use REST when filters can substantially reduce the payload, when the workflow is exploratory, or when pagination and reproducible requests are more useful than local storage. Use WebSocket for current monitoring and forward collection, not as a substitute for an already retained archive.

A workflow can combine purchased historical Parquet, REST for targeted queries and metadata, and WebSocket for current updates. Validate each delivery schema and venue-specific identifiers before joining them; file exports, REST responses and stream frames are not automatically interchangeable.