Columnar Formats for AI Telemetry: Parquet vs Vortex With DuckDB

An AI inference service makes a model available to an application through an API. When you ask an inference API a question, the application sends your prompt to the service. The service processes it and generates a response, often streaming the answer back in batches. Producing a correct answer is only part of the job. The service also needs to respond quickly, handle many requests, and use GPU compute resources efficiently. Measuring each incoming request helps services understand what users experience and investigate problems when performance bottlenecks occur.

AI Inference Metrics

Models process input in units called tokens. Hence, common metrics include time to first token (TTFT), time between tokens (TBT), and time per output token (TPOT). Other measurements could be end-to-end latency, input and output token counts, request rate, error rate, etc.   

Also, TTFT and TBT describe different analyses. A response might start almost immediately but stream slowly, or take several seconds to start and then finish. Measuring only total response time hides that distinction. Averages can also hide problems. Services often examine P95 latency. This helps reveal slower experiences that an average can mask.

To have a deeper understanding of such service behavior, a service can emit a simplified log record like the one below when a request finishes. One record explains one request. Millions of such records may be used to analyze the service and answer questions such as “Why did service-side latency increase during a particular time window?” “Did service errors increase with incoming traffic?” etc. This article explores using Parquet, Vortex, and DuckDB for such analytical queries.

{
   "event_time": "2026-09-01T12:30:00",
   "model": "llama",
   "input_tokens": 100,
   "output_tokens": 200,
   "ttft_ms": 100
}

Columnar Formats

Most telemetry queries need only a handful of columns; for example, a high-latency investigation may need only one hour of records. Columnar storage lets query engines access selected columns, and data layouts can help them skip irrelevant data. One such columnar format, Parquet, organizes data into row groups, column chunks, and pages. Another format, Vortex, takes a compressed array approach that supports some operations without fully decoding the data. Neither design guarantees faster queries; the outcome depends on the data, query implementation, and work being performed. This article uses DuckDB as the query engine because it can run SQL queries against both Parquet directly and Vortex through its extensions without setting up a database server.  

Scenario: Investigating High Latency

When an AI service feels slow, a common starting point is to examine requests from the affected time window. For example, “How many requests were handled?” “What was the P95 TTFT?” etc. 

This experiment explores how file format and row ordering affect the time it takes to query one hour of inference latency data. For this purpose, one million inference records are synthetically generated using the structure shown earlier. Each record contains an event timestamp, model name, input and output token counts, and time to first token (TTFT) in milliseconds. All timestamps are interpreted as UTC. 

A given record is written in two formats, i.e., Parquet with Zstandard compression and Vortex with default writer settings. For each format, time-sorted and shuffled versions are created. This produces the four files below. 

sorted.parquet - Time sorted records, Parquet with Zstandard compression.
shuffled.parquet - Shuffled records, Parquet with Zstandard compression.
sorted.vortex - Time sorted records, Vortex with default writer settings.
shuffled.vortex - Shuffled records, Vortex with default writer settings.

The following query calculates request count, average TTFT, and P95 TTFT for a populated one-hour window. The query is then repeated against the shuffled files. Request counts and latency statistics are checked across all four configurations before timing results are compared. 

SELECT
    COUNT(*) AS request_count,
    AVG(ttft_ms) AS avg_ttft_ms,
    QUANTILE_CONT(ttft_ms, 0.95) AS p95_ttft_ms
FROM read_parquet('data/sorted.parquet')
WHERE event_time >= TIMESTAMP '2026-09-01 12:00:00'
  AND event_time <  TIMESTAMP '2026-09-01 13:00:00';

For Vortex, only the file reader and path are changed:

FROM read_vortex('data/sorted.vortex')

Step 1: Generate the Data Files

from pathlib import Path
import duckdb

# Refuse to overwrite an existing dataset.
Path("data").mkdir(exist_ok=False)

con = duckdb.connect()
con.execute("SET threads = 1")
con.execute("SET preserve_insertion_order = true")
con.execute("INSTALL vortex")
con.execute("LOAD vortex")
print("Writer DuckDB:", duckdb.__version__)

con.execute("""
    CREATE TABLE logs AS
    SELECT
        -- One request per second; timestamps are interpreted as UTC.
        TIMESTAMP '2026-09-01'
            + i * INTERVAL '1 second' AS event_time,

        -- Synthetic model mix: 70%, 20%, and 10%.
        CASE
            WHEN i % 10 < 7 THEN 'llama'
            WHEN i % 10 < 9 THEN 'gemma'
            ELSE 'qwen'
        END AS model,

        -- Different hash constants give each field a distinct sequence.
        -- Input tokens: 100–8099; output tokens: 100–1099.
        CAST(100 + hash(i, 11) % 8000 AS INTEGER) AS input_tokens,
        CAST(100 + hash(i, 22) % 1000 AS INTEGER) AS output_tokens,

        -- Baseline TTFT: 100–499 ms; add 800 ms during the slow hour.
        CAST(100 + hash(i, 33) % 400 AS INTEGER)
            + CASE
                WHEN event_time >= TIMESTAMP '2026-09-06 12:00:00'
                 AND event_time <  TIMESTAMP '2026-09-06 13:00:00'
                THEN 800
                ELSE 0
              END AS ttft_ms

    FROM range(1000000) AS t(i)
""")

formats = {
    "parquet": "FORMAT PARQUET, COMPRESSION ZSTD",
    "vortex": "FORMAT VORTEX",
}

layouts = {
    "sorted": "event_time",
    # Timestamp breaks any hash ties, giving a deterministic order.
    "shuffled": "hash(event_time), event_time",
}

for layout, order in layouts.items():
    for fmt, options in formats.items():
        path = f"data/{layout}.{fmt}"
        con.execute(f"""
            COPY (
                SELECT * FROM logs ORDER BY {order}
            )
            TO '{path}' ({options})
        """)
        print("Created", path)

con.close()

Writer DuckDB: 1.5.5
Created data/sorted.parquet
Created data/sorted.vortex
Created data/shuffled.parquet
Created data/shuffled.vortex

Step 2: Run the Experiment

Each configuration receives one warmup, followed by five measured executions. Execution order is randomized across configurations, and DuckDB is configured to use one query worker. File generation, sorting, and extension loading are excluded from query timings. Median query latency is reported alongside the observed minimum and maximum. File size is recorded separately. Two different kinds of latency are involved: TTFT describes the synthetic AI requests, while query latency measures how long DuckDB takes to analyze those records. The experiment measures the latter, including filtering, aggregation, and fetching the small result.

import csv
import math
import random
import statistics
import time
from pathlib import Path

import duckdb

con = duckdb.connect()
con.execute("SET threads = 1")
con.execute("LOAD vortex")
print("Reader DuckDB:", duckdb.__version__)
print("Vortex:", con.execute("""
    SELECT extension_version FROM duckdb_extensions()
    WHERE extension_name = 'vortex'
""").fetchone()[0])

query = """
    SELECT count(*), avg(ttft_ms), quantile_cont(ttft_ms, 0.95)
    FROM {source}
    WHERE event_time >= TIMESTAMP '2026-09-06 12:00:00'
      AND event_time <  TIMESTAMP '2026-09-06 13:00:00'
"""

jobs = [
    (layout, fmt)
    for layout in ("sorted", "shuffled")
    for fmt in ("parquet", "vortex")
]
times = {job: [] for job in jobs}
rng = random.Random(42)
reference = None

# Round 0 is warmup; rounds 1–5 are measured.
for repeat in range(6):
    rng.shuffle(jobs)
    for layout, fmt in jobs:
        sql = query.format(
            source=f"read_{fmt}('data/{layout}.{fmt}')"
        )
        start = time.perf_counter()
        result = con.execute(sql).fetchone()
        elapsed = (time.perf_counter() - start) * 1000

        if reference is None:
            reference = result
        assert result[0] == 3600
        assert all(
            math.isclose(a, b, rel_tol=1e-9, abs_tol=1e-6)
            for a, b in zip(result[1:], reference[1:])
        ), "Results differ across configurations"

        if repeat:
            times[(layout, fmt)].append(elapsed)

rows = []
for (layout, fmt), values in sorted(times.items()):
    rows.append({
        "format": fmt,
        "layout": layout,
        "size_mib": Path(f"data/{layout}.{fmt}").stat().st_size / 2**20,
        "median_ms": statistics.median(values),
        "min_ms": min(values),
        "max_ms": max(values),
    })

with open("results.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=list(rows[0]))
    writer.writeheader()
    writer.writerows(rows)

print("\nRequests, average TTFT, p95 TTFT:", reference)
print("\nFormat    Layout      MiB     Median ms    Min–max ms")
for r in rows:
    print(
        f"{r['format']:9} {r['layout']:9} {r['size_mib']:7.2f}"
        f" {r['median_ms']:12.2f}"
        f"    {r['min_ms']:.2f}–{r['max_ms']:.2f}"
    )

con.close()

Reader DuckDB: 1.5.5
Vortex: 9a8eb67

Requests, average TTFT, p95 TTFT: (3600, 1100.8336111111112, 1281.0)

Format    Layout      MiB     Median ms    Min–max ms
parquet   shuffled     9.40        11.91    11.32–12.49
vortex    shuffled     6.60         7.14    6.99–9.14
parquet   sorted       6.79         2.75    2.30–2.86
vortex    sorted       4.10         1.68    1.38–4.12

Takeaway: File Format vs. Row Ordering

The selected hour contained 3600 requests, with an average TTFT of 1100.83 ms and a P95 TTFT of 1281 ms. These values describe the synthetic inference records, and the query times measure how long DuckDB took to calculate that summary. Time-sorted Vortex had the lowest median query time and smallest file size. 

However, ordering improved query performance substantially for both formats. Note that these findings apply to this synthetic dataset and hour window query after warmup. Timings varied across runs, and the results do not establish a universal format ranking or directly measure how much data was skipped.

Conclusion

AI inference services generate telemetry that helps explain token usage, identify latency changes, and investigate high latency issues. Efficient storage and querying can make that data less expensive to retain and faster to analyze. 

In this synthetic DuckDB experiment, time ordering improved median query performance. Sorted Vortex produced the smallest file and fastest median query, but sorted Parquet outperformed shuffled Vortex, highlighting the importance of organizing telemetry around its access patterns. These local warm-up-based results do not establish a universal format winner or demonstrate faster inference. Instead, they illustrate how format choice and data layout can support more efficient AI telemetry analysis, making it easier to investigate performance problems before deciding how to address them.

References

• Build a DIY AI Model Hosting Platform With vLLM
• Understanding Inference Time Compute
• File Format, Parquet 
• Vortex Documentation
• DuckDB Documentation, Vortex
• Reading and Writing Parquet Files, DuckDB Docs