Beyond JSON: Benchmarking TOON and TOON-LD for LLMs

JSON has been the default structured-data format for APIs, configuration, event streams, and application integration for decades. It is portable, readable, widely supported, and easy to validate.

However, JSON was not designed for LLMs.

When structured data is placed inside an LLM prompt, every quotation mark, repeated field name, brace, comma, and nested structure contributes to the prompt’s token count. For a small request, this overhead may be insignificant. For applications that send thousands of records, tool results, or knowledge-graph entities to an LLM, it can consume a meaningful portion of the context window.

Token-Oriented Object Notation, or TOON, proposes a different representation. It encodes the same objects, arrays, and primitive values as JSON but uses a compact, line-oriented syntax designed for LLM prompts. TOON combines indentation for nested structures with tabular representations for homogeneous arrays. Its strongest use case is a collection of objects that share the same fields.

TOON-LD applies a related idea to Linked Data. It is intended to represent JSON-LD knowledge graphs more compactly while retaining Linked Data constructs such as @context, @id, @type and @graph.

This tutorial explains the differences among JSON, TOON, JSON-LD & TOON-LD and shows how to benchmark their token consumption, serialized size, conversion overhead and round-trip correctness.

Why JSON Consumes Additional LLM Tokens

Consider the following incident records:

{
  "incidents": [
    {
      "id": "INC-000001",
      "service": "checkout",
      "severity": "critical",
      "region": "ap-south-1",
      "owner": "platform"
    },
    {
      "id": "INC-000002",
      "service": "payments",
      "severity": "high",
      "region": "eu-west-1",
      "owner": "payments"
    }
  ]
}

The field names id, service, severity, region, and owner appear in every record. An application parser needs those repeated keys to reconstruct each JSON object, but an LLM prompt pays for their repeated tokenization.

A corresponding TOON representation can declare the fields once and place the values in rows:

incidents[2]{id,service,severity,region,owner}:
  INC-000001,checkout,critical,ap-south-1,platform
  INC-000002,payments,high,eu-west-1,payments

The exact encoded output depends on the TOON specification and encoder version, so production applications should generate TOON through a library rather than manually constructing it.

The important difference is structural: JSON repeats the complete object syntax for every row, whereas TOON can amortize that structure across a uniform collection.

The TOON project describes the format as a lossless representation of the JSON data model and identifies uniform arrays of objects as its primary efficiency advantage. It also notes that deeply nested or non-uniform data may not receive the same benefit and can sometimes remain more efficient in JSON.

JSON and TOON Serve Different Architectural Purposes

TOON should not automatically replace JSON across an application. JSON remains appropriate for:

• Public and internal APIs
• Application configuration
• Persistent storage
• Event exchange
• Schema-based validation
• Browser and programming-language interoperability
• Observability logs and audit records

TOON is better evaluated as a representation used at the LLM boundary. A practical architecture is:

The application continues to use JSON internally. Only the structured context inserted into the prompt is converted to TOON. This approach reduces migration risk and confines the new format to the part of the architecture where token efficiency matters.

What Is JSON-LD?

JSON-LD is a W3C-standardized JSON-based format for Linked Data. It adds semantic meaning to ordinary JSON through globally identifiable concepts and relationships. The JSON-LD 1.1 specification is a W3C Recommendation and is designed to integrate Linked Data into JSON-based programming environments and web services. Consider the following example:

{
  "@context": {
    "ex": "https://example.org/",
    "affects": {
      "@id": "ex:affects",
      "@type": "@id"
    },
    "ownedBy": {
      "@id": "ex:ownedBy",
      "@type": "@id"
    }
  },
  "@graph": [
    {
      "@id": "ex:incident-101",
      "@type": "ex:Incident",
      "ex:severity": "critical",
      "affects": "ex:checkout"
    },
    {
      "@id": "ex:checkout",
      "@type": "ex:Service",
      "ownedBy": "ex:platform-team"
    }
  ]
}

This document contains more than two nested JSON objects. It describes a graph:

An LLM can use this structure for questions such as: Which team owns the service affected by incident 101?

JSON-LD is therefore useful for knowledge graphs, semantic search, Graph-RAG, interoperable metadata, and agent systems that must traverse relationships among entities.

What Is TOON-LD?

TOON-LD is an emerging format that extends TOON with Linked Data semantics. Its implementation describes TOON-LD as a compression representation for JSON-LD knowledge graphs used in LLM context windows. It supports JSON-LD constructs and provides conversions between JSON-LD and TOON-LD.

A simplified TOON-LD representation of a uniform graph may resemble:

@context:
  ex: https://example.org/

@graph[2]{@id,@type,ex:severity,ex:affects}:
  ex:incident-101,ex:Incident,critical,ex:checkout
  ex:incident-102,ex:Incident,high,ex:payments

The main optimization again comes from declaring a common shape once instead of repeating every JSON-LD field for every entity.

TOON-LD should nevertheless be assessed differently from JSON-LD. JSON-LD is a mature W3C standard with established processors and semantic-web tooling. TOON-LD is considerably newer and should be evaluated for library stability, interoperability, and semantic preservation before production use.

JSON, TOON, JSON-LD and TOON-LD Compared

TOON should be compared with JSON. TOON-LD should primarily be compared with JSON-LD. Comparing TOON-LD only with ordinary JSON would mix two different data models and could produce a misleading conclusion.

Designing a Fair Benchmark

Token-efficiency claims should not be evaluated with one carefully selected payload. The accompanying benchmark uses four datasets:

1. Flat homogeneous incident records
2. Nested homogeneous incident records
3. Irregular and sparse incident records
4. JSON-LD incident knowledge graphs

Each dataset is generated at multiple scales: 10 records,100 records, 1000 records, 10000 records

This exposes an important characteristic of token-oriented formats: their benefits can depend significantly on the shape and scale of the input. 

Flat Homogeneous Data

The flat dataset contains records with identical fields:

{
  "id": "INC-000001",
  "service": "service-01",
  "severity": "critical",
  "region": "ap-south-1",
  "owner": "platform",
  "latency_ms": 450,
  "retryable": true
}

This is likely to be the strongest scenario for TOON because the schema can be declared once and reused for all rows.

Nested Data

The nested dataset includes workload, metric, and status objects:

{
  "id": "INC-000001",
  "workload": {
    "namespace": "team-1",
    "deployment": "service-01",
    "pod": "service-01-000001"
  },
  "metrics": {
    "cpu_percent": 72,
    "memory_mib": 850,
    "latency_ms": 450
  },
  "status": {
    "severity": "critical",
    "acknowledged": false
  }
}

This tests whether TOON’s reduced punctuation compensates for indentation and nested structural markers.

Irregular Data

The irregular dataset intentionally varies fields across records:

[
  {
    "id": "INC-000001",
    "service": "checkout",
    "severity": "critical"
  },
  {
    "id": "INC-000002",
    "dependencies": ["postgresql", "kafka"],
    "retry_after_seconds": 30
  },
  {
    "id": "INC-000003",
    "error": {
      "code": 503,
      "message": "upstream unavailable"
    }
  }
]

This is important because tabular formats perform best when records share a schema. Sparse or heterogeneous structures can reduce or eliminate that advantage.

Linked-Data Graph

The final dataset contains incidents, services, teams, and relationships expressed through JSON-LD. This evaluates TOON-LD against the representation it is intended to optimize.

Metrics Used in the Experiment

The benchmark records the following metrics.

Serialized Characters

This is the number of Unicode characters in the encoded document. Character count is easy to understand, but it is not a substitute for token count. Different tokenizers divide the same text differently.

UTF-8 Bytes

The benchmark measures the encoded byte length using: len(serialized_value.encode("utf-8")). This helps estimate storage and network-transfer overhead.

Token Count

Token count is measured using the selected tokenizer. The repository defaults to the o200k_base tokenizer but allows another tokenizer to be configured.  For linked data, JSON-LD replaces JSON in the calculation. Token savings are tokenizer-specific. A result measured with one tokenizer should not be presented as universally applicable to every model family.

Encoding Latency

Encoding latency measures the time required to convert an in-memory object to JSON, TOON, JSON-LD, or TOON-LD. The benchmark reports:

• median encoding latency;
• 95th-percentile encoding latency.

Decoding Latency

Decoding latency measures the time required to reconstruct the application data from its serialized representation. This matters because reducing prompt tokens may introduce additional CPU overhead in the application.

Peak Memory

Python’s tracemalloc module records the peak memory observed during serialization.

Round-Trip Correctness

For every measured iteration, the benchmark verifies: source data == decode(encode(source data))

A format that produces a smaller prompt but cannot reliably reconstruct the source data is unsuitable for lossless interchange.

Running the Benchmark

Clone the repository:

git clone https://github.com/jojustin/json-toon-toonld-benchmark.git
cd json-toon-toonld-benchmark

Create a virtual environment:

python -m venv .venv
source .venv/bin/activate

Install the dependencies:

pip install -r requirements.txt

Run a small validation experiment first:

python -m src.run_benchmark --sizes 10 100   --iterations 5

Run the complete benchmark:

python -m src.run_benchmark --sizes 10 100 1000 10000   --iterations 30

To calculate percentage reductions and encoding overhead:

python -m src.summarize

Run the automated tests:

pytest -q

Why the Benchmark Uses Minified JSON

A TOON comparison can be exaggerated by comparing it only with pretty-printed JSON.

Pretty-printed JSON contains indentation and line breaks intended for human readability:

{
  "id": 1,
  "name": "Alice"
}

Minified JSON removes optional whitespace:

{"id":1,"name":"Alice"}

Since production systems can easily minify JSON before placing it in a prompt, minified JSON is the appropriate primary baseline. Pretty-printed JSON can still be reported as a separate readability baseline, but it should not be the only comparison.

Interpreting the Expected Results

The benchmark results show that token-oriented serialization is not uniformly more efficient than JSON. Its effectiveness depends strongly on the structure of the input data. TOON performs best when the input consists of flat, homogeneous records that share the same fields, while compact JSON remains more efficient for irregular and deeply nested structures. TOON and TOON-LD also introduce measurable conversion overhead because their encoders must analyze the input structure and generate a more specialized representation.

Token Efficiency

For the flat dataset, TOON reduced the token count from approximately 39,500 tokens to 23,000 tokens, corresponding to a reduction of about 42%.  This result represents TOON’s intended use case: a large collection of records sharing a common schema. Rather than repeating every field name for each record, TOON declares the fields once and represents the values in a tabular form.

The result was different for irregular data. Compact JSON required approximately 27,300 tokens, while TOON required about 33,000 tokens — an increase of approximately 21%. Because the records contained different fields and structures, TOON could not efficiently amortize a shared schema across the collection. The additional structural notation therefore outweighed the savings obtained by removing JSON punctuation.

A similar pattern appeared in the nested dataset. TOON used approximately 74,000 tokens compared with 64,000 tokens for compact JSON, representing an increase of around 16%. The result indicates that deeply nested objects are not necessarily well suited to tabular token-oriented encoding. Indentation, nested object markers, and repeated hierarchical structures can make TOON less compact than minified JSON.

For the linked-data dataset, TOON-LD reduced the representation from approximately 40,000 JSON-LD tokens to 28,500 tokens, a saving of about 29%. This demonstrates the potential of schema-aware linked-data compression. However, the token reduction must be interpreted together with the round-trip validation results. In the tested implementation, the reconstructed TOON-LD output did not preserve valid JSON-LD semantics. The observed token saving therefore represents compression potential, but not a verified lossless transformation for this workload.

Encoding Performance

JSON consistently encoded faster than TOON. For the flat dataset, compact JSON required approximately 6 milliseconds, whereas TOON required around 27 milliseconds. TOON was therefore about four times slower, despite producing a substantially smaller token representation.

The irregular dataset showed a similar pattern. JSON encoding took approximately 5 milliseconds, while TOON required nearly 30 milliseconds. In this case, TOON introduced significant processing overhead while also producing more tokens, making compact JSON preferable on both efficiency and runtime grounds.

For the nested dataset, JSON required approximately 12 milliseconds and TOON approximately 55 milliseconds. This was the highest TOON encoding time observed among the datasets. The additional processing required to traverse and represent deeply nested structures contributed to both higher runtime and higher token count.

JSON-LD encoding required approximately 6 milliseconds for the linked-data dataset, compared with about 17 milliseconds for TOON-LD. TOON-LD was therefore around three times slower to encode, although its absolute processing time remained below 20 milliseconds for 1,000 records.

These results show that reduced token count is not computationally free. TOON and TOON-LD shift some work from the LLM prompt to the application’s serialization layer.

End-to-End Conversion Overhead

For flat data, TOON introduced approximately 29.14 milliseconds of additional conversion time compared with JSON. For irregular data, the overhead increased to 31.94 milliseconds. The linked-data comparison produced the lowest overhead: TOON-LD added approximately 11.59 milliseconds relative to JSON-LD.

The nested dataset generated the largest conversion overhead at 58.93 milliseconds. This finding is consistent with the encoding-time and token-count results: nested structures were both slower to process and less token-efficient in TOON.

Although these overheads are small compared with the end-to-end latency of many remote LLM requests, they may still matter in high-throughput systems, local inference pipelines, or workflows that repeatedly serialize and deserialize large payloads. Conversion cost should therefore be evaluated relative to the expected inference savings and request volume.

Overall Interpretation

The combined results reveal three distinct workload categories.

The strongest result is that data shape is the primary determinant of TOON efficiency. TOON is effective for uniform, tabular collections because it avoids repeating field names. It is less suitable for sparse, irregular, or deeply nested data, where compact JSON can require fewer tokens and substantially less conversion time.

The linked-data result should be treated cautiously. Although TOON-LD reduced token usage and introduced relatively modest conversion overhead, the tested implementation failed semantic round-trip validation. It should therefore not be presented as a lossless JSON-LD replacement for this experiment.

A practical selection policy derived from the results is:

Flat and homogeneous records  → TOON

Irregular or nested records   → Compact JSON

Linked-data graphs  → JSON-LD unless TOON-LD semantic validation passes

Overall, the benchmark supports using TOON as a selective prompt-boundary optimization, rather than as a universal replacement for JSON. The appropriate decision should consider token reduction, conversion overhead, structural correctness, and semantic preservation together.

Extending the Benchmark With LLM accuracy

The repository focuses on deterministic, provider-neutral measurements. A second experiment can assess how well an LLM understands each representation. Use semantically identical questions for JSON and TOON:

List the IDs of all critical incidents owned by the platform team. Return only a JSON array of incident IDs.

For JSON-LD and TOON-LD, include multi-hop questions:

Which teams own services affected by critical incidents?

Measure:

• Input tokens
• Output tokens
• Time to first token
• Total response latency
• Exact-match accuracy
• Precision, recall, and F1
• Invalid-output rate
• Hallucination rate
• Cost per request

Keep these variables constant:

• Model and model version
• System prompt
• Question
• Temperature
• Maximum output tokens
• Dataset
• Number of repeated trials

Randomize the order of JSON and TOON trials so that temporary service conditions do not consistently favor one format.

When Should TOON Be Considered?

TOON is worth evaluating when:

• Large homogeneous datasets are repeatedly placed in prompts
• Prompt-token cost is significant
• Context-window capacity is constrained
• The application controls both encoding and decoding
• Structured context is primarily read by the model
• Benchmarked accuracy remains acceptable

TOON may be less attractive when:

• Payloads are small
• Objects are deeply nested or highly irregular
• Standard interoperability is more important than token savings
• The model must reliably generate complex TOON output
• Downstream tools require JSON directly
• Conversion complexity exceeds measurable savings

When Should TOON-LD Be Considered?

TOON-LD may be useful when:

• A Graph-RAG pipeline inserts many JSON-LD entities into prompts
• Repeated graph entities share common shapes
• A semantic agent receives linked relationships as context
• Preserving @context, identifiers, and graph relationships is essential
• JSON-LD token consumption limits useful graph size

It should be approached cautiously when:

• External systems expect standards-compliant JSON-LD directly
• RDF canonicalization and semantic round trips have not been tested
• Package maturity and long-term compatibility are critical
• The linked-data graph contains complex or highly heterogeneous structures

Security Considerations

Structured-data compression does not eliminate prompt-security concerns. Before inserting TOON or TOON-LD content into a prompt:

• Treat serialized values as untrusted data
• Separate instructions from retrieved content
• Validate decoded responses
• Enforce output schemas where possible
• Limit graph traversal and retrieved entity counts
• Prevent untrusted content from altering system instructions
• Log the canonical JSON or JSON-LD source for auditability

For TOON-LD, external contexts and linked identifiers should also be controlled. Applications should avoid dereferencing arbitrary remote contexts or URLs without appropriate allowlists, timeouts and content validation.

Conclusion

JSON remains the correct default for general-purpose application integration. It has unmatched interoperability, mature tooling, schema support and broad developer familiarity.

TOON addresses a narrower problem: reducing the token overhead of structured data passed to language models. Its strongest potential advantage is in large, homogeneous collections where repeated JSON keys consume substantial context.

TOON-LD applies the same general principle to JSON-LD knowledge graphs. It may allow Graph-RAG and semantic-agent systems to place more linked data in an LLM context, but it is newer and requires careful testing for semantic equivalence and implementation maturity.

The key decision should not be based on token reduction alone.

A production evaluation should measure:

• Token count
• Serialized bytes
• Encoding and decoding overhead
• Memory usage
• Round-trip correctness
• LLM comprehension
• Structured-output reliability
• End-to-end latency
• Cost at realistic request volumes

A practical adoption pattern is to retain JSON or JSON-LD as the canonical application representation and introduce TOON or TOON-LD only as an explicitly measured prompt-boundary optimization. The accompanying benchmark provides a reproducible starting point for making that decision with evidence rather than assumptions.