Token-Efficient APIs for the Agentic Era

As autonomous agents become primary API consumers, a subtle cost problem emerges. Traditional JSON serialization, optimized for human readability and broad compatibility, incurs significant token overhead when feeding data to language models. Every structural character (braces, quotes, colons, commas) gets tokenized and charged separately.

The issue compounds at scale. When agents query APIs hundreds of thousands of times daily, JSON's verbosity translates directly to infrastructure costs. Organizations running agent-heavy workloads are discovering that a substantial portion of their LLM token consumption is due to serialization overhead, not actual data transfer.

Understanding the Token Costs

JSON's verbosity creates a multiplicative cost problem when feeding data to language models. The root cause lies in how modern tokenizers handle structural characters.

Let's consider a simple user record:

{"user_id": 12345, "status": "active", "role": “admin"}

This tokenizes to roughly 13 tokens. But only three of those tokens represent actual data (12345, active, admin). The other 10 are structural characters: braces, quotes, colons, and commas. That represents significant overhead on token costs.

The problem compounds with arrays. A list of 1,000 users repeats every key 1,000 times. JSON payloads consistently show a substantial token multiplier compared to the semantic information being transmitted.

At a million-request scale, this overhead becomes the dominant cost factor. The token tax has three primary components:

• Delimiter overhead: Every {, }, ", :, and , consumes a token. In deeply nested structures, delimiters can outnumber data tokens substantially.
• Key repetition: Arrays of objects repeat identical keys across every record. A 10-character key like "created_at" appears N times for N records, each occurrence consuming multiple tokens.
• Whitespace handling: Even minified JSON has implicit spacing between elements that affects tokenization. Pretty-printed JSON with indentation multiplies the problem further.
  For organizations processing millions of agent requests daily, these inefficiencies translate to measurable infrastructure costs.

TOON: Columnar Serialization for Tabular Data

Token-Oriented Object Notation (TOON) eliminates key repetition by treating data as columnar, similar to CSV, but with better structure preservation.

JSON representation:

[
{"id": 1001, "name": "John Doe", "role": "architect", "active": true},
{"id": 1002, "name": "Chris Smith", "role": "engineer", "active": false}
]

TOON equivalent:

HEADERS: id, name, role, active
1001 | John Doe | architect | true
1002 | Chris Smith | engineer | false

TOON delivers a significant token reduction compared to JSON. Models trained on CSV and tabular data understand this format without accuracy loss. The pipe delimiter is chosen deliberately; it tokenizes as a single character and rarely appears in data values, unlike commas, which require escaping.

TOON works particularly well for analytics pipelines where agents process time-series metrics. The format also compresses effectively when combined with standard HTTP compression, as repeated patterns in column values are more easily compressed than scattered JSON keys.

TOON works well with:

• Homogeneous record sets (user lists, transaction logs, event streams)
• Data with consistent schemas across records

TOON breaks down with:

• Deeply nested objects (you can't represent hierarchy cleanly)
• Sparse data with many optional fields (empty cells waste space)

Implementation of TOON parsers is straightforward. The header row establishes the schema, and subsequent rows map positionally to those headers. Most LLMs handle this pattern naturally due to their exposure to CSV data during training.

TRON: Eliminating Keys Through Schema Contracts

TRON takes a more aggressive approach: remove keys entirely and rely on positional arguments, similar to constructor calls.

JSON representation:

{
"user": {
"id": 1001,
"profile": {"name": "Alice Chen", "email": "[email protected]"},
"roles": ["admin", "architect"]
}
}

TRON equivalent:

User(1001, Profile("Alice Chen", "[email protected]"), ["admin", “architect"])

TRON achieves a deeper token reduction than TOON. The structure is implied by the schema definition, not embedded in every payload. TRON resembles function calls or class instantiation syntax, which LLMs handle effectively due to extensive code training.

The critical trade-off: agent reasoning accuracy drops when semantic labels are removed. The model must infer that "[email protected]" is an email from the pattern itself, not from an explicit "email" key. For extraction tasks where precise field identification matters, this degrades performance measurably.

The accuracy degradation stems from the loss of semantic context. In JSON, the key "email" provides explicit type information. In TRON, the model relies on position (the second parameter of Profile()) and pattern matching (the presence of the @ symbol). This works for obvious patterns but fails on ambiguous data.

TRON excels in agent-to-agent communication where both sides operate under shared schema contracts. Multi-agent orchestration systems benefit most when a coordinator agent spawns worker agents and passes state; the schema is controlled on both ends. The token savings compound across thousands of inter-agent messages.

TRON works well when:

• Both the producer and the consumer are agents you control
• Schemas are stable and versioned
• Data is hierarchical with deep nesting
• Inter-agent communication volume is high
• The accuracy trade-off is acceptable for the specific workload

TRON is not suitable for:

• Public APIs or third-party integrations
• Human-readable logs or debugging output
• Dynamic schemas that evolve frequently

Implementation: Convert at the Boundary

Rewriting services to emit TOON/TRON natively creates technical debt and breaks existing clients. The migration path becomes complex, requiring coordinated updates across service boundaries.

The better pattern: JSON-in, TOON-between. Origin services continue speaking JSON. Conversion happens in the request path before data reaches agents.

Implementation typically occurs at the API gateway layer. When an agent sends an Accept: application/toon or Accept: application/tron header, the gateway converts the JSON response dynamically. Schema definitions are stored centrally and versioned through API metadata.

This approach delivers three benefits:

• Backward compatibility – Existing JSON clients work unchanged. No breaking changes required across the ecosystem.
• Gradual rollout – Enable TOON/TRON per endpoint as needed. High-value, high-volume endpoints convert first. Low-traffic or legacy endpoints remain JSON indefinitely.
• Schema evolution – Update schemas without service redeployment. Schema changes deploy independently of service code, enabling faster iteration.

The middleware intercepts responses before serialization, checks for format negotiation headers, and applies conversion if requested. Schema definitions are loaded from a registry at startup, enabling hot-reloading when schemas update.

Conversion overhead is minimal compared to the token savings on the LLM side. The processing cost of transformation is negligible relative to the reduction in inference costs. Gateway-level conversion adds minimal latency, while token reduction provides ongoing savings on every downstream LLM call.

Conclusion

TOON has proven effective for replacing JSON in internal agent-to-service calls, particularly for tabular data. The implementation is straightforward, the accuracy impact is minimal, and the cost savings are immediate.

TRON remains appropriate for specialized use cases. High-volume agent orchestration scenarios where schema stability is guaranteed and both endpoints are under direct control represent the primary application. It addresses a narrow but valuable use case.

The recommended starting point: implement TOON for tabular data. The risk is low, the savings are tangible, and the operational overhead is manageable once conversion is centralized at the gateway layer.
TOON adoption makes sense for systems with significant LLM infrastructure costs, particularly those handling large volumes of structured data through agent interfaces. The implementation overhead is modest relative to the ongoing savings in token consumption.

For modern agent architectures, optimizing for token efficiency represents a fundamental design consideration.