Cutting AI Token Costs With MgntUtils Stack Trace Filtering

A live production integration case study.

Introduction and Purpose of This Article

This article is written for mid- and high-level managerial and technical decision-makers. I am the author of the open-source Java library MgntUtils. The article presents an analysis of a real integration of the stack trace-filtering feature from that library into a live commercial production environment.

A few important clarifications up front:

• This is not a side-project pilot and not a lab demo. The feature was integrated into a production service of a company that serves a high volume of real customers. Due to legal constraints, I am not at liberty to name the company.
• This is not a how-to article for implementers. If you came looking for code samples or logging-framework wiring, please see the dedicated articles listed in the Disclaimer below.
• MgntUtils can be used in Java projects and in other JVM-based languages such as Kotlin.

Before diving into the production numbers, it is worth stating briefly what the feature does and why those numbers matter.

Server-side stack traces are usually full of framework and infrastructure noise — proxies, filter chains, containers, thread pools, and similar boilerplate — while the few lines that actually explain the failure are easy to lose in the pile. The MgntUtils filtering utility keeps the application frames and the exception / Caused by chain, and collapses that noise. The result is a much shorter stack trace without losing the information you actually need.

When those stack traces are later consumed — sent to an LLM for analysis, or opened by an engineer — that reduction can mean:

• Substantial AI token savings
• Typically more accurate AI root-cause answers, because the model has less framework noise to latch onto and hallucinate about
• A meaningful productivity boost for human triage

The rest of this article focuses on what was observed after integrating this feature in production: the measured benefits, how to interpret them, and the integration experience itself — including gotchas that only surfaced in a real live environment, as opposed to a pilot project.

Disclaimer

This article deliberately does not discuss the technical design of stack trace filtering or the technical details of the integration. Each of those topics has its own dedicated article:

Filtering Java Stack Traces With MgntUtils Library

• DZone: https://dzone.com/articles/filter-java-stacktrace-mgntutils
• DEV Community: https://dev.to/mgantman/java-stacktrace-filtering-utility-1c1i

Zero-Code-Change Stack Trace Filtering for Spring Boot: An Infrastructure-Level Integration

• DEV Community: https://dev.to/mgantman/zero-code-change-stacktrace-filtering-for-spring-boot-an-infrastructure-level-integration-3fk5

Production Results and Benefits

Below are the observations and conclusions from monitoring the live production system after the feature integration. The feature had been running for about a month, and filtering was also temporarily turned off for comparison.

What the Production Environment Looked Like

Anonymized sketch of the deployment (enough to judge fit, without identifying the company):

• High-traffic JVM/Spring Boot service in a commercial production estate
• Structured JSON logging to a major observability platform
• Observability billing dominated by per-event (not per-byte) pricing
• In a typical production day, that service emitted on the order of ~70,000+ log events carrying a stack trace

That is a large stream of stack trace payloads — expensive if fed to an LLM, and tiring if engineers open them by hand.

Stack Trace Volume Reduction Range in Production

Filtering was measured across production stack traces with filtering on vs off. Observed size/token reductions typically fell in roughly the ~75%–95% range:

• Toward the high end (~90–95%): framework-heavy request-handling traces (long security/container/proxy tails)
• Toward the lower end (~75%+): more application-dense traces, where a larger share of frames is your own code

The average reduction on a typical trace in this environment was about ~91%. The table below is a real before/after example — shown so you can see what that looks like in practice:

Every application frame in the business call path was retained; what disappeared was framework and infrastructure noise (proxies, filter chains, container/thread-pool frames, and similar boilerplate).

Stack traces tokenize poorly for LLMs — package separators, generated class names, and (File:line) markers all split into extra tokens — so the token reduction tracks the size reduction closely.

Root-cause readability was unchanged. In both versions, the failure was identifiable from the application frames and the exception message. Filtering did not remove diagnostic signal; it removed the large majority of the payload that never helped.

What Improved

AI analysis: cheaper and more accurate (when exceptions are analyzed).

For every exception sent to an LLM, the stack trace input payload shrank by roughly ~75–95% depending on the trace shape (~5,800 tokens saved on a typical ~91% trace). That saving repeats for every analyzed event. In an environment where tens of thousands of stack traces are emitted per day, any AI triage, clustering, or “explain this error” pipeline pays that tax over and over unless the noise is stripped first.

Cost is only half of the AI benefit. Filtering also improves answer quality. The removed frames are framework and infrastructure boilerplate — identical across many errors and unrelated to the application failure. When those frames remain in the prompt, models often latch onto them and hallucinate a root cause in the noise. With them collapsed, the model is steered toward the application frames and exception message that actually explain the failure — so analysis is not only cheaper, but typically more accurate.

Sensitivity calculator (illustrative — not this company’s AI spend).

If your org analyzes exceptions with an LLM, you can size token cost roughly as:

annual token saving ≈ (exceptions analyzed per year)
                    × (tokens saved per exception)
                    × (model input price per token)

Using ~5,800 tokens saved per exception (average on a typical ~91% trace) and an illustrative model input price of $3 per 1 million input tokens:

Plug in your own analysis volume, your place in the ~75–95% reduction range, and your model pricing. The production measurement that is firm is the observed per-exception reduction range, with application frames preserved.

Secondary AI upside: More errors per context window.

Because a typical filtered stack trace is so much smaller (~540 tokens vs ~6,300 in the example above), many more distinct exceptions fit into a single model call. That is a capability change, not just a cost saving: cross-error analysis — clustering failures, or asking “what went wrong in the last N hours?” — becomes practical instead of blowing the context window on framework noise. It is secondary to the per-exception token and accuracy benefits, but it matters for any AI workflow that looks at more than one error at a time.

Human triage productivity.

Engineers reading a filtered typical trace see the full application call path at the top (~18 lines in the example above) instead of scrolling through ~195 lines to confirm there is no hidden nested cause and to piece the business path together. For on-call and incident review, that is a direct readability win.

What Changed in Log Volume — and What Did Not

It helps to separate event count from bytes per event.

Event count did not change.

A stack trace is still one log event whether it is 195 lines or 18. If your observability vendor bills per event (or per indexed log line item), filtering does not reduce that charge. In this production environment, that was the dominant billing model — so there were no savings on a per-event bill.

Bytes per stack trace event did change.

Each filtered stack trace was roughly ~75–95% smaller than its unfiltered counterpart (commonly ~90% for framework-heavy traces). There is a real reduction in stack trace payload size.

How much that shows up in total log volume is not deterministic.

Overall space / ingested-byte savings depend on what share of all logs are stack traces:

overall byte reduction ≈ (stacktrace share of total log volume)
                         × (~75–95% reduction on those stacktraces)

In this company’s environment, stack traces were only about ~1% of total log volume — which is unusually low (an anomaly for many systems, but what we observed here). Cutting ~90% of that 1% yields only a fraction of a percent of total logs, which is easy to lose inside normal day-to-day traffic variance. That is why aggregate ingested-byte charts did not show a clear step when filtering was toggled.

In another organization where stack traces are a much larger share of log volume, the same per-trace cut would produce a more visible space saving. Those savings are real in principle, but variable by workload and not the main point of this case study.

The main point here is consumption cost.

The firm, repeatable benefit we are highlighting is what happens when a stack trace is analyzed by an LLM or read by an engineer: large payload reduction, same diagnostic signal. Treat log-space savings as a possible secondary effect, sized by your own stack trace-to-total-logs ratio — not as the success criterion for this feature.

How to Read These Results as a Decision Maker

The production evidence supports a clear, bounded claim: when stack traces are consumed, filtering delivers a large, repeatable reduction in payload size with no loss of application signal. Per-event log bills do not drop. Overall log-space savings may exist but depend on stack traces’ share of total volume — and are not the primary reason to adopt the feature.

Integration Experience

I started from an implementation I already had in the MgntUtilsUsage side-project repository — a runnable Spring Boot demo of MgntUtils features, meant to emulate real-life apps as closely as possible. It was a very good starting point. Still, as I worked through the live commercial integration, a few gotchas surfaced that a single-JVM demo simply does not force you to confront.

Gotchas That Showed Up in a Real Production Environment

1. Feature Toggle Storage Across Multiple Containers

My demo app runs in a single JVM. A real production service typically runs on several containers that scale in and out. In the demo, the on/off flag for stack trace filtering lived in memory — which is fine for one process, and useless once you have more than one. In a multi-container environment, you need an external, shared flag holder that every instance can read. Redis (or an equivalent shared store available to all containers) is a good candidate.

2. JSON Logging Adapters, Not Only the Classic Logback Pattern

When I first modified the Logback configuration, my demo mainly used conventional Logback pattern-based adapters. A real production app will most likely also use a JSON encoder for external logging systems such as Datadog (and similar platforms). That special adapter has its own throwable-handling path, so wiring the filter there is a must — otherwise you can end up with filtered console output locally and unfiltered stack traces in the system that actually matters.

3. Hardening the Fail-Safe Path

A fall-back option already existed for the case where anything goes wrong inside the filtering path. For production, that fail-safe had to be hardened a bit further to make it as bullet-proof as possible: if filtering ever fails, the system must still emit a full standard stack trace and must never drop the log event.

4. Logback Is Not the Only Popular Logging Framework

This company uses Logback, so that is what the production integration targeted. But Logback is not the only widely used option — my own favorite, for example, is Log4J. For the dedicated integration article (linked in the Disclaimer), I also had to provide Log4J instructions, even though Log4J was not used in this particular environment. Anyone planning an org-wide rollout should assume more than one logging stack may need to be covered.

Effort, Timeline, and Outcome

All in all, the integration was smooth, and the side-project was close enough to the final result in the real app.

• About 4–5 hours to get an integrated version up and running in the staging environment
• About one day of observing staging to make sure there were no unexpected behaviors
• Then deployment to production, with about another day of close monitoring before declaring the feature live

So roughly half a day of integration work, and about 1.5 working days of testing / staging observation / production monitoring.

Not a single bug was found. There are two contributing factors for that:

1. The stack trace-filtering feature itself is mature and battle-tested — I am tempted to say it has no bugs, but let’s just say it is highly stable and reliable.
2. The integration itself is simple enough.

The next integration should be even faster, since this one is now well documented (including the dedicated Spring Boot integration article linked in the Disclaimer).

If you are interested in integrating this feature into your project, the detailed integration instructions are in the article Zero-Code-Change Stack Trace Filtering for Spring Boot: An Infrastructure-Level Integration. If you are interested in support for the integration, feel free to contact me at or through my LinkedIn profile.

Conclusion

This case study supports a simple decision:

Adopt stack trace filtering if your organization already analyzes production exceptions with LLMs at a meaningful volume, or if engineers routinely open noisy stack traces during triage and on-call. In those cases, the live evidence is clear: typically about ~75–95% less stack trace payload (around ~91% on a typical trace), with application frames preserved — cheaper AI analysis, typically more accurate answers, and easier human reading.

Do not adopt it expecting your per-event observability bill to drop, or expecting a large automatic cut in total log volume. Event count does not change. Overall byte savings depend on how large a share stack traces are of all logs — and that varies by organization. Consumption cost is the main point; log-space savings are secondary and workload-dependent.

On effort and risk: in this live commercial integration, getting to staging took about half a day of work, followed by roughly a day and a half of staging observation and production monitoring. No bugs were found. The feature is mature, the integration is simple, and the demo-to-production gaps (shared toggle, JSON logging adapters, fail-safe hardening, and covering more than one logging framework) are now documented.

If that profile matches your environment — high exception volume that is actually consumed by AI or by people — this is one of the cheaper, lower-risk improvements available. If exceptions are mostly logged and rarely looked at, the benefit will be thin, and that is an honest reason to pass.