Best of Three Worlds: AI Agents Writing In Rust and Java Beat the C Library
How two pure-language image decoders: one in Rust and one in Java, ended up beating libheif, and what I learned about getting AI agents to improve each other's work.
Every app that accepts photo uploads meets HEIC sooner or later. iPhones save in it by default. Neither Java nor Rust can read it out of the box, and the usual answer, libheif, is a C library: native packages in every container image, a C toolchain in the build, and a parser written in C reading files that strangers upload.
I had the problem twice, in a Java product and in a Rust one. So I went the other way: decoders written in the language itself, built and improved by AI coding agents, with me steering. Here is how it went.
1. Make It Work, In Each Project
One agent per codebase, each starting from an existing open-source decoder in its own language: Openize.HEIC for Java, HEIC decoder for Rust. The first versions lived inside each app. They read the photos, and that was enough to ship. They were ready in no time, given there were upstream libraries to start from, even if some features had to be added to match libheif.
It was tempting to stop there. Nobody complains about an upload that works.
2. Extract and Iterate
Next, I had each decoder pulled out as a standalone library, open source, with its own tests and releases, aligned with the best practices for each language (and mine). Then I asked each agent to iterate on its own: measure, improve, measure again.
That alone went a long way. The Java decoder went from 19 seconds to 3.2 for our ten test photos.
3. Compare Everything With Everything
Working alone, each agent eventually ran out of obvious ideas. What changed the game was putting both decoders next to libheif on the same machine with the same photos, and then comparing them stage by stage.
Every gap became a lead. If another implementation did a step fast, there was a way to do it even faster, and the size of the gap told me whether it was worth chasing. A gap is a hint with a number attached.
The comparison also kept us honest. It showed where we were ahead, and it showed that part of libheif's lead came from doing less work: it copied color samples where our decoder interpolated them.
4. Make the Agents Talk
Here is the part that led to further improvements. Each agent, working in its own session, was told to talk with the other agent. I had the Rust agent brief the Java agent on what it had found, and the other way around. A gap one side had already closed became a recipe for the other. The Java decoder took the Rust decoder's entropy decoding approach; the Rust decoder took the Java decoder's parallel tiles and transform tricks. Later, the two agents, who were then aware of each other, even coordinated on their own, communicating which CPU cores each could benchmark on.
Connecting them was the most useful step in the project.
5. Repeat On the Gaps That Remain
Then we ran the same loop again: compare, find the largest remaining gap, close it, then measure. One more round on the Rust side closed the last single-core gap with libheif. The Java side tried the JVM's vector instructions and gained another 9 to 14%.
Not every idea survived. The trick the C library is famous for, wide vector instructions, gained almost nothing in Rust: its slow parts were elsewhere. We measured it, wrote it down, and dropped it.
Be Strict About Evidence
A computer is never idle. Browsers, chat apps, other agents: everything competes for the CPU, and the numbers move with it. When we measured the very same code twice, back to back, the second run came out about 0.4% slower. Noise with a direction looks exactly like progress, or like a regression.
So I asked for proof, not impressions: count CPU work instead of wall-clock time, run old and new in alternating order, split results by type of photo, and drop any "gain" smaller than the noise. Several changes didn't make it. One that looked good overall turned out to slow down rotated photos, which are most of what phones upload, until the agent found why.
Where They Stand Today
Measured together on the same machine, the same ten photos, all three doing the same work:
- 8 cores: Rust 0.46 s, Java 0.55 s, libheif 0.81 s
- 1 core: Rust 1.38 s, libheif 1.45 s, Java 1.96 s
- Rust can also interpolate color for sharper edges and still matches libheif: 1.45 s
Memory is on par with libheif or below, and a long way from where the two decoders started.
What I Take Away
- Iterating alone gets you far. Comparing gets you further. The gap between two implementations is the best to-do list.
- Cross-pollinating agents helps a lot. The ideas were there, in two separate sessions. Making them talk to each other triggered more optimizations.
- Be strict about evidence. Ask for measurements that hold up against noise, and be ready to throw away a change that doesn't.
- Two implementations in two languages beat one. Each made the other faster. Getting a third as reference (libheif) helped even more.
- Java is fast, too. “Java is slow” is a meme, not a measurement. Optimized the right way, it comes close to the C library on one core and beats it on eight.
Both decoders are open source:
Published at DZone with permission of Yannick Majoros. See the original article here.
Opinions expressed by DZone contributors are their own.