Reading GC Logs: Memory Leak or Undersized Heap?
A large live set after repeated full GCs doesn't prove a leak. Rising post-GC floor = accumulation; stable high floor = undersized heap.
It's 3 AM. Your phone buzzes. The pipeline that's been running fine for six hours just died with java.lang.OutOfMemoryError: Java heap space, and the on-call channel wants to know: is this a leak, or did we just under-size the executors? Someone suggests bumping memory from 8g to 12g and going back to sleep. Someone else suspects the new dedup cache from last week.
The GC log won't settle that argument by itself — but it tells you which diagnostic to run next, and that's worth more than a guess at 3 AM.
This is the first article in a series on JVM memory failures in data pipelines. A companion jvm-oom-field-guide repository contains small runnable programs for each failure mode, along with diagnosis notes and fixed versions. This article focuses on the first skill: reading the post-GC baseline and choosing the right next diagnostic.
Turn It on Today, Not at 3 AM
On JDK 11+, enable detailed GC logging and automatic heap dumps before an incident occurs:
-Xlog:gc*:file=/path/to/gc.log:time,uptime,level,tags:filecount=5,filesize=20M \
-XX:+HeapDumpOnOutOfMemoryError \
-XX:HeapDumpPath=/path/to/heap-dumps
That gives you timestamped GC events, plus an automatic heap dump on the next OutOfMemoryError (OOM). Set HeapDumpPath explicitly — the dump defaults to the working directory. Make sure both paths are writable, have enough disk space, and retain enough history to survive until someone looks. Add these now, while everything is healthy, so the evidence exists when things aren't.
Start With the Before-and-After Numbers
Here's a G1 log line, decoded:
[0.412s][info][gc] GC(4) Pause Full (Allocation Failure) 61M->58M(64M) 38.204ms
The field that matters is 61M->58M(64M): heap usage went from 61 MB before GC to 58 MB after GC, with a current reported heap capacity of 64 MB. The collector performed a full-heap collection and reclaimed only 3 MB from a nearly full heap. (The Allocation Failure reason string varies across JDK versions — on newer ones you may see G1 Compaction Pause. The before-and-after numbers are what matter.)
Now watch three of them in a row:
[0.412s][info][gc] GC(4) Pause Full (Allocation Failure) 61M->58M(64M) 38.204ms
[0.489s][info][gc] GC(5) Pause Full (Allocation Failure) 61M->60M(64M) 44.917ms
[0.571s][info][gc] GC(6) Pause Full (Allocation Failure) 62M->61M(64M) 51.330ms
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
(These lines come from the 64 MB demo in the last section — on a tiny heap, the whole pattern plays out in under a second.)
Repeated full GCs reclaim very little, leaving the heap close to capacity. That is strong evidence the JVM is retaining a large amount of live data — but retained memory is not automatically a memory leak. It could be an unbounded cache, a batch accumulating too much data, or a legitimate working set that simply doesn't fit the configured heap. The log shows retention; it doesn't tell you which one.
Compare the Floor, Not Just the Peaks
Here's a different JVM that also keeps hitting its heap limit:
GC(14) Pause Full (...) 61M->42M(64M)
GC(19) Pause Full (...) 62M->43M(64M)
GC(25) Pause Full (...) 61M->42M(64M)
This JVM repeatedly approaches its heap limit, but the post-GC occupancy returns to roughly the same level: 42 MB, 43 MB, 42 MB. That stable floor argues against unbounded accumulation. The application may have a legitimate 42 MB live set but insufficient headroom for temporary allocations or workload peaks.
Compare that with the earlier sequence — 58 MB, 60 MB, 61 MB after GC. Its rising floor suggests that more and more data remains reachable. That is evidence of accumulation, although the heap dump must still determine whether the retained data is unintended.
A compact mental model:
- Rising post-GC floor: suspect accumulation.
- Stable but high post-GC floor: suspect a large working set or insufficient headroom.
- Large peaks followed by a stable low floor: suspect allocation churn rather than retention.
The Diagnostic Method
Here is how to narrow it down, in order:
- Track the post-GC baseline. Is occupancy-after-full-GC rising across comparable workload intervals — say, as more records flow through? One caveat: the log lines above are heap-usage summaries, not old-generation measurements. To track the old generation specifically, use detailed GC logs or JVM metrics, and compare post-GC levels across similar workload windows rather than reading one line in isolation.
- Check what the collector can reclaim. Repeated full GCs that free almost nothing mean most of the occupied heap stays live across collections. That confirms retention — the open question is whether it's intended.
- Inspect the heap dump. Open it in Eclipse MAT (Memory Analyzer), sort the dominator tree by retained heap, and trace the top suspects' paths to GC roots. A collection growing with input volume is your prime suspect; a working set matching expected data sizes points to sizing.
The rule that matters: a rising post-GC baseline is evidence of accumulation, not proof of a leak. A stable baseline near capacity may mean a large working set or an undersized heap. Correlate the trend with workload volume and heap-dump evidence before deciding whether to fix retention or add memory. Increasing the heap may be appropriate when the expected live set genuinely needs more room. Reducing batch size or processing data incrementally may be better when the peak working set can be redesigned. For unbounded retention, additional heap only postpones the next failure.
One Distinction Worth Knowing
OutOfMemoryError: Java heap space– an allocation failed because the JVM could not obtain sufficient heap space. This can result from a leak, an oversized working set, or a single oversized allocation.OutOfMemoryError: GC overhead limit exceeded– HotSpot's overhead safeguard, triggered when roughly 98% of time goes to GC while recovering less than 2% of the heap over consecutive collections. It signals severe memory pressure, not necessarily a leak.
As a rough mnemonic, heap space is the fast death and GC overhead is the slow spiral — but treat that as a memory aid, not a diagnostic rule.
Go Run It Yourself
Reading about the pattern is fine. Watching it happen is better. The repo's scenario 01 is an unbounded in-memory cache under a 64MB heap — ./run.sh in broken/ reproduces the same diagnostic pattern in under a minute: repeated full GCs reclaiming almost nothing before OutOfMemoryError. The full diagnosis guide walks through heap-dump triage in Eclipse MAT and the bounded-eviction fix in fixed/.
Coming next in the series: oversized batch accumulation, leaked connections, static collection growth, ThreadLocal leaks, humongous object churn under G1, and shuffle-style sorts blowing past the heap — each one runnable, each one diagnosable, each one fixable. The goal is simple: next time your phone buzzes at 3 AM, the log tells you which question to ask first.
TL;DR
Repeated full GCs that reclaim little reveal a large live set, but they do not prove a memory leak. Track the post-GC floor across comparable workload windows: a rising floor suggests accumulation, while a stable but high floor suggests a large working set or insufficient heap headroom. Confirm what retains the memory with a heap dump before deciding between code changes, smaller batches, or additional heap.
Opinions expressed by DZone contributors are their own.