DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Securing Your Software Supply Chain with JFrog and Azure
Register Today

Trending

  • What Is Envoy Proxy?
  • From On-Prem to SaaS
  • Cucumber Selenium Tutorial: A Comprehensive Guide With Examples and Best Practices
  • Performance Comparison — Thread Pool vs. Virtual Threads (Project Loom) In Spring Boot Applications

Trending

  • What Is Envoy Proxy?
  • From On-Prem to SaaS
  • Cucumber Selenium Tutorial: A Comprehensive Guide With Examples and Best Practices
  • Performance Comparison — Thread Pool vs. Virtual Threads (Project Loom) In Spring Boot Applications
  1. DZone
  2. Coding
  3. Java
  4. Creative Way To Handle OutOfMemory Errors

Creative Way To Handle OutOfMemory Errors

Nikita Salnikov-Tarnovski user avatar by
Nikita Salnikov-Tarnovski
·
Mar. 28, 14 · Interview
Like (0)
Save
Tweet
Share
7.48K Views

Join the DZone community and get the full member experience.

Join For Free

This can’t be happening. This was pretty much the only thought in my head when staring at the log files. The JVM generating those logs was getting SIGTERM signals out of nowhere and disappearing without a trace.

Let me repeat the last sentence – several times a week someone was deliberately killing an otherwise perfectly decent Java batch job.

The mystery started to unwind when I managed to get access to the rest of the data from within this machine.

Apparently this java process was not the only inhabitant in this machine. I discovered another JVM running on the very same box. The second JVM was apparently running a small webapp deployed to the Tomcat app server. What immediately caught my eye was the correlation in availability of those applications. It seemed that whenever the problematic batch job died, Tomcat was also facing some sort of outage. But Tomcat seemed to recover shortly after, as the batch job remained dead.

An hour later I had went through the Tomcat log files and found another interesting pattern. Right before the Tomcat restarts the all-too-familiar java.lang.OutOfMemoryError: heap space was staring right into my face. So apparently the Tomcat was dying due to lack of memory. But it was still not explaining why the batch job was behaving like it was.

And then I found myself watching at the following parameter in the JAVA_OPTS used to launch Tomcat:

-XX:OnOutOfMemoryError='pkill java;/opt/app/tomcat/start.sh'

I was not even aware there is such an option available. But apparently you can indeed register a shell script to be executed when your JVM has run out of memory. The OutOfMemoryErrors indeed are a painpoint if they have deserved a special flag in the JVM.

But jokes asides, the author of this solution did not take into account the potential of having several Java processes in the same JVM. And so he fired a SIGTERM to all Java processes in this machine. Mystery solved.

Moral of the story? If there was a world cup for guys who are into hiding the symptoms instead of solving the underlying problem, this one would have made it to the semi-finals. Why on earth should you think it is a good idea to solve a shortage of memory and/or memory leak in such a peculiar way?

If you belong to the ranks of engineers who always try to get down to the root cause then subscribe to our Twitter feed for performance tuning advice.

Java (programming language)

Published at DZone with permission of Nikita Salnikov-Tarnovski, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Trending

  • What Is Envoy Proxy?
  • From On-Prem to SaaS
  • Cucumber Selenium Tutorial: A Comprehensive Guide With Examples and Best Practices
  • Performance Comparison — Thread Pool vs. Virtual Threads (Project Loom) In Spring Boot Applications

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: