DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Newsletter
Log In / Join
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Related

  • How AI Is Actually Changing SRE Tools, Part 2: ITOps, Chaos Engineering, and the Rest of the Job
  • Chaos Engineering Has a Blind Spot. Agentic AI Lives in It.
  • What AI Systems Taught Us About the Limits of Chaos Engineering
  • Why Agentic AI Demands Intent-Based Chaos Engineering

Trending

  • SEO Writing 101: A Simple Guide to Getting Found
  • Building Multi-Rail Payment Failover to Stop Latency Spikes and Double Debits
  • One Database, Two Models: Building AI-Friendly Java Applications With MySQL JSON Duality Views
  • Adding Graph Retrieval to a RAG Pipeline With Python

Draft This article hasn't been published, so readers can't see it.

In moderation This article is waiting for review, so readers can't see it yet.

Deleted This article has been deleted, so readers can't see it.

  1. DZone
  2. Software Design and Architecture
  3. Performance
  4. Chaos Engineering: Thread Leak

Chaos Engineering: Thread Leak

‘java.lang.OutOfMemoryError: unable to create new native thread’ will be thrown when more threads are created than the memory capacity of the device.

By 
Ram Lakshmanan
DZone Core CORE ·
Apr. 29, 21 · Tutorial
Post on X (opens in a new tab) Share on LinkedIn (opens in a new tab)
9.1K Views

Join the DZone community Save articles, join the discussion, and download Refcards and Trend Reports with a free account.

Join for free Already a member? Log in

In the series of chaos engineering articles, we have been learning to simulate various performance problems. In this post, let’s discuss how to simulate thread leaks. ‘java.lang.OutOfMemoryError: unable to create new native thread’ will be thrown when more threads are created than the memory capacity of the device. When this error is thrown, it will disrupt the application’s availability.

Sample Program

Here is a sample program from the open source BuggyApp application, which keeps creating an infinite number of threads.

public class ThreadLeakDemo {    
   
     public static void start() {            
         
            while (true) {          
                
                 new ForeverThread().start();      
               }   
       } 
}
 public class ForeverThread extends Thread {    

    @Override   
    public void run() {         

    // Put the thread to sleep forever, so they don't die.      
    while (true) {              

       try {                        

         // Sleeping for 10 minutes repeatedly            
         Thread.sleep(10 * 60 * 1000);         
     } catch (Exception e) {}      
   }   
  } 
}


You can notice that the sample program contains the ‘ThreadLeakDemo’ class. This class has start() method. In this method, ‘ForeverThread’ is created an infinite number of times because of the ‘while (true)’ loop.

 In the ‘ForeverThread’ class there is the run() method.  In this method, the thread is put to continuous sleep, i.e., thread is repeatedly sleeping for 10 minutes again and again. This will keep the ‘ForeverThread’ alive always without doing any activity. A thread will die only if it exits the run() method. In this sample program, the run() method will never exit because of the never-ending sleep.

Since ‘ThreadLeakDemo’ class keeps creating ‘ForeverThread’ infinitely and they never exit. Thus very soon, several thousands of ‘ForeverThread’ will be created. It will saturate memory capacity, ultimately resulting in ‘java.lang.OutOfMemoryError: unable to create new native thread’ problem.

How to Diagnose ‘java.lang.OutOfMemoryError: unable to create new native thread’? 

You can diagnose ‘OutOfMemoryError: unable to create new native thread’ problem either through a manual or automated approach.

Manual Approach

In the manual approach, you need to capture thread dumps as the first step. A thread dump shows all the threads that are in memory and their code execution path. You can capture thread dump using one of the 8 options mentioned here. But an important criterion is: You need to capture thread dump right when the problem is happening. As thread leaks cause production outage, your support/SRE team might restart the application before thread dumps are captured. If thread dumps are captured after the application is restarted, you won’t be able to identify the leaking threads.  Even if thread dumps are captured at the right point in time, you need to import the thread dumps from production servers to your local machine. Then you need to use thread dump analysis tools like fastThread, Samurai to analyze the thread dumps to identify the problem.

Automated Approach

You can use root cause analysis tools like yCrash – which automatically captures application-level data (thread dump, heap dump, Garbage Collection log) and system-level data (netstat, vmstat, iostat, top, top -H, dmesg,…). Besides capturing the data automatically, it marries application-level data and system-level data generates an instant root cause analysis report. Below is the report generated by the yCrash tool when the above sample program is executed:

 yCrash reporting 12,000+ are created

Fig: yCrash reporting 12,000+ are created and they can cause ‘OutOfMemoryError: unable to create new native thread’.
Line of Code in Which 12,000+ Threads Are Stuck

Fig: yCrash reporting the line of code in which 12,000+ threads are stuck.

From the report, you can notice that yCrash points out that 12,000+ threads are created, and they have the potential to cause ‘OutOfMemoryError: unable to create new native thread’ problem. Besides the thread count, the tool is also reporting the line of code, i.e., ‘com.buggyapp.threadleak.ForeverThread.run(ForeverThread.java:12)‘ in which all the 12,000+ threads are stuck. Equipped with this information, one can quickly go ahead and fix the problematic code.

Chaos engineering Dump (program)

Opinions expressed by DZone contributors are their own.

Related

  • How AI Is Actually Changing SRE Tools, Part 2: ITOps, Chaos Engineering, and the Rest of the Job
  • Chaos Engineering Has a Blind Spot. Agentic AI Lives in It.
  • What AI Systems Taught Us About the Limits of Chaos Engineering
  • Why Agentic AI Demands Intent-Based Chaos Engineering

Partner Resources

×

Comments

Loading comments

The comments didn't load

Please try again in a moment.

Likes

Loading

The next likes didn't load. Please try again.

The likes didn't load

Please try again in a moment.

See who liked this

Log in or join DZone for free to see who liked this article.

Log in Join for free
  • RSS
  • X
  • Facebook

ABOUT US

  • About DZone
  • Support and feedback
  • Community research

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 215
  • Nashville, TN 37211
  • [email protected]

Let's be friends:

  • RSS
  • X
  • Facebook