DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Building Scalable Real-Time Apps with AstraDB and Vaadin
Register Now

Trending

  • Using OpenAI Embeddings Search With SingleStoreDB
  • Database Integration Tests With Spring Boot and Testcontainers
  • What Is Envoy Proxy?
  • 13 Impressive Ways To Improve the Developer’s Experience by Using AI

Trending

  • Using OpenAI Embeddings Search With SingleStoreDB
  • Database Integration Tests With Spring Boot and Testcontainers
  • What Is Envoy Proxy?
  • 13 Impressive Ways To Improve the Developer’s Experience by Using AI

The Leprechaun Trap: A Thread Dump Analysis Pattern

During GC, the Finalizer thread will constantly seek out finalize methods. If you use the methods poorly, your Finalizer will get stuck. Beware the Leprechaun Trap!

Ram Lakshmanan user avatar by
Ram Lakshmanan
CORE ·
Jan. 23, 17 · Opinion
Like (5)
Save
Tweet
Share
8.14K Views

Join the DZone community and get the full member experience.

Join For Free

The world is full of patterns.

Starting from a fine grain of salt to complex cosmos, there are patterns everywhere. A true pattern withstands time. They are changeless in the ever-changing world. It requires a lot of hard labor, rich experience (both good and bad), laser focus, and perseverance to crystallize and create patterns from the noise. Fortunately, our computing world has been blessed to have such wonderful patterns created.

One of the classic examples is software design patterns: singleton, factory, visitor, observer, memento, etc. created by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. These four engineers have encapsulated their years and years of hard-learned lessons, refined them, and passed them onto us as easy-to-understand patterns. These patterns are universally applicable, irrespective of what programming language you use (Java, C, C++, PHP, Ruby, etc.), irrespective of what technology stack you run (JEE, .NET, LAMP, etc.), and irrespective of what type of application you build (mobile, web, SOA, microservices, batch, etc.).

Thread Dump Analysis Patterns

Inspired by those four great engineers in my limited capacity, in a humble way, I have crystalized, refined, and created, through my years and years of production battles, thread dump analysis patterns.

Thread dumps are vital components of RCA (Root Cause Analysis). Did your application become unresponsive all of sudden? Did your application’s CPU start to spike without an increase in traffic, without making any code changes, or any environmental changes? Did your application’s response start to degrade? Did your application start to experience memory problems after running for multiple days/weeks? Answers to several such complex problems are present in the thread dumps. But they are buried inside a mountain of details. To shed light on those hidden answers, I have created thread dump analysis patterns.

In this article, let me introduce you to the ‘Leprechaun trap’ pattern. Objects that have finalize() methods are treated differently during garbage collection than those that don't don’t have those methods. During garbage collection, objects with finalize() aren’t immediately evicted from the memory. Instead, as a first step, those objects are added to an internal queue of the java.lang.ref.Finalizer object.

There is a low priority JVM thread by the name of ‘Finalizer’ that executes the finalize() method of each object in the queue. Only after the execution of finalize() do objects become eligible for GC. Because of poor implementation of finalize(), if the Finalizer thread gets blocked, then it will have a severe, detrimental, cascading effect on the JVM.

If Finalizer gets blocked, then the internal queue of java.lang.ref.Finalize will start to grow. It would cause the JVM’s memory consumption to grow rapidly. It would then result in an OutOfMemoryError, jeopardizing the entire JVM’s availability. Thus, when analyzing thread dumps, it’s highly recommended to study the stack trace of the Finalizer thread.

Real-world example

Here is a sample stack trace of a Finalizer thread that got blocked:

"Finalizer" daemon prio=10 tid=0x00007fb2dc32b000 nid=0x7a21 waiting for monitor entry [0x00007fb2cdcb6000]
   java.lang.Thread.State: BLOCKED (on object monitor)
at net.sourceforge.jtds.jdbc.JtdsConnection.releaseTds(JtdsConnection.java:2024)
- waiting to lock 0x00000007d50d98f0 (a net.sourceforge.jtds.jdbc.JtdsConnection)
at net.sourceforge.jtds.jdbc.JtdsStatement.close(JtdsStatement.java:972)
at net.sourceforge.jtds.jdbc.JtdsStatement.finalize(JtdsStatement.java:219)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101)
at java.lang.ref.Finalizer.access$100(Finalizer.java:32)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:178)


The above stack trace was captured from a JVM that was using one of the older versions of the JTDS JDBC driver. Apparently, this version of the driver had an issue; you can see the finalize() method in the net.sourceforge.jtds.jdbc.JtdsStatement object calling the JtdsConnection#releaseTds() method. Apparently, this method got blocked and never returned back. Thus, the Finalizer thread got stuck indefinitely in the JtdsConnection#releaseTds() method. Due to that, Finalizer wasn’t able to work on the other objects that had finalize() methods. Due to that, the application started to suffer from an OutOfMemoryError . In the latest version of the JTDS JDBC driver, this issue was fixed. The point is that when you are implementing finalize() methods, be very careful.

Why named as Leprechaun Trap?

Kids in some western countries build Leprechaun traps as part of celebrating St. Patrick’s Day. Leprechauns are fairy characters — basically a very tiny old man wearing a green coat and hat who hoards and searches for gold coins. Kids build creative traps for Leprechauns, luring them with gold coins. Similarly, the anxious Finalizer thread is always in search of objects that have finalize() methods to execute them. If a finalize() method is wrongly implemented, it can trap the Finalizer thread. Because of this similarity, we have named it a Leprechaun Trap.

Tool

You can learn more such patterns here. On the other hand, to make it easy for the users, we have built a universal thread dump analysis tool: fastthread.io in which all the thread dump analysis patterns are incorporated. You just have to upload your thread dump to this online tool, and automatically, all the thread dump analysis patterns are applied and the root cause of the problem will be reported to you in flash.

Dump (program)

Opinions expressed by DZone contributors are their own.

Trending

  • Using OpenAI Embeddings Search With SingleStoreDB
  • Database Integration Tests With Spring Boot and Testcontainers
  • What Is Envoy Proxy?
  • 13 Impressive Ways To Improve the Developer’s Experience by Using AI

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: