Memory leaks in JEE applications

Memory leaks come in various types such as thread and ThreadLocal leaks, ClassLoader leaks, system resource leaks, and connection leaks. Approaches to memory leak detection typically involve examination of Java virtual machine tool Interface (JVMTI) or performance monitoring infrastructure (PMI) counters to watch for slow growth in Java or native heap use.

New feature:

WebSphere Application Server v8.5 provides a top down pattern-based memory leak detection, prevention, and action by watching for suspect patterns in application code at run time. WebSphere Application Server has some means of protection against memory leaks when stopping or redeploying applications. If leak detection, prevention and action are enabled, WebSphere Application Server monitors application and module activity performs diagnostic actions to detect and fix leaks when an application or an individual module stops. This feature helps in increasing application up time with frequent application redeployments without cycling the server.newfeat

Class loader memory leak

Many memory leaks manifest themselves as class loader leaks. A Java class is uniquely identified by its name and the class loader that loaded it. Classes with the same name can be loaded multiple times in a single JVM, each in a different class loader. Each web application gets its own class loader and this is what WebSphere Application Server uses for isolating applications.

An object retains a reference to the class it is an instance of. A class retains a reference to the class loader that loaded it. The class loader retains a reference to every class it loaded. Retaining a reference to a single object from a web application pins every class loaded by the web application. These references often remain after a web application reload. With each reload, more classes are pinned which leads to an out of memory error.

Class loader memory leaks are normally caused by the application code or JRE triggered code.

JRE triggered leaks

Memory leaks occur when Java Runtime Environment (JRE) code uses the context class loader to load an application singleton. These singletons can be threads or other objects that are loaded by the JRE using the context class loader.

If the web application code triggers the initialization of a singleton or a static initializer, the following conditions apply:

• The context class loader becomes the web application class loader.

• A reference is created to the web application class loader. This reference is never garbage collected.

• Pins the class loader, and all the classes loaded by it, in memory.

Application triggered leaks

Categories of application triggered leaks are as follows:

• Custom ThreadLocal class

• Web application class instance as ThreadLocal value

• Web application class instance indirectly held through a ThreadLocal value

• ThreadLocal pseudo-leak

• ContextClassLoader and threads created by web applications

• ContextClassLoader and threads created by classes loaded by the common class loader

• Static class variables

• JDBC driver registration: RMI targets

• Web application class instance stored in dynamic cache

• Web application-registered custom PMI component

For more information on application triggered links, see http://wasdynacache.blogspot.com/2012/01/websphere-classloader-memory-leak.html, http://www.websphereusergroup.org.uk/wug/files/presentations/31/Ian_Partridge_-_WUG_classloader_leaks.pdf, and http://www.ibm.com/support/docview.wss?uid=swg1PM39870.

WebSphere Application Server now has some means of protection against memory leaks when stopping or redeploying applications. WebSphere Application Server monitors application and module activity and performs diagnostic actions when an application or an individual module is stopped.

There are three parts to this memory leak feature in WebSphere Application Server: detection, prevention, and action.

1. Detection: Issue warnings when a memory leak is detected. Through a combination of standard API calls and some reflection tricks when a web application is stopped, undeployed or reloaded. WebSphere Application Server checks for known causes of memory leaks and issues warnings when an application leak is detected, as follows:

   [11/17/11 12:01:05:911 EST] 00000005 LeakDetection E   CWMML0015E: The web application [WasSwat#WasSwatWeb.war]  created a ThreadLocal with key of type [test.memleak.MyThreadLocal] (value [test.memleak.MyThreadLocal@216c691])
   and a value of type [test.memleak.MyCounter] (value [test.memleak.MyCounter@21942ff]) but failed to remove it  when the web application was stopped. 

2. Prevention is on by default and applies only to JRE triggered leaks. JRE triggered leaks are prevented by initializing singletons at server startup, when the application Server class loader is the context class loader.

3. Action: Take proactive action to fix memory leaks. These actions have reasonable defaults and are configured on a case-by-case basis.

   protected void com.ibm.ws.classloader.clearReferences(){
           if(ENABLE_CLEAR_REFERENCES_JDBC)
     clearReferencesJdbc();
    if(ENABLE_CLEAR_REFERENCES_THREADS)
             clearReferencesThreads();
           if(ENABLE_CLEAR_REFERENCES_THREADLOCALS)
     clearReferencesThreadLocals();
    if(ENABLE_CLEAR_REFERENCES_RMI_TARGETS)
             clearReferencesRmiTargets();
           if(ENABLE_CLEAR_REFERENCES_STATICS)
               clearReferencesStaticFinal();
   }

Leak cause	How to fix	WebSphere Application Server Java Virtual Machine properties for enabling and controlling
Threadlocal	Renew threads in the threadpool for a configurable period. Getting threads out of the pool enables the threads and threadlocals to be garbage collected.	com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesThreadLocal com.ibm.ws.runtime.component.MemoryLeakConfig.renewThreadPoolNames com.ibm.ws.runtime.component.MemoryLeakConfig.threadPoolRenewalDelayFactor com.ibm.ws.util.ThreadPool.DEFAULT_THREAD_RENEWAL_DELAY
HttpClient keep-alive threads	Switch thread to the parent class loader.	com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesHttpClientKeepAliveThread
Timer threads	Use reflection to stop any new tasks that might be scheduled.	com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesStopTimerThreads
Non JVM controlled threads	If the thread is started using an executor, shut down the executor or interrupt the thread.	com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesInterruptThreads
JDBC drivers	Unregister any JDBC drivers that are registered by the web application that the web application forgot.	com.ibm.ws.runtime.component.MemoryLeakConfig.preventJreMemoryLeak
ResourceBundle	Clear the ResourceBundle cache of any bundles that were loaded by this class loader or any class loader where this loader is a parent class loader.	com.ibm.ws.runtime.component.MemoryLeakConfig.preventJreMemoryLeak
RMI targets	Use reflection to clear the values in sun.rmi.transport.ObjectTable.implTable and sun.rmi.transport.ObjectTable.objTable.	com.ibm.ws.runtime.component.MemoryLeakConfig.preventJreMemoryLeak
Static class variables	WebSphere Application Server nullifies the value of all static class variables of classes that are loaded by the application or module class loader.	com.ibm.ws.runtime.component.MemoryLeakConfig.clearReferencesStatic com.ibm.ws.runtime.component.MemoryLeakConfig.filterPrefixes

Related tasks

Configure the memory leak policy

Configure the hang detection policy

Monitor performance with Tivoli Performance Viewer

Example: Adjusting the thread monitor to affect server hang detection Concept topic