Coffea Works

Tracking Memory Issues in Java Applications

Using Enerjy Memory Profiler

by Richard Sharpe

From its conception to its current invocation, one of the most favourable merits of the Java language (currently Java 5) is automatic memory management. Unlike other languages, developers no longer have to worry about the many mistakes that can occur from the more traditional languages, for example, allocating memory or forgetting to free dynamically allocated memory. The JVM will generally free up memory when it reaches a certain limit or when it needs more memory to continue execution of an application. Due to the mechanics of the garbage collector however, once an object is de-referenced there is no guarantee that the object will be cleaned up.

Memory consideration is one of the main issues in the design and performance of Java applications. Whether developing Web applications hosted on server farms or developing personalized applications for hand held devices, problems such as memory leaks, or memory limitations, can occur right from the start of development.

Introducing Enerjy Memory Profiler
Enerjy Memory Profiler is a method-based profiler that collects data from the JVM while the application runs. The data is reported in a clear and concise manner, thus allowing the developer to examine any memory issues the application has. The developer can focus on specific hotspots and even proceed directly to the source code to review or rework the code that has caused any problems.

Dynamic profiling techniques allow the developer to take a snapshot of memory usage any time during the life of the application. The developer can then clear the data, change the configuration to profile at a more granular or broader level, and even change the focus of the Profiler to examine different parts - all without having to restart the application. The benefits of on-the-fly profiling are evident when using the tool to monitor large, complex applications or J2EE applications where developers can concentrate on numerous areas while the user interacts with the system.

As with all tools from Enerjy Software, Enerjy Memory Profiler tightly integrates into the IDE. This means there is no tricky system configuration to endure, since the Profiler automatically uses the configuration from the IDE. The only configuration needed is the area of the application to profile and what sort of information is required.

Configuring Enerjy Memory Profiler is extremely straightforward. A window appears with two main sections; the first section allows the developer to select whether they want heap data and if so, how often to poll the heap to obtain the data. This will depend on the size of the application; for larger Web applications with user interaction, polling can be set to 500 - 2000 milliseconds. For smaller, automated applications this can be more frequent, say, 100 - 300 milliseconds. Selecting this checkbox allows the developer to see -- and take snapshots of -- a dynamic graph of heap usage whilst the application is still running.

Figure 1: Configuring Enerjy Memory Profiler

The second section allows more data to be collected so the developer can analyse what occurs while the application runs. As mentioned above, Enerjy Memory Profiler is a method based profiler and therefore the developer can select the methods appropriate to where in the application they wish to focus. Snapshots can be sent after each time the method is invoked, but a word of caution: this may result in a lot of snapshots if the method is invoked from a loop!

Metrics are then produced, providing data regarding any objects created from the methods (or called methods) selected. The checkbox "Detect objects created but not released by above methods" may contain any leaks in the application and the "Enable Goto Source" checkbox allows the developer to access the source code directly from the Memory Profiler data.

The overhead of profilers has always been an issue for developers. All code profilers have an element of overhead, for example, some profilers instrument the byte-code throughout the lifetime of the application. This adds lines to the byte code and therefore has an overhead, even in areas that the developer is not interested. Enerjy Memory Profiler reduces the overhead level by only instrumenting byte code when absolutely necessary. Other techniques such as using some of the more efficient features of JVMPI, filtering data to show the developer only what they need to know, sampling techniques, and on-the-fly profiling ensure that Enerjy Memory Profiler adds a minimal level of overhead so the results displayed are an accurate reading of how an application will perform.

Analysing the Results
The data produced by all code profilers in the market should be fairly similar, however interpreting these results can be an arduous task. It is the way Enerjy Memory Profiler displays the information to the user that differentiates it from other profiling tools. Enerjy Memory Profiler integrates into the IDE and allows the user to drill down from an overview of memory usage by the application in to the individual objects leaked and then directly to the source code.

At the developer's request, Enerjy Memory Profiler shows snapshots of the heap activity and again once the application has terminated. The snapshot of heap activity can quickly determine the level of memory used in the application and if there are any problems, such as memory leaks.

Figure 2: Enerjy Memory Profiler used within the Eclipse IDE.

The snapshot of the heap (taken from the dynamic runtime graph in the bottom pane) shows the memory allocated by the system to the JVM (green) and the actual memory used by the application (red). In Fig. 2 above, Enerjy Memory Profiler was executed on a self-contained application. The results clearly show something is wrong, as the level of memory utilised is not at the same level as before the application was executed (suspect memory leaks, perhaps). Now that a problem has been identified the user can re-run the profiler with more options to focus on why and where the problem is occurring.

By reconfiguring Enerjy Memory Profiler to enable leak detection, the user can re-run the application, click on the 'Leaks' tab of the snapshot (Fig. 3), and see details of any objects left in memory after the application has terminated determining any memory leaks.

Figure 3: Identifying Memory Leaks in Enerjy Memory Profiler.

In the 'Objects' pane any objects (grouped by class name) remaining in memory are shown. The leaks are shown at Parent Leak level rather than showing all leaks (although this option can be selected). This helps the developer fix the top-level leaks that will then automatically clear up any incidental leaks. For example, if an object has leaked containing various strings and arrays, listing the fact that the object, strings and arrays have leaked shows the developer that there is a problem. However, the developer may spend unnecessary time tracking down the incidental leaks (strings and arrays) just to find that by clearing up the main object the other leaks are automatically eliminated.

By selecting an element in the Objects pane, the developer can drill down in the Roots pane to discover the exact method that created the leaked object. Double clicking on the element takes the developer to the source code at the point the object was created. The developer can then begin to examine the code to determine why the Garbage Collector did not automatically collect this object. Once all leaks are fixed, the developer can use Enerjy Memory Profiler to determine if the application uses too much memory at any point.

By running Enerjy Memory Profiler on the application with only the "Collect heap data usage" box selected, the Profiler shows the Heap graph. From this graph the developer can determine the level of memory the JVM used at any point in the application. By selecting a range within the graph, the Outline pane details the methods that were invoked at this time. If the memory utilised is too high the developer can re-code to prevent this. For example, if the application in Fig. 4 was to be implemented on a device with only 12MB of RAM available, the developer can highlight a peak of the graph above this limit and view which methods were called. The developer can then decide how to change the code and possibly force a garbage collection in the MemExample.loop() method.

Figure 4: Using Enerjy Memory Profiler to determine memory limitations.

For reporting purposes, snapshots from all Enerjy Software products, can be exported as comma separated or tab-delimited files. These results can then be imported into a reporting tool of choice to display the various statistics of your application.

Figure 5: Exporting the data from Enerjy Memory Profiler into Microsoft Excel.

Profiling J2EE Applications
Traditionally, trying to profile J2EE applications results in large overhead to the point of the system slowing to a crawl, or even crashing, with an 'out of memory' error. One reason for this is that some tools profile everything, including the Application Server. At this level there is no interest in the Application Server, only the application. This can lead to a performance decrease of the system of up to ten times. Due to the on-the-fly profiling feature in Enerjy Memory Profiler the overhead for profiling J2EE applications is minimal. Start the profiling session with no options set (and therefore no overhead) and start the application server. When you are ready to profile the application, or any part of it, reconfigure Memory Profiler with the settings you require. This ensures you are in control of what you are profiling and reduces any redundant data reported.

Enerjy Memory Profiler can be utilised for J2EE applications in development, using software such as the WebSphere Application Server Testing Environment as part of WSAD 5.x, as well as remote profiling on a staging/test server. The configuration of Memory Profiler is similar in both cases although a one line 'Xrun' command is to be entered in the VM environment options of your J2EE certified Application Server.

Profiling Payoff
Profiling Java applications is critical to a developer's jobwhether they are the one writing the curly braces and semicolons or whether they are managing the development team that is creating the application. Knowing what to look for and how, and getting into the mindset of profiling in an iterative fashion - rather than when crisis strikes - will put the developer leagues ahead. Enerjy Software can help achieve this by providing a low cost, high quality Memory Profiler that can be easily integrated into the IDE and be effective.

There are a lot of conflicting opinions on the matter of profiling Java applications, but one area where everyone agrees is at the base level: profiling Java applications is a good thing. Run tests using Enerjy Memory Profiler on different Java code that has similar functionality, and once you have the optimal code, put it in your code library. It was noted at recent Java trade shows such as JavaOne 2004 (San Francisco, California) and JavaPolis 2004 (Belgium) that the interest in this area is high, although, surprisingly, the number of books on the subject is few. Use Enerjy Memory Profiler to add new strategies, techniques, and alternatives to create optimised, reliable applications.

IDE Support
Currently Enerjy Memory Profiler supports the following IDEs:

Eclipse (by eclipse.org), Release 2.0.x or 3.x
JBuilderTM (by Borland), Release 7, 8, 9, or JBuilder X
WebSphere Studio Application Developer (by IBM), Release 5.x

About Enerjy Software
Our Tools: Enerjy's suite of Java tools strives to provide tactical, comprehensive, andat their heartuseful solutions for the everyday Java developer. Our tools keep you inside your IDE, where you spend eight (ten?) hours a day, and get you back to your curly braces and semicolons, where the work really matters. You'll get simple-to-read tables and easily recognized graphs, and you'll always be a double-click away from your code.
Our Company: You'll find that we're a little different from other software tools companies. Instead of trying to sell you everything, an all-encompassing development solution, we're maniacally focused on giving you just the help you need, when you need it, and then getting out of your way. And we won't charge you an arm and a leg for our tools. We've built the company from the ground up with a view to keeping costs under control and passing those savings on to you.

About the Author
Richard Sharpe: Richard Sharpe is a technical support specialist at Enerjy Software. He previously worked for IBM in the E-Business Group for a number of years and then joined Aspective Ltd in the Mobile Business Group as a WebSphere/Java specialist. Richard holds a bachelor's degree in Computing Systems from Nottingham Trent University, England and is a certified WebSphere Application Server Systems Expert.

Reference