Profiling the
profilers
Tutorial Details:
Profiling the profilers
Profiling the profilers
By: By Laurence Vanhelsuwé
A comparative review of three commercial Java code profilers
odern software is such an unwieldy multidimensional beast that no single development tool can ever hope to give programmers the complete picture of their creations. Even performance?that elusive metric we all love to hate when our code struggles with second gear, and love, when our code travels at O(1) speed?consists of many devilishly interlocked facets.
Performance's proverbial tip of the iceberg consists of the subjective, user-tangible perception of program speed and responsiveness. If we temporarily eliminate the user from the equation, then we can equate performance with the sum effect of objective performance facets such as algorithm choice, overall memory usage, object allocation and de-allocation dynamics, and multithreading design and runtime behavior. Helping you to understand your program's dynamic behavior in these select dimensions is the burdensome job of code profilers.
In this article, I look at three commercial Java profilers and determine which ones come close to satisfying your, and my, needs:
Borland's Optimizeit Suite
Quest Software's JProbe Suite
ej-technologies' JProfiler
Profiler basics
Not surprisingly, all three products have a lot in common. All modern profilers begin from an identical starting point and constraint: the Java Virtual Machine Profiler Interface (JVMPI) (see sidebar, " The Java Virtual Machine Profiler Interface "). This Sun Microsystems API lets tool vendors interface or connect with a JVMPI-compliant JVM, and monitor the workings and key events of a JVM running any Java program?from standalone application to applet, servlet, and Enterprise JavaBeans (EJB) component.
Given that JVMPI imposes a standardized, level playing field for all profiler tool vendors, it is no surprise that the main differentiating factor vendors compete on boils down to the tools' metafeatures (i.e., features that add significant value to raw JVMPI data and functionality, and, even more important, the graphical user interface (GUI) fronting those features).
As you'll see in the rest of this product review, the products' make-or-break GUIs each have an individual approach to the core problem of how to exploit the raw JVMPI features to maximize analytic and debugging productivity. Unfortunately, as for so many applications tasked with visualizing large datasets in an intuitive and truly user-friendly way, not every profiler convinced me that its makers succeeded in that respect.
The three reviewed profilers have almost identical profiling session configuration capabilities, so I briefly mention them here and move on to the comparison:
JVM selection
To-be-profiled program selection
CLASSPATH and source path selection
To start any profiling session, all three products let you select the JVM on which you normally run your application. Once you select a JVM, you must specify your program's main class, or executable jar file, and what arguments, if any, your program expects. Finally, setting the CLASSPATH for a profiling session also typically lets you point the tool at your source code hierarchy. Figure 1 shows a typical session configuration dialog.
Figure 1. A typical profiling session configuration dialog; in this case, JProbe's dialog. Click on thumbnail to view full-size image.
Launching a program in a profiler implies the generation, capture, and visualization of overwhelming volumes of data, so all profilers include diverse approaches to control this data flood by filtering on various criteria, typically on a per-package basis. This is done using flexible regular expression-style patterns like java.util.* or even jav* .
A quick product comparison
Before I explore profiler-specific features (and anti-features), Table 1 shows an attribute matrix summarizing each offering's key points:
Table 1. Attribute matrix
Optimizeit Suite
JProbe Suite
JProfiler
Version
5.0
5.0
2.2.1
Price
$1,599
$2,000¹
$499
Free evaluation
Yes
Yes
Yes
Online (built-in) help
Yes
Yes (JavaHelp)
Yes (JavaHelp)²
Is help context-
sensitive?
Yes
Yes
Yes
Built-in tutorials
Yes
Yes
No³
Paper documentation
No
Yes
No
Number of tool modules
3 (Profiler, Thread Debugger,
Code Coverage)
4 (Profiler, Coverage,
Memory Debugger,
Threadalyzer)
0 (all-in-one)
Tool modules sold
separately?
No
Yes
No
CPU profiler
Yes (not real time)
Yes (not real time)
Yes (real time)
Object/heap profiler
Yes
Yes
Yes
Thread profiler
Yes
Yes
Yes
Deadlock detection
Automated and visual
Automated
Manual
Race condition
detection
No
Yes
No
Code coverage
Yes
Yes
No
Multi-JVM support
Yes
Yes
Yes
Drill-down to source
Yes
Yes
Yes
Drill-down to bytecode
No
Yes
Yes
Remote profiling*
Yes
Yes
Yes
Automated profiling**
Yes
Yes
Yes
IDE integration
Yes
Yes
Yes
Report generation
Yes
Yes
Yes
Host platform
licensing policy
Multiplatform and
single-platform licenses
Single platform
Multiplatform
Website
www.borland.com/optimizeit
www.jprobe.com
www.jprofiler.com
Ease of use
7/10
4/10
8/10
Explanatory table notes:
* Remote profiling: The ability to profile a Java program executing on a machine other than your development machine
** Automated profiling: The ability to perform unattended overnight profiling sessions; in other words, command-line-driven operation with no GUI
¹ JProbe Suite price includes one year of Gold Support (technical support)
² ej-technologies' JProfiler Online Help contains almost no screenshots of views or dialogs
³ ej-technologies' lack of explicit tutorials is partly compensated by some demo sessions
Test platform
I was pleasantly surprised by the profilers' broad support for diverse platforms, both from a host operating system (OS) and Java implementation point of view. In fact, most profilers support every commercially relevant host and/or JVM implementation (due to the large number of permutations; see the vendors' product Websites for precise details). One exception is that neither Borland's nor Quest Software's profiler suites support Windows 98. This is, I was told, because Windows 98 isn't a "serious" OS when it comes to method timing accuracy. (Windows 98 apparently only offers 50-ms tick accuracy via its public API, and sure enough, many methods will fall through such a coarse timer's net.) So, I tested all three contenders on a standalone PC built around a 900-MHz Athlon CPU, 256 MB RAM, running Windows XP (Service Pack 1).
To keep this review manageable, I restricted myself to testing standalone Java 2 Platform, Standard Edition (J2SE) applications. I didn't test Java 2 Platform, Enterprise Edition (J2EE) applications, although all three vendors try their best to sell into J2EE markets by including product features that explicitly support servlet profiling or EJB components running on various application servers.
You must have intimate knowledge of an application's architecture and implementation to expect to gain new insights from using a profiler, so I relied mainly on two of my own real-life applications as profiling guinea pigs (see Table 2 below).
Table 2. Profiled applications
Program name
World-on-a-Disc
Slave
Description
Map-based multimedia
engine for CD/DVD-ROM
See www.worldonadisc.com
Generic pluggable file
and directory processor
See www.lv2.clara.co.uk/slave.html
Performance
"Achilles' heel"
CPU-bound
I/O-bound
Number of classes
35
60+
All three profilers come with small demonstration applications. I definitely found it necessary to familiarize myself with each profiler by playing with these demos before letting the tools loose on my own programs.
Borland Optimizeit Suite
Borland's Optimizeit Suite is the most mature, full-featured profiler package reviewed here. At $1,599 a seat, Borland is clearly not trying to grab the lone developer market. The suite consists of three loosely coupled components: Optimizeit Profiler, Optimizeit Thread Debugger, and Optimizeit Code Coverage.
Borland Optimizeit Suite's core features
Borland's Optimizeit Profiler is the combined tool façade for CPU and heap/object profiling. Figure 2 shows a typical GUI screenshot.
Figure 2. Optimizeit Profiler's class instances view: Easy on the eyes, but an eye opener nevertheless. Click on thumbnail to view full-size image.
Figure 2's class instances view tabulates the distribution of live objects, grouped by class and sorted by number of objects. If you've never used a heap profiler before, be prepared for an epiphany: Not only will the underlying reality of your program viewed through this lens completely disorient you, but once you come to terms with it, you'll never view your program's source code the same way again.
Borland's Profiler lets you click on any class and view precisely where each instance of that class has been allocated. Not only does Profiler tell you in which methods the allocations occurred (Figure 3), but if you double-click any method name, a source code viewer pops up with the allocating statement line highlighted.
Figure 3. Optimizeit Profiler object allocation backtrace view. Click on thumbnail to view full-size image.
This feature is invaluable when you suspect your program suffers from object allocation hot spots (i.e., parts of the program that allocate too many objects), or want to track down the source of live objects you think shouldn't even exist in your program (e.g., Abstract Window Toolkit (AWT) color objects in an XML parser or Swing objects cluttering up a command-line-only utility!).
The Memory Leak Detector lets you compare two snapshots of your program's heap, and thus hopefully uncover loitering objects that anchor potentially crippling numbers of referenced objects. Note that, despite this feature's name, detecting memory leaks remains hard, manual work, although this feature vastly reduces the size of the needle's h
Read
Tutorial at: Click here to view the tutorial
Rate Tutorial: Profiling the
profilers
View Tutorial: Profiling the
profilers
Related
Tutorials:
 The Java HotSpot
Performance Engine is set to break new records - JavaWorld
The Java HotSpot
Performance Engine is set to break new records - JavaWorld
|
JavaWorld
December 1999
JavaWorld
December 1999
|
Use the JVM
Profiler Interface for accurate timing -
JavaWorld
Use the JVM
Profiler Interface for accurate timing -
JavaWorld
|
The smart approach to distributed performance monitoring with Java - JavaWorld September
2000
The smart approach to distributed performance monitoring with Java - JavaWorld September
2000
|
Tweak your IO performance
for faster runtime - JavaWorld November
2000
Tweak your IO performance
for faster runtime - JavaWorld November
2000
|
Design for
performance, Part 1: Interfaces
matter - JavaWorld January
2001
Design for
performance, Part 1: Interfaces
matter - JavaWorld January
2001
|
Performance books put to the test - JavaWorld March 2001
Performance books put to the test - JavaWorld March 2001
|
Develop a generic caching
service to improve performance - JavaWorld July
2001
Develop a generic caching
service to improve performance - JavaWorld July
2001
|
Diagnose common runtime problems with
hprof
Diagnose common runtime problems with
hprof
|
Step into
the J2EE architecture and process
Step into
the J2EE architecture and process
|
I want my AOP!, Part 3
I want my AOP!, Part 3
|
Put Java in the fast lane
Put Java in the fast lane
|
Java Tip 130: Do you
know your data
size?
Java Tip 130: Do you
know your data
size?
|
Profiling CPU
usage from within a Java application
Profiling CPU
usage from within a Java application
|
My kingdom for
a good timer!
My kingdom for
a good timer!
|
Profiling the
profilers
Profiling the
profilers
|
YourKit Java Profiler 2.5.2 Released
YourKit Java Profiler 3.2 Released
With help of YourKit Java Profiler, an outstanding tool for Java professionals, you can easily solve wide range of CPU and memory related performance problems in J2EE and J2SE applications.
|
Java Application Instrumentation with Log4J
Java Application Instrumentation with Log4J
Application metrics, such as performance metrics, are key to understanding and improving application efficiency. Profiling and monitoring tools yield valuable information on CPU and resource usage, including OS
|
YourKit Java Profiler 3.2 Released
With help of YourKit Java Profiler, an outstanding tool for Java professionals, you can easily solve wide range of CPU and memory related performance problems in J2EE and J2SE applications.
|
HPROF: A Heap/CPU Profiling Tool in J2SE 5.0
Having performance problems with your Java Application? Using too much Java heap space and don't know why? Sometimes simple software tools are all that's needed.
|
|
|
|
