It's a situation nearly every Java developer faces - Murphy's Law strikes at the most inconvenient moment: a critical application upon which everything depends suffers from an elusive heap memory leak and begins throwing OutOfMemoryErrors.
After fruitless hours spent trying to re-create the conditions spawning the leak in a debugger, nothing is gained. If you're like me, you check the Java APIs one more time in vain, just to remember that the System.getFreeMemory() method is about the only indication you have of when memory is tight. "Of course, the memory's running thin!" you shout to yourself. "What I really need to know is where the memory is going!" Even worse than this is a critical deadlocking of threads, leaving a JVM staring blankly at frustrated programmers. It's generally around this time that a developer starts wishing that the JVM had a way to police its resource usage at a fine-grained level. It would be wonderful to have some sort of a "profiler object" that could provide information about the JVM and let the developer register actions based on it.
Sun definitely recognizes this shortcoming of the Java programming language and is taking some strides to try and overcome it. For example, the new Logging API in JDK 1.4 can be used in a limited fashion to gain post facto information about the internal operations of a running program. In addition, JDK 1.4 makes available some interesting new ways to get a snapshot of the thread activity in the JVM, including information on the locks the threads hold. Regardless of this though, a "pure Java" solution for peering inside the JVM and taking action based on what is found there is still not available.
Consider the following situation: an application server or other piece of infrastructure becomes memory intensive, and just one dynamically loaded component using it has starved the heap of memory. Situations like this call for more detailed attention to memory use, and while the Java APIs may not make this easy, the answer is available in the form of JNI and its cousin API, the Java Virtual Machine Profiling Interface, or JVMPI. As Sun has kept relatively quiet about JVMPI, this article serves as an introduction to the technology and will start developers on the road to better memory and thread management.
At first glance, you may wonder why a developer would even need to police the memory heap. After all, isn't the garbage collector supposed to take care of this? Unfortunately, the answer is "not exactly." It's the garbage collector's job to reclaim objects that are not being referenced, but greedy Java programs are creating new objects without releasing the old ones when their use has run out. The garbage collector might be good, but it's not a mind reader.
Another argument against this kind of memory protection has a point - micromanaging the running environment does not fit in with the idea of Java. This may be the case, but it doesn't help anything when memory is leaking and programs are crashing. Moreover, that argument has, in some ways, held Java back as a language for writing fault-tolerant servers. Consider the example of a Web application server that loads a memory-greedy object. The entire server suffers since it lacks knowledge of the source of the memory leak. The best memory protection a "pure Java" application server can hope for is to periodically check the heap's free space and, after too much has been used up, to politely ask loaded components to release extraneous object references. Not only is this strategy less than optimal, it can actually cause loaded components that are conforming to suffer at the hands of a greedy component. Clearly, a technique for finding the trouble spots and eliminating them is necessary.
However, possibly without even realizing it, Sun has given the dedicated developer a way to deal with these very issues. The answer lies in an unusual place - JVMPI. JVMPI is a mechanism through which a developer can receive notifications of extremely low-level events such as object allocations and deallocations. Its structure makes it clear that Sun intended JVMPI to be used for simple profiling of the JVM at runtime but, when coupled with JNI, JVMPI becomes the means through which an application developer can micromanage the virtual machine. With the power to easily record (and act on) low-level JVM events, any Java application can implement fine-grained system resource reporting and protection. Such a program can use strategies designed to eliminate components that misbehave, control thread behavior, and even guarantee completely deterministic system behavior. JVMPI is a powerful tool, but programming with it requires extreme care.
JVMPI is a native-code Java API. What this means, in essence, is that it is a way in which the developer can create a plugin to the Java Virtual Machine. The system profiling agent (the JVMPI code that monitors JVM activity) has to be written in C or C++. After the JVMPI code has been compiled into a shared library (a DLL or SO file), it's ready for use. Launching a JVM with a user-supplied memory profile requires the -Xrunxxx argument, where "xxx" is the name of the shared library without its extension. Once the JVM is up and running, the JVMPI agent will receive notifications about JVM activity. Since the agent receives notifications about the JVM launching and terminating, it has everything it needs to track system resource usage throughout the life of the program. In and of itself, this little-known feature of the JVM might be useful for the simple profiling of applications. In fact, Sun's reference implementation of a JVMPI agent (called "hprof") is built completely around the paradigm of reporting memory use at critical points, such as at JVM shutdown.
Thanks to JNI, though, JVMPI can be used for so much more. The secret lies in JNI's ability to dynamically bind native method calls to Java method calls through the registerNatives() function. It's through the use of this method that a Java developer can write a Java object to serve as an elegant bridge between Java code and a JVMPI profiler. Through this proxy object, a Java program can become aware of its behavior to a degree not offered by any "pure Java" solution and can also become capable of performing tasks not available with Java or even Java and JNI. There can be some pitfalls in this process, and this article will shed some light on some of them, but curious readers are encouraged to review the additional resources mentioned at the end of this article.
The first step in creating a runtime-accessible agent is to create the agent. The agent doesn't need to be very complex at this point, as the purpose is to get a simple agent that will properly launch at runtime. After including the jvmpi.h and jni.h header files, two functions have to be implemented: JVM_OnLoad() and NotifyEvent(). The former uses a pretty common bit of code to prepare the JVMPI environment when the JVM loads. The latter is the mechanism by which the agent receives event notifications from the JVM. Once these two functions are implemented, the profiler is usable, even if it doesn't really do anything. Compile the agent into a shared library using your favorite compiler (I've found the GNU C Compiler to work just fine), place it in the appropriate location for your system (PATH for Win32 systems, LD_LIBRARY_PATH for most UNIX systems), and run a Java application with it using the command line argument mentioned earlier.
Now that a stub agent has been made, things can start getting interesting. The principles of this technique are common to JNI. The secret lies in the fact that a JVMPI agent is, to the JVM, just another shared library. Because it's a shared library, JNI techniques for representing Java methods in native code allow a developer to call a method in Java that acts on data in the JVMPI agent. The first step is to decide which kind of API you're going to use with your memory profiler. For starters, a simple, one-method mechanism for returning data about memory use works. Undoubtedly, just getting a snapshot of the JVM on demand is a necessary first step to even a complex memory management system.
Next, the object encapsulating this method needs to be made. The method is not going to be implemented in the object; it should be declared native and go unimplemented. Now, return to the source code for the agent and implement the method there as a JNI function.
Now that the method is implemented in JNI and has its corresponding native method available in Java, all that remains is to suture the two together using the JNI function registerNatives(). This requires at least a little more programming of the agent (which is good, since the agent does nothing as of yet). To find the right moment at which to link the native method, the agent must listen for class-loading events. In JVMPI, class loads happen in a single-threaded context where the garbage collector is off. This means that the agent can spot the loading of the profiling API object and, when the class is loaded, register the native method implementation to its corresponding method in Java. The process can take a little time to do for someone who is unfamiliar with JNI, but in abstract, it's simple - when a class-loading event comes into the NotifyEvent() method, check to see if it is a class-loading event. If it is, check the name of the class being loaded. If it's the one you're looking for, execute code designed to call the registerNatives() method.
There it is! Now it's possible to have a means of communicating with the JVM at a very fine-grained level. It should be noted that in order for the program in question to run, it must always run with the agent loaded (via the -Xrun argument) or proper care must be taken not to load the API object when the profiler isn't loaded, otherwise an UnsatisfiedLinkError will be thrown. Since it's possible to ensure that these conditions are met with simple launching scripts or compile-time variables (depending on the situation at hand), keeping the UnsatisfiedLinkError at bay is not difficult.
Now the agent is live and running, and can be interacted with via Java method calls. Everything is in place except for something very important - the actual implementation of the agent. The actual behavior of the agent will vary wildly as to its purpose, and a proper discussion of various agent programming techniques is beyond the scope of this article. Instead, this article will focus on some important issues relevant to good agent development that may not be immediately intuitive.
First and foremost, it's important to remember that when you're programming a JVMPI agent, you're not in Java anymore. This may seem obvious, but it's important to remember that C and C++, even with JNI, are not Java. There's no garbage collector. Passing and dereferencing pointers become important habits again. For the most part, all data structures are homebrew. Thread safety requires the explicit use of locks. These are all things that you have to keep in mind to avoid being a slave to Java programming habits. As a corollary to this, it's important to remember that a JVMPI agent is part of the JVM. What this means is that many of the checks the JVM builds against memory and threading issues are not present. For the most part, the JVM is going to accept the behavior of the agent even if the behavior is bad. Extreme care must be taken to ensure that deadlock and memory leaks do not occur.
Most memory management issues involved with C and C++ programming can be resolved only with conscientious programming. Far more important than memory management concerns, though, are thread-related concerns. The first important thing to remember is that most JVMPI events are sent in the same thread as the event that triggers them. Activity in an agent is generally not synchronous, which means that the code for an agent may be processing many events at the same time. The same design-level care must be taken with regard to resources shared across threads as would be used in Java; however, the implementation of synchronization must be done explicitly using the monitor methods of JVMPI (RawMonitorCreate(), RawMonitorEnter(), RawMonitorExit(), RawMonitorWait(), RawMonitorNotifyAll(), RawMonitorDestroy()).
Using monitor locks on frequently received events (such as Method
Entered) can damage the performance of the JVM in general, and so another technique is needed. JVMPI thus provides an additional mechanism for implementing synchronization techniques known as "thread local storage." Thread local storage is similar to the concept of the ThreadLocal object in Java - it's a means of storing data that is bound to the scope of a specific thread. For each thread running in the VM, JVMPI makes available a pointer that can be used to store profiling data specific to that thread. By initializing a data structure to use as local storage when a thread is made, and utilizing a thread's local storage throughout its life cycle, a developer can record thread-specific information in data structures that are relatively immune to threading issues. At an appropriate time, such as when the thread dies, the local storage can be easily retrieved and merged into global data structures.
In addition, the agent must deal with JVM states that Java programs can effectively ignore. For example, an agent can enable and disable the garbage collector. Some events even broadcast while the garbage collector is disabled (for example, class loads). When the garbage collector is disabled, threads executing in the agent will block if they execute code that causes the creation of Java objects or forces the garbage collector to run (JVMPI allows for deterministic garbage collector calls). In addition, blocking dependencies between threads can become more complicated - if thread A is holding a lock and attempts to run the garbage collector when thread B disables the garbage collector and tries to acquire thread A's lock, both threads deadlock unless some third thread enables the garbage collector again.
The other JVM state a profiling agent must carefully execute under is "thread-suspended mode," during which the only thread executing is the one in the agent. This situation is easier to cope with than operating with the garbage collector disabled, but still carries its own special conditions. When a thread is suspended, it continues to hold its locks (it's for this reason that the suspend() method is deprecated). When all the threads are suspended, locks that the agent cannot check against may be held. These may be anywhere, including potentially blocking calls in the standard C library. Developers must take care not to block the running thread while in thread-suspended mode. Failure to do so will cause unrecoverable deadlock.
If many of these cautions make it sound like programming in JVMPI can be difficult, that's because it can be. JVMPI is essentially an API for writing a plugin to the JVM, and there's almost no limit to what can be done with a mixture of C, Java, JNI, and JVMPI. Even without venturing deeply into native code programming, some amazing things can be done. JVMPI allows for some very tight control and micromanagement of the JVM. One method available in JVMPI that can be very useful is GetThreadStatus(), from which the state of a thread, including information regarding what it's waiting on, can be discerned. It may also be preferable to see the actual amount of execution time that various threads are taking so greedy threads can be identified. With such information, it's possible to implement policies in a Java program for dealing with various threading issues in ways not available before.
Such examples, however, are minor compared to the real power that JVMPI offers - the ability to clearly know how the JVM is being used. Since JVMPI is running as a part of the JVM, it can gather memory and resource use information in ways that aren't available using the standard Java API. Just how this is leveraged varies from application to application, but a common strategy is to break down object allocation and use by thread. When it's found that memory is running tight, a quick call to an agent proxy object (built using the RegisterNatives() technique mentioned earlier) can identify threads that may be excessively using memory so they can be dealt with. By identifying an errant thread (or whatever would be identified as a "resource hog"), application logic designed to perform forceful resource cleanup can be targeted where it's needed, which effectively could not be done without JVMPI.
Where do you go from here? Curious developers are encouraged to look at the example sources referenced in the resources section - one is written in highly portable C and the other in C++. Both are designed for basic memory and thread profiling, and while they may not employ fine-grained resource management strategies, they will definitely provide a good education in the underpinnings of profiling that are necessary for such a task.
JVMPI is definitely not for the faint of heart, but the kinds of projects with runtime profiling of the JVM (application servers, component architectures, Java development tools, etc.) are not either. If you've ever wished you could have absolute control of the JVM or that you could see what was really going on in your program, or if you're writing some sort of a component architecture or server where identifying and handling regions of code that misappropriate objects could mean a boost in performance and reliability, JVMPI is by far the best way to fly.
Interested parties are encouraged to visit http://java.sun.com/j2se/1.4/
docs/guide/jvmpi/jvmpi.html for documentation on JVMPI. This page includes links to the source code for a sample profiler to which the technique in this article can be applied.
C++ programmers are encouraged to visit http://starship.python.net/
crew/garyp/jProf.html to see an excellent example of JVMPI programming in C++.
J. Rhett Aultman has been a Java developer for three years, offering his talents to the open source community both by himself and through Weatherlight Technologies, a co-op of independent developers (www.weatherlight.com).