HomeDigital EditionSys-Con RadioSearch Java Cd
Advanced Java AWT Book Reviews/Excerpts Client Server Corba Editorials Embedded Java Enterprise Java IDE's Industry Watch Integration Interviews Java Applet Java & Databases Java & Web Services Java Fundamentals Java Native Interface Java Servlets Java Beans J2ME Libraries .NET Object Orientation Observations/IMHO Product Reviews Scalability & Performance Security Server Side Source Code Straight Talking Swing Threads Using Java with others Wireless XML

This is not another article about the Proxy API as introduced in JDK 1.3. Well, okay, it is, but only indirectly. The Proxy API is implemented using techniques that can seem a lot like magic if you're not familiar with them.

That's really what I want to share: how and where to dig for more information. I'll spoil as much of the magic as I can.

Generative programming is a class of techniques that allows for more flexible designs without the performance overhead often encountered when following a more traditional programming style.

Individual generative techniques have been around for decades, but programming systems like Java make them much easier to build. In recent years there has been an explosion in their use.

A JSP engine is an obvious example. A generative program is simply a program that generates another program. If you've ever made a syntax error in a JSP page, you're no doubt (painfully) aware that the underlying implementation of a JSP is a servlet. You might say that the servlet was generated from your JSP on its first run.

In the implementations I've seen, it's quite easy to determine how the JSP engine works because the generated servlet code is actually written to a .java source file somewhere on disk.

On the other hand, java.lang.reflect.Proxy is a generative program that isn't quite as transparent - a perfect topic for the curious.

First, in case you're unfamiliar with the Proxy API, let's recap a little. Consider a class "Bug" with methods like escalate() and resolve(). Actually, pretend it has a lot more methods, or that there are a lot of classes similar to it with the same problem. Or, if you're like me, pretend that you really, really, really hate repetition. Listing 1 shows "Issue," the interface that Bug implements.

Now you have to implement a history-tracking module that can log all the actions taken on bugs. One way is to go into every method in Bug, add log() calls:

public void resolve( Person by, String desc ) {
log( "Bug", "resolve", by+","+desc, null );
resolved = true;
}
and hope you don't forget to change the "resolve" string for each call or to log a parameter or leave out a method or class entirely, and that no one adds or exposes new methods in the future. Forget being able to easily change the log() call to replay or rollback actions later.

You're always better off trading thinking for typing: the less typing you do, the less gunk lying around that you have to maintain. Besides, it's more fun to use an automatic Proxy. A Proxy is an automatically generated class that implements an interface that you choose. When any of the methods it implements gets called, the Proxy object will call an InvocationHandler's invoke() method. invoke() can then do whatever it wishes, including logging the parameters and forwarding the call onto the "real" object. See Listing 2 for an example.

The "method.invoke()" line uses Java reflection to invoke the appropriate method on the "real" underlying Bug object and then store the return code. The rest of the method is organized so it can log even if an exception is thrown. Our log() will write the log entry in this format: com.dotcom.Bug.resolve( Jim, fixed now).

As I mentioned earlier, this is not an article about the Proxy API, so I've deliberately left out a lot of details. For the curious, the complete example is available for download below.

If you've read anything about Aspect-Oriented Programming (AOP), not only will you recognize that logging is a popular example, you'll also realize that this use of the Proxy class eliminates the need to scatter logging code throughout a program. This provides something coined separation of concerns: keeping different aspects of the program separate. This should increase the clarity and general maintainability of the code you write because smaller pieces of it need to be grasped when looking at any particular method. When all is said and done, you've typed less. Actually, if you enjoy these concepts, you'll love AOP (look at AspectJ at www.aspectj.org).

One thing about adding lots of log() calls is that by the time you finish typing, you know, in painstaking detail, what gets logged and how. It's nice to know "how things work"; debugging is easier and design decisions are, if not actually better, at least more informed.

Perhaps it's time for a little investigation into how our magic Proxy works. Unlike your options in the Microsoft world (which begin and end with SoftICE and some of the utilities at www.sysinternals.com), Java includes source code in the form of an src.jar that ships with the JDK. Once you know it's there, it doesn't take much ingenuity to extract it and pull up src/java/lang/reflect/Proxy.java. I use WinZip (because I'm lazy), but "jar -xvf src.jar" also works.

If you're good at squinting and looking at the big picture, you will be able to determine that Proxy builds a Java class into a byte array and then loads it into the JVM with a private native defineClass0 method.

Incidentally, if you ever have the urge to do the same, java.lang.ClassLoader provides the API you'll need. This technique is more useful than you might initially think; Stu Halloway's excellent book, Component Development for the Java Platform, covers the topic extensively.

Aside from the broad approach, this source code doesn't tell us that much. For instance, we can't speculate as to how much space a Proxy object will take up or what its performance might be like. All the interesting work is being done by a sun.misc.ProxyGenerator class for which there is no corresponding source file, at least in src.jar.

If you've put in your time on Sun's Web site, you'll know that they make the JDK source available for "look but don't touch" use through the Sun Community Source License, or SCSL. As this source can be compiled into an actual JDK, we can rest assured that the code will be in there this time. If you're so inclined, take a look at src/share/classes/sun/misc/ProxyGenerator.java.

Sometimes your questions are most easily answered by pulling up the source code to the Sun JVM. This, sadly, is not one of those times.

When presented with the full complexity of this implementation, you'll realize (as I did) that you don't really care how Proxy works beyond the broad details. The algorithm used to build a class file byte by byte is actually uninteresting compared to the generated class itself.

If you want to learn about javac, don't study the source, create examples and study the bytecode it produces. I doubt I'll ever find it necessary to read the code to the Hotspot JIT, although I have spent a week of quality time with the x86 assembly it produces. I daresay that far fewer people care how Hotspot optimizes than what optimizations it really performs.

Unfortunately, Proxy doesn't leave a .java or even a .class file on disk for us to examine, so we'll need a way to intercept the class bytes that it generates. An annoyingly easy way would be to modify the code to Proxy to dump the generated class bytes to disk.

Every so often I hear of someone trying to prevent decompilation by encrypting class files on disk and decrypting them when his or her application is run. This tool should be a graphic demonstration that this approach is a waste of engineering effort.

The consensus seems to be that if you need protection for your class files, simple obfuscation can be used. If you were at JavaOne 2001 you may have heard the Hotspot team mention that more advanced code obfuscations (such as control-flow obfuscation) can actually produce slower programs: the obfuscated bytecode is more difficult to optimize.

Nothing prevents you from recompiling Proxy and either replacing it in jre/lib/rt.jar or prepending it to the JVM bootclasspath. The bootclasspath is like the classpath except it's for system classes. To prepend an entry to the bootclasspath, you need to run Java with a command line like:

java -Xbootclasspath/p:C:\newSysClasses com.dotcom.LoggingTest

I won't go into too much detail. Ted Neward has an excellent explanation of the bootclasspath at www.javageeks.com/papers. Although I think it's okay to use such hacks while exploring, there should be a more elegant way to capture class files.

If you have a tendency to read obscure parts of the API, you might be aware of a class_load_hook callback hidden inside the Java Virtual Machine Profiling Interface (JVMPI). JVMPI is an API used by tools vendors for profiling products such as DevPartner/Java, JProbe, or Optimizeit. It's a great way to learn about the inner goings-on of the VM and can be quite entertaining (interpret that how you will).

The ClassCapture Tool
If you're good at JNI, writing a simple JVMPI agent that captures class bytes is quite easy, once you get over the chutzpah of it all. Here are the steps:
1. Create a DLL called "capture.dll" (or a shared object called "libcapture.so" if you're on a Unix system) that exports a JVM_OnLoad() function. JVM_OnLoad() is the function that the JVM will call when it first loads our DLL.
2. JVM_OnLoad() should initialize JVMPI and enable the class_load_hook event. We'll also pass a base directory to JVM_OnLoad so it knows where to write the .class files (see Listing 3).
3. Create a notifyEvent() method to listen for class_load_hook and write the class file bytes to disk (see Listing 4). (The omitted methods are contained in the ClassCapture source download.)
4. When you run a program, add "-Xruncapture:outputPath" to the Java command line. "-Xrun" is a prefix that tells the VM to load the named DLL - in this case "capture". The colon is used to pass arguments to the DLL; we're using it to pass in our output directory.

You might execute:

java -Xruncapture:C:\proxy\captured
Classes -cp . com.dotcom.LoggingTest
From then on, you'll get a call to notifyEvent() every time a class is loaded, regardless of where it's loaded from. Remember, classes can have many sources: normal .class files on disk, from a compressed .jar file, from a URL in Australia, or even dynamically generated, as we've discovered.

The callback gives you the actual bytes of the class (the same format found in a .class file on disk) and the opportunity to change them prior to their being loaded. (The ClassCapture tool [source and a Win32 binary] is available for download on the JDJ Web site.)

Having done all this work, let's take ClassCapture for a test drive. Going back to our original example of the bug history module, the ClassCapture tool will create a directory structure like this:

C:\proxy\capturedClasses
+--com
+--dotcom
+--java
+--io
+--lang
+--reflect
+--net
+--security
+--cert
+--util
+--jar
+--zip
+--sun
+--misc
+--net
+--www
+--protocol
+--file
+--reflect
+--security
+--action
As you can see, even system classes are captured, except for those that were loaded before capture.dll, such as java.lang.Object or java.lang.String.

In C:\proxy\capturedClasses there's a $Proxy0.class file. If you're fluent in Java bytecode you can disassemble it with "javap -c -classpath . captured-Classes $Proxy0". If you don't habitually read about mysterious binaries in jdk/bin, you might not be aware that javap is a Java class file disassembler that ships with the JDK.

JVM bytecode is very simple when compared to traditional chip-based instruction sets (like x86), which makes it easy to examine and reason about, if you're into that kind of thing. Otherwise, you're still in luck because it also makes it easy to decompile. There are many (free and commercial) Java decompilers available and I haven't spent enough time with any of them to start picking favorites. A search on Google for "Java decompiler" will give you everything you need. The decompiled (okay, I airbrushed it a little) Proxy class is shown in Listing 5.

How Proxy Works
It turns out that Proxy works as we might have guessed. It doesn't generate a .java file on disk and then compile it the way a JSP engine does, so we don't have to worry that creating lots of different Proxy objects will run multiple compiles and grind away on even the fastest machine. Instead, it efficiently builds a .class file in memory and loads it directly into the JVM.

The class it generates uses the Reflection API to look up methods, but this is done only once in the static initializer. A reflective lookup is much slower than a reflective invoke once you have a method reference.

By looking at the getPriority() call you can see how it casts the object that the handler returns into integer and then calls intValue() to return a primitive int.

The escalate( Person by, int amount ) method shows the boxing (new Integer()) that is generated in order to pass a primitive int into the handler method as an object.

The defer() method is a good way to see that RuntimeExceptions, Errors, and checked exceptions (such as CannotDeferException) are simply rethrown, but other exceptions are wrapped inside an UndeclaredThrowableException.

The implementations of toString(), equals(), and hashCode() show that the Proxy will properly call the invocation handler even for java.lang.Object methods.

Conclusion
We've gone about as far as we can down this path, laid bare the internals of the Proxy API, and hopefully learned a couple of neat tricks along the way. Since you've already read this article I can admit, now, that we didn't discover anything about the Proxy API that couldn't be found in its documentation. But, having done all this work you might find that you're no longer quite as impressed by java.lang.Proxy.

Proxy is a neat concept, and it does use a couple of esoteric Java APIs, but it isn't anything beyond our reach, hidden in the depths of the JVM. It isn't even that farfetched to imagine writing something similar, should the need arise. If you do so, or if you stumble across someone else who has (perhaps someone without the documentation diligence of the JDK team), maybe you'll have a couple of the pieces of the puzzle already in place when you have to try and work out what on Earth is going on.

Resources

  • Generative Programming: www-ia.tu-ilmenau.de/~czarn/generate/engl.html
  • Sun Community Source License: www.sun.com/software/communitysource/java2/index.html
  • Ted Neward's explanation of the Java BootClassPath: www.javageeks.com/Papers/BootClasspath/index.html
  • Component Development for the Java Platform by Stuart Halloway: http://cseng.aw.com/book/preface/0,3829,0201753065,00.html

    Author Bio
    Paul McLachlan is the Java technical lead for Compuware's DevPartner product, which features tools for Java performance profiling and code coverage as well as thread and memory analysis. [email protected]

    	
    
    
    Listing 1: Issue interface: Issue.java
    
    package com.dotcom;
    
    
    public interface Issue
    {
            public void escalate( Person by, int amount );
            public void resolve( Person by, String resolution );
            public void defer( Person by, String resolution )
                    throws CannotDeferException;
            public int getPriority( Person by );
    }
    
    
    
    Listing 2: Generic handler method: LoggingInvocationHandler.java
    
    package com.dotcom;
    
    
    import java.lang.reflect.*;
    
    
    class LoggingInvocationHandler
            implements InvocationHandler
    {
            private Class clazz;
            private Object forObject;
    
    
            public LoggingInvocationHandler( Class c, Object o )
            {
                    this.clazz = c;
                    this.forObject = o;
            }
    
    
            public Object invoke( Object proxy,
                                  Method method,
                                  Object[] args )
                    throws IllegalAccessException,
                           InvocationTargetException
            {
                    try
                    {
                            //  execute the underlying method:
                            Object ret = method.invoke( forObject, args );
    
    
                            //  log the call & return code:
                            LogManager.log(
                                    clazz.getName(),
                                    method.getName(),
                                    args,
                                    ret );
    
    
                            return( ret );
                    }
                    catch( InvocationTargetException ex )
                    {
                            //  log the call & exception thrown:
                            LogManager.log(
                                    clazz.getName(),
                                    method.getName(),
                                    args,
                                    ex.getTargetException() );
    
    
                            throw ex;
                    }
            }
    }
    
    
    
    Listing 3: Class capture: JVM_OnLoad
    
    extern "C" JNIEXPORT jint JNICALL JVM_OnLoad(
            JavaVM *jvm,
            char *options,
            void *reserved )
    {
            //  read options into global base path variable
            parseArguments( options );
    
    
            //  get a pointer to the JVMPI interface
            if( jvm->GetEnv( (void **)&jvmpi, JVMPI_VERSION_1 ) < 0 )
            {
                    trace( "Error enabling JVMPI\n" );
                    return JNI_ERR;
            }
    
    
            jvmpi->NotifyEvent = notifyEvent;
            jvmpi->EnableEvent( JVMPI_EVENT_CLASS_LOAD_HOOK, NULL );
            return JNI_OK;
    }
    
    
    
    Listing 4: 
    
    Class capture: notifyEvent()
    void notifyEvent( JVMPI_Event *event )
    {
            if( event->event_type != JVMPI_EVENT_CLASS_LOAD_HOOK )
                    return;
    
    
            //
            // We're passed an event structure like:
            //
            //  struct {
            //        unsigned char *class_data;
            //        jint class_data_len;
            //        unsigned char *new_class_data;
            //        jint new_class_data_len;
            //        void * (*malloc_f)(unsigned int);
            //  } class_load_hook;
    
    
            unsigned char* class_data =
                    event->u.class_load_hook.class_data;
    
    
            jint class_data_len =
                    event->u.class_load_hook.class_data_len;
    
    
            //  Leave the class unchanged
            copyUnchangedClassBytes( event );
    
    
            //  Getting the name of a class from the class
            //  bytes isn't exactly straightforward, I'll
            //  spare you the details:
            const char* path = getClassPath( class_data );
    
    
            //  make directories if necessary
            ensurePathAvailable( path );
    
    
            //  Write out the class bytes as we were given them
            FILE* f = fopen( path, "wb" );
    
    
            delete[] const_cast<char*>( path );
            path = 0;
    
    
            if( !f ) return;
    
    
            fwrite( class_data, class_data_len, 1, f );
            fclose( f );
    }
    
    
    
    Listing 5: Decompiled proxy class: $Proxy0.java
    
    import com.dotcom.Issue;
    import com.dotcom.Person;
    import java.lang.reflect.*;
    
    
    public final class $Proxy0
            extends Proxy implements Issue
    {
            public $Proxy0(InvocationHandler h)
            {
                    super( h );
            }
    
    
            private static Method m_defer =
                    Issue.class.getMethod(
                            "defer",
                            new Class[] { Person.class, String.class }
                    );
    
    
            private static Method m_escalate =
                    Issue.class.getMethod(
                            "escalate",
                            new Class[] { Person.class, Integer.TYPE }
                    );
    
    
            private static Method m_getPriority =
                    Issue.class.getMethod(
                            "getPriority",
                            new Class[] { Person.class }
                    );
    
    
            private static Method m_resolve =
                    Issue.class.getMethod(
                            "resolve",
                            new Class[] { Person.class, String.class }
                    );
    
    
            private static Method m_hashCode =
                    Object.class.getMethod(
                            "hashCode",
                            new Class[0]
                    );
    
    
            private static Method m_toString =
                    Object.class.getMethod(
                            "toString",
                            new Class[0]
                    );
    
    
            private static Method m_equals =
                    Object.class.getMethod(
                            "equals",
                            new Class[] { Object.class }
                    );
    
    
            public final void defer(Person p, String s)
                    throws CannotDeferException
            {
                    try
                    {
                            h.invoke(
                                    this,
                                    m_defer,
                                    new Object[] { p, s } );
                    }
                    catch( Error err ) { throw err }
                    catch( RuntimeException rte ) { throw rte; }
                    catch( CannotDeferException cde ) { throw cde; }
                    catch( Throwable t )
                    {
                            throw new UndeclaredThrowableException( t );
                    }
            }
    
    
            public final void escalate(Person p, int i)
            {
                    try
                    {
                            h.invoke(
                                    this,
                                    m_escalate,
                                    new Object[] { p, new Integer( i ) } );
                    }
                    catch( Error err ) { throw err; }
                    catch( RuntimeException rte ) { throw rte; }
                    catch( Throwable t )
                    {
                            throw new UndeclaredThrowableException( t );
                    }
            }
    
    
            public final int hashCode()
            {
                    try
                    {
                            return (
                                    (Integer) h.invoke(
                                            this,
                                            m_hashCode,
                                            null )
                            ).intValue();
                    }
                    catch( Error err ) { throw err; }
                    catch( RuntimeException rte ) { throw rte; }
                    catch( Throwable t )
                    {
                            throw new UndeclaredThrowableException( t );
                    }
            }
    
    
            public final String toString()
            {
                    try
                    {
                            return (String)
                                    h.invoke( this, m_toString, null );
                    }
                    catch( Error err ) { throw err; }
                    catch( RuntimeException rte ) { throw rte; }
                    catch( Throwable t )
                    {
                            throw new UndeclaredThrowableException( t );
                    }
            }
    
    
            public final boolean equals(Object o)
            {
                    try
                    {
                            return (
                                    (Boolean) h.invoke(
                                            this,
                                            m_equals,
                                            new Object[] { o } )
                            ).booleanValue();
                    }
                    catch( Error err ) { throw err; }
                    catch( RuntimeException rte ) { throw rte; }
                    catch( Throwable t )
                    {
                            throw new UndeclaredThrowableException( t );
                    }
            }
    
    
            public final int getPriority(Person p)
            {
                    try
                    {
                            return (
                                     (Integer) h.invoke(
                                            this,
                                            m_getPriority,
                                            new Object[] { p } )
                            ).intValue();
                    }
                    catch( Error err ) { throw err; }
                    catch( RuntimeException rte ) { throw rte; }
                    catch( Throwable t )
                    {
                            throw new UndeclaredThrowableException( t );
                    }
            }
    
    
            public final void resolve(Person p, String s)
            {
                    try
                    {
                            h.invoke(
                                    this,
                                    m_resolve,
                                    new Object[] { p, s }
                            );
                    }
                    catch( Error err ) { throw err; }
                    catch( RuntimeException rte ) { throw rte; }
                    catch( Throwable t )
                    {
                            throw new UndeclaredThrowableException( t );
                    }
            }
    }
    
      
     
    

    Additional Source Files (~ 40.6 KB ~Zip File Format)

    All Rights Reserved
    Copyright ©  2004 SYS-CON Media, Inc.
      E-mail: [email protected]

    Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.