HomeDigital EditionSys-Con RadioSearch Java Cd
Advanced Java AWT Book Reviews/Excerpts Client Server Corba Editorials Embedded Java Enterprise Java IDE's Industry Watch Integration Interviews Java Applet Java & Databases Java & Web Services Java Fundamentals Java Native Interface Java Servlets Java Beans J2ME Libraries .NET Object Orientation Observations/IMHO Product Reviews Scalability & Performance Security Server Side Source Code Straight Talking Swing Threads Using Java with others Wireless XML

As part of building the infrastructures for a large J2EE project, we've spent the last few months designing and implementing a JDO-based O/R persistency framework. This framework provides our business logic programmers with the following features: an interface-based abstract view of the data-layer with full OO semantics, zero-need knowledge of the object-to-database mapping details, "delta" support, and more. This article presents the persistency framework that we've built on top of JDO, and offers a commentary on the current advantages and shortcomings based on our experience with the JDO specification.

Choosing an Object Persistency Layer
Choosing the right object persistency layer is a major milestone in almost every J2EE project. Given the state of today's market, you should follow the "buy, don't reinvent" rule (assuming your object persistency demands haven't reached the "you must be dreaming" line). But though today's component server-side frameworks surely make the architect/developer's life simpler, there are still some issues to consider when designing your data layer, namely performance, sound API toward the business logic, and standardization. Limiting ourselves to the object-relational mapping market per se, the choice is between:

  • Using a proprietary yet powerful and proven tool from one of the veteran O/R mapping vendors such as TopLink (www.webgain.com), CocoBase (www.thoughtinc.com), or Castor (http://castor.exolab.org)
  • Relying on one of the newer yet standard object persistence Java specifications: EJB entity beans (ThoughtInc., WebGain, and others are offering entity beans-based solutions as well) or Java Data Objects (JDO)

    While all the above vendors offer (more or less) the features listed in Table 1, since we're especially zealous about standardization and vendor independence, we decided on JDO, a strategic decision. Left with choosing between entity beans and Java Data Objects (it was about a year ago that we made this decision), we went again for the second choice. Entity beans seemed rather too bulky and heavyweight for our purposes, obscuring the simple Java model while providing (unnecessarily in our case) factory-wrapped network interfaces and limited inheritance capabilities. We felt that JDO provided a simpler, cleaner API with pure OO syntax. Still another advantage of JDO lies in its relative lightness. (Note that by saying that we're certainly not trying to slight EJB entity beans. Indeed, the initial doubts about their true usefulness seem to have died out, especially after the improvements introduced in the EJB 2.0 spec.)

    Short Overview of JDO
    In the remainder of this article we'll share the experience gained while building an O/R persistency framework on top of JDO, designed and implemented in a large organization. Let's start by briefly summarizing the main features of JDO.

    The JDO specification provides a flexible method for handling persistent objects. It originates from JSR-000012, approved in July 1999 - those not yet familiar with the spec should consult the "Resources" section of this article for the appropriate URL - and was designed to integrate with existing frameworks (most notably J2EE), offering more complete transactional facilities. Indeed, session beans can directly manipulate JDO persistent instances, while entity beans can delegate business logic and persistence management to JDO classes. In our project we took the first approach.

    Working with JDO, classes that can be made persistent implement the PersistenceCapable interface, but the class author need not explicitly declare this in the code. All a developer needs to do is to provide a companion XML file, which contains metadata about the class (e.g., the names of the fields that are to be made persistent). This XML metadata is used by a "class enhancer" to generate a persistence-capable copy of the original class. (JDO ensures binary compatibility between persistence-capable classes, even if generated by different vendors' enhancers.)

    As a result, all the details of the class's actual persistence management happen behind the scenes. Each JDO instance of a persistent class has its own unique identity in the datastore and hence can be shared between different applications concurrently. When a reference is followed from one persistent class to another, the JDO implementation transparently instantiates it as necessary into the JVM. When fields of a persistent object are accessed or modified, the JDO implementation transparently marks them for change in the datastore. Almost all user-defined classes can be made persistent. While this excludes most system classes (like those in java.io or java.net), some are handled specially by a JDO implementation. For instance, the java.util.Collection interface is supported, as are immutable classes like Integer, Float, String, and Date - in addition to arrays.

    The PersistenceManager is the primary interface for JDO-aware applications. It enables users to store and fetch persistent objects from the datastore and allows users to perform synchronized transactions and queries on those objects. JDO uses a neutral query syntax similar to OQL (Object Query Language).

    Pros and Cons of JDO
    The JDO specification has honestly won its territory, being a compact, smart, and truly pure object-oriented persistency specification. However, being a newcomer also means being young, suffering from childhood fallacies, and, sadly enough, having older enemies.

    JDO touts as an advantage its back-end and datastore neutrality. This is indeed a virtue of the specification. But on the other hand it entirely ignores various important O/R issues such as object locking paradigm and O/R mapping instructions (e.g., schema structure, referential integrity, constraints, and so on). That is, JDO can be categorized as a general object persistence framework, with no special interest in being "baptized" as a full-blown O/R persistence specification. Furnishing no O/R mapping guidelines (let alone standards) and leaving this important role to the different product vendors makes the application developer's life somewhat confusing. For instance, Kodo (www.solarmetric.com), ObjectFusion (www.prismtech.com), and IntelliBO (www.signsoft.com) don't use database-referential constraints. But other vendors may choose to do so. And while IntelliBO lets you define code mappings (translation of pre-defined values into more compact database representations), it requires you to use proprietary XML tags to do it.

    Aside from its current "O/R uncertainties," there's still some uncertainty surrounding JDO. Sun, by not making a very good job of telling us when to use EJB entity beans and when to use Java Data Objects, has added to the overall confusion. Furthermore, the current JDO vendors tend to be "niche vendors" in the Java industry. A major step toward the acceptance of JDO would happen when the "big guns" like IBM, Oracle, and BEA begin to follow suit and join them - something that's not yet happened. Finally, JDO has also managed to win itself some sour enemies. The infamous duel on TheServerSide.com between JDO specification lead Craig Russel and Thought Inc. CTO Ward Mullins is just one example (see "Resources").

    Balancing the pros and cons of JDO, we believe that in spite of its current shortcomings JDO will mature into a pervasive first-class object persistency solution. An up-to-date portrait of the JDO market can be found on the JDO official site (see "Resources").

    A Simple Approach to Working with JDO
    As can be seen in our UML class diagram (see Figure 1), the superclass of the entities in our example is LifeForm. Man, Dog, and Flea are all LifeForms, where Dog also references a collection of Fleas and holds a reference to his best friend, Man. Name and Age are aggregated types in a LifeForm where an Age is always positive and a Name consists entirely of English letters. Note that Name and Age are "primitive wrappers" (of a String and Integer, respectively) supplying us with better abstraction, type safety, and basic validation in the type itself (validated during construction).

    The Java code pertaining to the UML specification is presented in Listing 1. Examining the syntax of class Age, a question arises: Is the use of an Integer necessary? Can't we simply use an int? Using the latter would, curiously enough, make it impossible to create an ageless life form, since there's no Java null value for int, thus no datastore NULL value for age. Of course, you can allocate a prespecified int value representing null (e.g., -1) but this would make writing your queries a tad peculiar.

    Writing the Java code, our next step is to run the vendor's JDO enhancer tool on the classes. Using the XML metadata file supplied in Listing 2, the enhancer will make these classes PersistenceCapable by applying direct bytecode manipulation. Some enhancers will automatically generate DD SQL statements (or directly create the database schema given a JDBC connection) as part of the enhancing process, while other vendors will politely ask you to run their schema-building tool. Notwithstanding, some vendors let you capture a preexisting schema and declare the O/R mappings yourself.

    Robust OO models should follow the real world. Thus, when our dog passes away, its fleas should gracefully pass along with it. The dog's best friend, however, won't (we hope). In JDO terms (see Listing 2) Dog and Man are First Class Objects (FCOs) - i.e., they possess an independent life cycle. On the other hand, Fleas are embedded in Dog as Second Class Objects (SCOs). SCOs have no JDO identity of their own and are stored/deleted from the datastore as part of their owning FCO. To complete the picture, note that Age and Name are embedded as SCOs in LifeForm.

    Finally, some business logic code using our data objects can be found in Listing 3. Observe that in deleteDog(Dog) it's enough to explicitly delete the Dog object. Its fleas are implicitly deleted from the datastore along with its Name and Age objects.

    Performance Analysis (Round 1)
    Albeit simple, the naive implementation results in poor runtime performance. The database tables generated by a typical schema tool are depicted in Figure 2. The exact table and field names are, not surprisingly, vendor-proprietary as are the referential constraints imposed on the tables. Nevertheless, they would follow the general structure presented here. Looking at Figure 2 we can see that fetching a Dog object results in a minimum of four fetches and three joins (assuming no fleas, no friend, but proper identification). Fetching a LifeForm translates to three fetches and two joins. This is highly inefficient.

    You're probably curious why Name and Age aren't represented as columns in LifeFormT, even though they've been declared as "embedded" SCOs. Well, in order to be consistent with the JDO specification, an O/R vendor must make the life cycle of Name and Age dependent on their owner, but there are no restrictions regarding the actual O/R mapping. All the vendors we've tested have failed to exploit the "embed" hint into a smart datastore embedding.

    It should be clear that the issue that causes a problem isn't the creation of the NameT and AgeT tables. Since a vendor's knowledge of the application semantics is limited to that defined in the metadata XML file, it would rightfully choose to generate the NameT and AgeT tables. Indeed, Name and Age could be used as an FCO or as elements of Collection in another part of the application. It's the lack of O/R embedding capabilities in cases analogous to LifeFormT that causes the problem.

    Introducing Persistency Framework Based on JDO
    Looking back at the naive approach above, you can spot a three-layered persistency architecture consisting of the JDO enhanced classes, the JDO implementation, and the datastore

    Our JDO-based persistency framework introduces two additional layers, namely the interface layer and the wrapper layer, thereby attaining the following added-value goals:

  • A: Interface-based abstract view of the data layer
  • B: Tuned O/R mapping providing superior performance
  • C: "Delta" support and Delta-driven semantics (described below)
  • D: Change tracking during the transaction

    Figure 3 shows the architecture of our framework. A vertical view of the framework reveals two major components: data entities (generated from the repository) and persistency management (an adapter, delegating the application persistency instructions to the concrete JDO implementation - e.g., persistence by reachability, JDOQL query engine, integration with EJB container, and so on). The horizontal view unveils the framework's layers: interface layer, wrapper layer, JDO layer, JDO implementation, and datastore.

    We won't go any further into the persistency management component, since it's mainly just an adapter. Readers eager to learn about persistency features are wholeheartedly directed to the JDO specification.

    Figure 4 presents the life form example, refactored to fit our framework. Relations connecting different layers are, as you can see, framework driven, while relations within a layer are OO-model driven. In fact, each layer encapsulates a "replica" of the life form model, using its own abstraction level to do so. Note that whereas Dog, Flea, and Man are explicitly represented throughout the layers, Name and Age are missing from the JDO layer. The reason for this will be made clear later.

    Interface Layer
    It's hard to overstate the importance of using a metadata repository within any large J2EE project. Our project uses one ("home made") in which we define our data entities, their fields, properties, and much more. The metadata repository is the source for our active code-generation tools, which generate our entities' interfaces and implementations and also much of the framework's generic code.

    The interface layer (see Listing 4) provides a fašade toward the business logic, allowing it to work in terms of abstract interfaces (Goal A). The clear separation between interfaces and implementing classes is not just a good programming practice, but also allows for a more loosely coupled compilation model. Moreover, as our data entities can implement multiple interfaces, we're able to provide different business logic modules with personalized local views of the data and enable the creation of generic mechanisms/algorithms operating on interfaces. For instance, one could declare Dog and Man to implement a Trainable interface. Doing that, it's possible to write a generic algorithm working in terms of trainable life forms.

    Wrapper Layer
    The wrapper layer is the middleware between the abstract interfaces presented to the business logic and the persisted objects actually stored in the database. It provides the concrete implementation of the interfaces defined in the interface layer through a symbiotic relationship with the JDO-layer classes (the enhanced classes), which act as the actual value holders. The wrapper layer's chief responsibilities focus on attaining the remaining goals, namely:

  • Concrete implementation of the interface layer, employing smart object embedding (Goal B)
  • Delta support (Goal C)
  • Change tracking during the transaction (Goal D)

    Wrapper objects make use of OO mechanisms such as reflection, dynamic invocation, dynamic proxy, lazy wrappings, and more. Their concrete code, being somewhat intricate (yet generic and actively generated), is not listed.

    Object Embedding
    In Figure 5 (a subdiagram of Figure 4) you can see the symbiotic relationship between the wrapper instances and the JDO-layer instances. The business model views a Man instance as an entity composed of two wrapper fields, Name and Age, that respectively reference a String object and an Integer object. The JDO model, being less abstract - in order to be efficiently persisted (Goal B) - has no explicit Name or Age objects. Instead, a ManJdo instance shares the same String/Integer instances with its ManImpl wrapper. The following code fragment (line numbers correspond to tag numbers in Figure 6) illustrates this sharing. The code creates a man and assigns it a name. Figure 6 illustrates the procedure, marking each element with the code line that created it. Note that, when updated, the ManImpl wrapper internally synchronizes its ManJdo accordingly:

    1 public static void main(String[] args)
    {
    2 Man man = new ManImpl();
    // implicit creation of ManJdo
    3 Name name = new NameImpl("John Smith");
    4 man.setName(name);
    // implicit update of ManJdo
    5 }
    Not surprisingly, we call this feature embedding, since the persisted ManJdo class actually embeds fields pertaining to several wrapper objects (e.g., NameImpl, AgeImpl). In fact, we use our metadata repository to define which fields in an entity are embedded and which are simply referenced. Our framework also supports (to some extent) the embedding of "complex" types, themselves containing multiple types, though the concrete implementation details would be out of scope. For instance, an Address composed of street, city, zip code, state, and country can be declared embedded in a Person. And while the business logic would work in terms of person and address interfaces, the only class to be actually persisted would be PersonJdo.

    Delta Support and Change Tracking
    Our project's functional requirements turn delta support (Goal C) and change tracking (Goal D) into important technical mechanisms. However, given that they're not the principal focal point of this article, we offer the following concise review. (A thorough investigation of these mechanisms would require an article of its own.)

    Delta Support
    Supporting remote clients that work on local copies of data entities, we let our client-side code build and transmit an "entity-delta" (the entity subgraph defined only by those fields that have actually been made dirty) to the server. The server-side business logic can now pursue "delta-driven" (i.e., "change-driven") semantics. So, for example, a typical session bean method would fetch the matching persistent entity, apply the delta, or perform some additional logic depending on the delta values, and commit (see Listing 5 for an example of this).

    As Listing 5 shows, each entity interface (implementation) has a matching - actively generated - delta interface (implementation) exposing only the getter (accessor) methods. This is both type-safe and logically firm.

    Our entities (in fact, the wrapper objects) support the creation and application of deltas by continuously keeping track of the changes made on them during the transaction. The delta-build and delta-apply code is entity-specific - i.e., it's actively generated along with each data entity. We chose this method over a general mechanism that uses reflection due to complexity, maintainability, and speed issues.

    Change Tracking
    Our server-side framework often needs to be handed the entities changed so far (created, updated, or deleted) during the transaction. It needs these in order to perform certain generic functional mechanisms. The persistency framework helps out by supplying a ChangeTracker component, which cooperates with the wrapper layer in tracking these entities (JDO provides instance callbacks that help maintain this information).

    JDO and Database Layers
    The JDO layer is made up of the classes that are actually persisted (enhanced), i.e., the JDO-suffixed classes (see Figure 4). JDO regards dates, strings, and Java primitive wrappers like Integer and Float as fundamental SCOs. Consequently, the typical vendor will choose to embed such fields in their owning entity's table. In our case, since the enhanced LifeFormJDO contains fields referencing a String and an Integer, the correspondent LifeFormT table directly contains the Name and Age columns. We can thus see that, due to the embeddings performed by their wrappers, the JDO layer classes - and so too the conforming database tables - are better suited to efficient O/R mappings. The database schema depicted in Figure 7 conforms to the JDO layer classes of the life form example.

    Performance Analysis (Round 2)
    If you compare Figure 7 with Figure 2, you can see that the table structure has improved since Round 1. It has become more compact, resulting in faster, more efficient runtime behavior. Fetching a LifeForm requires only a single fetch as opposed to three fetches plus two joins, as required by the naive implementation. When looking at the toy example, the improvements might seem minor. But the framework's smart-embedding capabilities become much more important when you're confronted with large data sets and a variety of entity and field types, as is the case with e-businesses.

    The Figures for this article have been placed together below:

    Figures 1 - 7
    Figures 1 - 7

    Conclusion
    Summing up, the JDO specification is a promising newcomer in the object persistency arena, even in light of its current shortcomings. Its pure semantics and light model seamlessly integrate within the J2EE environment, making it a viable candidate for small as well as large J2EE projects. However, on projects that involve a rich set of interfaces, classes, and types, in addition to large runtime data sets, developers should take care when building their persistency layer around a JDO product to ensure abstraction and performance.

    In this article we've tried to contribute from our experience of doing just that by presenting the JDO-based framework we've built for our own project. We demonstrated how our framework attains the important added-value goals of abstraction and performance, while providing additional capabilities such as code generation, delta-oriented business logics, and more.

    Resources

  • Java Data Objects Public Access (JDO official site contains JDO specification): http://access1.sun.com/jdo
  • Rettig, M.J., with Fowler, M. (2001). Reflection vs Code Generation: www.javaworld.com/javaworld/jw-11-2001/jw-1102-codegen_p.html
  • J2EE specification: http://java.sun.com/j2ee/
  • The infamous "Java Data Objects vs Entity Beans" duel: www.theserverside.com/discussion/thread.jsp?thread_id=771

    Author Bios
    Yaron Telem is an IT/J2EE architect at Mamdas, involved in the construction of large enterprise systems. Yaron received BS and MS degrees in computer science and mathematics from Tel-Aviv University. [email protected]

    Shay Litvak, also an IT/J2EE architect at Mamdas working on large enterprise systems, holds a BS in computer science from the Israel Institute of Technology. [email protected]

    	
    
    
    Listing 1: LifeForm entities - naive implementation
    
    
    // LifeForm.java
    public abstract class LifeForm {
      // private members
      private Age _age;
      private Name _name;
    
    
      // methods
      public Name getName() {
        return _name;
      }
      public void setName(Name name) {
        _name = name;
      }
      public Age getAge() {
        return _age;
      }
      public void setAge(Age age) {
        _age = age;
      }
    }
    
    
    // Man.java
    public class Man extends LifeForm {
    }
    
    
    // Flea.java
    public class Flea extends LifeForm{
    }
    
    
    // Dog.java
    public class Dog extends LifeForm {
      // private members
      private Collection _fleas = new Vector();
      // Conveniently enough, we initialize the Fleas collection in the Dog
      // constructor, thus a Dog with no Fleas (assuming one exists?)                                     will hold
      // an empty Collection
    
    
      private Man _bestFriend;
    
    
      // methods
      public Collection getFleas() {
        return _fleas;
      }
      public Man getBestFriend() {
        return _bestFriend;
      }
      public void setBestFriend(Man bestFriend) {
        _bestFriend = bestFriend;
      }
    }
    
    
    // Name.java
    public class Name {
      // private members
      private String _name;
    
    
      // constructor
      public Name(String name) {
        if (!validate(name)) {
          throw new IllegalArgumentException(
            "Invalid value: name must contain only letters");
        }
       _name = name;
      }
    
    
      // methods
      public String stringValue() {
        return _name;
      }
      private boolean validate(String name) {
        for (int i = 0 ; i < name.length() ; i++) {
          if (!Character.isLetter(name.charAt(i)))
            return false;
        }
        return true;
      }
    }
    
    
    // Age.java
    public class Age {
      // private members
      private Integer _age;
    
    
      // constructor
      public Age(Integer age) {
        if (age.intValue() < 0) {
          throw new IllegalArgumentException(
            "Invalid value: age must be a positive integer");
        }
        _age = age;
      }
    
    
      // methods
      public int intValue() {
        return _age.intValue();
      }
    }
    
    
    
    Listing 2: XML file used by JDO enhancer
    
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE jdo SYSTEM "jdo.dtd">
    <jdo>
      <package name="lifeforms">
        <class name="Name"/>
        <class name="Age"/>
        <class name="LifeForm">
          <field name="_name" embedded="true"/>
          <field name="_age" embedded="true"/>
        </class>
        <class name="Man" persistence-capable-superclass="LifeForm"/>
        <class name="Flea" persistence-capable-superclass="LifeForm"/>
        <class name="Dog"  persistence-capable-superclass="LifeForm">
          <field name="_fleas">
            <collection element-type="Flea" embedded-element="true"/>
          </field>
        </class>
      </package>
    </jdo>
    
    
    
    Listing 3: Session Bean working on life forms
    
    public class DogServices implements SessionBean {
      // A CMT stateless session bean
    
    
      public void createDog(Age age, Name name,
                            Collection fleas, Man bestFriend) {
        PersistenceMnager pm = JDOFactory.getPersistenceManager();
        Dog dog = new Dog();
        dog.setAge(age);
        dog.setName(name);
        dog.getFleas().addAll(fleas);
        dog.setBestFriend(bestFriend);
        pm.makePersistent(dog);
      }
    
    
      public void addFlea(Object dogId, Flea flea) {
        PersistenceMnager pm = JDOFactory.getPersistenceManager();
        Dog dog = pm.getObjectById(dogId);
        dog.getFleas().add(flea);
      }
    
    
      public void deleteDog(Dog dog) {
        PersistenceMnager pm = JDOFactory.getPersistenceManager();
        pm.deletePersistent(dog);
      }
    
    
      // ...
    }
    
    
    Listing 4: LifeForm interfaces
    
    
    // LifeForm.java
    public interface LifeForm {
      public Age getAge();
      public void setAge(Age age);
    
    
      public Name getName();
      public void setName(Name name);
    }
    
    
    // Age.java
    public interface Age {
      Integer integerValue();
    }
    
    
    // Name.java
    public interface Name {
      String stringValue();
    }
    
    
    // Flea.java
    public interface Flea extends LifeForm{
    }
    
    
    // Man.java
    public interface Man extends LifeForm {
    }
    
    
    // Dog.java
    public interface Dog extends LifeForm {
      public Collection getFleas();
      public Man getBestFriend();
      public void setBestFriend(Man man);
    }
    
    
    
    Listing 5: Sample code, using Delta
    
    // Client side code
    public void someFunc(Dog dog) {
      ...
      dog.setAge(new AgeImpl(7));
      dog.getFleas().remove(dog.getFleas().iterator().next());
      DogServices services = ServiceFactory.getDogServices();
      // send dog's delta to server
      services.updateDog(dog.buildDelta());
      ...
    }
    
    
    // Server side code
    public void updateDog(DogDelta dd) {
      ...
      PersistenceMnager pm = JDOFactory.getPersistenceManager();
      Object oid = dd.getObjectId();
      Dog dog = pm.getObjectById(oid);
      if (dd.getAge() != null)
    Dog.applyDelta(dd);
      else ...
      ...
    }
    
      
     
    

    All Rights Reserved
    Copyright ©  2004 SYS-CON Media, Inc.
      E-mail: [email protected]

    Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.