HomeDigital EditionSys-Con RadioSearch Java Cd
Advanced Java AWT Book Reviews/Excerpts Client Server Corba Editorials Embedded Java Enterprise Java IDE's Industry Watch Integration Interviews Java Applet Java & Databases Java & Web Services Java Fundamentals Java Native Interface Java Servlets Java Beans J2ME Libraries .NET Object Orientation Observations/IMHO Product Reviews Scalability & Performance Security Server Side Source Code Straight Talking Swing Threads Using Java with others Wireless XML

One minute of system downtime can cost an organization anywhere from $2,500 to $10,000 per minute. Using that metric, even 99.9 data availability can cost a company $5 million a year.
The Standish Group

Internet commerce demands 24x7 Web site availability. We need systems that provide zero planned downtime, so during routine maintenance, system upgrades, etc., the system can still serve client requests. We also need systems with zero unplanned downtime one application server crashing or one computer accidentally turned off should not stop the system as a whole from working. We need to be able to handle far more concurrent requests than one server can handle. To do all this we need clustering.

These objectives cannot be met by strict adherence to J2EE APIs. It's not something the application does, but something the container or infrastructure provides.

A cluster is a set of server nodes that cooperates to provide a more scalable and fault-tolerant server infrastructure for stateful and stateless components. To external clients, a cluster appears as a single server that services requests with a single point of entry.

In this article we'll discuss clustering with regard to the J2EE platform, focusing on the Web tier. We'll discuss the various tiers of clustering, the mechanisms, and their performance/scalability ramifications, and illustrate how to design highly available, scalable, fault-tolerant, and performant systems.

Tiers of J2EE Clustering
Clustering is not limited to any single tier. It can be achieved in every tier cache, Web (servlet, JSP), EJB, and even at the database (see Figure 1).

Figure 1

Caches sit in front of Web servers caching Web content and acting as virtual servers. They reduce the response time, offload the back-end servers, and help the Web tier handle the huge volume of client requests, thus increasing scalability.

The Web tier sits behind the cache layer. In a typical J2EE solution, the Web tier is made up of servlets and JSPs that are responsible for dynamic content generation.

Beyond the Web server tier is the EJB tier that runs stateless session beans, stateful session beans, and entity beans.

Even the last layer of the database can now be clustered using a database cluster, as with Oracle9i Real Application Clusters.

Aspects/Facets of J2EE Clustering
The two major aspects of clustering are:

  1. Load balancing or front ending
  2. Fault tolerance or reliability
Load Balancing or Front Ending
The idea behind load balancing is to distribute the load (from client requests) to multiple back ends. This enables a cluster (collection of cooperating servers) to handle more requests/clients than an individual server, thus providing scalability. A load balancer sits in front of the server nodes and receives requests, then redirects these client requests to the various back ends.
  • This distribution of the workload boosts performance (shorter response time).
  • Individual servers in the cluster can go offline for maintenance without causing the system to halt or fail (zero planned downtime).
  • This provides scalability because the cluster can handle more clients than any individual server.

    Group Membership
    Server nodes need to be registered with the load balancer to have requests routed to them from the load balancer (see Figure 2). These registrations can happen statically or dynamically.

  • In static registration the load balancer is configured statically beforehand with information about its target list of servers to register. To change this list, the load balancer must be restarted.
  • In dynamic registration, server nodes register dynamically at runtime with their target load balancer. This enables you to add new servers to the mix dynamically without bringing down the load balancer.

    Figure 2

    Usually the dynamic registration is done using bidirectional notification at startup.

    Server nodes notify the load balancer at startup and register with the target front end. The load balancer in turn lets all potential servers know when it comes up so that server nodes seeking to register with it can do so. The load balancer is the single point of entry to a cluster and this becomes a single point of failure.

    To add reliability to the load balancer and ensure availability, a process monitor daemon can ping the load balancer; if the load balancer does not respond, the daemon can restart it. The load balancer can be behind a hardware load balancer that's using a virtual IP. To prevent a restart from interrupting the request routing, it's important for the load balancer to be stateless. In other words, all routing information for the session is transferred as a cookie with each request.

    Load Balancing Strategies

  • Random: As the name indicates, the requests get dispatched to back ends in a random manner.
  • Round-robin: Refers to a distribution on a round-robin basis. This can result in a bad situation: if we have two servers and our request pattern is a big heavy workload request followed by a light workload request. In a round-robin situation, all heavy requests go to one back end and all lightweight requests go to another.
  • Weighted: Any of the algorithms can have weights associated with them. For example, I want twice as many requests dispatched to server A as to server B. This implies a weight of 2/3 : 1/3 for A:B.
  • Equal request: Load balancer maintains a counter for each node ensuring all nodes receive an equal number of requests (or weighted).
  • Equal client: Requests get dispatched based on the client requesting it. This proves useful for stateful sessions. The load balancer sticks a session to a server node, which means all requests from a given session will be handled by the same server node (in failover cases this will hold until the server fails).
  • Equal workload: The load balancer keeps track of the workload handled by each server node by polling them at regular intervals, and weights can be adjusted for distribution. So workload statistics, if fed to an adaptive weighted system on a load balancer, can load balance an equal workload.

    Even though this may seem the ideal way of load balancing, the overhead of calculating the workload results in performance hits that may make it undesirable.

    Failover and Reliability
    When an application server in a cluster fails to serve client requests, the load balancer reroutes the requests to its peer(s). This is termed as failover and provides the basis for fault tolerance and reliability in the cluster. Each server node of a cluster names one of its peer as its secondary server node. If a server node fails, the load balancer finds its secondary/secondaries and reroutes the requests.

    The advantages of failover are:

    • User is immune to system crashes
    • Reliability
    The disadvantages of failover are:
    • Redundancy
    • Performance hit due to replication
    Failover is achieved through replication. Replication is done using either:
    • Point-to-point using direct socket communication
    • Multicast the message to the entire group
    Stateless session failover is tantamount to a simple request redirection. Stateful session failover on the other hand requires both request redirection and session replication.

    Replication is a two-step process of transmission and consumption. Transmission from the sending server VM occurs according to some strategy while message consumption on the receiving VM occurs lazily. The bytestream received is not deserialized until needed. Because it's not materialized into the HttpSession, stateful sessions tend to be sticky. We see that the entire system is an n serialize (serialization happens many times on the transmitting side), or a 0 or 1 deserialize (message is consumed lazily only if needed, i.e., in case of a failover) system. Stickiness of sessions is done using the jsession_id cookie.

    Replication Methods
    Replication can happen over different transports.

    Point-to-point transport replication is typically done for a single primary and a single secondary. Here you typically use a TCP socket connection between the primary and a secondary server. In case of a failover, the client can actually failover to any node in the cluster. Thereafter the node receiving the request can request the session from the cluster and then the secondary will send the session and continue to serve as the secondary for the new primary.

    An alternative to point-to-point replication is to replicate using UDP multicast. However, the primary concern with it is the reliability of the message delivery. TCP packets have an acknowledgment built into the system but, on top of UDP, application server vendors often build a NAK (negative acknowledgment). Each packet can be sequenced with an ID, and if the receiving VM finds that it received packets 1, 2, 3, and 5, then it knows it missed packet 4. UDP-based systems can have multiple secondaries or an island (as in Oracle9i Application Server) unlike point-to-point systems. Since UDP-based systems are not waiting for acknowledgment, the replication can happen asynchronously and so such a system can be more performant.

    In this system, state replication happens within an island and load balancing happens within and across islands. However, once a stateful session is bound to an island, it should continue to work against the island. A stateful session once bound to a back end sets cookies that identify the back end as well as the island to which it was bound.

    Web Application Failover
    Web application failover requires that:

  • The application is marked distributable.
  • Objects in the HttpSession are serializable or remote.
  • Instance/static variables should not store state. Such variables do not make sense when we failover to a different Java Virtual Machine (server node).
  • ServletContext should not store state.
  • EJBs/database should store long-lived application state.

    public void doGet (...) ... {

    HttpSession session = request.getSession(true);
    Cart cart = (Cart)session.getAttribute("cart");
    if (cart == null) {
    cart = new Cart();
    session.setAttribute("mycart", cart);
    } ....

    In the Web clustering replication most application server vendors currently replicate when setAttribute/removeAttribute is called on HttpSession.

    This implies that if we do a setAttribute for cart, then update it and don't setAttribute again, the secondary/secondaries won't have the updated state.

    There are times when you may want to use load balancing without failover:

  • In pure performance when we're not prepared to take the hit of replication
  • When we have nonserializable objects in HttpSession
  • When the requests are stateless

    Common Pitfalls
    In general, failover is transparent to the end user except in some cases that I'll mention later.

    Static variables in a servlet should not be used, but in most servlet containers users can get away with using static variables. However, in distributable applications, static variables will be reset in the new/failed-over VM and will therefore be different from the values in the original VM.

    Most people assume that the init method of a servlet is executed only once. This is true in one VM; however, in a failover scenario the init method of a servlet executes once per failover (once on each VM).

    Remember that in case of a failover, we go back to the last state in which the primary did a setAttribute for each attribute. If after certain updates a setAttribute was not done, we won't find those updates on the secondary/secondaries.

    Performance Optimizations
    The price of plurality lies in replication and its related network overhead.

    The system is not n serialize n deserialize. Almost all clustering solutions use n serialize 1 deserialize, so anything to speed up serialization helps.

    In the course of serialization a serialVersionUID computes the version of an object and this can be a time-consuming operation that's being undertaken for every serializationSerialVersionUID. This is a private static long defined in a class.

    Preferably, provide a readObject/writeObject for the class being serialized. This can save on time spent in reflection to get the attributes/state variables to serialize. When writing a point object the writeObject can just write the values of x and y; for the default serialization mechanisms to serialize an object of type point, it must reflect on the class Point and find out that x and y are the attributes to serialize followed by reflecting to obtain the values of x and y.

    If a network is the constraining factor, it may be useful to find the difference between previously serialized streams and transmit it. However, this typically has the overhead of computing the delta or difference between the streams.


  • Multisect the problem space so that the secondary/secondaries do not have common causes of failure.
  • Use as much redundancy as possible.
  • Eliminate common causes of failure, such as a common source of power.
  • For a session, preferably pick a secondary so that both primary and secondary are not on the same machine.

    In this article, we looked into clustering, focusing on the Web tier. We also discussed the two major aspects of clustering: load balancing and failover. To provide high availability, load balancing, scalability, and reliability, it's important to choose the correct application server infrastructure and configure it appropriately. An insight into the underlying mechanisms can prove invaluable. Application servers, such as BEA WebLogic and Oracle9i, provide clustering. In subsequent articles we'll discuss clustering in the EJB tier.

    Author Bios
    Ashok Banerjee is a technical lead at BEA working on WebLogic Server. Prior to this he worked at Oracle Corporation for four years on Oracle9iAS, and at IBM for two years. A regular presenter at JavaOne and other conferences, Ashok also teaches Java design patterns at the San Jose State University. [email protected]

    Ganesh Kondal is a senior engineer at Modulant Solutions. He has been working on distributed computing and Java for the past four years and is now moving into Web services and Linux. His interests are distributed computing, Web services, and design patterns. Ganesh is currently working on a master's degree in science and engineering at San Jose State University. [email protected]

    Sunil Kunisetty is a technical lead and lead developer for Oracle Web Services. He's the expert member of JSR 109 (Enterprise Web Services for J2EE) and WS-I Basic Profile communities from Oracle. He has over six years of distributed computing experience with an emphasis on Java. Sunil is a frequent speaker at Oracle OpenWorld, JavaOne, and Oracle iDevelop conferences. [email protected]

    All Rights Reserved
    Copyright ©  2004 SYS-CON Media, Inc.
      E-mail: [email protected]

    Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.