To the uninitiated, those unversed in the art of computing, 'building to scale' sounds like something architects do when they take one of those models to a board meeting for approval. It sounds a million miles away from what we understand in computing.

Just for the record what do we mean when we talk about "building to scale"? We mean that the computer system (the software and hardware) performs predictably under a documented range of conditions. Oddly enough it isn't any different for the architect with the model or an engineer creating some new valve for a process control plant. Good practice and common sense are in short supply and not often used. Building something simple and making it scalable requires an understanding of the dimensions of the problem. So a search algorithm that's bounded may be best implemented by a simple bitmap, whereas one that's not bounded can't use this technique. Imagine trying to deliver a complex system that scales. The complexity itself makes it more difficult for many people to see the shape and dimensions of the problem they're trying to solve. Web services enables us to focus on the problem by reducing the noise and clutter through the adoption of standards. In this article we'll be applying good practice and common sense to expose some of the design patterns that ensure Web services can be scaled to deliver predictable performance over a range of conditions. The use of the Java Messaging Service API (JMS) and the emerging Java Caching API (JCACHE) are complementary to Web services and form part and parcel of a broad architectural solution space.

The Basic Technologies
Let's delve into a bit of history. But before we do, it's worth stating why history is so important. From the early days of computational science the progress that's been made has been firmly predicated on all that happened before. The saying, "No experience is a bad experience; it's at least a good experience of a bad experience" really tells us to value those things that clearly work and to understand those that don't. The things that don't work often tell us more than those that do. So history is important because it offers us experience and allows us to draw on that experience to forge new solutions, which are often previous experiences recast in a different setting.

Distributed systems are what Web services are all about. Fundamentally they're about passing information from one system to another to achieve some goal. What they offer is an open standards-based canvas upon which to paint a solution. Web services deal with standards to describe what service is offered is how to connect to that service, and what protocol is used to enact some business goal. The fundamental technologies behind Web services are the Simple Object Access Protocol (SOAP), the Web Service Description Language (WSDL), and the Universal Description and DIscovery repository (UDDI).

SOAP
SOAP is an XML schema for encoding function calls and their parameters. It has its genesis, unsurprisingly, in XMLRPC; which is an XML encoding for remote procedure calls. SOAP is at the core of Web services technology. The SOAP protocol is synchronous - although an asynchronous mechanism can be built on top, similar to those in the 1980s when X-Windows and call-backs were all the rage. Using a CORBA analogy, SOAP is akin to an XML encoding of the IIOP protocol.

WSDL
WSDL is an XML encoding for the description of Web services. A WSDL description includes information about what protocols (HTTP, HTTPS, JMS, JAXM, and so on) a Web service is prepared to accept. WSDL also records details of the Web service itself, such as the SOAP requests it offers. In effect it is the IDL of Web services.

UDDI
A UDDI repository is a repository that contains information about what a Web service provides. UDDI repositories can be considered Web services themselves, as they offer SOAP interfaces. A UDDI repository acts as a broker of information about various Web services. Users of the repository run queries against it to locate trading partners or Web services of interest. To use the CORBA analogy again, UDDI is akin to the interface repository.

The Problem Domain
To understand how to scale a Web service we need to be fully cognizant of what it is trying to achieve. In its raw form its aim is, as we stated before, to support the execution of a business goal. That itself, though, isn't enough to understand the shape of the problem - and that shape is the key to scalability. What distinguishes one Web service from another is the interaction model it's required to support. Some Web services may be read-only - that is, they provide information to the requester and don't actively participate in a business transaction. On the other hand, a Web service may actually be running a business transaction for you. It might be taking in bids and offers, matching them, and reporting back the trade confirmations as they happen.

In the case of a read-only Web service, it might be providing price changes from a wire feed for financial service trading. On the other hand, it might be providing a calendar service to check the working days between two dates, the result of which is used to compute interest accrual. These two examples illustrate different interactions. The demand for pricing information is huge whereas the demand for calendar information is much smaller - because many requesters will monitor price changes, but only when a trade is being executed will a request be sent to the calendar.

In the case of a transactional Web service it might provide bids and offers for something quite simple in which the response (the confirmation) is a single response. In a more complex scenario, several partial confirmations might well occur, all of which make up the business transaction. These examples also exhibit different characteristics. The former only requires a request/ response pair since the order is the request and the confirmation is the response. The latter requires state information to be retained by the requester and Web service. The order is still a request, but now the responses need to be coordinated and matched so the correct confirmations are matched with the correct order.

Architectural Patterns
What can JMS and JCACHE do for us? They can help us, as architects, to connect applications pretty effectively. They can even help us reuse legacy-messaging investments through a more open JMS framework approach and so provide standards-based multiplug asynchronous connectivity, a key component in addressing high fan-out delivery of information and asynchronous interaction. SOAP, together with JMS, gives us a standard synchronous functional protocol over an asynchronous communication mechanism for Web services. JCACHE provides a developer with a set of interfaces that, coupled with JMS, provides the bedrock for elaborating standard architectural patterns that are geared to alleviating the problems of server bottlenecks.

If we look at the technologies described so far, they fall into two camps: there are those based on a (request/response) synchronous paradigm, and those that fall into a (publish/subscribe) asynchronous paradigm. Messaging is generally asynchronous. It has to be to promote fire-and-forget messaging and so support a decoupled solution. But applications are generally built using synchronous mechanisms. The challenge is how to combine these paradigms into an architecture that engenders scalability. In order to do this we need to understand the problems. In part they're a consequence of history, which is why history is important.

The Role of MOM and JMS
The Java Messaging Service API was born in late 1997 and the first commercial implementation (from SpiritSoft) came out during Easter of 1998. Since then a wider market has grown up around the standard as Fiorano and Progress (now Sonic Software) entered the marketplace. IBM offered a JMS interface to MQSeries shortly after and finally, late in 2001, Tibco Software joined the club.

The adoption of JMS as the first vendor-independent MOM standard testifies to the power of asynchronous messaging as a fundamental technology for delivering distributed solutions. The move toward message-driven beans and the mandatory status of JMS in J2EE 1.3 is a huge step forward - the ability of many JMS products to offer a scalable, load-balanced solution to the distribution, marshalling, and management of requests to an application server supports this. But it's only the start. Life gets much more interesting when you start to apply MOM technology through a JMS standard to caching and so start to understand the basic architectural patterns that it promotes.

What Does JMS Do For Us?
So what is JMS and what does it do for us? The Java Messaging Service is an API, an application programming interface. It's defined as a set of interfaces for producing and consuming messages based on both publish/subscribe notification and point-to-point queuing models. The JMS specification enables subscribers or receivers to specify a quality of service for messages, so that messages can be reliably delivered or guaranteed. The specification also describes some semantics that JMS vendors must conform to. Amongst these are semantics that dictate how a message is received. Thus a queue allows one receiver and one receiver only to receive a particular message (i.e., it's delivered once and only once), while many subscribers to a topic might receive the same message. What JMS does for Web services is to provide the basic asynchronous delivery mechanism for high fan-out data (like prices and calendar information) as well as to provide the transactional secure delivery of orders and confirmations.

JCache and the Role of Caching
Although JCACHE (JSR107) is still making its way through the Java Community Process, several vendors have announced or will announce caching products based on it. What JCACHE will provide is a standard set of APIs and some semantics that are the basis for most caching behavior. According to its functional requirements, JCACHE "allows applications to share objects across requests and across users and coordinates the life cycle of the objects across processes."

JCACHE provides a fairly rich set of APIs, enabling control to be exercised over cache loading, cache eviction, and cache validation. It deals with basic caching patterns in which caches can be fed by other caches. A consequence of this is that it allows implementers to take advantage of distribution technology such as that offered by JMS as well as any other distribution technology deemed appropriate.

According to JCACHE, a cache is "specific to each process." This seemingly simple statement has profound implications and is one of the reasons why the proposed solution is flexible. Each process can implement its own mechanisms for cache loading and replacement or share the necessary functionality as required. When an object is invalidated or updated in one process, the name and associated information can be broadcast to all other instances of the cache; this is where JMS comes to the fore, by acting as the standard distribution interface for a cache. And this in turn allows the entire system of processes to stay synchronized without the overhead of centralized control.

The long and the short of it is that JCACHE is a smart approach for read-only data that's essential to both the browsing nature of passive business transaction involvement, and to the repeatability of business transactions and active involvement of read-only data.

Shape
Understanding a problem is all about understanding its form or shape. When we fully appreciate the shape of a problem, we can find a solution that truly fits it. It's the difference between effecting a cure and merely relieving symptoms. All of us have experienced being pressed for time on a software project. A bug has been holding us up for days so we decide to change this or that based on intuition... and the problem goes away. We may well have fixed the problem but, because we don't understand the shape, we can't be sure. On the other hand a bug holds us up for days and suddenly, perhaps because we were not looking too hard, we have further insight and understand the shape of the problem. It's what John Bentley calls "an aha! moment," when we truly understand the nature of a problem and so the solution hits us in the face.

The problem with the synchronous and asynchronous paradigms is that enterprises are neither totally synchronous nor totally asynchronous. They reflect both paradigms in specific ways that underpin their business model. The shape is there and the architecture needs to reflect that shape. In this way we can build systems that reflect and support the business rather than the business being a reflection of the computer systems that support it. A synchronous solution would necessitate all components of an enterprise to participate in some form of distributed transaction. This is a legacy from history, encompassed by the old mantra that "all solutions start with a database," whereas solutions are bounded by a problem.

Server Bottlenecks
Request/response systems are, by their very nature, pull-based and are passive. The applications pull data from the server. Many of these applications manipulate the same logical data many times over within the same time period and across many transactions. As the client population increases, so does the number of requests to the application or DBMS server. The server becomes the bottleneck, swamped by too many spurious requests. Application server, DBMS, and hardware vendors suggest using replication to alleviate the problem and thus reduce the client-to-server ratio. But this just relieves the symptoms and does nothing toward effecting a cure. As the enterprise grows, so the need for more replication and more servers will grow - and so the costs will escalate.

Web services don't fare any better, because they're synchronous. They do help deliver systems, getting you there fast, because they offer standard protocols and standard interfaces, but they do nothing to alleviate the inevitable problems of server bottlenecks in large scale Web services within and between enterprises. The more complex the interaction model, the more obvious this becomes.

Transactional Islands
The first pattern is a reflection of how many organizations can be broken down into logical units. In banking this was always based on splitting the enterprise into front, middle, and back office functions. Each functional unit owns its own transactional space. Each transaction space is then connected by a JMS-bus. Depending on the semantics of the messages published on the bus, different qualities of service (QoS) may be used. Thus for an order we would want to ensure it is always delivered, but for a price change we might be willing to accept the last price published (see Figure 1). In this way each functional unit of the business is responsible for its own scalability issues. The use of JMS enables them to proceed at the rate they need to. The only functional unit that dictates overall performance is the order entry system, and that's now free to enter orders without having to worry about the other functional units of the enterprise. The flow of messages (or events) can then be controlled as a system-to-system workflow and so provide a better controlling and monitoring environment.

Figure  1:

Simple Caching
The next architectural nuance is to recognize that many systems spend a high pro-portion of their time reading and then have flurries of activity as they transact and data is changed, added or deleted. During the mid 1990s when financial services took up the challenge of decoupled architectures, many also looked to use MOM to provide a distribution mechanism for data. It was a natural step to try to cache the data nearer the application that needed it and so reduce even further the server bottlenecks.

This next pattern simply recognizes that a typical layer between a consumer of messages from a JMS bus and the bus itself is a read-only cache (see Figure 2). The read-only nature of a cache is important here because the cache is decoupled from the underlying data source. Updating the objects doesn't change the source; updatability is a more advanced pattern. What this pattern requires is a container that understands what we want, how to get it and how to propagate the changes that it receives. This is the basis for traditional caching behavior.

Figure  2:

Once we take the view that the applications can have a cache of information from the application server or DBMS, and that the cache can be up to date or active, then the application no longer needs to pull data that it has already requested. Applications in an active architecture pull once for any given object (or objects) and the server simply pushes thereafter. This seemingly simple change has a major impact on performance and scalability and has a major impact on the design of the applications. Applications now need to become event driven and so, themselves, become active.

Hierarchical Caching
A more complex form of caching can be derived from the simple form above. If caches can be connected in a hierarchy with each cache subscribing to a JMS bus (see Figure 3), we can use the natural hierarchical shape of an enterprise and some counterintuitive logic to model a hierarchical caching solution across n-tier architecture.

Figure  3:

In an n-tier architecture, the nearer to the data source you are, the finer the granularity you need to identify the data you want. The further away you are, the more localized the data becomes. Consider this: as all the requests for data converge onto a data source, the data source needs to identify what each requester needs. But further out toward the edge, the requesters already know what they need, be it through object identity or through predicate definition. Thus the counterintuitive principle is to have fine grain subscription closer to the data source and coarse grain subscription further away from the source. The fact that JMS based solutions offer topic hierarchies makes them highly suited to this form of traffic shaping and complementary hierarchical caching.

Active Queries
The more an active cache is decoupled from a server, the more scalable the solution becomes. Active queries provide a flexible way of describing what we want in our cache, through the use of predicates. This changes the way we build applications. In this fashion, subscription to a cache, and the cache's subscription to a data source, are based on these predicates. Notification of change is based on specific objects or entities, and the results set that this information is moved into (see Figure 4).

Figure  4:

Updatable Caches
Decoupled caches don't lend themselves to conventional methods of updating information. The very fact that they're decoupled means that the transaction consistency of the cache is sacrificed. Conflicts that arise from inconsistency need to be dealt with as exceptions in the business flow that underpins an enterprise. This is what many financial institutions do anyway and is what transactional islands recognize too. What we can do is provide patterns that ensure that the window of opportunity in which these problems may arise is as small as possible. In this way we can create an architecture in which exceptions are exceptional and most of the investment in software deals with the bread and butter.

How do we close this window? There are two basic mechanisms. The first uses the concept of a proxy to provide a synchronous update mechanism. If the business object or entity is a proxy within the cache, and that proxy has all it needs to update the information source, then the proxy can update the information source directly (in standard RMI type fashion). When the synchronous update call returns, the proxy can publish the update through the cache to other interested parties (see Figure 5).

Figure  5:

This pattern is fine for low-frequency updates. The synchronous calls to the information source guarantee that all updates are synchronized and so are transactionally safe, but the price to be paid is the latency of the synchronous call and the attendant server bottleneck. While the read frequency remains high and the update frequency low, this pattern has considerable merit.

The second mechanism uses a totally decoupled cache for read and update. When caches are loaded, the information source is no longer the source but is just treated as a subscriber to change. Changes occur in the network by publishing the changes onto the network but the information source uses a durable subscription to ensure that it updates the information source. This does carry with it a need to perform conflict resolution when updates to the same object overlap; this can be easily detected by carrying the pre and post images of the object - but the conflict resolution is application defined.

This pattern works for both higher read and high update frequencies. But it does incur the cost of conflict resolution. As long as the number of conflicts is low, the conflicts can be costed as part of the overall exception handling that systems generally have to perform. If the update profile is segmented so that different parts of an organization update different data, then the pattern is very effective in increasing overall throughput in the face of change.

Figure  6:

Moving Up the Stack
How does all this fit into the real world? All the patterns mentioned, from transaction islands to decoupled updatable caches, have relevance in today's Web applications, as well as having relevance to the Web services of tomorrow. The inability of Web servers to cache information effectively suggests that caching based on a JMS distribution model is highly applicable. The same would be true of a SOAP-based Web service or for any internal application in which reading is a large part of what the app is used for.

What JMS has done is to trigger the debate and move us up the stack. We're now able to identify patterns and so expose further APIs and develop the related JMS tools market. If you like, it has moved the debate beyond JMS in the same way that the introduction of SQL gave rise to database tools in the 1980s.

In Search of Greater Flexibility
The marriage of JMS and JCACHE and the use of topics and selectors provides an open standards-based solution to a common architectural problem. What it does is allow us to abstract out some of the architectural patterns that underpin the construction of large-scale distributed systems. For example, the JMS selectors are what enables the messaging and caching to deliver what we need based on a predicate. It's simply the ability to distribute change events, apply a condition to the change, decide what to change in the cache, and then inform an application of the change to the cache. It's a case of managing events, conditions, and then actions - and applying it to JMS and JCACHE. Flexibility, the provision of runtime change to such approaches, is what we all seek.

Summary
What we have shown is that the coupling of JMS and JCACHE offers a solution to the very real problems inherent in delivering scalable Web services. The patterns we've described are a testament to this. Furthermore we've shown that such solutions can be flexible by adding a RuleML ingredient to the architecture to provide personalized caching, which moves us closer to reflecting the changing needs of users and so keeping them at the core of our architecture.

Author Bios
Steve Ross-Talbot is CTO of SpiritSoft, Inc. steve.ross-talbot@spirit-soft.com

Gary Brown is Advanced Technology Architect, SpiritSoft, Inc. gary.brown@spirit-soft.com