As Web services technology begins to impact business process management (BPM) and application-to-application integration, a question arises: "What business and technical challenges will we face applying Web services technology to BPM and application integration?" This article presents the seven dominant principles of good Web services design. The principles are based on 15 years of software technology evolution, combined with practical experience from today's deployed Web services applications.

The technical motivation for Web services is shown in Figure 1. In the '90s, the Internet enabled a network of browsers to access a flow of content. Today, the Internet enables a network of applications to access a flow of business transactions and processes. Web services standards promise to enable that shift.

The business imperative is also clear: yesterday's technological approaches to integration have failed - they are too expensive, too difficult to implement, and too vendor-specific.

So let's address these technical and business challenges proactively by looking at a series of principles that can guide your enterprise toward a better approach to integration.

Principle 1: Plan for a Virtual Enterprise
Industry pundits bifurcate the integration space into internal (enterprise application integration, or EAI) and external (business-to-business, or B2B) application integration. In practice, this distinction is useless. The operations of corporations are becoming virtual - within an enterprise, functions like customer relationship management, sales force automation, and billing are outsourced to third parties. New insight into the operations of partners makes you more knowledgeable and efficient. The notion of a purely internal application - one that doesn't traverse firewalls or public networks - is disappearing.

The distinction between internal and external applications is hazy from a technology point of view as well. Both require process-flow management, packaged application adaptors, messaging, and security.

So, forget the pundits and think in terms of business problems, not market classifications.

Principle 2: Be Aware of the Hidden Integration Foil: Heterogeneity
The hidden foil to integration is heterogeneity, which rears its ugly head during implementation. Why? In any integration, the applications involved typically:

• Have different representations of the same data
• Have different notions of similar business processes
• Are written in multiple programming languages
• Execute on more than one operating system
• Use more than one communication medium
• Were developed by teams with varying degrees of skill
• Use proprietary integration technologies

Some might say the answer lies in the emerging platforms for e-business: .NET and J2EE. Unfortunately, as we can see from the points of heterogeneity above, these environments don't come close to addressing our technology diversity. We don't really expect Microsoft software to run on IBM's OS/390 systems, Visual Basic developers to hack the Linux kernel, or COBOL applications to be rewritten in Java. We have no choice but to embrace our diversity.

Simple awareness of the depth and breadth of heterogeneity issues is what Principle 2 is all about. The remaining principles work together to mitigate the heterogeneity problem.

Principle 3: Describe Business Processes Succinctly
When compared to most applications, which affect subsets of an enterprise or pieces of a supply chain, the impact of BPM is pervasive. It spans organizations within an enterprise, impacts scores of trading partners and consumers, and employs a vast array of technologies.

Figure 2 describes a way to think about business processes as a combination of real-world and technology models:

• The process model describes the steps that occur in the real world (e.g., the trucks that deliver goods from point A to point B, the schedules and locations of the drivers).
• Workflow models describe the technology interactions that support, interact with, or implement the real-world process model (e.g., system X sends request for purchase order to system Y).

The workflow models themselves should also be expressed succinctly, although English may not be the best language for expression. Industry-standard languages and modeling tools are often better choices to express these models, but it's still critical for everyone to understand and agree that they're accurate.

Effective organizations follow Principle 3 and begin with clear, well-understood business process models.

Principle 4: Leverage Standards
The software industry's solution to the heterogeneity problem is middleware. Vendors sell middleware as a "software bus" that isolates applications from the complexity of a heterogeneous environment (see Figure 3, Vendor view). Simply develop a single interface to the middleware, and other applications can plug in to the bus, like an appliance plugs in to an electrical outlet for power.

Unfortunately, traditional middleware has failed to achieve that promise (see Figure 3, Actual view). While EAI technology is useful for system-to-system integration, it requires homogeneous technology on each end of the connection and tends to create proprietary islands of integration. Proprietary tools and custom applications add more moving parts to the ugly reality of the enterprise technology infrastructure.

But the software industry has a history of neutralizing proprietary architectures once they become unwieldy with standards. From relational databases (SQL) to networks (TCP/IP) to operating systems (Linux), standardization - industry-driven or de facto - reduces our reliance on proprietary techniques and simplifies architecture.

Web services standards (see Figure 3, Web services standards) define "middleware for middleware." While proprietary products will not go away, proprietary interfaces between them will. Standardization cleans up the heterogeneity mess not by eliminating it, but by defining standard interfaces and services. Proprietary systems that support these standards can participate in the universal bus architecture that liberates software assets.

Despite the fact that Web services standards are still evolving, they provide real value today. Most popular tools support Web services already, so developers can use the tools they already know. Since the tools use the same standards, they require less proprietary training - think of the impact SQL had on the accessibility of databases. Standards are simpler to learn, which means developers of all levels can fully participate in the Web services party.

Principle 4 is about embracing standards early and incrementally - the sooner you incorporate Web services integration methodologies into your architecture, the sooner you'll liberate assets from proprietary tools. Furthermore, you'll begin to bridge the islands of integration that have formed and mitigate the heterogeneity challenge (Principle 2: Be aware of the hidden integration foil: heterogeneity).

Principle 5: Anticipate Change with Adaptive Infrastructure
When designing the interfaces for an application integration, think a year or two ahead. Envision a system that can withstand changes in the business, in the application, and in the underlying technology infrastructure. Systems that can adapt best to change are best suited for application integration.

Why is it important to develop adaptive architectures, from a business perspective?

1. The best way to optimize integration projects is to avoid them.
2. If your interfaces adapt, subsequent applications can be integrated more easily.
3. The longer your interfaces last, the better job you have done defining your requirements (Principle 3: Describe business processes succinctly).
4. Stable interfaces imply you have accomplished a high level of granularity (Principle 7: Focus on Granularity, below), which is a natural use for BPM technology.

While the requirements phase of the first project was increased by four weeks, the time to integrate the three systems was reduced because:

• Each development team utilized native standards-based interfaces in the tools they were already using - no training was required.
• Each project phase capitalized on Web services provided by systems on different software and hardware (heterogeneity was mitigated).
• Although new requirements were added for Phase 2, the Phase 1 system didn't break because the XML interface was extensible.
• Web services standards provided the architecture to support self-describing, extensible architectures.

Principle 6: Think Integration Platform, Not Integration Applications
An application is installed and used as a black box - without regard for its inner workings. A platform, by contrast, provides services to other applications so they can operate more effectively and can scale as more applications are added, load is placed on the system, and more or less resources are required.

Enterprises with hundreds of applications have unique platform needs. For example:

• The ability to distribute and balance the load across many instances of the same application
• The ability to manage many applications, each running in its own environment, as you would manage a single homogeneous system
• Security management, where each system has its own way of managing security (yet another heterogeneity dimension)
• The ability to allow small-scale integration projects to interoperate with these complex environments as well.

Principle 6 orients you to think of the Web services "universal bus" as a platform that provides support for these classes of service.

Principle 7: Focus on Granularity: Think Faxes, Not Phone Calls
Good workflow design utilizes large-grain interfaces to exchange complex information. Let's say you want to buy running shoes. You could call a shoe salesperson and have a conversation, or send for a catalog and place an order. The differences in these two approaches illustrate the final principle of good business process management: proper selection of granularity.

The fine-grain approach is like a phone call between two people. Consider the following conversation, or business process flow:

• "Do you have Saucony racing flats?"
• "No."
• "How about Nikes?"
• "Yes."
• "Do you have Nike Zoom Rival D?"
• "Yes, but only in sizes 10 and 9."
• "Great, I'm a 10; do you have three pairs in stock?"
• "No, just two."
• "OK, I'll take two pairs then."
• "That'll be $149.96."
• "What kind of credit cards do you accept?"
• ... and so on

• Simple data is exchanged: "shoe type," "quantity," "size," and "price."
• Lots of data is exchanged, in small chunks.
• The conversation is tightly coupled: Replies are meaningless without the context of the question. The phrase "sizes 10 and 9" tells you nothing useful on its own.
• You have to understand to whom you're talking: You have different conversations with a shoe salesperson than a waiter.
• Synchronous communication: Because the conversation is tightly coupled, you must be able to associate replies with requests.

Things to note about the large-grain approach:

• Complex data is exchanged: structured, self-describing data is exchanged. You got an entire shoe catalog. You knew it was a catalog by reading it, not by knowing that it was a response to my request.
• Large chunks of data are exchanged infrequently.
• Loose coupling: You didn't need to directly associate the catalog and your previous request because the catalog is sufficiently self-describing to anyone who knows how to read a catalog (understands the schema).
• Asynchronous communication: Enabled by the self-describing data and loose coupling. You could not know directly that the envelope containing the catalog was a response to your request.

It may seem obvious that large-grain system interfaces would tend to be more stable, easier to describe, and easier to integrate - so why would any developer develop fine-grained interfaces? First, specifying high-level, abstract interfaces is not easy. Developers tend to start exposing systems by mapping existing, low-level interfaces directly to Web services, since it's the most straightforward approach. Another factor conspiring to lead us astray is today's tools, which tend to encourage this simplistic approach to creating Web services. For example, most Web services "wizards" read existing service descriptions, objects, or component models and simply map them one-to-one to fine-grain Web services.

With such a confluence of factors steering developers toward a less desirable approach, it's easy to see why integration projects tend to be expensive, late, and error-prone. However, by observing Principle 7, we can avoid these issues by proactively specifying course grain architectures that ensure more loosely coupled, highly cohesive systems.

Pulling It All Together
To pull the pieces together, let's quickly describe an effective Web services architecture for business process management:

• It addresses the needs of the virtual enterprise - internal applications as well as applications that run on external networks (Principle 1: Plan for a virtual enterprise).
• It's constructed with the full awareness of the heterogeneity challenge (Principle 2: Be aware of the hidden integration foil: heterogeneity).
• Business processes are well-documented and understood (Principle 3: Describe business processes succinctly).
• Standards adherence is central to the architecture; not an afterthought (Principle 4: Leverage standards).
• Change is anticipated; the solution is designed to be adaptive. Interfaces are designed to be stable and withstand inevitable change (Principle 5: Anticipate change with adaptive infrastructure).
• The solution employs a platform, not point-to-point applications or proprietary architectures (Principle 6: Think integration platform, not integration applications).
• Applications have large-grain interfaces. They implement loosely coupled, highly cohesive interfaces, rather than low-level, tightly coupled conversations (Principle 6: Focus on granularity: think faxes, not phone calls).

Author Bio
Mark has over 14 years of experience in technology, most recently as CTO of YouthStream Media Networks where he led all technology initiatives, from internal operations to the creation of the Sodalis platform for integrating and supporting several hundred of YouthStream's partners, including leading colleges and universities in the United States. MARK.PALMER@IONA.COM