SOAP is at the heart of all Web services as the way to deliver messages between two applications or systems. SOAP in its various versions is well known and often discussed. However, SOAP on its own is often not enough, especially if the message is sent outside the enterprise where privacy of the message content and reliable message delivery become much more important.

Even security and reliability may not be sufficient. You may also need to limit the life of a message so that it "expires" at some point, or specify the type of long-running business transaction of which the message is part.

Efforts are under way to provide these and other useful features in various SOAP standards efforts.

Why Do These Standards Matter?
Standards matter because standards lead to better solutions at lower cost, but why?

First, it's not easy to build solutions to problems such as reliably sending a message over the Internet or making sure that someone can't pretend to be you when sending a SOAP message over the Web.

Solving these problems requires a lot of effort from knowledgeable individuals, and even if you could develop your own solutions to these problems, you'd still need to persuade your partners to implement technology to support it.

On the other hand, if a solution to these problems has been developed in a standards group, then not only will you probably have the leading experts in the world involved, but you should have a "better solution."

Multiple vendors developing a standard should also result in multiple vendors offering solutions. So you should get lower costs because of the competition.

Finally, if "good" solutions are available from multiple vendors, then you don't need to build them. It also means that your partners will be implementing compatible solutions that lower their costs so that you can connect to them easier and faster.

So What Are We Waiting For?
Sadly, standards take time to develop – especially if they're to be "done right." You also need to know which standards to implement; there are so many and they're still evolving.

Which brings me to the purpose of this article, which describes all of the features that are, or will be, needed to use SOAP to "deliver the message" for linking businesses. It then describes the leading standards that are being developed to support those features under four headings: Message Packaging, Metadata, Addressing, Reliable Delivery and Security (see Figure 1) and finally suggests an architectural approach to solving the problem of multiple evolving standards. Note that the rest of this article references many different standards initiatives. Brief descriptions and links to these initiatives are available below.

Message Packaging:
How to Include Data Inside a Message
Packaging describes how all the different parts of a SOAP message are assembled into the string of characters that is sent "over the wire." Similar concepts are needed as in the real world. This means you need to define the electronic envelope into which the main content of the message is packed – the equivalent of the envelope used by logistics companies such as UPS or FedEx; how the message is transported over the Net and errors reported – the equivalent of the delivery systems run by postal services and logistics companies; and, how to handle multiple documents at the same time – for example, if you wanted to send an electronic book and an electronic delivery note in one SOAP message.

Standardization of message packaging has progressed well with SOAP as the definitive electronic envelope. Version 1.1 of SOAP has been widely deployed, with SOAP version 1.2 published in final form in summer 2003. Basically, SOAP standardization is nearly complete. The SOAP specifications also provide detailed descriptions of the "delivery system," (i.e. how to send and receive a SOAP message over HTTP).

Handling multiple documents still needs work. "SOAP with Attachments" is in widest use although it is not part of any standardization process. The Direct Internet Message Encapsulation (DIME) specification developed to handle attachments by Microsoft did not receive widespread support and seems to have been abandoned. The most likely replacement will be the SOAP Message Transmission Optimization Mechanism (SMTOM) specification being developed in the W3C, although the first draft was published only last summer.

At the same time, the Web Services Interoperability Organization (WS-I) published a profile in August 2003 that defines how SOAP 1.1, WSDL 1.1, and UDDI 2.0 should be used together. WS-I also conducts interoperability testing to help assure that solutions from different vendors work together.

Metadata: Useful Data About a Message
Metadata is additional information about a message that is often needed to process the message correctly. It covers such things as identifying a message with a unique ID; relating (or correlating) one message to another, such as relating a request and its reply; linking together all the messages in a long-running transaction, such as relating the messages containing an order with the related messages containing the delivery note and invoice; the type of long-running business transaction being followed, such as should a supplier that receives an order reply with an invoice or should they send an order response that indicates whether they can satisfy the order; and finally agreement information that identifies the agreement or rules that the sender and receiver of the message have agreed to follow when exchanging messages.

Several specifications include how to identify and correlate messages, for example the WS Reliable Messaging specification developed in OASIS; the WS MessageData specification developed by BEA; and the WS Addressing specification developed by IBM, Microsoft, and BEA. The last two specifications have intellectual property restrictions on their use and have not been submitted to a standards group.

Independent of this, Sun, Oracle, and others have developed the WS-Context specification. It describes how to do correlation and relate the messages in long-running transactions. This specification has been submitted to OASIS.

However, no standards activities currently exist that identify the type of business transaction being followed or how to identify any agreement that is being used.

Addressing: Specifying the Destination for a Message
Addresses on messages serve a similar purpose to addresses on letters. For example, they can be used to check that the message has been delivered to the correct recipient; work out how to route a message to its correct destination; and identify alternative methods of delivery if the original method did not work.

Addresses can be of several types, including logical addresses that specify who or what should receive a message without specifying its physical location; physical addresses that specify exactly where the message should be sent rather than who should receive it; return addresses that specify where replies should be sent and address routes that specify the precise route that a message should follow in order to get from its origin to its destination. Standardization of Addressing has not made as much progress as most implementations assume that where you send a SOAP message is defined in the WSDL definition for the Web service and it's always fixed. This means that where you send a message to is always worked out at design time and then hard-coded into the resulting program. This is fine if you're using Web services to build applications, but if you want to use Web services for delivering business messages it does not work as well since addresses change – businesses are bought, or new systems are implemented. With a physical address, this requires that the programs change; on the other hand, if a logical address is used, then it can be mapped to the actual physical address at runtime.

There are some specifications that include addressing. These include the WS Reliability specification, which specifies SOAP Headers that identify the logical address and physical address for the message. The WS Addressing specification also provides SOAP Headers that identify logical addresses and physical addresses as well as a return address. The WS Callback specification also defines a return address. The WS Routing specification from Microsoft defines how to route a message through one or more intermediary SOAP processors.

Only WS-Reliability is being developed in a standards group.

Reliable Delivery:
How to Ensure a Message Gets There
Reliable Delivery includes the features that make sure a message is delivered and processed properly. It includes Delivery Receipts where a recipient of a SOAP message sends back another SOAP message confirming the original message was delivered; Reliable Messaging, where the sender of the SOAP message tries to make sure the message is delivered by resending it if they don't receive an acknowledgment back from the recipient; sequencing, which includes sequence numbers in messages to make sure they are delivered to the application in the same sequence the messages were sent – even if they arrive out of sequence; Expiry, which is a way of indicating in a message that it should not be processed if it is received after a certain time; and two-phase commit, which is based on distributed database ideas and includes many of the ideas of Reliable Messaging but also describes what to do to reverse the effect of changes if the message was not delivered.

Standardization of Reliable Delivery is making good progress. For example, the WS Reliability specification provides an acknowledgement message that can be used as a Delivery Receipt. Delivery Receipts are also provided by the WS Reliable Messaging and WS Acknowledgement specifications. The WS Reliability and WS Reliable Messaging specifications also provide support for reliable messaging and message sequencing.

Expiry of messages is supported by the WS Reliability specification.

Two-phase commit is supported by the Business Transaction Protocol (BTP) activity within OASIS and the similar WS Transaction specification developed by IBM, Microsoft, and BEA. Two-phase Commit is also an idea supported by the Web Services Transaction Management (WS-TXM) specification developed by Arjuna, Fujitsu, IONA, Oracle, and Sun.

What's interesting is that the WS Reliability and WS Reliable Messaging specifications are almost identical – the first is being developed in OASIS, the second is not in any standard group. Of the other specifications, only the BTP and WS-TXM specifications are being developed in standards groups.

Security: How to Secure the Information in a Message
Message security covers ensuring the privacy of the message content, knowing who sent the message, and making sure messages aren't altered after they're sent. It includes transport security, digital signatures, message encryption, authentication, authorization, and key management.

Transport security uses a communication protocol such as SSL to put an "opaque pipe" around the SOAP message so no one can see what's flowing or alter the message while it is in the pipe without detection. However, once the message pops out the end of the pipe, all privacy is gone and the message can be changed at will.

Digital signatures, although they don't help with privacy, can ensure that the message has not changed even after it has been received and its contents unpacked. They work in much the same way as signing a letter helps make sure that the recipient knows the letter has not changed.

Message encryption goes further than transport security and digital signatures by encrypting the message itself in much the same way that "spies" code their messages. This means that even if you can see the message you can't understand it unless you have the correct "key" to decode it. Also, if you can successfully decode or decrypt the SOAP message then you also know it was unaltered as any alterations would lead to a decryption failure.

Even if you receive a SOAP message that is digitally signed and encrypted, it doesn't automatically tell you who or what created and sent the message. Authentication takes the next step by checking the digital signature on a message to make sure it is genuine, or for encryption, checking who gave you the "keys" that allowed you to decrypt the message.

Authorization goes the final step and checks not only who the sender of the SOAP message was but also what the authority of the sender is to make the request that receiving the message implies. For example, a driver's license can be used to identify someone (authentication), but it also indicates that the owner is "authorized" to drive.

With a driver's license, you trust that the authority that issued it checked that the holder is who they say they are. But this only works if you know what a driver's license looks like, who issued it, and whether or not you trust the issuer. For example, after 9/11 it was discovered that some of the terrorists had falsely obtained Virginia driver's licenses with only minimal checks. Similarly, when a SOAP message is received over the Web, you need to have received, in advance, copies of the codes and certificates from a source you trust, that allow you to check the validity of the signatures and decrypt the message.

This distribution of certificates and codes in advance is known as "key management" and often ultimately requires that copies of the certificates and codes be sent out-of-band using separate security management solutions.

A lot of progress has been made on the development of standards for security. Secure Sockets Layer (SSL) is "the" standard for transport security and the XML Signature and XML Encryption standards are mature, stable specifications that can be used to digitally sign and encrypt any XML document including a SOAP message. However, they don't specify exactly how to sign or encrypt a SOAP message so the WS-Security initiative in OASIS is specifying how to do this as well as how to authenticate the sender of a SOAP message.

The Liberty Alliance is an initiative that is specifying how to determine the identity of businesses and individuals so that they can then be used to authenticate SOAP messages.

Trust and key management is one of the hardest areas to address, although some specifications, such as WS Trust, exist for establishing trust relationships; WS Secure Conversation for establishing a secure context for exchanging SOAP messages; and WS Federation for brokering trust between organizations. However, none of these specifications have been submitted to a standards group although all the other security-related specifications have.

What You Should Do Next
The description of the SOAP features needed and the current state of standards development shows that not all the standards needed are under development, for some areas there are competing standards, and finally, some of the specifications have been published outside of a standards group and with intellectual property restrictions.

Gaps in standards and competing standards mean that it will probably be some time before stable standards are established.

Also, the intellectual property restrictions on some standards mean that any user should be wary of committing to those standards in anything more than an experimental way. The recent lawsuits raised by SCO around Linux illustrate that what might appear to be "legally safe" in fact is not. Even if these suits are resolved in favor of Linux, the uncertainty makes the use of IP-constrained standards a risk.

Adopting the Right Architectural Approach
However, waiting for standards to mature will mean missing the benefits of "better lower cost solutions" that were described earlier.

Fortunately, adopting the "right" architectural approach to the solutions you build or buy makes managing the problems easier (see Figure 2).

First, your Web services architecture should have three levels: the application that contains the business logic; the Web services middleware that understands Web services technology; and in between, a rules-driven "interoperability bridge."

The key component is the interoperability bridge as it takes as input a "businesslevel" request to send a message that specifies the data to be sent and who (not where) the data should go to.

The interoperability bridge can then look up rules to determine what standards the destination wants and the physical address the message should be sent to. It then uses the Web service–level APIs to invoke the Web services middleware that does the actual sending and receiving of the message. A similar approach can be followed for the return path.

This approach separates the business application from the Web services standards so that each can evolve relatively independently. It also means that when changes occur they can be applied in one place – the interoperability bridge – at lower cost and more consistently.

About the Author
David Burdett is director of Standards Strategy, Web Services for Commerce One. He is responsible for directing Commerce One's use of Web services standards. He is also a consultant with over 20 years' experience in managing and motivating teams in IT strategy, development, architecture, and organization. david.burdett@commerceone.com

Using Web Services for Business by David Burdett
WSJ Vol 03 Issue 11 - pg.39