Web services are platform-independent interfaces that allow
communication with other applications using standards-based Internet
technologies, such as HTTP and XML. They provide an opportunity for
organizations to reduce the costs and complexities of application
integration inside the firewall and create new possibilities for
legacy applications to participate in e-business. One problem with traditional integration techniques is the
proliferation of point-to-point communication and data conversions
that must change as new applications are integrated or data formats
change. The problem quickly gets complicated when you add business
partners, subsidiaries, mergers, or acquisitions to the integration
mix. Web services simplify integration by reducing the number of APIs
to one (SOAP) and the number of data formats to one (XML). While
there is often variation in the syntax of the XML payload, such as
the FinXML, FIXML, OFX, and IFX standards used within financial
services, the fundamental knowledge and skills needed to work with
each of these standards is the same.
Web services are standards based and platform independent.
Independent standards bodies control the direction of the core
components, rather than individual corporations pushing their own
technologies. By using standards-based technologies and widely
available skill sets, Web services allow companies to develop
flexible integration solutions that can evolve as needs change.
Web services sound promising for new applications that support
built-in Internet
technologies, but what about legacy applications, like CICS? How can
they benefit from Web services integration and what does it take to
integrate legacy applications as Web services?
CICS handles more than 30 billion transactions/day valued at
over $1 trillion/week
Figure 1 shows a high-level taxonomy of CICS applications.
CICS represents the broadest category of mainframe applications. CICS
transactions fall into two subcategories: "visual" and "nonvisual." A
"visual" transaction is one that expresses a presentation interface
to an end user at a terminal. You could also refer to a "visual"
transaction as a "terminal-oriented" transaction. In contrast,
"nonvisual" transactions don't interact with an end user. Instead,
another program invokes these transactions. (This type of transaction
is also referred to as a "COMMAREA transaction" because the
input/output parameters are passed to/from the transaction using an
area of storage referred to as the "communication area," or COMMAREA.)
The nature of CICS applications makes them complementary to Web
services technologies. Nonvisual COMMAREA applications take one
request and return data to the requesting application in a single
step. This maps well with the Web services model because a single
SOAP request would yield the required host data. Meanwhile, the
majority of visual applications use a component of CICS called Basic
Mapping Support (BMS). BMS essentially handles the presentation logic
of the transaction and relieves the application developer of having
to encode and decode 3270 terminal data streams. BMS expresses fields
and data as name/value pairs, which can be converted to XML for
consumption by Web services.
While mainframes continue to be the most reliable and scalable
platforms for handling large amounts of data and large numbers of
transactions, many organizations want to move to newer technology and
stop relying on the dwindling numbers of COBOL developers and system
administrators with mainframe skills. However, in most cases these
same organizations realize that their legacy applications "work",
their business processes rely on them, and that making changes to
decades of business logic is both time-consuming and risky. Moreover,
integrating applications is usually cheaper than rewriting them to
operate on other platforms. Now by enabling your legacy applications
as Web services you can deliver integration projects faster and
cheaper because your legacy application groups and Internet groups
use their existing knowledge to integrate the applications.
Integration Models
There are two basic models for integrating CICS applications as Web
services, both of which include the use of adapters. The differences
between these models depend upon where the Web services exist, how
they operate under the covers, and the types of applications you want
to integrate. In this article, we will refer to these models as
connectors and gateways.
Gateways run off the mainframe on middle-tier servers and
often use traditional methods, such as screen-scraping (that
is, capturing legacy data based on row/column coordinates of the application screen).
In the purest sense, adapters might seem antithetical to the Web
services model because the target application is not itself acting as
a Web service. Nevertheless, when you face the daunting task of
rewriting trillions of lines of code and millions of legacy
COBOL/Assembler applications, the need for adapters becomes apparent.
The question then becomes "what type of adapter do you want to use?"
The choice between using connectors or using gateways often depends
upon the types of applications you need to integrate. Integration
vendors have traditionally provided facilities that allow you to use
gateways to access CICS. These gateways commonly use screen-scraping
techniques. However, with the release of CICS Transaction Server, IBM
began providing facilities that allow the use of connectors to access
legacy applications, so you can choose between connector and gateway
models based on your needs.
The recent availability of connectors that support both visual
(terminal-oriented) and nonvisual (COMMAREA) CICS application types
allows remote applications to invoke almost any CICS application as a
web service. Because most shops have a mix of CICS application types,
companies should seek out this kind of connector to avoid multiple
software licenses and additional training on how to integrate the
different application types within your organization.
In the connector and gateway examples below, we focus on using Web
services with visual (terminal-oriented) applications because they
are typically more difficult to integrate than nonvisual (COMMAREA)
applications.
Connector Model for Web Services
Connectors allow you to transform your legacy applications into Web
services without requiring the use of additional hardware, without
changes to the legacy application, and without falling back upon
brittle techniques like screen scraping. Compared to gateways,
connectors yield better performance by running on the host, and more
reliable operation due to the elimination of the many layers data
must pass through due to screen-scraping (Figure 3 details these
layers). Connectors also provide enhanced access to application
information such as state and error codes. This information is lost
when data is sent to a terminal emulation technology such as a 3270
emulation client.
Figure 2 shows the basic model for accessing legacy applications as
Web services through connector technologies. It illustrates a Web
services architecture: a Provider that supplies the Web service, a
Requester that uses a Web service, and a Broker that finds Providers
for Requesters. In this model, the legacy application is the
Provider. The following steps represent how to find and use a Web
service connector (steps 1-3 are optional).
1. The Provider uploads a WSDL specification to publish its Web service with a Broker.
2. The Requester (usually a Java or .NET application) queries the Broker for a Web service by name or category.
3. The Broker selects a Provider and returns the Provider information to the Requester.
4. The Requester uses the information from the Broker to format and send a SOAP message to the service provider.
5. The Provider returns a SOAP/XML response to the Requester with the legacy application data enclosed.
Gateway Model for Web Services
Unlike connectors, gateways typically run on a physical or logical
middle tier. Where the gateway runs is important because there are so
few options for accessing the host from the middle-tier servers,
which means gateways usually involve some form of screen-scraping.
The solution is tightly coupled in that the integration is between
the gateway and a specific application. Any changes to the
application will break the integration.
When gateways communicate with terminal-oriented legacy applications
they open a terminal session with the legacy application, send a
request to the application, receive the terminal datastream, use
HLLAPI to capture the screen data, process the screen data, convert
the contents to XML, and ship the XML document to the requester. (A
variation of the gateway model is to use FEPI on the mainframe
instead of a middle-tier terminal emulation client. This simply moves
the middle tier onto the mainframe.) The most common components of
the gateway model appear in Figure 3.
Gateways allow you to get an application into the Web services mix,
but screen scraping creates performance bottlenecks and multiple
points of failure between the legacy application and the Web service.
For this reason, gateways are best for short-term projects, either as
a transition to using connectors or as a stopgap measure during
application reengineering or platform migration.
Figure 4 shows the basic model for accessing legacy applications
indirectly using a gateway technology such as a screen scraper.
Again, the diagram in Figure 4 illustrates a Web services
architecture: a Provider that supplies the Web service, a Requester
that uses a Web service, and a Broker that finds Providers for
Requesters. The legacy application is not itself a Web service, but
is accessed by an off-host Provider. The following steps represent
how to find and use a Web service gateway (steps 1-3 are optional).
1. The Provider uploads a WSDL specification to publish its Web service with a Broker.
2. The Requester (usually a Java or .NET application) queries the Broker for a Web service by name or category.
3. The Broker selects a Provider and returns the Provider information to the Requester.
4. The Requester uses the information from the Broker to format and send a SOAP message to the Provider.
5. The Provider starts an emulation session and conducts a series of transactions with the legacy application to collect the requested data.
6. The Provider converts the legacy data stream into an appropriate SOAP/XML response and returns the response to the Requester with the legacy application data enclosed.
Dynamic Connectors vs Code Generators
As we have seen, connectors provide several advantages over gateways
when it comes to Web services integration. How those connectors
operate can be just as important. Products that implement the
connector model for Web services usually fall into two camps: those
that use code generation and those that are dynamic. Code generators
have been around for some time and require developers to adhere to a
fairly structured process. In the case of CICS applications,
something like the following takes place:
1. Download program source code, copybooks, or screen maps to a
workstation running a proprietary tool.
2. Use a proprietary tool to generate Web services "wrapper" code.
3. Upload generated code to the mainframe.
4. Compile, install, and test the generated code.
5. Repeat this process for each program and, again, whenever a
program changes.
While generated connectors can jump-start the initial development
process, they create several maintenance problems. First, the
integration developer is generating code off the host and uploading
the code to the mainframe. In most cases, this developer will know a
lot about Web services and XML, but will know little about the
mainframe. Thus, any changes to the mainframe application require
coordination between the Web developer, the application developer,
and the CICS administrator. Second, generated connectors create "net
new code" that must be managed. Changes to either the legacy
application or to the Web service will require repetition of the code
generation process to keep the generated code in sync with the
integrated application. Without proper management, you are likely to
drown in a sea of generated code.
These issues led to the development of dynamic connectors. Dynamic
connectors operate with little or no configuration and changes to
legacy applications are automatically incorporated into the SOAP/XML
output. In many cases there is no configuration required, while some
cases may require a single-step process to specify Web service
information for each application. As a result, there is no generated
code and there is a clear division of labor: the CICS administrator
installs the connector and the Web developer simply invokes the
connector as a Web service.
Conclusion
The subsystem under which your legacy application runs determines
your top-level integration choices. IBM's CICS Transaction Server
includes facilities that allow third-party vendors to create
connectors that can immediately enable legacy applications as Web
services. These facilities provide additional benefits over gateways,
such as improved performance and increased stability compared to
their screen-scraping counterparts. By using the same
industry-standard technologies as Web services, some connectors make
it possible for applications to transparently invoke CICS
transactions within a Web services architecture and receive the
resulting data as well-formed XML. For organizations that want to
retain the value of their CICS applications, the combination of
XML-enabling connectors and Web services offers a practical and
powerful integration solution.
Web services are not a trend, but an industry-wide movement that can
provide a long-term solution for companies that want to integrate
legacy applications and data with new e-business processes. In the
end, companies need to assess the value of the data contained in
their CICS applications. Most companies have already determined that
such data is highly valuable and they are looking for ways to
preserve their investments. Given that recent surveys show the top
strategic priorities of CIOs and CTOs are integrating systems and
processes, the use of Web services for legacy integration will grow
rapidly.
Author Bio
During his 23-year career, Russ Teubner has focused much of his creative energy on solving the difficult problems associated with the integration of legacy systems with technologies that have emerged more recently, such as XML. Russ currently is CEO of HostBridge Technology.
russ@hostbridge.com
All Rights Reserved
Copyright © 2004 SYS-CON Media, Inc.
E-mail:
info@sys-con.com
