As companies have viewed new technologies historically, the smart enterprise adopted new technologies while improving on its existing technology investments. Enterprises that embrace this methodology can drive more value out of both the new and the old technology. Why spend millions on buying and implementing a new system if you can repurpose your existing applications and data to deliver the same benefits of a new system? Today, companies are finding they can get more life out of mainframe legacy systems by exposing valuable business logic as Web services. Web services are allowing legacy applications to be integrated seamlessly throughout the enterprise.
"Many existing mainframe organizations are reevaluating the value proposition of this platform," said Dale Vecchio, research director at Gartner. "The ability to leverage existing systems in a services-oriented way requires technology to easily encapsulate these systems as services, and then manage them with the same quality-of-services expectations."
Web services offer a solution to the classic application development quandary - how to universally integrate disparate programs spanning programming methodologies, languages, and computing platforms. Web services allow the enterprise to securely expose core business logic both inside and outside the firewall. As an enabler for application-to-application interaction, Web services have no equal. No other technology has delivered on this "holy grail" of universal application integration.
The immediate impact of Web services will be in unlocking the hidden treasure of the Global 2000 - a multi-decade investment in what is now termed legacy-based applications. These applications, residing on the IBM mainframe and mid-range platforms, were written in COBOL and RPG. They may be cumbersome to use, provide dated user interfaces, and appear unable to easily integrate with new technologies. Legacy applications are all this and more. But more importantly, they are also finely tuned repositories of corporate business logic and data. These applications have evolved with the needs of the business over the last 30 years and cannot be replaced easily. A decision to buy a multimillion dollar packaged application or to invest in a costly rewrite will not guarantee the successful duplication of these applications. The simple fact is, these applications are the lifeblood of business computing and are not going away in the near future.
Web services provide the key to integrating the legacy applications that still power the Global 2000 with any new application or technology. Web services from legacy applications, partnered with a Web services management solution, deliver seamless integration throughout the enterprise. Let's explore the seemingly disconnected worlds of mainframe applications and Web services solutions.
The Mainframe Is Alive and Kicking
Enterprises have invested much time and money in legacy systems. According to industry analysts, over 70% of the world's data is contained in legacy systems. IBM states that there are more transactions processed by CICS systems today than by the entire Internet. CICS handles more than thirty billion transactions per day, and is used by 492 of the Fortune 500. Nine out of ten ATM transactions are done using COBOL. IT departments in large corporations manage over 30,000 CICS code modules written in COBOL, Assembler, or PL/I. The oldest modules are 25-30 years old, but still process mission-critical data every day and almost certainly will continue to run for years to come. You get the idea - legacy systems still have an important role to play in business today.
This enormous amount of code, running on mainframe and AS/400 systems, is collectively known as "legacy applications." Between 70 and 80% of IT budgets are spent yearly on maintaining and evolving this code.
Many corporate IT departments have turned to Windows NT and Unix-based machines as a complement to their existing mainframes for developing and deploying Web and composite applications. The mission-critical legacy applications residing on the mainframe must be leveraged to provide both data and business logic to power these new Web and composite applications. It is too costly to simply recode legacy applications in languages designed for these new platforms.
Legacy application integration is the solution. Existing legacy applications must be made available to these new applications. Corporate IT departments must be able to isolate specific business transactions from within one or more applications and expose it to the Web or create composite applications. Web services is the technology powering this new form of application integration.
How to Unlock Legacy Applications
The difficulty in integrating legacy mainframe applications stems from how they were written. The vast majority of legacy applications were designed with the business logic and presentation layer tied together. This prevents several problems for programmers attempting to integrate these applications with new development projects. Let's explore the common approaches used for access to legacy application data.
Two primary methods have been used:
- Direct database access
- Emulation-based access
Direct database calls to the raw legacy data sound like the ideal solution. They deliver lightning-quick response times, and allow the developer to repackage the data upon retrieval in any format needed. Unfortunately, all is not as it seems! Here's why: the original developers of legacy systems were faced with a shortage of hard-drive storage space. To get around the tremendous costs of storage on the mainframe, most legacy databases were stored in various compressed and very confusing formats. Additionally, by accessing the data directly, you lose the real benefit of most legacy applications - the business logic. Remember, the business logic is tied into the user interface layer, so by going to the raw data you lose the ability to decipher the meaning of the raw data. You also run the risk of corrupting the data when writing back to the legacy database, since many applications have data that is updated from batch (or offline) applications.
Emulation-based access has been the solution of choice for most developers who need to integrate with mainframe legacy applications. Emulation-based access, commonly referred to as "screen-scraping," is the process of taking the data from the screens of a legacy program that was formatted for a "green-screen" terminal and reformatting the data for use in a new user interface. The input from the new user interface must also be reformatted to allow the underlying legacy program to understand the request.
Screen-scraping products range widely in their degree of sophistication and scalability. Low-end products exist that simply use the geometry of the screen (row and column) to provide a new presentation or programmatic interface in a two-tier deployment. They do not allow any reengineering of the legacy business logic to allow for more complex integration scenarios. Several vendors have released products introducing the concept of a mid-tier application server to the screen-scraping market. These vendors offer high-end legacy integration solutions with automated development environments that leverage the capabilities of a middle-tier legacy application server to provide elaborate reengineering or "intelligence" around the navigation of the legacy screens. These tools are based on terminal emulation, but provide many additional features designed to improve both the development and deployment of applications based on legacy integration.
Emulation-based legacy integration tools have proven valuable in application integration projects. Enterprises have used tools such as the ClientSoft ClientBuilder product line to unlock legacy applications and expose them as Java or Microsoft components, and now as Web services. These components or services can then be integrated into new J2EE applications and other packaged applications, such as a Siebel CRM application or a new portal application.
While emulation-based legacy integration tools have been the only sure-fire way to access legacy transactions they do have some considerations. The performance and scalability of the new interface is a primary concern. The underlying use of terminal emulation is a limiting factor of application performance and scalability. The datastream can only handle 1920 characters of data at any one time, based upon the 80x24 size of a legacy host screen. This fundamentally limits how much information can be passed to and from the legacy application via terminal emulation. A second consideration is the "dual maintenance burden". The geometry of the host screen requires programmers to engage in manual row-and-column based development projects that fail if the position of the fields on the underlying host screens is altered. This forces application developers to modify the emulation-based interface any time a change is made to the underlying legacy application.
While emulation-based legacy integration solutions work, there is a better way to accomplish this integration without any of the concerns discussed above - a direct integration approach. Direct integration to the host applications provides a higher-performing integration solution and increased stability. Enterprises want to maximize the performance of their legacy integration solutions and remove some of the weaknesses associated with emulation-based approaches.
Vendors that offer a direct integration solution include ClientSoft, HostBridge Technology, InnerAccess Technologies, and SofTouch Systems. Each of these vendors offers a two-tier solution, with a runtime that resides on the mainframe and a generated client on the front end. This client can be an HTML GUI, a JavaBean or Enterprise JavaBean, or a Web service. ClientSoft and InnerAccess Technologies offer direct integration solutions coupled with a Web service generation option. Let's examine how these vendors accomplish direct access legacy integration.
The development environment is Windows-based: This environment allows direct interaction with the mainframe CICS and IMS applications through one of the following APIs - COMMAREA, FEPI, or 3270 Bridge - not via terminal emulation. The use of each of these APIs will vary depending on the vendor's solution and the backend host type.
Runtime component resides on the mainframe inside of a CICS region: This allows the direct access solution to interact with the transactions using direct APs and not emulation. There is no middle-tier requirement - just the mainframe-based runtime and the component or Web service.
The products provide a generated interface, whether presentation or programmatic: HostBridge provides XML from legacy applications. SofTouch Systems provides a browser-based GUI application from legacy applications. ClientSoft and InnerAccess Technologies focus on programmatic integration, offering components and Web services that can be used stand-alone or in an application server requirement with WebSphere or another J2EE application server.
Direct mainframe integration solutions remove all of the traditional considerations associated with typical emulation-based approaches. A direct legacy integration approach delivers substantial performance gains coupled with a lower risk of application failure at runtime. The combination of direct legacy integration and Web services provides seamless access to the hidden treasures of the corporate data center - legacy applications.
Introducing Web Services
Web services are not mysterious. According to Gartner, Web services are "loosely-coupled software components delivered over Internet-standard technologies."
The challenge is in how to deliver Web services from legacy applications. Several ISV products have emerged to allow the encapsulation of legacy business logic as a Web service. Integration vendors ClientSoft, Jacada, Seagull, and WRQ all offer Web services products that utilize a terminal emulation approach to legacy access. These tools are subject to the considerations detailed above. Both ClientSoft and InnerAccess offer legacy integration products that do not utilize terminal emulation, and provide automatic delivery of Web services from mainframe legacy applications. Let's examine one of these solutions further.
ClientSoft Tanit Objects (CTO) provides a graphical Windows-based development environment. This development environment allows developers, whether their skill-sets are COBOL or another modern language, to utilize three different APIs to connect to mainframe applications:
The choice of API used to connect to the legacy system is based on the type of legacy application. Visual applications, those applications that traditionally utilize a 3270-based presentation layer, are accessed via the 3270 Bridge or FEPI APs. Nonvisual applications, those applications that are modularized and don't have the presentation layer and business logic integrated, are accessed via the DPL API using the COMMAREA. The important thing here is that you do not need to have mainframe expertise with these APIs.
The next step is to utilize the tool to isolate the business transaction you would like to expose as a Web service. When working with visual, 3270-based applications the isolation of the business transaction is done with the 3270 Recorder/Mapper. This simulates "green-screen" interaction with the legacy application and allows the developer to easily define the entire unit of work involved in the relevant business transaction and define the input and output parameters for the Web service (see Figure 1). When working with nonvisual modularized applications, the developer uses the Tanit Parser to automatically parse the COBOL copybook and then define the entire unit of work with the appropriate input and output parameters describing the complete business process.
The final step is to use the Tanit Generator (see Figure 2) to have the Web services automatically generated from the development environment.
Tanit Objects will generate the complete WSDL (Web Services Description Language) file based on the series of Input and Outputs defined through either the Parser or Recorder/Mapper. Additionally, it will generate a "starter program" which allows the developer to test the newly created Web service. The generated Web service resides on the mainframe, taking advantage of the inherent security and performance of this platform.
Managing Your Way to Web Services Success
Web services technology promises to finally usher in a world of service-oriented integration, thereby revolutionizing the business of application integration both inside and across enterprise boundaries. And while the technology behind Web services standards is interesting, technologies with similar purposes have come and gone in the past in the form of CORBA, DCE, and DCOM. It is the nearly universal vendor support for Web services standards that will finally make this promise a reality.
Native support for Web services-based connectivity is emerging in packaged enterprise applications, application servers, portal solutions, and application development tools. The leading solutions in each of these software categories today offer the native ability to connect with other software systems via Web services standards. Legacy integration vendors make the mainframe systems that continue to serve as the backbone of many enterprise-computing environments available to these newer software systems.
As more software systems are made available for integration via Web services standards, more connections based on these standards will be built. But left unmanaged, the widespread leverage of Web services can lead to massive complexity and cost. Figure 3 illustrates the exploding set of interconnections that can exist between systems connected via Web services standards.
Without a sound and uniform management strategy, this growing set of direct (point-to-point) connections between Web service providers and consumers can create insurmountable problems for an enterprise including:
Complex "point-to-point" network of interconnected systems leads to exploding (n2) costs and risks of system-wide failure:
- Security risks: Security configuration is duplicated at each provider system
- complex, coarse-grained and error prone
- Multiple points of Web services control: Access control, monitoring, interoperability support - solving the same problem many times
- Lack of administrative consistency between various provider systems
Impaired visibility of Web services activity and availability leads to downtime, frustration, and security risks:
- Difficulty detecting failures and troubleshooting
- No monitoring of service availability and quality
- Hard to aggregate and audit historical Web service usage information for analysis and potential billing
- Missed opportunity to monitor business activity
Interoperability challenges narrow the reach of Web services:
- Varying schemas/semantics between application systems: "My purchase order system generates purchase orders in this format, can you accept that?"
- Varying standards interpretations and implementations
Noninsulated, brittle direct connections between consumers and providers impedes flexibility and limits business agility:
- Upgrades or changes to Web service providers impacts downstream consumers
- Expensive one-off strategies to support migration
These costs tend to increase in a nonlinear fashion as each new system begins to depend, directly or indirectly, on a growing set of existing systems. Eventually the number of connections can cause the system at large to collapse under its own weight. Skyrocketing costs mean that beyond a point, each new system leveraging a Web service becomes unprofitable to support. Figure 4 illustrates this concept.
If Web services are to usher in an era of truly universal enterprise application connectivity, then management of these environments must be a consideration from day one.
Recognizing these issues, legacy integration vendors are partnering with Web services management companies to ensure that customers reap the substantial benefits of Web services while avoiding the pitfalls of unmanaged Web service deployments.
Legacy integrators should look for a Web services management provider that offers a platform-neutral, unified solution for Web services monitoring, life cycle management, interoperability, traffic management, and security policy enforcement. In addition, the solution should allow IT organizations to:
Substantially mitigate security risks through fine-grained Web service access control
Gain end-to-end visibility of Web services activity and identify root cause of Web services failures, thereby maintaining uptime
Manage Web services through their entire life cycle without disrupting the operation of systems that grow to rely on those services
Enhance the value of Web services by cost-effectively growing their potential user community
Dramatically lower management costs
Enforce, monitor, and control service availability, Quality of Service (QoS), and Service Level Agreements (SLAs)
Scale services to meet the needs of demanding and varying sets of consumers
Monitor business activity as represented by Web services interactions
Audit the use of Web services in support of development planning, billing, charge backs, and capacity planning
Managed Legacy Access - A Natural First Step
There is substantial momentum behind the push toward Web services as the architectural underpinning of any robust enterprise application integration strategy. A natural first step in the adoption of this strategy is to unlock the systems and assets already in place in the enterprise. Legacy integration vendors make it possible to extend the life of existing mainframe-based applications by unlocking these assets as Web services.
But before embarking on a Web services-based initiative, strong consideration must be given to how these critical business services will be managed. The foundation for success lies in providing a comprehensive system for the enterprise-wide management of Web services.
Author Bios
Brian Anderson is the senior product marketing manager at ClientSoft (www.clientsoft.com), a leader in delivering fast and secure integrated applications to legacy systems. He is responsible for ClientSoft's product marketing and management
functions. Before joining ClientSoft, Anderson worked in product marketing for Jacada. Anderson received a BS in geography and computer science from Florida State University and pursued graduate work in marketing at the University of Notre Dame.
banderson@clientsoft.com
James Phillips is senior vice president of marketing and product management at Actional. He has worldwide responsibility for Actional's product and market strategy and market execution. In more than 17 years of software industry experience, James has held senior marketing management, software engineering, and business development roles with Intel, Intuit, Synopsys and Central Point Software. He is a frequent speaker and editorial contributor on Web services-related issues and serves on the international advisory board for Web Services Journal.
jamesp@actional.com
All Rights Reserved
Copyright © 2004 SYS-CON Media, Inc.
E-mail:
info@sys-con.com