There are two fundamental attributes developers look for in a given technology to ease their development tasks: extensive functionality and code reusability.

Look at the success of the Unix operating system. A fundamental part of its success is that it can be written in a machine-independent language and ported to multiple hardware platforms. Thus it provides robust functionality that can be reused on multiple platforms. However, despite the extensive similarity between most of the popular Unix flavors, there's no guarantee that an application written for one Unix platform will be portable to another without making significant changes to the application's code. Thus, it falls short as far as code reuse is concerned.

A key design goal for the Java platform from its inception was to address this code portability issue. The "Write Once, Run Anywhere" mission of Java is paramount to its success. A developer no longer needs to code to a specific-hardware platform; rather, the code is interpreted and run in a virtual machine that shields the developer from the underlying hardware and operating system.

The Java 2, Enterprise Edition (J2EE) platform extends this portability to a set of enterprise components that run in a J2EE server. A developer coding to these APIs now not only has the assurance that the code will run on any platform supporting a JVM, but that the components will be portable across any container supporting the J2EE APIs.

Plus, J2EE component developers can get further assurance that this is true through the J2EE Compatibility Test Suite (CTS). The CTS was created to ascertain the portability of developers' enterprise components by testing whether an application server meets each individual requirement set forth in the J2EE specifications.

This article presents the types of tests addressed by the J2EE CTS, the future of the CTS, and further explores the benefits as well as the limitations of the CTS for the J2EE developer.

The CTS
The J2EE CTS includes the tests and tools necessary to ensure that a J2EE-based application server works as established in the J2EE specifications. It consists of several thousand tests that explore whether all the required APIs are present, the necessary services are provided, and the contracts between the required components work as expressed in the specifications. Once the CTS is successfully completed by a given vendor, that vendor's application server is considered J2EE-compatible and receives the J2EE brand. Being J2EE- compatible implies that an application server meets the requirements that guarantee the stability of a J2EE application component in any J2EE environment.

The CTS tools consist of a GUI console for executing and monitoring the tests (see Figure 1) and an associated testing harness. The testing harness consists of a portability component that allows a vendor to plug in a set of classes that govern the deployment and removal of the test applications. In addition, the harness defers to vendor-specific classes for the creation of URLs that access the Web-tier components of a test. Finally, vendor-supplied porting kit classes govern JNDI lookups of deployed test application components.

Figure  1:

There are four major server components in a J2EE server environment: application components, containers (e.g., EJBs, application clients, and servlet/JSP), resource manager drivers (e.g., JDBC), and one or more databases. The J2EE specifications define a set of standard services that each of these components must support. Application components access these services through APIs provided by the J2EE containers. The CTS thoroughly exercises each of these server elements via direct API tests or via end-to-end integration tests that may repeat the execution of a given test in each of the server's containers.

The J2EE CTS ensures that the application servers abide by the requirements set forth in the constituent set of APIs as shown in Table 1.

Table  1:

Types of Tests
There are three types of compatibility tests: signature, API, and end-to-end or integration. Signature tests are executed across all the J2EE technologies to ensure that the correct APIs are available to the application components running in the server. API tests are executed for each specific technology to ascertain whether the application server meets the requirements of the associated specification (see Table 2). Finally, the integration tests examine the interaction between various application components to guarantee that request invocations are propagated appropriately (e.g., resource references are obtained successfully, transactions are propagated).

Table  2:

The API and integration tests are executed in a variety of execution contexts to ensure support for the required application components; a given test is repeated from an application client, an EJB, a servlet, and a JSP.

J2EE 1.3 CTS
In the recent release of the J2EE specifications (version 1.3), several specifications are either modified significantly or are added to the platform. These platform modifications resulted in added complexity in the compatibility process. Therefore, the J2EE 1.3 CTS includes newer sets of tests to verify the compatibility of J2EE 1.3 application servers, including:

Connectors
Connector tests verify that an application server has all the required pieces that allow any compliant resource adapter (RA) to be deployed and work correctly. A resource adapter is a system-level software driver that's used by a Java application to connect to an Enterprise Information System (EIS) such as CICS or IMS.

EJB 2.0
The EJB 2.0 specification contains some dramatic changes from earlier specification versions. The most notable change is the new container-managed persistence (CMP) model, the implementation of which relies on some new technologies such as the new EJB query language.

J2EE CTS 1.3 tests are required to check the conformance of an application server with the EJB query language requirements of the EJB 2.0 specification. These tests focus on the runtime execution of the query and not explicitly on the language syntax. To be able to accomplish the majority of the queries needed in the test cases (derived from the extensive number of queries defined in the specs), an abstract persistence schema had to be implemented. This schema provides the necessary relationships, dependent objects, and entity bean classes that provide the necessary support for those queries.

Other changes include a new bean type called message-driven beans (MDBs), home methods independent of a specific EJB instance, and requirements for security and transactional interoperability. In addition, new Local and LocalHome interfaces were added to support the CMP 2.0 model. The interoperability tests verify that security propagation, transaction propagation, remote method invocation, and naming services can access all work and interoperate across multiple J2EE-compliant platforms.

Java Message Service (JMS)
The tests in this area verify compliance with the JMS functionality expressed in the EJB 2.0 specification, the J2EE platform specification version 1.3, and the JMS specification version 1.0.2. These tests are derived from the requirements to provide message-driven beans with JMS access from the various J2EE containers, and the capability to perform JMS operations in a distributed transaction. The appropriate JMS API tests are also included.

JDBC 2.0
The new JDBC tests added in CTS 1.3 verify that an application server implements the required JDBC 2.0 extensions expressed in the J2EE 1.3 specifications.

Java Transaction API
The new transaction-related tests in CTS 1.3 check compliance with the new CMP model introduced in the EJB 2.0 specifications. Specifically, tests will be included to validate access to multiple databases within the context of a transaction.

Security
Security testing in CTS 1.3 covers the compatibility requirements in multiple specifications, including J2EE 1.3, EJB 2.0, Servlet 2.3, and JMS 1.0.2 specifications. Security coverage for the J2EE connector architecture is not included in these tests.

JSP 1.2/Servlet 2.3
The JSP and servlet tests cover the new functionality related in the new specifications such as the new tag library extensions, JSP precompilation, and servlet filtering.

CTS - What It Is (and Isn't)
Implementing a J2EE API requires reading and interpreting the core API specifications. Although the specifications are for the most part clearly written, there are a number of ambiguities. These ambiguities, which may not become apparent until multiple implementations exist, often lead to varying interpretations on the part of the J2EE licensees. A process is needed that not only validates the various implementations, but also the fundamental interpretations that formed the basis of those implementations.

The tests included in both the current and upcoming versions of the CTS provide a concrete representation of the requirements listed in the respective J2EE specifications. The application server vendors use these tests to validate their understanding of the specifications. A licensee is much more apt to successfully implement a given technology when presented with concrete examples showing how components interact with the given API. In addition, the CTS provides a forum for the vendors to resolve specification ambiguities.

One of the major challenges for the developers creating the CTS tests is that at their inception, no actual implementations existed other than Sun Microsystems' J2EE Reference Implementation (RI). Therefore, the tests are executed and validated against the RI.

However, the RI may have some inherent specification assumptions and interpretations that are erroneous, leading to tests that are unsound. If a vendor disagrees with a given test, a dialogue is initiated with Sun's CTS team to resolve the issue. This process ultimately resolves whether the test represents a misinterpretation of the specification, is flawed and could never be passed, or whether the vendor's implementation does not support the requirement represented by the test. If a test is deemed unsound for any of these reasons, it's either patched (if removing it would compromise the compatibility process) or added to an exclusion list (i.e., a list of tests that are no longer executed by the CTS).

The process outlined above presents the J2EE component developer with numerous benefits, as well as some distinct disadvantages. Generally, a developer can regard the J2EE CTS process as a standardization effort that ultimately normalizes the behavior of his or her application components. For example, if multiple vendors pass the same set of tests that address the functions of the servlet context, a developer can be confident that his or her use of the servlet context will manifest the same results, no matter which server is utilized.

However, the fact that certain tests have potentially been excluded implies that the current suite of tests may not have complete coverage of the J2EE APIs. Although vendors are obligated by their participation in the J2EE branding process to provide a fully compliant J2EE server, without a thorough and complete set of tests there's no guarantee that such compliance has been achieved.

In addition, certain APIs may not lend themselves to automated testing and may require manual intervention. For example, the Secure Sockets Layer (SSL) requirements of JavaServer Pages (JSP) can't be tested without a fully SSL-compliant browser invoking the requests.

It's also important to note that the CTS was not created to address the performance, robustness, ease-of-use, or scalability of a given J2EE application server. The CTS is a methodical examination of compliance with the requirements of the J2EE specifications. It makes no attempt to time the tests or run the tests under a significant client load (performance metrics will be addressed by JSR 4 of the JCP - the ECPerf Benchmark Specification). To make a judicious choice of an application server, a developer should not only ascertain whether the server is J2EE-compliant, but also how the server performs, how easily it's configured, and whether it's fault tolerant.

Moving forward, the developers of the CTS have a substantial challenge to keep up with the breadth of the J2EE platform. The overwhelming number of new and modified requirements will translate to a need to substantially increase the number and scope of the associated tests. Ensuring that all the requirements are addressed and sufficiently tested will present a major effort.

Finally, developers need to be aware that the J2EE CTS is not backwards-compatible, although most of the API specifications under the J2EE umbrella do require old application components to continue to operate; in other words, an application server that has any J2EE 1.3 component support can no longer be considered J2EE 1.2-compatible and must proceed on the path to achieving J2EE 1.3 compatibility. To developers, this means they may be confronted with releases of application server products that no longer display the J2EE brand even if a previous release was officially compatible. In some cases, J2EE 1.3 platform implementations may be released with a "beta" or "early release" label. Generally, developers should rest assured that the vendors are continuing to support the J2EE APIs and will release compatible versions as soon as they can support the large number of new requirements in the J2EE specifications.

Summary
The compatibility testing process encompassed by the J2EE CTS provides a comprehensive test of an application server based on the requirements and design guidelines established in the J2EE specifications. The CTS also attempts to provide full coverage of the strict requirements of each J2EE-related specification.

For developers, the CTS provides a significant level of assurance that coding to the J2EE APIs will yield an application that can be deployed on any certified J2EE application server, so developers no longer need to learn vendor-specific proprietary APIs. This also means that developers have a much more portable skillset.

The CTS also holds significance for corporate IT managers, even though the importance of portable code for people in these positions is probably negligible. As a corporate IT manager, if you purchase and deploy a certified J2EE application server (that is, one that has passed the CTS), your "hiring pool" - the candidates qualified to write and deploy your applications - increases exponentially. Instead of spending time and resources looking for developers with skills specific to one vendor, you need only look on a candidate's résumé for experience coding to J2EE standards. In addition, the decision as to which application server will host the resultant application can be made parallel to the development effort; the CTS goes a long way toward ensuring that the application will deploy successfully and run on a certified J2EE application server.

The CTS isn't a guarantee of any kind, nor should it be the only criteria used to select an application server. Rather, the CTS is an equalizer; it provides application server vendors with a platform from which they can differentiate their products through functionality, implementations, and so on.

In other words, the CTS isn't the answer, but it's an important part of the whole equation.