There are two fundamental attributes developers look for in a
given technology to ease their development tasks: extensive
functionality and code reusability.
Look at the success of the Unix operating system. A
fundamental part of its success is that it can be written in a
machine-independent language and ported to multiple hardware
platforms. Thus it provides robust functionality that can be reused
on multiple platforms. However, despite the extensive similarity
between most of the popular Unix flavors, there's no guarantee that
an application written for one Unix platform will be portable to
another without making significant changes to the application's code.
Thus, it falls short as far as code reuse is concerned.
A key design goal for the Java platform from its inception
was to address this code portability issue. The "Write Once, Run
Anywhere" mission of Java is paramount to its success. A developer no
longer needs to code to a specific-hardware platform; rather, the
code is interpreted and run in a virtual machine that shields the
developer from the underlying hardware and operating system.
The Java 2, Enterprise Edition (J2EE) platform extends this
portability to a set of enterprise components that run in a J2EE
server. A developer coding to these APIs now not only has the
assurance that the code will run on any platform supporting a JVM,
but that the components will be portable across any container
supporting the J2EE APIs.
Plus, J2EE component developers can get further assurance
that this is true through the J2EE Compatibility Test Suite (CTS).
The CTS was created to ascertain the portability of developers'
enterprise components by testing whether an application server meets
each individual requirement set forth in the J2EE specifications.
This article presents the types of tests addressed by the
J2EE CTS, the future of the CTS, and further explores the benefits as
well as the limitations of the CTS for the J2EE developer.
The J2EE CTS includes the tests and tools necessary to ensure
that a J2EE-based application server works as established in the J2EE
specifications. It consists of several thousand tests that explore
whether all the required APIs are present, the necessary services are
provided, and the contracts between the required components work as
expressed in the specifications. Once the CTS is successfully
completed by a given vendor, that vendor's application server is
considered J2EE-compatible and receives the J2EE brand. Being J2EE-
compatible implies that an application server meets the requirements
that guarantee the stability of a J2EE application component in any
The CTS tools consist of a GUI console for executing and
monitoring the tests (see Figure 1) and an associated testing
harness. The testing harness consists of a portability component that
allows a vendor to plug in a set of classes that govern the
deployment and removal of the test applications. In addition, the
harness defers to vendor-specific classes for the creation of URLs
that access the Web-tier components of a test. Finally,
vendor-supplied porting kit classes govern JNDI lookups of deployed
test application components.
There are four major server components in a J2EE server
environment: application components, containers (e.g., EJBs,
application clients, and servlet/JSP), resource manager drivers
(e.g., JDBC), and one or more databases. The J2EE specifications
define a set of standard services that each of these components must
support. Application components access these services through APIs
provided by the J2EE containers. The CTS thoroughly exercises each of
these server elements via direct API tests or via end-to-end
integration tests that may repeat the execution of a given test in
each of the server's containers.
The J2EE CTS ensures that the application servers abide by
the requirements set forth in the constituent set of APIs as shown in
Types of Tests
There are three types of compatibility tests: signature, API,
and end-to-end or integration. Signature tests are executed across
all the J2EE technologies to ensure that the correct APIs are
available to the application components running in the server. API
tests are executed for each specific technology to ascertain whether
the application server meets the requirements of the associated
specification (see Table 2). Finally, the integration tests examine
the interaction between various application components to guarantee
that request invocations are propagated appropriately (e.g., resource
references are obtained successfully, transactions are propagated).
The API and integration tests are executed in a variety of
execution contexts to ensure support for the required application
components; a given test is repeated from an application client, an
EJB, a servlet, and a JSP.
J2EE 1.3 CTS
In the recent release of the J2EE specifications (version
1.3), several specifications are either modified significantly or are
added to the platform. These platform modifications resulted in added
complexity in the compatibility process. Therefore, the J2EE 1.3 CTS
includes newer sets of tests to verify the compatibility of J2EE 1.3
application servers, including:
Connector tests verify that an application server has all the
required pieces that allow any compliant resource adapter (RA) to be
deployed and work correctly. A resource adapter is a system-level
software driver that's used by a Java application to connect to an
Enterprise Information System (EIS) such as CICS or IMS.
The EJB 2.0 specification contains some dramatic changes from
earlier specification versions. The most notable change is the new
container-managed persistence (CMP) model, the implementation of
which relies on some new technologies such as the new EJB query
J2EE CTS 1.3 tests are required to check the conformance of
an application server with the EJB query language requirements of the
EJB 2.0 specification. These tests focus on the runtime execution of
the query and not explicitly on the language syntax. To be able to
accomplish the majority of the queries needed in the test cases
(derived from the extensive number of queries defined in the specs),
an abstract persistence schema had to be implemented. This schema
provides the necessary relationships, dependent objects, and entity
bean classes that provide the necessary support for those queries.
Other changes include a new bean type called message-driven
beans (MDBs), home methods independent of a specific EJB instance,
and requirements for security and transactional interoperability. In
addition, new Local and LocalHome interfaces were added to support
the CMP 2.0 model. The interoperability tests verify that security
propagation, transaction propagation, remote method invocation, and
naming services can access all work and interoperate across multiple
Java Message Service (JMS)
The tests in this area verify compliance with the JMS
functionality expressed in the EJB 2.0 specification, the J2EE
platform specification version 1.3, and the JMS specification version
1.0.2. These tests are derived from the requirements to provide
message-driven beans with JMS access from the various J2EE
containers, and the capability to perform JMS operations in a
distributed transaction. The appropriate JMS API tests are also
The new JDBC tests added in CTS 1.3 verify that an
application server implements the required JDBC 2.0 extensions
expressed in the J2EE 1.3 specifications.
Java Transaction API
The new transaction-related tests in CTS 1.3 check compliance
with the new CMP model introduced in the EJB 2.0 specifications.
Specifically, tests will be included to validate access to multiple
databases within the context of a transaction.
Security testing in CTS 1.3 covers the compatibility
requirements in multiple specifications, including J2EE 1.3, EJB 2.0,
Servlet 2.3, and JMS 1.0.2 specifications. Security coverage for the
J2EE connector architecture is not included in these tests.
JSP 1.2/Servlet 2.3
The JSP and servlet tests cover the new functionality related
in the new specifications such as the new tag library extensions, JSP
precompilation, and servlet filtering.
CTS - What It Is (and Isn't)
Implementing a J2EE API requires reading and interpreting the
core API specifications. Although the specifications are for the most
part clearly written, there are a number of ambiguities. These
ambiguities, which may not become apparent until multiple
implementations exist, often lead to varying interpretations on the
part of the J2EE licensees. A process is needed that not only
validates the various implementations, but also the fundamental
interpretations that formed the basis of those implementations.
The tests included in both the current and upcoming versions
of the CTS provide a concrete representation of the requirements
listed in the respective J2EE specifications. The application server
vendors use these tests to validate their understanding of the
specifications. A licensee is much more apt to successfully implement
a given technology when presented with concrete examples showing how
components interact with the given API. In addition, the CTS provides
a forum for the vendors to resolve specification ambiguities.
One of the major challenges for the developers creating the
CTS tests is that at their inception, no actual implementations existed other than Sun Microsystems' J2EE Reference Implementation (RI). Therefore, the tests are executed and validated
against the RI.
However, the RI may have some inherent specification
assumptions and interpretations that are erroneous, leading to tests
that are unsound. If a vendor disagrees with a given test, a dialogue
is initiated with Sun's CTS team to resolve the issue. This process
ultimately resolves whether the test represents a misinterpretation
of the specification, is flawed and could never be passed, or whether
the vendor's implementation does not support the requirement
represented by the test. If a test is deemed unsound for any of these
reasons, it's either patched (if removing it would compromise the
compatibility process) or added to an exclusion list (i.e., a list of
tests that are no longer executed by the CTS).
The process outlined above presents the J2EE component
developer with numerous benefits, as well as some distinct
disadvantages. Generally, a developer can regard the J2EE CTS process
as a standardization effort that ultimately normalizes the behavior
of his or her application components. For example, if multiple
vendors pass the same set of tests that address the functions of the
servlet context, a developer can be confident that his or her use of
the servlet context will manifest the same results, no matter which
server is utilized.
However, the fact that certain tests have potentially been
excluded implies that the current suite of tests may not have
complete coverage of the J2EE APIs. Although vendors are obligated by
their participation in the J2EE branding process to provide a fully
compliant J2EE server, without a thorough and complete set of tests
there's no guarantee that such compliance has been achieved.
In addition, certain APIs may not lend themselves to
automated testing and may require manual intervention. For example,
the Secure Sockets Layer (SSL) requirements of JavaServer Pages (JSP)
can't be tested without a fully SSL-compliant browser invoking the
It's also important to note that the CTS was not created to
address the performance, robustness, ease-of-use, or scalability of a
given J2EE application server. The CTS is a methodical examination of
compliance with the requirements of the J2EE specifications. It makes
no attempt to time the tests or run the tests under a significant
client load (performance metrics will be addressed by JSR 4 of the
JCP - the ECPerf Benchmark Specification). To make a judicious choice
of an application server, a developer should not only ascertain
whether the server is J2EE-compliant, but also how the server
performs, how easily it's configured, and whether it's fault tolerant.
Moving forward, the developers of the CTS have a substantial
challenge to keep up with the breadth of the J2EE platform. The
overwhelming number of new and modified requirements will translate
to a need to substantially increase the number and scope of the
associated tests. Ensuring that all the requirements are addressed
and sufficiently tested will present a major effort.
Finally, developers need to be aware that the J2EE CTS is not
backwards-compatible, although most of the API specifications under
the J2EE umbrella do require old application components to continue
to operate; in other words, an application server that has any J2EE
1.3 component support can no longer be considered J2EE 1.2-compatible
and must proceed on the path to achieving J2EE 1.3 compatibility. To
developers, this means they may be confronted with releases of
application server products that no longer display the J2EE brand
even if a previous release was officially compatible. In some cases,
J2EE 1.3 platform implementations may be released with a "beta" or
"early release" label. Generally, developers should rest assured that
the vendors are continuing to support the J2EE APIs and will release
compatible versions as soon as they can support the large number of
new requirements in the J2EE specifications.
The compatibility testing process encompassed by the J2EE CTS
provides a comprehensive test of an application server based on the
requirements and design guidelines established in the J2EE
specifications. The CTS also attempts to provide full coverage of the
strict requirements of each J2EE-related specification.
For developers, the CTS provides a significant level of
assurance that coding to the J2EE APIs will yield an application that
can be deployed on any certified J2EE application server, so
developers no longer need to learn vendor-specific proprietary APIs.
This also means that developers have a much more portable skillset.
The CTS also holds significance for corporate IT managers,
even though the importance of portable code for people in these
positions is probably negligible. As a corporate IT manager, if you
purchase and deploy a certified J2EE application server (that is, one
that has passed the CTS), your "hiring pool" - the candidates
qualified to write and deploy your applications - increases
exponentially. Instead of spending time and resources looking for
developers with skills specific to one vendor, you need only look on
a candidate's résumé for experience coding to J2EE standards. In
addition, the decision as to which application server will host the
resultant application can be made parallel to the development effort;
the CTS goes a long way toward ensuring that the application will
deploy successfully and run on a certified J2EE application server.
The CTS isn't a guarantee of any kind, nor should it be the
only criteria used to select an application server. Rather, the CTS
is an equalizer; it provides application server vendors with a
platform from which they can differentiate their products through
functionality, implementations, and so on.
In other words, the CTS isn't the answer, but it's an
important part of the whole equation.
Jonathan Maron is
engineer with Hewlett-Packard's middleware division. He was the lead
on HP Bluestone's J2EE 1.2 compatibility effort, and also
principal architect of the Bluestone EJB 1.1 server. Jonathan is a
member of the JavaSoft EJB and J2EE expert groups.