HomeDigital EditionSys-Con RadioSearch Java Cd
Advanced Java AWT Book Reviews/Excerpts Client Server Corba Editorials Embedded Java Enterprise Java IDE's Industry Watch Integration Interviews Java Applet Java & Databases Java & Web Services Java Fundamentals Java Native Interface Java Servlets Java Beans J2ME Libraries .NET Object Orientation Observations/IMHO Product Reviews Scalability & Performance Security Server Side Source Code Straight Talking Swing Threads Using Java with others Wireless XML

With the rapid adoption of J2EE has come the realization that more than just J2EE expertise is needed to successfully develop enterprise applications.

The following are critical to the success of J2EE projects:

  • A well-defined component integration strategy
  • The ability to select the right J2EE platform vendor
  • Good understanding of internal and external expectations
  • Evaluation of technical and nontechnical challenges and risks
  • Availability of resources and tools
  • Capabilities of the team
  • Realizing that the best J2EE design practices have to be tailored to meet the needs of the specific project
The ultimate goal of any development project is to produce quality software that meets the requirements of the customer, and is on time and under budget. J2EE adds another dimension to this challenge. Distributed Java projects are highly componetized. Componentization has many advantages as well as challenges, the biggest of which is bringing it all together when different components have been built in parallel.

J2EE projects typically combine a variety of components built on EJBs, servlets, JavaBeans, JSPs, HTML pages, and the database. Each tier of an application is developed separately and follows a different "life cycle" in the development process. For example, a minor change in the underlying database schema or an EJB can cause serious ripple effects in the application, whereas changing a servlet or a JSP does not constitute a risk of the same degree.

This article captures our recent experience in successfully managing a six-month, 22,500 man-hour J2EE project in a startup organization. It focuses on engineering processes that had to be set up to support the development of enterprise software. While we borrowed from different development and design processes, we also had to evolve our own to custom-fit into our environment, as is normally the case with real-world projects.

The primary motivation for writing this article was to pen down the processes, pitfalls, and challenges of successfully completing a fairly complex J2EE project in a bootstrap environment. We describe a practical approach to release and change management for a successful J2EE project including setting up the environment; defining project teams, the tools and processes used; and J2EE-based design, development, and deployment.

The Environment
It goes without saying that the right level of J2EE expertise is a must to make a J2EE project successful. Equally important is having the correct environment for the team to be the most productive. The development environment typically includes:

  • An IDE that supports Java development
  • Configuration management tools and processes that support various distributed components
  • A version-control system that supports Java applications
  • A bug-tracking system
  • Build processes and an overall release methodology
    ­The release methodology has to incorporate the fact that business components are built and released in a different manner than Web components built using servlets and JSPs.
    ­The dependencies of the business layer and the presentation layer have to be taken into consideration in the release process.

    The release methodology in our project was customized to support the multitiered nature of J2EE architecture. The development and release of each tier was done separately so that a defect in the components produced by one team would have minimal impact on the rest of the team.

    The development teams for this project had a couple of seasoned Java architects and developers as well as junior engineers. The end game was to develop a software product that was highly customizable and would adapt to different industry verticals. BEA WebLogic Server was used as the application server. Specific J2EE technologies used in this project are discussed later in this article.

    The project consisted of five major independent applications that reflected real-world functionality. An important objective of the application suite was to ensure that these components could be run as independent products as well as an integrated suite. They were developed as different applications within the same application server.

    The development team was split into five subteams; each team had between two and four developers and was responsible for one application. This turned out to be the optimum team organization for a company of less than 30 employees.

    In all, three different types of environments were set up to support the release management process (see Figure 1).

    Figure 1
    Figure 1

    Development Environment
    The boxes in the left section of Figure 1 depict independent Windows 2000 desktops and laptops. Each developer's personal desktop was used as the development environment. It had access to a database schema that could be used for development purposes and was focused on a particular tier of the application ­ the JSP and HTML tier, the servlet tier, the EJB and business component layer, and the database layer. Each developer worked on a specific layer at a given time.

    Integration Environment
    The middle section of Figure 1 shows the integration environment. A single application server machine was set up to run all five applications within BEA's application server. The application server pointed to a dedicated database schema reserved for integration testing. Components developed separately in the development environment were brought together in the integration environment.

    Staging Environment
    The staging environment consisted of two application servers and a database server. This closely resembled the production. Once the application was integrated and packaged into an installable application, it was promoted to the staging environment.

    Project Life Cycle
    The project was broadly divided into a development phase, an integration phase, and a test phase, as is the norm. However, since we were in startup mode, it was crucial to expect and accept changes in direction in order to market and sell the product suite. While testing was started in parallel with the development phase, it was not until the integration phase that the entire application was available for testing.

    Being a startup organization, an interesting distraction was to support ongoing sales activities during the entire development phase. Demos had to be built in short periods of time for the sales team and required the entire product suite to function in an integrated manner. Such unscheduled activities can potentially derail a well-planned development project. Instead of treating the demos as distractions, we used them as opportunities to develop prototypes for the yet undeveloped features. This enabled the sales team to present a more complete solution to the prospective client. Even as the demos proved to be a great advantage, there were many challenges. The top challenges were:

  • To ensure that the development effort continued while part of the team was diverted to build the demo. Existing activities were reprioritized so that they would have minimal impact on the schedule.
  • To evaluate new requirements (generated from the demos) and their impact on the existing design, functionality, and schedule. If major changes had be made to the underlying data model or in the EJB layer, they were less likely to make it into the release. On the other hand, if new requirements could be implemented by changing servlets or JSPs, there was a greater chance that they would make it into the release.
  • To roll back ad hoc enhancements into the development cycle.

    Due to this continuous feedback, the team was able to redesign certain components to better reflect the requirements from the field. At least one engineer from each team was assigned to work on the demo. Putting together an entire demo was another interesting activity, since it was always a hybrid of well-designed features as well as those features that were just prototyped for the demo. J2EE is an ideal platform for managing a changing codebase because of its inherent support for decoupled components.

    J2EE Technology Components
    The project conformed to the standard three-tier J2EE architecture. The application suite was built using the following:

    • JSPs, servlets, and JavaBeans formed the presentation layer.
    • EJBs encapsulated the business logic.
    • JMS was the messaging layer for the asynchronous exchange of business events.
    • BEA WebLogic Server was used as the application server.
    • BEA WebLogic Integration was used as the J2EE platform for business process management.
    • Oracle 9.1 database housed the business data.
    • Several third-party libraries were deployed to support charts, graphs, etc.
    Change Management
    We used Ant-based compile scripts to set up the integration environment. The staging environment was set up so that the application could be installed via the installer. Microsoft's Visual Source Safe (VSS) was used as the version control system.

    During the development phase, the integration environment was built on a regular schedule. Earlier in the project, the frequency of a new build was once a week. The build frequency increased to 3­4 builds a day toward the end of the release. The steps involved in rebuilding the integration environment were:

  • Check out the latest code base from VSS for the entire product suite along with all necessary configuration files and incremental schema changes if applicable.
  • Build the entire product suite using Ant scripts.
  • Configure all the necessary properties files.
  • Apply incremental schema changes to the database.

    Since each application could be run as a standalone system, the following changes had to be made via configuration files to execute them within the same environment:

  • By default, each application was set up with its own set of libraries, JAR files, classpaths, property files, etc. The applications had to be configured so that they shared a common environment. Thus a common configuration file had to be created that would be used by all the applications, and a common "lib" directory had to be created that contained the shared libraries and JARs.
  • Application-specific EJBs and JAR files were local to each application.
  • By default, each application pointed to a different schema. The properties files had to be updated so applications shared the same database.
  • Some of the HTML files and JSPs had to be modified to provide an integrated user interface.

    Figure 2 shows how the directory structure was set up to ensure that all common components were shared. In Figure 2, application 1 has its own set of classes, JARs, config files, JSPs, etc. It also shares the common classes, JSPs, JARs, etc., from the Common Environment folder.

    Figure 2
    Figure 2

    Each component was developed and tested in its own environment. Once the key features were developed, they were integrated in a separate "integration environment." Once the applications were integrated, they were promoted to the staging environment. The defects found in this environment were reproduced and analyzed in the integration environment.

    Figures 3 and 4 show the change management steps across various environments.

    Change Management of Shared Components
    Shared components implement functionality such as user management, common logger, and audit trail. Common functionality is shared across many applications. All EJBs, property files, JSPs, or database entities that support such common functionality have to be carefully managed since it impacts more than one application. Additional coordination is necessary to ensure that other teams can continue to be productive. The changes were completely tested in the integration environment before checking them into VSS. In other words, the next steps were followed to ensure the quality of the shared components:

  • Obtain the most current code base from VSS.
  • Obtain the modified shared components from the developer's environment and overwrite the code base from VSS.
  • Compile the application using the integrated build script.
  • If the functionality works as expected, immediately check the modified components into VSS.

    Change Management of the Business Logic Layer
    Sometimes changes in the EJB layer meant changing the presentation layer. JSPs and servlets may have to be fixed to reflect new changes in the EJB layer. Such changes were coordinated by the development team and implemented in the development environment. Developers working on the presentation layer used the updated EJB JARs and ensured that the presentation layer worked. All the changes were then checked in at the same time. Figure 3 shows the steps involved.

    Change Management of the Database Schema
    Typically, schema changes made to support new features are most likely to break the system. Schema changes made to fix defects must ensure that existing features are not impacted. However, they were discouraged in this project, and were made only if they impacted critical functionality. Change management of schema changes is shown in Figure 3.

    Figure 3
    Figure 3

    Change Management of the Presentation Layer
    Figure 4 shows the change management steps in the presentation layer. It can be seen that some changes are rolled forward from the development to the staging environment and some changes have to be rolled back from the staging to the development environment. Since the presentation layer has the least number of dependencies, it can be changed easily in any environment. During testing, it was sometimes easier to tweak a JSP in the test environment itself. These changes were then applied to the source in VSS. The challenge here was to be disciplined and roll the changes back into the development environment.

    Figure 4
    Figure 4

    Tools and Processes
    Various tools and processes have to be in place to support software development. Some of the key management aspects that must be addressed are time, process, version control, configuration, and defect. Being a startup organization, the team had to set up each and every process and tool from the ground up. The following are some of the criteria used to select these tools and processes:

    • Should adequately support the requirements
    • Be easy to use, configure, manage, and measure
    • Be cost effective
    • Be scalable to support a growing team for at least two years
    • Have a relatively low learning curve for the team
    • Be practical to use and not create additional work
    Configuration Management
    Configuration management controls and monitors change throughout the software development life cycle. Since we had to branch out often to support ongoing demos, it was extremely important for the team to follow a consistent configuration management process. Visual Source Safe (VSS) was used for configuration management and version control. Every element of the project was under configuration management right from the get-go, including:
    • Requirements documents
    • Configuration files
    • External libraries
    • Source code
    Defect Tracking
    There are several defect-tracking tools available under public domain that work well in small organizations. A tool called codeCharge code generator was used to build the bug-tracking tool. CodeCharge generator allows a developer to quickly create a Web-based application in a variety of languages including ASP, ASP.net, JSP, and ColdFusion. It supports all standard relational databases including MS Access. This tool enabled us to incorporate new features without much down time.

    Build Process
    Each developer set up his or her own build environment. The solutions ranged from using the built-in functionality of an IDE to using Ant and the basic batch script. Developers were not allowed to check in functional enhancements without ensuring that the entire application compiled and the new functionality worked in the local environment. A common build script using Ant was developed for each application. Thus, getting the most current version was as simple as obtaining the current code base and building it with the Ant scripts. The build scripts failed very rarely since the developers validated each and every code change.

    Test and QA Process
    You rarely have the luxury of a dedicated QA team in a startup environment. While the development team was busy building the new product, the domain experts were busy creating a functional test plan. A "war room" environment was set up for the entire development team for two weeks following the integration phase. With help from subject matter experts, two weeks were dedicated to testing the new product. Defects were documented and classified. Defects that prevented the execution of key functionality were classified as "show stoppers" and fixed immediately. The remaining defects were prioritized and either fixed following the test cycle or documented and scheduled for the next release.

    Challenges and Compromises
    Embarking on a new J2EE project is always challenging, especially in a small growing environment where the success or failure of the project depends on each decision ­ from selecting the right tools and setting up processes to making hard decisions and managing expectations. This project was no different. Some of the project management challenges were:

  • Requirements Management: Since the sales team was able to provide feedback well into the development cycle, requirements were never really frozen. Each new requirement was carefully evaluated and prioritized. If a new requirement was deemed to be critical for the release, it would be further analyzed and the impact on the schedule and design evaluated.
  • Testing and QA: A unique environment involving the entire team had to be created to test the product. The war room environment helped the team focus on the task at hand and iron out the rough edges.
  • Expectation Management: Since this was a new team and each person brought in unique experiences from prior jobs, managing the team's expectations was a high priority. The team had to be reminded frequently that the integration environment was for integration testing, not development, and that changing requirements have a huge impact on the quality of the product, etc.
  • Time to Delivery: As with most development projects today, time-to-market was the most critical metric for this project. Even though it was challenging, the project was completed on schedule by making certain compromises.
  • Mentoring: The project plan had to include mentoring activities, given the varied levels of J2EE expertise in the development team. Since there was a learning curve, productivity metrics had to adjusted for the entire team.

    To meet the goal of delivering a quality product on time, several compromises had to be made. Several performance-related issues were identified, but not resolved as this would have pushed the project beyond the release date. Portability was postponed to a future release. Compromises also had to be made when selecting tools and processes, since the bootstrap environment did not justifying buying commercial tools such as Rational Rose, ClearCase, WinRunner, and TestDirector. In some cases, functionality that should ideally be developed in the EJB layer was developed in the servlet layer to meet the time-to-market pressures.

    Conclusion
    J2EE projects can be completed on time by following the basic principles of project management. They can prove to be more challenging, as they target enterprise applications that require large integration efforts. Identifying potential risks and then taking quick actions to mitigate them keeps the project under control. This prevents small "fires" from destroying the entire project. Technical issues are relatively easy to resolve as compared to people and process issues. It's the responsibility of project management to get a buy-in from the stakeholders on the processes and tools to be used.

    It's human nature to resist change, especially when you don't see value in the process. Using best practices as a guideline and making pragmatic decisions go a long way in making any project successful. Fortunately, J2EE is mature enough, so you can learn from other people's experiences.

    Author Bios
    Kunal Shah is the director of product management at Controlling Factor. He is well versed in Java-based product management, ranging from small to large enterprise projects [email protected]

    Ajit Sagar is the J2EE editor of JDJ and the founding editor of XML-Journal. He is the director of engineering at Controlling Factor, a software firm in Dallas. He is well versed in Java, Web services, and enterprise technologies. [email protected]

    All Rights Reserved
    Copyright ©  2004 SYS-CON Media, Inc.
      E-mail: [email protected]

    Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.