HomeDigital EditionSys-Con RadioSearch Java Cd
Advanced Java AWT Book Reviews/Excerpts Client Server Corba Editorials Embedded Java Enterprise Java IDE's Industry Watch Integration Interviews Java Applet Java & Databases Java & Web Services Java Fundamentals Java Native Interface Java Servlets Java Beans J2ME Libraries .NET Object Orientation Observations/IMHO Product Reviews Scalability & Performance Security Server Side Source Code Straight Talking Swing Threads Using Java with others Wireless XML

Making A Mountain Out Of An Anthill, by Neal Ford

An unfortunate consequence of the pace of technological advancement is the lack of knowledge among new developers concerning the lore and tools of previous generations. While much of old technology is quaint and should be left behind (how many programmers do you know that miss PDP 11 Assembly language?), some good ideas were there as well.

One of the not-so-old and not-completely-forgotten programmer utilities of the past is the make utility. I'm not suggesting that make isn't still used (especially in the UNIX world), but the percentage of programmers who know what a make utility is and what it does is pretty small. Yet the need for its functionality has never been greater, particularly for large-scale Java projects. I hope this article will return it to the forefront where it belongs.

First, for the unenlightened, a little background on what a make utility does. Then I'll discuss the best thing to happen to Java since cream and sugar - Ant.

Make Utilities
Back in the dark ages of software development (about 10 years ago), most code was written in a good editor (back when the editor was the IDE). To make sure it was all compiled and linked correctly, a make utility was used. A make utility is a command line utility that uses a text file that contains a bunch of rules to transform one thing into another thing. For example, the make file would define the relationship between a ".c" source file and the ".OBJ" file generated by the compiler. The make file would then follow the rules (i.e., how to invoke the compiler) to make the transformation happen. You could also define dependencies to control the order in which artifacts were created. For example, you must compile the source files before you deploy them. Make files don't flow from the top of the file to the bottom. You tell the make file what the ultimate target is and it executes the necessary steps (in the correct order) to fulfill the target.

An example is in order. Listing 1 shows a simple make file for a C++ project (all code listings can be found below). The top of the make file defines what files make up the project. Next, compiler flags are listed as properties of the make file, and the compiler executables are defined as properties. In the last part of the make file, each line defines a transform for how to convert one file into another file. Thus, to convert a .cpp file to an .obj file, you use the compiler command (with the flags) listed in the make file. You'll also notice that the dependencies are implied. In other words, you can't make the .EXE file until the .OBJ files have been created.

Even though this is a simple concept, make files can perform a great service. They exist to handle repetitive, multistep processes that change in content (source files) but not in tasks (how to compile, deploy, etc.). Make utilities exist for all different platforms. In fact, most versions of UNIX have a make utility built into the operating system.

Traditional make utilities are a little ornery, however. The make file syntax must be in an exact format, and the format is specific to the version of the make utility. Different implementations of make utilities require slightly different syntax. As you can see, the syntax of the make file is somewhat daunting - it's like learning an entirely new development language. The make file also defines special wildcards (like the "@&&!" wildcard in Listing 1).

The bigger concern is that make utilities aren't well optimized for Java. For example, Java has very special requirements for package and directory structures. Make utilities don't natively understand this relationship. This isn't to say they haven't been used quite successfully for Java development. However, what was really needed was a new make utility that understands Java. That's where Ant enters the picture.

Ant
Ant, a make utility written in Java for Java, is part of the impressive Jakarta project at Apache. In fact, it's one of the Jakarta subprojects. The Apache developers needed to be able to automate the build process and they couldn't do it gracefully with traditional make files, so they wrote their own. It turned out to be so useful that they released it as its own subproject. And because it's from Apache, it's open source - the source code is freely available. You can find Ant in both binary and source form at http://jakarta.apache.org/ant". To install ant, you unzip it into some directory structure and take the following three steps:

  1. Set the ANT_HOME environment variable to point to the installation directory.
  2. Add the ant/bin directory to your system path (Ant is started with a bat file in the /ant/bin directory, so you'll probably want to put that directory on your path for easy access).
  3. Set the JAVA_HOME environment variable to point to your JDK installation directory.
Ant differs in several ways from traditional make utilities. First, the make file itself isn't in a proprietary file format but is an XML document called build.xml by default. Within this document you specify a series of targets. Each target encapsulates some task or set of tasks that you need to perform. Ant also defines a set of built-in tasks to handle everything from copying files and directories to e-mailing someone. For a short partial list of the built-in tasks see Table 1. A full list of the tasks is included with the docs that come with Ant (as of the current version, there are 47 predefined tasks). Moreover, because it's written in Java, you can create your own Ant tasks by subclassing built-in tasks.

table 1
Table 1: Summary of a few predefined Ant tasks

Generally, you define a build.xml file as a series of targets, each consisting of one or more tasks. Listing 2 shows a simple build file for Ant. Notice that the top of the file starts with an "init" task. This is typical in that it allows you to define properties that can be used throughout the build document. You can think of these as global variables that can be used anywhere within the build document. This is typically where directories, compiler settings, and flags are set up. By defining them at the top of the file, you can make changes to the values that flow throughout the entire document. The next target, named usage, allows the developer to invoke the Ant make file and find out what targets are available. This is help for this particular make file. The third target is called prepare; the fourth, prepare-src.

Automated builds can sometimes take a long time for large projects. It would be convenient if you could continue working on the project while the build is proceeding. Thus one of the early tasks typical in a build file is a task to create a build working directory and copy all the source files into it. The build utility can then work on the source code in the new make directory while you make changes to the mainline code.

Notice how Ant handles directory structures. Because Ant was written for Java, it understands how Java packages and directories are related to one another. Ant defines a special file-based wildcard: "**". This is like the normal wildcard "*", which matches anything in the current directory. However, the "**" wildcard matches everything in this directory and all its subdirectories. So if you tell Ant to copy /src/**/*.java, it will copy all the Java source files in all the subdirectories of the /src directory. In addition, when Ant copies source files, it leaves the directory structure in tact.

The next target in the build file is the Java compiler invocation. It will compile all the source files and place them in the destination directory defined by a property above. This task also shows how you can specify classpath information for tasks that need a classpath. You can define the classpath attribute in two ways. You can use a semicolon-delimited list, much like the classpath environment variable is defined. But that creates an attribute that's hard to read.

The alternative, shown here, is to use a nested property. In this case the classpath is defined as a special attribute, which has its own attributes, making it much easier to see what's on the classpath and make changes.

A Javadoc task is defined next. You probably don't want the Javadoc process to run every time you build your project. You'd probably rather run Javadoc only at certain stages in the project. To invoke Ant for a particular target, you can pass the name of the target on the command line. Ant will then try to fulfill that target. However, each target may have a "depend" attribute. Ant will ensure that every dependent target is run successfully before executing the next target. Notice that the "compile" target lists "prepare-src" as the target it's dependent on. So, like traditional make files, the flow of execution doesn't necessarily go from the top of the document to the bottom. The dependencies between the targets determine the order of execution. You can also define a default target at the top of the file - this is the target that will be fulfilled if no specific target is passed on the command line. Thus the Javadoc target will be executed only when you specify that you want Javadoc.

The last target is the "clean" target. It goes through and cleans up all the working directories created by Ant.

When you run Ant, you can pass it an XML document and a target. If you've named your build file "build.xml" and it's in the directory where you start Ant, you don't have to specify the build file name. Moreover, if you're building the default target, you don't have to specify a target either. Most of the time, therefore, you'll just invoke Ant on the command line. Ant will show each target that it executes in turn, printing status messages for individual tasks. Figure 1 is an example of a typical successful build. However, if any task can't complete successfully, the entire build process is aborted. Figure 2 shows an example of a build that failed.

figure 1
Figure 1:  Ant shows the status of each task and whether or not the build suceeded.

figure 2
Figure 2: When a build fails, Ant stops on the task that fails.

Because Ant is written in Java, it can be extended using Java. If you have a particular tool that you need to support in Ant, you can build an Ant task to handle it. Some of these custom tasks have already been written and are available from the Ant Web site. For example, many developers use Visual SourceSafe as their version control package. Ant has support for CVS built in, but there's no support for VSS. However, if you download the optional.jar file from the Ant site, you'll discover that someone has already written the integration for VSS for Ant. The optional JAR file contains several tasks that were written and contributed to the Ant project that aren't mainstream enough to become built-in tasks but are still useful. To use one of the optional tasks (or one of your custom tasks), you have to make Ant aware of what class to use to define the task. This can be done with a taskdef task, generally placed in the init section of the build file. A sample task registration is shown below:

<taskdef name="mytask"
classname="com.mydomain.MyVeryOwnTask"/>
You can use the MyVeryOwnTask as if it were a built-in task. If you want to register a new task with Ant more permanently, you can add the task to the default.properties file in the org.apache.tools.ant.taskdefs package. To use the optional VSS task, add a taskdef line to your build file and use the following syntax to invoke it:
<vssget localPath="C:\mysrc\myproject"
recursive="true"
label="Release1"
login="me,mypassword"
vsspath="/source/aProject"
writable="true"/>
Details on both the optional tasks and how to build your own tasks are included in the Ant documentation.

Enterprise Deployment
As you can see, Ant makes it very easy to consolidate all project construction information in a single location. You can define classpaths and other build artifacts once and reuse them throughout the build file. For simple projects like the one shown above, however, most modern IDEs handle some of these chores (although none that I have seen handle it as well as Ant). Ant really begins to shine when you look at more complex applications that have more complex dependencies or complicated distribution requirements.

A major headache in building Web applications in Java concerns the deployment of the required files to all their appropriate locations. For example, in most Web projects you have a mixture of HTML, JSPs, JavaBeans, and servlets. All these files (except HTML and JSP) go into different destinations. Each Web server has its own definitions of where these files belong. Although this has gotten better since Sun introduced the concept of a WAR file and the ensuing directory structure, the deployment headache still exists. If you forget to copy even one of these files when you deploy your project, the magic just doesn't work. This sort of tedious task is exactly why tools were built. Obviously, you can build an Ant build file that can handle not only the compilation chores for each Web project, but these annoying deployment chores as well.

One of the nice characteristics of Ant is that it does the minimum amount of work necessary. So, if a file already exists in the target location and it has the same timestamp, Ant won't bother copying it. This is contrary to the solution that many developers currently use, the lowly batch or script file. The disadvantage of batch and script files is that they generally do the most amount of work necessary rather than the least. Some scripting languages allow you to build in more intelligence, but you must remember to add it. Ant does the smart thing automatically.

If you look at the build and deployment process for multiple Web projects, it looks as though they do basically the same things, just for different sources and destinations. Rather than build a separate Ant build file for every Web project, it seems as though you could build one Ant build file and change just a few characteristics to make it work for almost all Web projects. This generic build file for Web projects (see Listing 3) makes a couple of assumptions about the way you have your project defined.

First, it assumes that there is a source directory (named /src, but this detail is easy to change) that consists of a directory/package structure containing your Java source files. It also assumes that the JSPs and HTML are defined in the /src directory without being inside the package hierarchy. Last, it assumes that the JavaBeans used as library classes by your servlets are not in the same directory or directories as your servlets. You may want to alter the build.xml file to change some of these assumptions.

A nice Ant feature demonstrated in Listing 3 is the ability to define Ant properties outside the Ant build file and import them. Listing 4 shows the properties for a specific Web project. Ant uses standard java.util.Properties files and a special property attribute to import properties. To import the properties file in Listing 4, the following task line is defined in the init target:

<property file="project.properties"/>

This imports the properties in the named file, which become indistinguishable from properties defined directly in the Ant build file. This mechanism makes it easy to define generic build files that can be customized without even editing the build.xml file.

The other monstrous build and deployment task typically found in enterprise Java development involves Enterprise JavaBeans. They must be developed, jarred, handed to a deployer tool for the application server, copied to a deployment location, and optionally unjarred in the appropriate place so the client has access to stub code. This is obviously a job for Ant! In fact, there is a relatively new set of Ant tasks defined especially for EJB tasks. Currently, the tasks defined for EJBs support only WebLogic 4.5.1 and 5.1, although I'm sure other tasks will appear as needed. If you don't use WebLogic, here's your chance to add to the open source movement! Of course, as long as your application server supports command line tools, you can use the exec task in Ant to invoke the tools for your application server.

In fact, before there were specific tasks for EJBs, I wrote an Ant task to execute WebLogic's ejbc compiler to generate all the necessary code for deployment. Just like WebLogic, most vendors give you Java versions of their deployment tools, all of which are easy to launch from within Ant. One advantage this has over batch files is that Ant can spawn the VM in a new thread rather than in a new process. Thus the start-up overhead is much less when Ant launches a Java application versus when a batch file launches the same application. In an organization that used batch files to check out, build, and deploy a large number of EJBs, I wrote an Ant build file that reduced the amount of time required from 47 minutes to 15 minutes! The entire time savings were realized by doing the minimum amount of work rather than the maximum and by spawning threads instead of processes.

The last supported task type in Ant that I'll mention is integration with the open-source testing framework JUnit. In a nutshell, JUnit defines a framework to automate unit testing for your Java classes. It includes both a command line and a graphical test environment. The JUnit task is defined in the optional JAR file and can invoke JUnit tests as part of the build process. The typical scenario involves building classes and defining tests, and making sure they're run each time the project is built. If subtle bugs appear during subsequent development, the unit test fails and can optionally fail the entire build process. Alternatively, you can define a task to e-mail the build manager whenever a class fails its regression test.

Ant's ability to guarantee consistent builds and correct deployment, and its automatic unit testing make for a powerful tool indeed. It automates part of the development process that should be automatic. Developers are too busy with more important jobs than making sure that files are copied correctly. Ant is powerful, extensible, convenient. And the source is included and the price is right!

Author Bio
Neal Ford is vice president of technology, DSW Group, Atlanta, Georgia, He is also the designer and developer of applications, instructional materials, magazine articles, and video presentations, and author of the books Developing with Delphi: Object-Oriented Techniques and JBuilder 3 Unleashed. Neal has been the featured speaker for the Borland Delphi/C++Builder/JBuilder World Tours, and has spoken extensively at annual Borland Developers Conferences worldwide. He can be contacted at:[email protected]

Download Assoicated Source Files (Zip format ~ 5.79 KB)

All Rights Reserved
Copyright ©  2004 SYS-CON Media, Inc.
  E-mail: [email protected]

Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.