Testing is usually an afterthought in the development process. The developer's main focus is to design and write code.
Of course, the developer runs the program many times during development to make sure the code runs and produces the expected results; however, this testing has no real structure and the main goal is to ensure the program runs at that moment. Most developers rely too much on QA or the end user to make sure the program works properly and meets requirements.
Extreme Programming has taken the "build a little, test a little" philosophy to a new level by requiring that all classes have unit tests that are run on a regular basis. A unit test is a structured set of tests that exercises the methods of a class. Unit testing is a good idea for the following reasons:
As unit tests are built for each class, they can be collected and run as a group on a regular basis to track the progress of the project and find bugs as they occur.
- Gives the developer confidence that the class is working properly
- Quickly finds the source of bugs
- Focuses the developer on the design of a class
- Helps refine the requirements of the individual classes
Although there are many good reasons to create unit tests, most developers don't believe it's worth the time or don't want to take the time to develop a unit-testing framework. Luckily Erich Gamma and Kent Beck have developed JUnit, a simple unit testing framework that makes developing unit tests almost painless. You can download a copy of JUnit at www.junit.org.
The JUnit Framework
JUnit is a set of classes that allows you to easily create and run a set of unit tests. You use JUnit by extending its classes. Figure 1 shows JUnit's main classes and their relationships.
The central class of JUnit is TestCase. TestCase represents a fixture that provides the framework to run unit tests and store the results. It implements the Test interface and provides the methods to set up the test condition, run tests, collect the results, and tear down the tests once they're complete. The TestResult class collects the results of a TestCase. The TestSuite class collects a set of tests to run. JUnit also provides a command line class (junit.textui.TestRunner) and a GUI class (junit.ui.TestRunner) that can be used to run the unit test and display the results.
To illustrate how to use JUnit, let's write a simple unit test to test some methods of the Java String class. The code, along with a batch file needed to run the unit test, can be found below in a zip file format. Listing 1 shows the code for the unit test. The class is called StringTester and is a subclass of TestCase. The constructor has one argument, name (the name of the unit test being run), that will be used in any messages displayed by JUnit. The next method in the listing is setUp(), which is used to initialize a couple of strings.
The next three methods (test-Length(), testSubString(), and testCon-cat()) are the actual tests. The test methods must be no argument methods that test a specific condition using one of JUnit's assert methods. If the conditions of the assert are true, JUnit marks the test as passed, otherwise it's marked as a failed.
The suite() method is used by JUnit to create a collection of tests. There are two ways to create a suite:
Adding tests individually to a suite is a good way to create a subset of tests to try to isolate a bug. Both JUnit classes used to run the unit tests require the suite method to be present.
- Create an instance of the TestSuite and then add the individual tests to the suite.
- Create an instance of TestSuite by passing the test class in the constructor; this creates a suite that contains all the methods in the test class that start with the word test. This is the preferred way to create a suite, since it's simpler and helps prevent the accidental omission of tests.
The class has a main method that can be used to run the unit tests. The main method calls the junit.textui.TestRunner.run method to run all the tests of the suite. The main method is useful if the unit tests are run by a batch file. The tests in this class can also be executed using one of the following commands in a console window:
java junit.textui.TestRunner String-Tester
java junit.ui.TestRunner StringTester
These assume that the junit.jar file is in the classpath. The first command runs the unit tests in command line mode and prints the results to the screen. The second command brings up the JUnit GUI, which shows the results graphically (see Figure 2).
The StringTester class has an intentional bug in it to show what happens when a test fails. In the testLength class the correct length of the string is really 12, not 11. When a test fails, JUnit reports which test failed and prints out the expected and actual values that failed.
Having an error in the test code brings up a good point. The unit tests should be tested to make sure there are no errors in the test code. The best way to do this is to write the tests first and then stub out the code to be tested. When the unit test is first run, all the tests should fail. Then as the code is filled in, executing the individual unit test helps ensure not only that it passes but also that the test is correct.
That's all there is to writing a unit test with JUnit. The framework is very easy to use. The hardest part is getting used to writing the tests as you write the actual classes. The process of writing unit tests will initially take more time during development, but it will pay great dividends when regression testing needs to be done or a high-risk change needs to be made to the code.
The Value of Unit Testing in the Real World
To show the value of JUnit in a real development environment let's look at what a set of completed unit tests for a moderately complex piece of software looks like and how JUnit improves software development.
This software calculates the position of a planet for a given date. There are four main classes that perform the following functions:
Calculate the of the number of days between 1/1/1980 and the given date.
Calculate the heliocentric coordinates of the planet.
Convert the heliocentric coordinates to right ascension and declination.
- CalcPlanetPosition: This is the main class used to calculate the position of a planet given a planet number and a date, and it involves three major steps:
Each class has a corresponding unit test class. The name of the unit test class is the name of the class with the word "Tester" added. The DateUtils class has three main functions. It determines whether the user entered a valid date (isDateValid), calculates the number of days between two dates (calcDateDiff), and calculates the difference between 1/0/1990 and a given date (Calc-NumOfDays).
- DateUtils: Calculates the number of days between 1/0/1900 and the given date
- HelioPos: Calculates the heliocentric coordinates of the planet
- ConvertUtils: Converts the heliocentric coordinates to right ascension and declination
The unit test for the isDateValid method tests a valid date of 8/1/1989 and an obviously invalid date of 13/32/0000 to test basic functionality. It also tests some not so obvious invalid dates like 9/31/1992 and 2/29/2001. For the calcDateDiff method, tests have been written to ensure that the number of days between two dates work when the dates are separated by a day, week, month, and year. There's also a test to ensure that the calculation works in a leap year.
Once the framework is set up it's easy to add new test cases as bugs are found. The complete set of source code, along with the unit tests, is contained in the PlanetPos.jar file and can be downloaded from the JDJ Web site.
The unit tests for the other classes follow in a similar manner. A special class called SystemTester runs the entire set of tests for the project. The class is a subclass of TestSuite and has a main() and a suite() method. The class creates a test suite that contains an entry for each of the unit test classes.
Once completed, the tests can be run on a regular basis to check for bugs as the development process continues. Once the program is fully working and tested, it's sent to QA where more rigorous testing occurs.
In this example, even though the code was written according to the requirements and passed all the unit tests, there are still two significant bugs that have been found by QA. The first problem is that the calculation of the planet position is not accurate enough for certain planets for certain dates. The second problem is that there's no way to enter dates before 1 AD.
To find the source of the bugs the first step is to write a set of unit tests that reproduces these bugs. First, new test cases are added to test the main class (CalcPlanetPostion) to confirm the results that QA has found. These new test cases will fail, but they accurately communicate the conditions of the bug. The next step is to write new test cases that test the calculations made for the other three classes (DateUtils, HelioPos, and ConvertUtils) for the failure conditions.
At this point the developer may need to consult with the user who wrote the original requirements to determine the pass/fail criteria for the new test cases. Again, the test case serves to communicate the information needed from the user in order to refine the original requirements. Once the new pass/fail criteria are determined, they become new requirements on the system. In this fashion, as bugs are found and features added, the test cases become a set of requirements of higher and higher fidelity that gets tested each time the unit tests are run.
In this particular case it turns out the date format used throughout the project assumes that the dates entered will be after 1 AD. Also, the formula used by calcHelioPosition in the HelioPos class is not good enough to produce accurate results for all cases. A number of methods will have to be rewritten. The rewritten code and unit tests are in the file PlanetPos_Reworked.jar available on the JDJ Web site.
At this point it's late in the development cycle and rewriting these core methods is high risk. But the unit tests that are now written make the risk much lower because once the changes are made, the full set of unit tests can be run and the developer can be confident that he or she has "done no harm."
It's important to be methodical and consistent when developing unit tests for a piece of software. The development of a unit test should be treated with the same care that's taken when developing other code. A number of best practices should be employed to keep the unit tests manageable and useful.
1. All unit test classes should have a main().
Although the unit tests will normally be run as a group, there are times when it's necessary to run just one of the unit test classes to isolate a bug. Having a main method that just calls the junit.textui.TestRunner.run method makes that easy.
2. Make sure your test cases are independent.
As the test cases in a unit test are developed, it's important to make sure that each test case can be run independently. This is especially true when testing database applications where the state of the database at the start of a test case must be consistent in order for the test to be effective. It's possible that a test case will incorrectly handle an exception that occurs during a test run and corrupt the database. This can cause all subsequent test cases to fail. Nothing is more annoying than spending hours debugging an application that appeared to be failing because the test cases were not set up properly.
Don't count on the test cases being run in a certain sequence because the JUnit framework doesn't guarantee the order of test method invocations. The setup and teardown methods are called before and after each test case is run. They can be used to set up a database to a known state before a test is run and restore it to a known state after the test is complete. It's also very important to make sure that test methods make every effort to clean up after themselves in every conceivable exit condition.
3. Minimize the amount of code in setup/teardown methods.
Since the setup and teardown methods are called before and after each test case, it's important to make sure they're efficient. Some test suites can contain 50, maybe even 100, test cases. This includes making sure that these methods don't output unnecessary log messages and make it hard to follow the execution path. Making these methods efficient will also speed up the time it takes to run the test, which allows the tests to be run more often.
4. Use fail() instead of assert() to catch test case error.
If the test case fails because of an error or exception as opposed to an assert that fails, it's better to use the Assert.fail(String msg) than assert(true,false). This will make it easier to wade through the debug statements and can decrease the amount of time required to understand and fix a problem. Using Assert.fail(String msg) with a description of the failure instead of calling assert(boolean boolValue) can more directly describe the type of failure occurring. The message passed to fail(String msg) will be printed out by the TestRunner in the stack track of failures. This allows the developer to quickly see what's really failing in a test case.
5. Properly document test code.
As with any code being developed, it's important that the test code be properly documented with Javadocs. Just as it's easier to maintain well-documented code, well-documented test cases are easier to update.
Things That JUnit Can't Do
JUnit does not separate test data from test code since the data is hard-coded in JUnit. This could be a problem when testing database-driven applications. The JUnit ++ extension solves this problem by creating a test data repository.
JUnit can't unit test Swing GUIs, JSPs, or HTML. But there are JUnit extensions, JFC Unit and HTTP Unit, that can do these things.
JUnit tests don't take the place of functional testing. If all your unit tests pass, it doesn't mean the software is ready to ship. Functional testing still needs to be done to ensure that the software meets the full set of requirements including performance criteria, hardware requirements, and usability.
Testing should not be an afterthought in the development process. It's a well-known fact that the cost of fixing a bug goes up drastically the later it is found. Unit testing provides a way to find bugs almost as they occur. It also improves the design process and provides an effective way to communicate bugs and requirements between members of the development team. JUnit is an easy-to-use framework that can be used to test any Java class.
Although you may not favor all aspects of Extreme Programming, unit testing will improve any development process and the quality of software produced.
Gamma, E., and Beck, K. "JUnit: A Cook's Tour."
Thomas Hammell is a senior developer at Hewlett-Packard/Bluestone
Software and is part of the tools group that develops various
tools for HP middleware products. He has over 15 years of
experience developing software. Tom holds a BS in electrical
engineering and a MS in computer science from Steven's
Institute of Technology.
Robert Nettleton is a software engineer at Hewlett-Packard/Bluestone
Software. He currently works as the test lead for HP/Bluestone's
Enterprise JavaBean container. He has 5 years of experience
developing and testing object-oriented software in C++ and Java.
Bob holds a BA in computer science from Rutgers University.
Download Source Files (~ 18.1 KB ~Zip File Format)