HomeDigital EditionSys-Con RadioSearch Java Cd
Advanced Java AWT Book Reviews/Excerpts Client Server Corba Editorials Embedded Java Enterprise Java IDE's Industry Watch Integration Interviews Java Applet Java & Databases Java & Web Services Java Fundamentals Java Native Interface Java Servlets Java Beans J2ME Libraries .NET Object Orientation Observations/IMHO Product Reviews Scalability & Performance Security Server Side Source Code Straight Talking Swing Threads Using Java with others Wireless XML

Graphical user interface (GUI) testing is a potentially problematic area because constructing effective test cases is more difficult than the corresponding application logic. The roadblocks to effective functional GUI testing are:

  • Traditional test coverage criteria like "80% coverage of the lines of code" may not be sufficient to trap all the user interaction scenarios.
  • End users often use a different user task interaction model than the one conceived by the development team.

    Functional GUI testing needs to deal with GUI events as well as the effects of the underlying application logic that results in changes to the data and presentation.

    The common methods for functional GUI testing are the "record and execute" script technique and writing test programs for different scenarios. In the "record and execute," the test designer interacts with the GUI and all the events are recorded in a script. The script can later be replayed to re-create user interactions for a particular scenario. In the test programs, the test designer tries to understand and write tests for the various GUI decision points.

    This article discusses how Abbot can be used to quickly and effectively come up with a comprehensive GUI test framework for Swing GUI applications. Abbot (http://abbot.sourceforge.net/) is a JUnit extension for Swing GUI testing. It provides an interesting framework that can be used for test case generation as well as "record and execute" scripting.

    Introduction to Abbot
    Abbot builds upon the java.awt.Robot class to provide an automated event generation and validation framework for Swing GUI components. The framework can be used to create, record, and execute scripts and programmatic test cases in Java. Abbot also has a script editor called Costello that facilitates the creation of scripts in XML. The framework can also be well integrated to run with JUnit.

    To illustrate the use of Abbot to create a GUI test infrastructure, this article will make use of the following two scenarios.

    1.   GUI already exists: The GUI has been coded and the application needs to be tested, but no unit tests are available. This scenario will primarily make use of the "record and execute" style and then focus on how test suites can be created and run with JUnit.

    2.   GUI has not been coded: The GUI has been designed on paper; however, no working code exists. This scenario will primarily focus on creating test cases in Java. This method will create a homogeneous suite of tests both for the back-end application code and the GUI.

    The two scenarios focus on the end-point cases in functional GUI testing. The principles mentioned can be mixed and matched to suit the needs of the project. To get started with Abbot, download the JAR files and place them in the lib directory of the project.

    Scenario One: GUI Already Exists
    In this scenario, most of the GUI has already been coded. Automated testing is not available and the testing is mainly being done by hand. The goal is to come up with a test suite in the shortest time possible so that the quality metrics for the project can be met effectively.

    The script editor Costello will be used to develop a battery of tests. Costello provides the "record and execute" functionality that will allow us to record different user interaction scenarios with the GUI and test those scenarios efficiently. The scripts that record the user interactions are saved as XML. The salient features of the scripts that help in rapidly creating a functional test framework are:

  • Focuses on test creation: The XML-based scripts can be written rapidly (directly or by using Costello) and are dynamically interpreted by Abbot. There's no need to write new code and compile test cases.
  • Focuses on validating GUI decision points: Abbot allows the test creators to add assertions to check for values resulting from user interactions. The assert tag is used to check for valid results from a user interaction before the script proceeds. This feature is very useful, as it helps pinpoint the step in which the GUI fails and aids in regression testing.
  • Provides infrastructure to create robust scripts: Some GUI "record and execute" tools produce very fragile scripts. These tools not only store the events but also the GUI component position information, such as coordinates, to identify them. These scripts are very fragile because any change in positional attributes (changes in layout or running the script on a different platform) breaks the scripts. Abbot uses a number of attributes to dynamically identify the component without depending on any positional attributes. These attributes in aggregate form a component reference, which can be used within the script even before the component itself necessarily exists.
  • Provides a high-level abstraction over the Java Robot class: Abbot builds on top of the Java Robot class by providing a layer of abstraction. The abstraction layer allows end users to create test cases that can capture high-level semantics like the OK button, Select CO from the list of states, etc. This makes the scripts easy to understand and enhances them in the future.

    An example of a test script in XML is provided in Listing 1 .

    With this background about the features of the script, it's time to look into how to use the script editor to create the test cases. The sample GUI shown in Figure 1 is an example from the Java Swing tutorial available at java.sun.com.

    figure 1

    The GUI is a Celsius-to-Fahrenheit converter that takes in a numeric value and on the button click displays the converted result in Fahrenheit. To test this GUI, we can write some simple GUI tests to verify the results on the input of a positive number, a negative number, and a nonnumeric input.

    The script editor will be used to create the test cases. Costello, the script editor, can be started by:

    java -jar lib/abbot.jar

    Once the GUI is running, the first step is to create a new script (File->New Script). Each script has a launch line that specifies the method (containing the GUI) to be tested. The launch line is edited with the information that can launch the GUI. The launch information requires the method name, arguments for the method, class information, and the classpath (see Figure 2).

    Figure 2

    method="main"
    args="[]"
    class="CelsiusConverter"
    classpath="src/demo"

    Note that the XML test script example has the same information in the launch tag.

    With the launch information in place, the GUI can be launched (Test->Launch). This will prepare the framework to record user actions. To record user actions, press F2 (Capture->All Actions). With this step we can record the first case, which inputs a positive number, and then press convert.

    Once the user interactions have been captured, we can move the actions out of the sequence block for better readability. With the script looking the way we want, it's now time to add an assertion. To do this, we'll use the Hierarchy tab to navigate to the GUI component. Upon selection of the GUI component, the name-value pair to be validated can be selected and the Assert Property = Value button will add the assertion (see Figure 3).

    The other two test cases can be created similarly to form a test suite. The test cases can run within the JUnit test harness utilizing either the command line or the GUI test runner. The junit.extensions.abbot.ScriptFixture class is subclassed to create the CelsiusConverterTest.

    Figure 3

    public class CelsiusConverterTest
    extends ScriptFixture

    The junit.extensions.abbot.ScriptTestSuite class is used to autogenerate a suite based on test scripts matching a certain criteria. In this case, all scripts residing in a particular directory will be used. The test ScriptTestSuite class is subclassed from the junit.Framework.TestSuite class.

    public static Test suite() {
    return new ScriptTestSuite
    (CelsiusConverterTest.class,
    "src\demo\scripts\CelsiusConverter");
    }

    The main method of the CelsiusConverterTest class invokes the JUnit test runners.

    public static void main(String[] args){
    args = Log.init(args);
    String[] names = {
    CelsiusConverterTest.class.getName()
    };
    if (args.length == 1 &&
    args[0].equals("--gui"))
    junit.swingui.TestRunner.main(names);
    else
    junit.textui.TestRunner.main(names);
    }

    This approach facilitates the creation of a variety of user interaction scenarios that can be easily integrated with JUnit. Another interesting and handy feature is the ability to insert existing scripts into another script. Some high-level GUI tasks can be broken down into smaller tasks. In these cases, I've created scripts for the most granular tasks and then created different interaction scenarios by inserting the scripts for different tasks in a different order or by adding additional user interactions in between the tasks. Since the scripts are XML-based, they're very easy to read and they help in rapidly generating different scenarios.

    Scenario Two: GUI Is Ready to Be Coded
    In this scenario, the GUI tasks have been identified and the specification is ready to be coded. A good way to begin coding is to write the tests that need to be passed before the GUI is created. The XML-based scripts can be used to create different test cases; however, to demonstrate the full power of Abbot, this section will discuss how the JUnit test cases can be written for the GUI.

    The Abbot Java API will be used to write traditional JUnit test cases. The salient features of the API that help in creating test cases are:

  • Standard JUnit test case structure: Every test case is a subclass of junit.framework.TestCase. The test case has a setUp() and tearDown(), and the tests are structurally similar to JUnit.
  • Provides the ability to simulate user events: Swing GUI testing is different from testing standard Java classes. For standard Java classes, the JUnit framework can run in a separate thread and invoke the methods on the Java classes. However, JUnit doesn't provide a framework that meets the testing needs of Swing applications. For Swing applications, the tests need to wait for the GUI to run and then execute the different user-generated events. Abbot provides the ComponentTester class. This allows the test case to invoke the GUI and generate user events on the AWT thread of the application (simulating a user interaction), without having to explicitly perform any synchronization. The flow of the test code is a simple series of user actions and GUI state checks.
  • Provides component-specific testers: Provides classes to simulate the various events that are specific to different Swing components. For example, JMenuItemTester would enable the simulation of various user-generated events within the test for each of the JMenuItems in the GUI. These classes are available in the abbot.tester package.

    The Celsius-to-Fahrenheit converter example presented in the previous section is used to walk through the steps involved in creating a test case using the Abbot Java API. In this test case, the input of a negative number to the Celsius input will be tested.

    Like JUnit test cases, the CelsiusConverterTest class will subclass the junit.Framework.TestCase class and the constructor will initialize the name of the test.

    public class CelsiusConverterTest
    extends TestCase {
    public CelsiusConverterTest
    (String name) { super(name); }
    }

    The setUp() and tearDown() methods can be used to initialize the frame in which the GUI will run. The setUp() method may be used to populate any arrays or other defaults that are required by the GUI to be fully functional.

    private ComponentTester tester;
    protected void setUp() {
    tester =
    ComponentTester.getTester
    (CelsiusConverter.class);
    // wait for the window to be displayed
    tester.waitForFrameShowing
    ("Convert Celsius to Fahrenheit")
    }

    To write the test, we create a method called testNegativeNumberInput(). This test method needs to get a reference to each of the GUI components that are being tested. One way Abbot gets a reference to a GUI component is by looking up a component of the appropriate class. This allows Abbot to continue to refer to different GUI components even if the layout changes.

    ComponentReference ref =
    new ComponentReference
    ("tempCelsius", JTextField.class);
    Component tempCelsius =
    getFinder().findComponent(ref);

    The next step is to initialize the ComponentTester so that the user interaction event of typing in a negative number can be generated

    tester = ComponentTester.
    getTester(tempCelsius);
    tester.actionKeyString(-45);

    Once the sequence of user interactions is in place, we would need to verify if the result is correct by adding assertions. The reference to the label is obtained and then the value of the label is tested using assetEquals()

    JLabel fahrenheitLabel =
    (JLabel)getFinder().findComponent(ref);
    assertEquals("-49 Fahrenheit",
    "-49 Fahrenheit",
    fahrenheitLabel.getText());

    To run the test, the TestHelper class available in the test package in Abbot will be handy. The TestHelper class also provides automatic test suite generation functionality.

    public static void main(String[] args){
    TestHelper.runTests
    (args,CelsiusConverterTest.class);
    }

    This method can be used in conjunction with JUnit test cases for the application logic to create a homogeneous test suite. The decision points in the application logic can be tested using normal JUnit tests. Test cases generated using Abbot will test for different user interaction scenarios.

    Conclusion
    Writing functional GUI tests can be a tedious task. The concepts from both of the testing scenarios mentioned can be combined to produce a comprehensive suite of tests. Abbot as a framework provides both scripting functionality and a Java API. The framework easily integrates with the JUnit test harness and therefore, during application development, the functional GUI tests can become a part of the test suite. All these features of Abbot make it an effective framework for rapidly creating a comprehensive test framework.

    Resources

  • Abbot: http://abbot.sourceforge.net/
  • JUnit: www.junit.org
  • Abbot Java API: http://abbot.sourceforge.net/doc/api/index.html
  • Java Swing Tutorial: http://java.sun.com/docs/books/tutorial/uiswing

    Author Bio
    Satadip Dutta is a software design engineer at Hewlett-Packard and has been programming in Java since 1997. His areas of interest include user interface design, distributed software architecture, and Web applications. Satadip holds an MS in computer science from Virginia Tech. sdutta@vt.edu

    "Abbot - A Friendly JUnit Extension for GUI Testing"
    Vol. 8, Issue 4, p. 12

    	
    
    
    
    Listing 1: Example of a Test Script
    
    
    
    <AWTTestScript>
      <component id="Convert Celsius to 
                 Fahrenheit Root Pane" 
                 class=
                 "javax.swing.JRootPane" 
     index="0" 
     title=
     "Convert Celsius to 
     Fahrenheit" 
     tag="Convert Celsius to
     Fahrenheit Root Pane"/>
       ...
      
      <launch    classpath="src/demo" 
     args="[]" 
     class=
     "CelsiusConverter" 
     method="main" />
      <action    args="45"
     method=
     "actionKeyString"/>
      <action    class=
                 "javax.swing.
                 AbstractButton"
     args="Convert..." 
     method="actionClick"/>
      <assert    value=
                 "113 Fahrenheit"
           component=
           "113 Fahrenheit" 
           method="getText"/>
      <terminate/>
    </AWTTestScript>
    
     
    

    All Rights Reserved
    Copyright ©  2004 SYS-CON Media, Inc.
      E-mail: info@sys-con.com

    Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.