Parasoft's .TEST product aims to automate the creation and execution of
testing for quality assurance of .NET code. Specifically, it can produce
four kinds of tests: white box, black box, regression, and static analysis.
White box tests are able to interact with the private members and
variables in a software component to verify that it has been built properly.
Black box tests, on the other hand, limit themselves to only the
externally-visible portions of components, to verify that they interact
properly with their environment.
Regression testing occurs after all of the white boxes and black boxes
have passed successfully for the first time. After this, every code
modification should involve the repetition of all previous white and black
box tests, to verify that no errors have been introduced by the
The final kind of testing that .TEST can automate is static analysis
testing. In this type of test, your .NET code is compared to a set of
built-in code design rules to assess its quality. If your code violates
these rules, quality points are deducted. When it follows the rules, quality
points are awarded.
How .TEST Performed
I'll give you the good news first. Dynamic testing white box, black
box, and regression is made infinitely easier and more valuable by .TEST.
After creating my simple test application, I was able to automatically
generate a whole suite of unit tests for my code in all of about 10 minutes...
and it only took that long because I didn't bother to read the directions!
The unit tests seemed to be of genuinely high quality. Each method in my
application wound up having several unit tests automatically generated for
it. Only white box tests were generated for the private methods. Both white
and black box tests were generated for the public methods. So, if nothing
else, there could be little or no doubt about the adequacy of test coverage.
Upon drilling down into the source code for the various tests, I was
pleasantly surprised by the quality of their contents. Although there are
obvious limits to exactly how specific tests that are generated
automatically by third-party software can be, the range of automatically
generated tests I saw seemed to be about as good as they could get. Savvy
developers who supplement many of these tests with just a few of their own
will almost certainly produce the most reliable code possible.
The static analysis functions worked quite well (see Figure 1). Thanks
to the product's tight integration with Visual Studio .NET, I was able to
simply click a button to quickly verify the .NET code I had produced against
a built-in database of apparently several hundred rules (see Figure 2).
My code didn't exactly pass (or even come close, really), but that
wasn't my main gripe with this feature. What I really couldn't understand
was why anyone would pay money for this functionality, when a very similar
product is available free from Microsoft's GotDotNet Web site
(www.gotdotnet.com/team/fxcop). Admittedly, though, the rules here seemed significantly more extensive than in Microsoft's free download, and it was nice to have both kinds of testing unified in the same tool.
In the final analysis, I felt the value of the automatically created and
run dynamic unit tests alone were enough to justify the purchase price of
the product. Therefore, the relative value of the static code analysis was
immaterial. I considered myself a very happy camper and I intend to continue
using .TEST as long as I can before my evaluation license expires!
101 E. Huntington Dr.
Monrovia, CA 91016
Windows NT/2000/XP. Note: .TEST fully integrates with Microsoft Visual Studio .NET
Microsoft .NET Framework 1.1. The .NET Framework 1.1 is included in Microsoft's Visual Studio .NET 2003. If you already use Visual Studio .NET, you can download and install the .NET Framework 1.1 from Microsoft's download site.
PC with a Pentium II class processor, 450 MHz, minimum; Pentium III class, 600MHz or better, recommended.
256 MB RAM is a required minimum;
512 MB RAM recommended.