Measuring the effectiveness of software testing
ISSN der Zeitschrift
Testing of component-based systems and software quality
Regular Research Papers
Gesellschaft für Informatik e.V.
In 1978 Harry Sneed set up the first commercial software test laboratory in Budapest charging DM 75,- per test case and DM 100,- for each error found. The laboratory was used to test the Integrated Transport Steuerung system of the German railroad and the BS2000 operating system of Siemens. Today, some 26 years later, managers are looking for a means to justify the cost of testing. While working as a test consultant for a Viennese software house from 1998 until 2003, Harry Sneed conceived a set of metrics for measuring the effectiveness of the test operations there. These metrics were intended to measure the performance of the test department, but they are equally valid for measuring test operations anywhere. In fact, with these metrics it should be possible to convert software testing from an art as perceived by Glenford Meyers in 1975 to a science as defined by Lord Kelvin in 1875. The metrics were obtained using the Goal/Question/Metric Method of Basili and Rombach and were refined through three years of practical application. In effect, they are a continuation of the test measurement work Sneed began as a young test entrepreneur in 1978. They are supported by a set of tools designed for both static and dynamic analysis as well as for evaluating the results of both. Working as a test team leader at the Wirtschaftskammer in Vienna, Sneed applied these metrics to successfully predict the test effort required to test a complex web application. From this presentation the attendants will be exposed to the experience of 30 years of software testing.