Looking for background information? Take a look at these foundational books about combinatorial testing.
We measured the coverage of combinatorial design test sets for 10 Unix commands: basename, cb, comm, crypt, sleep, sort, touch, tty, uniq, and wc. [] The pairwise tests gave over 90 percent block coverage.
[D. M. Cohen et al., 1996]


[] a set of 29 pair-wise AETG tests gave 90% block coverage for the UNIX sort command. We also compared pair-wise testing with random input testing and found that pair-wise testing gave better coverage.
[D. M. Cohen et al., 1997]


The block coverage obtained for [pairwise] was comparable with that achieved by exhaustively testing all factor combinations []
[I. S. Dunietz et al., 1997]


Our initial trial of this was on a subset Nortel's internal e-mail system where we able cover 97% of branches with less than 100 valid and invalid testcases, as opposed to 27 trillion exhaustive testcases.
[K. Burr and W. Young, 1998]


Compared to a traditional company that would use the quasiexhaustive strategy, an innovative company using the [Combinatorial] strategy would reduce its system level test schedule by sixty-eight percent (68%) and save sixty-seven percent (67%) in labor costs associated with the testing.
[J. Huller, 2000]


[Evaluating FDA recall class failures in medical devices we established that] [...] out of the 109 reports that [were] detailed [enough], 98% showed that the problem could have been detected by testing the device with all pairs of parameter settings.
[D. R. Wallace and D. R. Kuhn, 2001]


More than 70% of bugs were detected with two or fewer conditions (75% for browser and 70% for server) and approximately 90% of the bugs reported were detected with three or fewer conditions (95% for browser and 89% for server). [...] It is interesting that a small number of conditions (n<=6) are sufficient to detect all reported errors for the browser and server software.
[R. Kuhn and M. J. Reilly, 2002]



Maintained by Jacek Czerwonka, Last updated: October 2016