2011/1/6 Evan Laforge
QuickCheck especially is great because it automates this tedious work: it fuzzes out the input for you and you get to think in terms of higher-level invariants when testing your code. Since about six months ago with the introduction of JUnit XML support in test-framework, we also have plug-in instrumentation support with continuous integration tools like Hudson: Incidentally, I've never been able to figure out how to use QuickCheck. Maybe it has more to do with my particular app, but QuickCheck seems to expect simple input data and simple properties that should hold relating the input and output, and in my experience that's almost never true. For instance, I want to ascertain that a function is true for "compatible" signals and false for "incompatible" ones, where the definition of compatible is quirky and complex. I can make quickcheck generate lots of random signals, but to make sure the "compatible" is right means reimplementing the "compatible" function.
I should say that this reimplementation would be good. If you can compare two implementations (one in plain Haskell and second in declarative QuickCheck rules) you will be better that with only one. We did that when testing implementations of commands in CPU model. Our model was built to specification and we have to be sure we implement it right. One problem was in CPU flags setup, specification was defined in terms of bit manipulation, we wrote tests that did the same but with ordinary arithmetic. Like carry = (a+b) `shirtR` 8 was compared with carry = bit operandA 7 && bit operandB 7 && not (bit result 7). We found errors in our implementation, we fixed them and there was almost no errors found after that. Doing two implementation for testing purposes can be boldly likened to code review with only one person.