[Haskell-cafe] Letting the darcs test fail, if QuickCheck tests fail
When following the description on http://www.haskell.org/haskellwiki/How_to_write_a_Haskell_program#Add_some_a... then darcs will run the QuickCheck tests on each 'darcs record', but the new patch is also accepted by darcs if one of the tests fail. What is the most simple way to let 'darcs record' fail, when a QuickCheck test fails?
On 10/30/07, Henning Thielemann
When following the description on http://www.haskell.org/haskellwiki/How_to_write_a_Haskell_program#Add_some_a... then darcs will run the QuickCheck tests on each 'darcs record', but the new patch is also accepted by darcs if one of the tests fail. What is the most simple way to let 'darcs record' fail, when a QuickCheck test fails?
The same thing bit me when I prepared a package recently. The way I solved it was to call the function quickCheck' instead of test. It returns a boolean indicating if the test was successful or not. If it's false I call exitWithFailure. I posted some code to the wikibook: http://en.wikibooks.org/wiki/Talk:Haskell/Packaging Note that quickCheck' is only available in QuickCheck 2. All the best, /Josef
On Tue, Oct 30, 2007 at 05:24:21PM +0100, Henning Thielemann wrote:
When following the description on http://www.haskell.org/haskellwiki/How_to_write_a_Haskell_program#Add_some_a... then darcs will run the QuickCheck tests on each 'darcs record', but the new patch is also accepted by darcs if one of the tests fail. What is the most simple way to let 'darcs record' fail, when a QuickCheck test fails?
You can do this with QuickCheck 2 using quickCheck', but I don't know how to do this with QuickCheck 1. xmonad uses a function "mytests", which I guess is pretty much copied from the code of QuickCheck 1, with tracking of errors added in. It's ugly, but it's only a few dozen lines. Another option would be to grep the output of the test suite to look for failure. -- David Roundy Department of Physics Oregon State University
On Tue, 30 Oct 2007, David Roundy wrote:
On Tue, Oct 30, 2007 at 05:24:21PM +0100, Henning Thielemann wrote:
When following the description on http://www.haskell.org/haskellwiki/How_to_write_a_Haskell_program#Add_some_a... then darcs will run the QuickCheck tests on each 'darcs record', but the new patch is also accepted by darcs if one of the tests fail. What is the most simple way to let 'darcs record' fail, when a QuickCheck test fails?
You can do this with QuickCheck 2 using quickCheck', but I don't know how to do this with QuickCheck 1. xmonad uses a function "mytests", which I guess is pretty much copied from the code of QuickCheck 1, with tracking of errors added in. It's ugly, but it's only a few dozen lines.
Another option would be to grep the output of the test suite to look for failure.
I didn't want to introduce the dependency on QuickCheck2 just for quickCheck', so I went for the 'grep' solution. I also added your hints to http://www.haskell.org/haskellwiki/How_to_write_a_Haskell_program#Running_th... Admittedly, the grep solution will be certainly restricted to Unix. Even more I became aware, that grepping for 'Falsifiable' only catches unsatisfied tests, but if a test fails with 'error' this slips through. I could add some handling for this case, but I feel that I'm missing something, again.
participants (3)
-
David Roundy -
Henning Thielemann -
Josef Svenningsson