Thomas Tuegel
There have been two separate suggestions (of which I am aware) of ways to integrate tests into Cabal. One is to build the tests into their own executable which uses an error code on exit to indicate test failure.
I personally prefer this suggestion: for my graphviz library at least, I define at least one QuickCheck property that I cannot as yet actually use (as my Arbitrary instances aren't guaranteed to produce valid Dot code, so I can't run the test that attempts to pass a value through to dot/neato).
I propose to build a test suite as its own executable, but to avoid the problem of granularity by producing an output file detailing the success or failure of individual tests and any relevant error messages. The format of the file would be standardized through library routines I propose to write; these routines would run tests with HUnit or QuickCheck and process them into a common format. Cabal, or any other utility, could read this file to determine the state of the test suite. Perhaps Cabal could even warn the user about installing packages with failing tests.
All well and good, but your sample code (which I've ommitted for the sake of brevity) doesn't seem to lead to much room for customisation: for graphviz's test suite, I include various comments about the purpose of the test, etc. as well as changing some of QuickCheck's paramaters; does your proposal allow the inclusion of such customisations? As an aside, I question the necessity of this kind of proposal: how many people actually run tests for packages they download from Hackage? graphviz's test suite runs for 110 minutes, and I mainly use it to test that my changes/inclusions in what it prints and parses is consistent: I don't expect a user to bother running it (and would question why anyone would). How does the inclusion of a "test" option to Cabal allow any substantial benefits to developers over building the package with a build-time flag to enable building a test suite? -- Ivan Lazar Miljenovic Ivan.Miljenovic@gmail.com IvanMiljenovic.wordpress.com