Re: [Haskell-cafe] Re: GSoC: Improving Cabal's Test Support

On Wed, 2010-04-28 at 10:42 -0400, Thomas Tuegel wrote:

On Wed, Apr 28, 2010 at 4:54 AM, Duncan Coutts wrote:

...

On 28 April 2010 09:24, Richard G. wrote:

...

I think that formatted plain-text output would be much better than XML, something that is human-readable and relatively easy to parse via machine. Something similar to the GHC error output would work well because developers are familiar with it.

I have previously advocated a library interface as a detailed testsuite interface (in addition to a lowest common denominator interface of stdio+exitcode). That is a test stanza in a package .cabal file would specify a module containing an entry point of the right type (like main but using a more interesting type).

That way, cabal or any other tool could run the testsuite and produce results in whatever format it likes.

I appreciate the elegance of this method, but it seems to me that it requires dynamic loading, which is currently in a sorry state.

Actually it doesn't require dynamic loading. It just requires compiling a stub program that imports the user's library and some test-runner code. Cabal is good at doing that kind of thing already (eg Setup.hs scripts).

One way or another, cabal will need to provide a data structure it expects test suites to use for results. Is there a substantial advantage to a library interface, versus providing Read/Show instances for the test result data structure?

Yes, it means the testing agent (cabal-install or some other program/system) can do more than simply run all the tests. It means it can enumerate them and not run them (think a GUI or web interface), run a subset of tests, run them in parallel etc. Duncan