Re: [Haskell-cafe] Re: GSoC: Improving Cabal's Test Support

I apologize for spamming, but it only just occurred to me how to get the best aspects of both our ideas: On Tue, Apr 6, 2010 at 8:28 PM, Thomas Tuegel wrote:

On Tue, Apr 6, 2010 at 7:43 PM, Rogan Creswick wrote:

...

The existing Executable sections may serve the need fine, if we could specify how to run the tests in a different way.  Perhaps a list of test commands could be specified instead, eg:

...

TestCommands: foo-test-ghc6.6,                            foo-test-ghc6.8,                            foo-props --all

Anyhow, just food for thought.

One of the reasons I prefer implementing a dedicated 'Test' stanza is that it makes it easier to tell which executables to install.  There may be situations where we want to install test programs, but there will _always_ be situations where we _don't_ want to.  (Maybe '--{en,dis}able-tests' passed to 'cabal install' should control this?)  Could we take a 'TestCommands' list and parse out the options to get the executable names?  Yes, but relying on that to work makes me uneasy.  I think the more robust way to specify executable-specific options instead would be to add a field to the 'Test' section ('run-options' seems to be consistent with the naming scheme) that doesn't exist in 'Executable'.

Suppose we adopt your suggestion and let test programs be ordinary executables in ordinary 'Executable' sections, and make 'Test' sections that look like:

Test foo-1 exe-is: foo options: --enable-bar --disable-baz

Test foo-2 exe-is: foo options: --enable-bar --enable-baz

Test some-other-test exe-is: some-other-test options: --something-completely-different

Now we get named sets of options but don't have to worry about parsing through to find the executable names! It also solves the problem of running test suites with multiple sets of options at runtime. -- Thomas Tuegel