Hello, I've been playing about with the option to add tests to packages and have modified the Setup.hs as follows.. import Distribution.Simple import MyPackage main :: IO () main = defaultMainWithHooks (simpleUserHooks {runTests = myTests}) myTests :: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO () myTests _ _ _ _ = <whatever> Which I think is the right thing to do if I understand the docs. The first problem I come up against is that in order to do this practically all the modules from the package need compiling by runghc (even during the configure stage presumably). As I guess most people will be using runHaskell/runghc to do this it seems like this will all take quite a bit of time. In my case the process fails completely because the package library files need CPPing and is seems runghc doesn't do this (by default at least). Is this really the way tests are supposed to be included? It doesn't seem workable to me. I wonder if it would be better to just build the lib first (without tests) and then separately build one or more test executables? What does everyone else do about this? Thanks -- Adrian Hey
On Wed, 2008-01-09 at 10:27 +0000, Adrian Hey wrote:
Is this really the way tests are supposed to be included?
It doesn't seem workable to me. I wonder if it would be better to just build the lib first (without tests) and then separately build one or more test executables?
What does everyone else do about this?
I've only seen one package that actually uses tests integrated into the Setup.hs. So it seems everyone just runs tests manually. This is a fairly large hole in the Cabal story at the moment imho. Part of the problem is that we don't support custom Setup.hs stuff very well, we encourage people to use the simple build system. But for tests, even if it used the simple build system we'd end up making it use the custom flavour just to run some tests. It's not very satisfactory. I wonder if we shouldn't move tests into their own file completely. The problem always with custom Setup.hs scripts is that they're the first point of entry, and if it does not compile then everything is stuffed and you don't get a good error message. Duncan
On Wed, 9 Jan 2008, Duncan Coutts wrote:
On Wed, 2008-01-09 at 10:27 +0000, Adrian Hey wrote:
Is this really the way tests are supposed to be included?
It doesn't seem workable to me. I wonder if it would be better to just build the lib first (without tests) and then separately build one or more test executables?
What does everyone else do about this?
I've only seen one package that actually uses tests integrated into the Setup.hs. So it seems everyone just runs tests manually. This is a fairly large hole in the Cabal story at the moment imho.
Part of the problem is that we don't support custom Setup.hs stuff very well, we encourage people to use the simple build system. But for tests, even if it used the simple build system we'd end up making it use the custom flavour just to run some tests. It's not very satisfactory.
I wonder if we shouldn't move tests into their own file completely.
... as it is planned for executables, right? I used the Cabal test mechanism once, but it was not portable between different versions of Cabal. Thus I added an Executable stanza to the Cabal file and run the resulting test binary by darcs. Of course I think that in the long run tests should be managed by Cabal somehow in order to let Hackage check uploaded packages against their test suites or in order to let compiler implementors check whether new compiler versions break existing code.
Duncan Coutts wrote:
On Wed, 2008-01-09 at 10:27 +0000, Adrian Hey wrote:
Is this really the way tests are supposed to be included?
It doesn't seem workable to me. I wonder if it would be better to just build the lib first (without tests) and then separately build one or more test executables?
What does everyone else do about this?
I've only seen one package that actually uses tests integrated into the Setup.hs. So it seems everyone just runs tests manually. This is a fairly large hole in the Cabal story at the moment imho.
Part of the problem is that we don't support custom Setup.hs stuff very well, we encourage people to use the simple build system. But for tests, even if it used the simple build system we'd end up making it use the custom flavour just to run some tests. It's not very satisfactory.
I wonder if we shouldn't move tests into their own file completely. The problem always with custom Setup.hs scripts is that they're the first point of entry, and if it does not compile then everything is stuffed and you don't get a good error message.
One thing I've been trying today is building the test program as a separate executable and then having Setup.hs run this.. import Distribution.Simple import Distribution.PackageDescription(PackageDescription) import Distribution.Simple.LocalBuildInfo(LocalBuildInfo) import System.Cmd(system) main :: IO () main = defaultMainWithHooks (simpleUserHooks {runTests = myTests}) myTests:: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO () myTests_ _ _ _ = system "myTestProgram" >> return () But I don't still don't seem to be able to build myTestProgram.hs because it depends on the lib package which has just been built but not yet registered. But I guess there might be some cabal file or ghc-options magic that would make this possible without ghc trying (and failing) to recompile the entire library. Can anyone explain how to do this? Thanks -- Adrian Hey
participants (3)
-
Adrian Hey -
Duncan Coutts -
Henning Thielemann