I'm working on a little library package (purely for my own consumption) that's built with Cabal, and I have a couple of questions about the pragmatics of using HUnit for it. First, I'd like to be able to run my tests via "cabal test" from the shell prompt. I've seen http://hackage.haskell.org/trac/hackage/wiki/UpgradingTests and followed that, and it basically works. I'm curious if I'm doing it in the best (or at least most idiomatic) way. I've included my cabal file and test driver below, for specifics. Two questions: First: the web page I cite above describes the interface that the test binary must support to work with cabal, specifically w.r.t. the binary's exit code. "Your test suites likely already fit this model. However, if you are using an old version of QuickCheck or HUnit, your executable may not be returning the correct error code." This seems to me to suggest that recent versions of HUnit automatically take care of generating the exit code, but I've found that I have to examine HUnit's results and synthesize the exit code manually, as in the driver program below. (I'm running HUnit 1.2.4.2, but the interface for 1.2.4.3 doesn't appear to differ on this point.) Am I misinterpreting the wiki page, or am I missing something in HUnit's API that generates the exit code automatically? Second: Am I specifying the Build-Depends correctly for the Test-Suite? Specifically: do I need to state a dependency on the library defined in the same package, or does it pick that up automatically? Further, foo-tests doesn't use parsec directly. Is the transitive dependency automatically provided for me, or do I need to list it explicitly as below? Thanks much, Richard My cabal file: Name: foo Version: 0.0 Cabal-Version: >= 1.2 Author: Richard Cobbe Synopsis: Sample cabal package for HUnit integration Build-Type: Simple Library Exposed-Modules: Foo, Foo.Parser, Foo.Show Build-Depends: base >= 4.3.1.0 && < 5, parsec >= 3.1.2 Test-Suite foo-tests main-is: foo-tests.hs type: exitcode-stdio-1.0 Build-Depends: base >= 4.3.1.0 && < 5, parsec >= 3.1.2, HUnit >= 1.2.4.2 and foo-tests.hs: module Main where import Test.HUnit import qualified Foo.Parser import System.Exit main :: IO () main = do c <- runTestTT ("Foo" ~: Foo.Parser.tests) if (errors c == 0 && failures c == 0) then exitWith ExitSuccess else exitWith (ExitFailure (-1))
First: the web page I cite above describes the interface that the test binary must support to work with cabal, specifically w.r.t. the binary's exit code. "Your test suites likely already fit this model. However, if you are using an old version of QuickCheck or HUnit, your executable may not be returning the correct error code."
This seems to me to suggest that recent versions of HUnit automatically take care of generating the exit code, but I've found that I have to examine HUnit's results and synthesize the exit code manually, as in the driver program below. (I'm running HUnit 1.2.4.2, but the interface for 1.2.4.3 doesn't appear to differ on this point.) Am I misinterpreting the wiki page, or am I missing something in HUnit's API that generates the exit code automatically?
AFAIK, you have to do it explicitly. But you can shorten it to something like this: when (errors c /= 0 || failures c /= 0) exitFailure Personally I'd use some test framework that is build on top of HUnit and QuickCheck. My weapon of choice is Hspec [1], but there are other options.
Second: Am I specifying the Build-Depends correctly for the Test-Suite? Specifically: do I need to state a dependency on the library defined in the same package, or does it pick that up automatically? Further, foo-tests doesn't use parsec directly. Is the transitive dependency automatically provided for me, or do I need to list it explicitly as below?
You only have to add dependencies on what you actually use, so if you depend on your library and only use stuff from your library, you do not need to depend on parsec. On a broader scope, you have two options, either (a) Depend on your library in your Cabal test section. or (b) Include the source files of your library in your Cabal test section. Whether you want (a) or (b) depends on the circumstances. Here is a (possibly not complete) list of differences: - (a) Is suitable when you only want to test the public interface of your library, but it does not allow you to test stuff that is not exposed. In contrast, (b) allows you to test stuff that is not exposed. - (a) usually gives you a short test section in your Cabal file (and is more DRY). (b) requires you to repeat all dependencies, options, etc. of your library section in your test section. - Compilation is slower with (b), because the source files of your library are compiled twice. - (b) even works in the rare case, when your test framework depends on your library (e.g. if you use HUnit to test a dependency of HUnit). I'm assuming here, that your library sources and test sources live in different directory hierarchies, and I'm not sure what the exact behavior is, if they don't! Let's look at an example. If your library sources are in '.' and your test sources are in 'test' option (a) looks like so: test-suite foo-tests type: exitcode-stdio-1.0 main-is: foo-tests.hs hs-source-dirs: test build-depends: base >= 4.3.1.0 && < 5 , foo -- depend on your library , HUnit >= 1.2.4.2 And option (b) looks like so: test-suite foo-tests type: exitcode-stdio-1.0 main-is: foo-tests.hs hs-source-dirs: . -- include the sources of your library , test build-depends: base >= 4.3.1.0 && < 5 , parsec >= 3.1.2 -- inculde the depencencies of your library , HUnit >= 1.2.4.2 Hope that helps. Feel free to ask, if anything is unclear. Cheers, Simon [1] http://hspec.github.com/
On Thu, Jul 05, 2012 at 04:17:33PM +0200, Simon Hengel wrote:
First: the web page I cite above describes the interface that the test binary must support to work with cabal, specifically w.r.t. the binary's exit code. "Your test suites likely already fit this model. However, if you are using an old version of QuickCheck or HUnit, your executable may not be returning the correct error code."
This seems to me to suggest that recent versions of HUnit automatically take care of generating the exit code, but I've found that I have to examine HUnit's results and synthesize the exit code manually, as in the driver program below. (I'm running HUnit 1.2.4.2, but the interface for 1.2.4.3 doesn't appear to differ on this point.) Am I misinterpreting the wiki page, or am I missing something in HUnit's API that generates the exit code automatically?
AFAIK, you have to do it explicitly. But you can shorten it to something like this:
when (errors c /= 0 || failures c /= 0) exitFailure
Ah, good suggestion. Thanks!
Second: Am I specifying the Build-Depends correctly for the Test-Suite? Specifically: do I need to state a dependency on the library defined in the same package, or does it pick that up automatically? Further, foo-tests doesn't use parsec directly. Is the transitive dependency automatically provided for me, or do I need to list it explicitly as below?
<snip>
Let's look at an example.
Many thanks for your advice -- and I particularly appreciate the examples! I haven't had a chance to try them out yet, but I hope to be able to soon. Thanks a bunch for your help, Richard
participants (2)
-
Richard Cobbe -
Simon Hengel