Hi, Am Mittwoch, den 21.12.2011, 13:31 +0900 schrieb Kazu Yamamoto:
Especially to Johan and Milan,
I'm thinking to introduce more automatic tests for the containers library. Let's take Data.Map as example:
1) doctest
Some haddoc documents include the following style.
-- > Data.Map.null (empty) == True
If we modify it like
-- >>> Data.Map.null (empty) -- True
we can use doctest. And we can remove corresponding unit test from test/*-properties.hs.
I’m not sure what the benefit is here. containers is a “high profile” library which sees lots of effort. Introducing a new testing library (doctest) just to avoid having to add those test cases to the test suite does not seem to gain much, but takes away some flexibility while writing good documentation. Maybe the “==” form is more readable to some users?
2) test-framework-th
If we replace "test_" with "case_", we can use test-framework-th to generate the long list of test cases. We don't need to update it when we add "prop_" and "case_".
We can specify the number of QuickCheck test as follows:
runghc -i.. -DTESTING map-properties.hs --maximum-generated-tests=500
Since this is test stuff, we don't have to add doctest and test-framework-th to containers.cabal.
What do you think? If you like this, I will work and send you pull request.
For the same reason as above, I don’t think it is worth it. Maintaining the lists of cases is not nice, but the amount of work is not high in relation to the efforts going into the library, so the gain is litte. The downside is that the test suite cannot be run on architectures where Template Haskell is not supported. Given that containers does some bitshifting that can exhibit architecture dependent bugs, this could put us in the bad situation that the tests cannot easily be run on those machines that need the tests the most. If you would like to work on the test suites, maybe you can identify parts of the code that are under-tested and come up with test cases? Greetings, Joachim -- Joachim "nomeata" Breitner mail@joachim-breitner.de | nomeata@debian.org | GPG: 0x4743206C xmpp: nomeata@joachim-breitner.de | http://www.joachim-breitner.de/