Hi All, Trying to create unit test suite.. work in progress. Comments are more than welcome! http://sites.google.com/site/rizsotto/patches/cabal-ticket-212-v1.darcs-send As an easy start, I wrote unit tests with QuickCheck on the following files: Distribution/License.hs Distribution/Package.hs Distribution/Compiler.hs Distribution/System.hs Distribution/ModuleName.hs Distribution/PackageDescription.hs I wrote simple properties like this, for all data type
prop_parse x = x == parse $ display x
I did merged the existing QuickCheck test cases into one file. I found it easier to reuse the generator functions for different types. My findings: - The System.Platform did not passed parse tests. The reason: both System.OS and System.Arch allowed the dash character in the name. But System.Platform text representation was concatenate the two strings separated by a dash. So I just simple removed the dash from them. Solved! ;) - VersionRange does not passes parse tests, iff more than two ranges is present. As far as I could see, the parse method is wired to handle only two ranges. Not corrected, because found no good solution. Need help on this! - Package.Dependency also not passes parse tests. (It is depend on VersionRange success.) - Compat.ReadR gather property is obsolete and does not compile. Can't figure it out how could I rewrite it. Need help! - PackageDescription does not passes simple parse test. I guess the problem is with the test itself! ;) I used showPackageDescription method to generate string representation of this type and parse it back. I could not judge it is an expected behaviour or a bug. Need some explanation. My problem is mainly how to run these tests? As unit tests they are pure, does not manipulate any files on disks, does not open socket or anything. (Michael Feathers use the unit test word as strict as this.) So, I expected to run them by anyone, who could compile the sources. #1 'cabal test' could be the best way to exercise them. But the user hook is so complicated for a simple 'compile and run'. Found others ignoring the arguments and call System.Cmd, execute _the compiler_. The compiler name is wired and dependencies also not visible enough, not to mention the extension handling. Basically compile by hand is against cabal! :) (Pro: the best way to invoke; Con: very hard to do) #2 Create testing flag in the cabal file, which enable an executable to be build. (Xmonad guys doing this way.) It compiles easy, developer need not worry about different compilers. Running the test can be done by hand (or darcs) or the test user hook can execute it. If the main product is a library, then it does not link against that, since the library is not installed. Instead developers specify the source dir and it recompiles the tested sources. (Pro: easy to do; Cons: does not test against library) #3 Create another cabal file. Roughly speaking it is same as #2. In my opinion it is suite best for functional tests. (Pro: easy to do; Cons: testing target must be installed first) #4 Use the QuickCheck script. Based naming convention this script runs through the sources, 'grep' the properties, and exec an interpreter to run them. (Pro: easy; Cons: interpreter name is wired in the script) #5 As a proposed long term solution: I might add a new parameter to the 'executable' region describe the target is a unit test. Compile (and run) it only on 'cabal test' invocation. (Con: It does not exists and might be hard to implement :)) Thanks for any help! Regards, Laszlo