Peter, Pleased to hear that you are planning to work towards some advances in the Hat tools.
Cover
It will use hs-colour to try and show which functions are being used for a set of inputs. If the new tracing library is used, and it supports taking the union of traces, this will be used to show coverage from lots of traces. This can be tied into quickcheck.
Excellent, that's a really useful tool for me. You might find some useful code in hat-delta and in hat-cover. Hat-cover contains code that will give you full coverage for a call, while hat-delta contains code that will generate the coverage in the top level function call.
Hat-cover is pretty limited. Just in the last few weeks I have implemented a new and more powerful coverage tool called hpc in collaboration with Andy Gill at Galois. Hpc is not actually a Hat tool; its coverage records and index files are much simpler than Hat traces. But the approach is similar as we have a source transformation, run-time recording via a library, and separate viewing tools. Hpc seems to work quite nicely with QuickCheck, and a simple script can be used to keep testing until coverage reaches a fixpoint. Operations to combine coverage records (eg. union, intersection, difference) are still on our to-do list, but should be straightforward. I attach a README and a tar-ball of our current development version. Comments welcome. We have a paper in preparation and I'd be happy to send you a copy when the first draft is finished. I suppose there could be scope for an hpc GUI too! :-) Colin R HPC PRE-RELEASE PROTOTYPE ========================= We are distributing the prototype of a Haskell tool called hpc to a few potential users, such as yourself. We hope you will try it, and give us your reactions. *** Please do NOT distribute this prototype any further. *** Quite soon we expect to release a version of hpc for general use. It will be open-source. WHAT IS HPC? ------------ Hpc is a tool-kit to record and display Haskell Program Coverage. Hpc includes tools that instrument Haskell programs to record program coverage, run instrumented programs, and display the coverage information obtained. Hpc works by applying a source-to-source transformation; this transformation also generates as a by-product a program-index file (.pix) and module-index files (.mix). The transformed program is compiled with a library; in addition to its usual run-time behaviour the program generates a coverage record in a program-ticks file (.tix). If the program is run more than once, coverage data is accumulated to reflect all runs. Hpc provides coverage information of two kinds: source coverage and boolean-control coverage. Source coverage is the extent to which every part of the program was used, measured at three different levels: declarations (both top-level and local), alternatives (among several equations or case branches) and expressions (at every level). Boolean coverage is the extent to which each of the values True and False is obtained in every syntactic boolean context (ie. guard, condition, qualifier). Hpc displays both kinds of information in two different ways: textual reports with summary statistics (hpc-report) and sources with colour mark-up (hpc-source). For boolean coverage, there are four possible outcomes for each guard, condition or qualifier: both True and False values occur; only True; only False; never evaluated. In hpc-source output, highlighting with a yellow background indicates a part of the program that was never evaluated; a green background indicates an always-True expression and a red background indicates an always-False one. INSTALLING HPC -------------- Requires ghc (6.4*); hmake (3.10); linux or MacOSX; bash. There is not yet a slick installation kit. Pick a suitable directory in which to build hpc and try this: 1. tar xf hpcproto.tar 2. add HPCROOT=$(pwd)/hpc to your environment 3. cd hpc 4. make 5. add $(pwd)/bin to your PATH USING HPC --------- There is no fancy packaging of the hpc tools, just a few shell scripts. Try this: 1. cd to the source directory for a Haskell program -- let's assume the main module is in Main.hs 2. hpc-build Main [build options such as -i..., -I... or -package ...] -- transforms for coverage and builds new executable -- in a subdirectory .hpc, along with index files -- and copies of original sources 3. hpc-run Main [args for Main] -- runs the transformed Main, recording coverage info 4. hpc-report Main [module-name...] -- writes a textual summary of coverage to stdout -- if no modules specified, reports on all 5. hpc-source Main [module-name...] -- generates highlighted <module-name>.hs.html for each module -- if no modules specified, generates all See the ${HPCROOT}/test directory for examples, including a nice way to use hpc with QuickCheck (hpc-fix). LIMITATIONS ----------- * The source transformation is not informed by operator priorities. Operands are treated as the immediate subexpressions of any chain of infix applications. * Most ghc extensions are not yet supported. * Various minor glitches -- you will encounter them soon enough! * Developed using ghc under linux. However, as hpc uses very few extensions beyond Haskell 98 it should port readily to other compilers and environments. EXTENSIONS ---------- There are various ideas for hpc that we have discussed, put aside for now, but might come back to. They include: * extending boolean coverage to boolean arguments of Prelude functions; * extending boolean coverage to functions and argument positions specified by the user; * generalising boolean coverage to value-coverage for other datatypes; * success and failure coverage of argument patterns; * path coverage in the functional dependency graph; * indicating frequency of use, not just coverage; * combining information from more than one coverage record. Andy Gill (andy@galois.com) & Colin Runciman (colin@cs.york.ac.uk) 19 July 2006 (revised 28 July 2006)