On Wed, 2011-10-05 at 12:09 +0200, Tillmann Rendel wrote:
Hi,
Johan Tibell wrote:
[...] I'm convinced we need support for building benchmark using Cabal. [...]
Could test sections be used for benchmarks, too?
Step 1. Add basic support for building simple benchmark suites e.g. modelled after test suite sections:
This should work immediately.
Step 2. Add a detailed benchmark interface that allows the benchmark to associate a real valued *measure* (optionally with upper and lower bounds) with a benchmark name.
This could be supported by a protocol where a "test" reports a real instead of a boolean value.
Test bench-foo type: benchmark-1.0 main-is: bench-foo.hs build-depends: base
Except for the more specialized keywords, there seems to be no need to distinguish tests from benchmarks.
Hmm, that's an interesting idea, to just make benchmarks a different testsuite protocol. Certainly we could prototype it like this. Duncan