#12941: Extend nofib with benchmarks focused on compiler performance -------------------------------------+------------------------------------- Reporter: michalt | Owner: michalt Type: task | Status: new Priority: normal | Milestone: 8.4.1 Component: NoFib | Version: benchmark suite | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- GHC seems to be missing a set of benchmarks focused on the compiler performance that would contain some realistic examples and make it easy to test even small changes in the compiler (by having a simple "before and after" comparison). The two main use cases would be: - Anyone hacking on GHC could quickly check even for small performance differences (running the benchmark with and without their change). - Tracking performance of GHC over time (potentially as part of perf.haskell.org) We already have two collections of performance tests/benchmarks: - `tests/perf/compiler` which contains actual tests (part of `validate`) and seems to be focused on larger regressions (if the bounds are too tight, it can generate a fair amount of noise). - `nofib` which is mostly focused on benchmarking the code produced by GHC (the vast majority of the modules compile within 400ms). The current idea is to extend `nofib` with some modules that are only checking the performance of GHC (e.g., might be a library module that doesn't contain `main`). Please see: https://mail.haskell.org/pipermail/ghc- devs/2016-December/013323.html for an initial discussion about this. Also related: - Ticket focused on improvements to `tests/perf/compiler` #12758 - Proposal of having a similar tests but hosted in a separate repo on github: https://github.com/ghc-proposals/ghc-proposals/pull/26 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12941 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler