[GHC] #12941: Extend nofib with benchmarks focused on compiler performance

7 Dec 2016

      #12941: Extend nofib with benchmarks focused on compiler performance
-------------------------------------+-------------------------------------
           Reporter:  michalt        |             Owner:  michalt
               Type:  task           |            Status:  new
           Priority:  normal         |         Milestone:  8.4.1
          Component:  NoFib          |           Version:
  benchmark suite                    |
           Keywords:                 |  Operating System:  Unknown/Multiple
       Architecture:                 |   Type of failure:  None/Unknown
  Unknown/Multiple                   |
          Test Case:                 |        Blocked By:
           Blocking:                 |   Related Tickets:
Differential Rev(s):                 |         Wiki Page:
-------------------------------------+-------------------------------------
 GHC seems to be missing a set of benchmarks focused on the compiler
 performance that would contain some realistic examples and make it easy to
 test even small changes in the compiler (by having a simple "before and
 after" comparison). The two main use cases would be:
 - Anyone hacking on GHC could quickly check even for small performance
 differences (running the benchmark with and without their change).
 - Tracking performance of GHC over time (potentially as part of
 perf.haskell.org)

 We already have two collections of performance tests/benchmarks:
 - `tests/perf/compiler` which contains actual tests (part of `validate`)
 and seems to be focused on larger regressions (if the bounds are too
 tight, it can generate a fair amount of noise).
 - `nofib` which is mostly focused on benchmarking the code produced by GHC
 (the vast majority of the modules compile within 400ms).

 The current idea is to extend `nofib` with some modules that are only
 checking the performance of GHC (e.g., might be a library module that
 doesn't contain `main`).

 Please see: https://mail.haskell.org/pipermail/ghc-
 devs/2016-December/013323.html for an initial discussion about this.

 Also related:
 - Ticket focused on improvements to `tests/perf/compiler` #12758
 - Proposal of having a similar tests but hosted in a separate repo on
 github: https://github.com/ghc-proposals/ghc-proposals/pull/26

--
Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12941
GHC http://www.haskell.org/ghc/
The Glasgow Haskell Compiler