On Mon, Dec 5, 2016 at 12:00 PM Moritz Angermann <moritz@lichtzwerge.de> wrote:

Hi,

I’ve started the GHC Performance Regression Collection Proposal[1] (Rendered [2])
a while ago with the idea of having a trivially community curated set of small[3]
real-world examples with performance regressions. I might be at fault here for
not describing this to the best of my abilities. Thus if there is interested, and
this sounds like an useful idea, maybe we should still pursue this proposal?

Cheers,
moritz

[1]: https://github.com/ghc-proposals/ghc-proposals/pull/26
[2]: https://github.com/angerman/ghc-proposals/blob/prop/perf-regression/proposals/0000-perf-regression.rst
[3]: for some definition of small

Interesting! I must have missed this proposal. It seems that it didn't meet

with much enthusiasm though (but it also proposes to have a completely separate

repo on github).

Personally, I'd be happy with something more modest:

- A collection of modules/programs that are more representative of real Haskell

programs and stress various aspects of the compiler.

(this seems to be a weakness of nofib, where >90% of modules compile in less

than 0.4s)

- A way to compile all of those and do "before and after" comparisons easily. To

measure the time, we should probably try to compile each module at least a few

times.

(it seems that this is not currently possible with `tests/perf/compiler` and

nofib only compiles the programs once AFAICS)

Looking at the comments on the proposal from Moritz, most people would prefer to

extend/improve nofib or `tests/perf/compiler` tests. So I guess the main

question is - what would be better:

- Extending nofib with modules that are compile only (i.e., not runnable) and

focus on stressing the compiler?

- Extending `tests/perf/compiler` with ability to run all the tests and do

easy "before and after" comparisons?

Personally, I'm slightly leaning towards `tests/perf/compiler` since this would

allow sharing the same module as a test for `validate` and to be used for

comparing the performance of the compiler before and after a change.

What do you think?

Thanks,

Michal