Hi all, As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known. That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 : https://travis-ci.org/ku-fpg/data-reify tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements. Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you. Regards, -- Gaël Deest Tweag I/O
Note that Travis CI time is a sum of all operations, including
fetching/saving caches etc.
Opening the build log and looking for the compilation might be more
revealing. For example, 7.10 seems ~10 seconds, 8.2.1 seems ~50 seconds.
There are two issues I can see with this, that should be addressed by
benchmarking:
1) The Travis CI build hosts might show varying performance across reruns
due to shared machines.
2) Tuning the RTS params when invoking GHC as a compiler can yield
significant benefits, at least on larger compilations (such as -A128m).
It would probably be doable to fetch a set of packages from *ackage and
build with various compilers under controlled circumstances?
Robin
2017-07-04 9:16 GMT+02:00 Deest, Gaël
Hi all,
As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known.
That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 :
https://travis-ci.org/ku-fpg/data-reify
tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements.
Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you.
Regards,
-- Gaël Deest Tweag I/O
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
I tried the package with 8.2.1 and 8.0.2 and both build in less than 5
seconds with quite a large amount of variability. As Robin points out,
there are varying factors which can account for longer build times on
Travis.
On Tue, Jul 4, 2017 at 8:16 AM, Deest, Gaël
Hi all,
As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known.
That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 :
https://travis-ci.org/ku-fpg/data-reify
tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements.
Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you.
Regards,
-- Gaël Deest Tweag I/O
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Thanks Gael.
As you may know, we’ve focused a lot on compile time in the last year or so, and 8.4 is a lot faster. I think there are still plenty of bad cases, but things are better.
Our compile-time perf page is here: https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler
It links to a list of open perf tickets: https://ghc.haskell.org/trac/ghc/query?status=!closed&failure=Compile-time+performance+bug
What helps above all are reproducible test cases. Package X compiles much more slowly with 8.2 than with 7.10, say. You did exactly that, thank you – can you open a ticket explaining exactly how to reproduce it, and what measurements you took. (Matthew’s numbers seemed two orders of magnitude different; I wonder why?)
Another thing that is really helpful is people who characterise what the problem is:
* Is GHC generating a lot more code? -dshow-passes can reveal this.
* Is it GHC at all, or is it link-times, or some other part of the tool chain?
* If it is GHC, and the output of compilation is no bigger than before, where is the time going? -dshow-passes gives a rough idea of that too.
* Is there a particular characteristic of the program that seems to trigger the poor behaviour? Can you accentuate that characteristic to make it behave wildly badly?
Everyone: GHC is your compiler! We need your help.
Simon
From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Deest, Gaël
Sent: 04 July 2017 08:17
To: ghc-devs@haskell.org
Cc: Manuel Chakravarty
participants (4)
-
Deest, Gaël -
Matthew Pickering -
Robin Palotai -
Simon Peyton Jones