Hi all, As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known. That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 : https://travis-ci.org/ku-fpg/data-reify tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements. Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you. Regards, -- Gaël Deest Tweag I/O
Note that Travis CI time is a sum of all operations, including fetching/saving caches etc. Opening the build log and looking for the compilation might be more revealing. For example, 7.10 seems ~10 seconds, 8.2.1 seems ~50 seconds. There are two issues I can see with this, that should be addressed by benchmarking: 1) The Travis CI build hosts might show varying performance across reruns due to shared machines. 2) Tuning the RTS params when invoking GHC as a compiler can yield significant benefits, at least on larger compilations (such as -A128m). It would probably be doable to fetch a set of packages from *ackage and build with various compilers under controlled circumstances? Robin 2017-07-04 9:16 GMT+02:00 Deest, Gaël <gael.deest@tweag.io>:
Hi all,
As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known.
That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 :
https://travis-ci.org/ku-fpg/data-reify
tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements.
Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you.
Regards,
-- Gaël Deest Tweag I/O
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
I tried the package with 8.2.1 and 8.0.2 and both build in less than 5 seconds with quite a large amount of variability. As Robin points out, there are varying factors which can account for longer build times on Travis. On Tue, Jul 4, 2017 at 8:16 AM, Deest, Gaël <gael.deest@tweag.io> wrote:
Hi all,
As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known.
That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 :
https://travis-ci.org/ku-fpg/data-reify
tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements.
Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you.
Regards,
-- Gaël Deest Tweag I/O
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Thanks Gael. As you may know, we’ve focused a lot on compile time in the last year or so, and 8.4 is a lot faster. I think there are still plenty of bad cases, but things are better. Our compile-time perf page is here: https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler It links to a list of open perf tickets: https://ghc.haskell.org/trac/ghc/query?status=!closed&failure=Compile-time+p... What helps above all are reproducible test cases. Package X compiles much more slowly with 8.2 than with 7.10, say. You did exactly that, thank you – can you open a ticket explaining exactly how to reproduce it, and what measurements you took. (Matthew’s numbers seemed two orders of magnitude different; I wonder why?) Another thing that is really helpful is people who characterise what the problem is: * Is GHC generating a lot more code? -dshow-passes can reveal this. * Is it GHC at all, or is it link-times, or some other part of the tool chain? * If it is GHC, and the output of compilation is no bigger than before, where is the time going? -dshow-passes gives a rough idea of that too. * Is there a particular characteristic of the program that seems to trigger the poor behaviour? Can you accentuate that characteristic to make it behave wildly badly? Everyone: GHC is your compiler! We need your help. Simon From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of Deest, Gaël Sent: 04 July 2017 08:17 To: ghc-devs@haskell.org Cc: Manuel Chakravarty <manuel.chakravarty@tweag.io> Subject: Compilation time Hi all, As you are probably well aware, GHC performance has been a growing concern over the last few years. Many Haskell programmers complain that build time has significantly increased over the last few releases. However, to our knowledge, there isn't much data available to substantiate this claim and the severity of these problems is not well known. That's why we would like to bring some anecdotal evidence to your attention that seems to indicate really major performance regressions. We stumbled upon the CI of the data-reify package, which is built against all GHC releases since 2010 : https://travis-ci.org/ku-fpg/data-reify<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftravis-ci.org%2Fku-fpg%2Fdata-reify&data=02%7C01%7Csimonpj%40microsoft.com%7C1bb3bed0e1b040d80b4508d4c2acb803%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636347494457518816&sdata=Gd9NDAt7jus3ewaGX0GgWwB0%2FjkjX3ZW0cB%2B5LEWE2E%3D&reserved=0> tl;dr: Build time has gone from 1 min 32s for GHC 7.0 to 4 min 35s for GHC 8.2. The 8.2 release alone seems to have increased compilation time by almost 2 minutes, with the current development branch bringing only minor performance improvements. Of course, this single data point is not sufficient to establish how severe and widespread these problems are. More data could probably be gathered from other packages. However, it certainly matches our (subjective) experience and we felt important to report it to you. Regards, -- Gaël Deest Tweag I/O
participants (4)
-
Deest, Gaël -
Matthew Pickering -
Robin Palotai -
Simon Peyton Jones