How big are your data types in the file In question? Do you ghc generics or deriving or template Haskell? Could you share a link to the file in question ? On Mon, Feb 15, 2021 at 12:06 PM Troels Henriksen wrote:

Henning Thielemann writes:

...

On Mon, 15 Feb 2021, Troels Henriksen wrote:

...

I have a program with about 220 modules. One of these modules takes _much_ longer to compile than the others (several minutes longer). I don't understand why. It compiled much faster with GHC 8.8.3, but got orders of magnitude slower when I changed to GHC 8.10.2. I don't understand why, since I don't think there is nothing special about the module - e.g. no type-level programming.

Does the problem persist with GHC-8.10.4?

Yes. I'll try with GHC 9.10.1 once my dependencies catch up.

...

I would try to out-comment parts of the module and watch when compilation becomes faster.

I will try that. I also noticed that other modules are affected as well, although to a lesser degree, and by passing '--ghc-options=-dshow-passes' I obtained this information:

!!! Desugar [Futhark.Pass.ExtractKernels.DistributeNests]: finished in 255448.24 milliseconds, allocated 363811.623 megabytes

So apparently the "desugar" pass takes over four minutes. Apart from lowering Haskell into Core, what exactly does this pass do that is so expensive? The following Simplifier passes take almost no time in comparison.

-- \ Troels /\ Henriksen _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Ccing ghc devs since that’s a better forum perhaps Crazy theory: this is a regression due the the partial changes to pattern matching coverage checking in 8.10 that finished / landed in ghc 9 Why: Desugaring is when pattern/ case statement translation happens I think? And the only obvious “big thing” is that you have some huge , albeit sane for a compiler, pattern matching I’d first check if the new ghc 9 release doesn’t have that regression in build time that you experienced. And if it does file a ticket. I may be totally wrong, but that seems like a decent likelihood ! On Mon, Feb 15, 2021 at 12:20 PM Troels Henriksen wrote:

Carter Schonwald writes:

...

How big are your data types in the file In question? Do you ghc generics or deriving or template Haskell? Could you share a link to the file in question ?

The file does not define any large data types itself, but it operates on some fairly large data types (an AST, it's a compiler). But so do many other modules that work just fine. It uses no generics, nontrivial deriving, or Template Haskell. It's this file:

https://github.com/diku-dk/futhark/blob/master/src/Futhark/Pass/ExtractKerne...

I also found a handful of other modules in my project that are significantly slower to compile in GHC 8.10, and seemingly also because of the desugarer, but none of them have any obvious smoking guns like generics or TH.

The only commonality I can find is that the affected modules contain functions with a relatively large typeclass context. I use ConstraintKinds to make them more concise, and I guess unfolded there may be 30-40 equality/class constraints in total. Like this:

type DistLore lore = ( Bindable lore, HasSegOp lore, BinderOps lore, LetDec lore ~ Type, ExpDec lore ~ (), BodyDec lore ~ () )

where the 'Bindable' constraint in particular then implies a further handful of "~" constraints:

class ( ASTLore lore, FParamInfo lore ~ DeclType, LParamInfo lore ~ Type, RetType lore ~ DeclExtType, BranchType lore ~ ExtType, SetType (LetDec lore) ) => Bindable lore where ...

FParamInfo/LParamInfo/etc are type families. Are such constraints particularly slow?

-- \ Troels /\ Henriksen

Hi, I'm not sure I see all the context of the conversation, but it is entirely possible that code with many local constraints regresses the pattern-match checker (which is accounted to Desugaring in the profile emitted by -v2), I'm afraid. That simply has to do with the fact that we now actually care about them, previously they were mostly discarded. I'd be glad if you submitted a relatively isolated reproducer of what is fast with 8.8 and slow with 8.10 (even better 9.0). I hope that things have improved since we fixed https://gitlab.haskell.org/ghc/ghc/-/issues/17836, which is part of 9.0 but not of 8.10. Cheers, Sebastian Am Mo., 15. Feb. 2021 um 19:04 Uhr schrieb Troels Henriksen < athas@sigkill.dk>:

Carter Schonwald writes:

...

Ccing ghc devs since that’s a better forum perhaps Crazy theory:

this is a regression due the the partial changes to pattern matching coverage checking in 8.10 that finished / landed in ghc 9

Why: Desugaring is when pattern/ case statement translation happens I think? And the only obvious “big thing” is that you have some huge , albeit sane for a compiler, pattern matching

I’d first check if the new ghc 9 release doesn’t have that regression in build time that you experienced. And if it does file a ticket.

I may be totally wrong, but that seems like a decent likelihood !

You may be right! Another module that regressed is also mainly characterised by large-but-not-insane case expressions:

https://github.com/diku-dk/futhark/blob/d0839412bdd11884d75a1494dd5de5191833...

I'll try to split these modules up a little bit (I should have done so a while ago anyway) and maybe that will make the picture even clearer.

-- \ Troels /\ Henriksen _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Don’t forget ghc 9 is already out! :) On Mon, Feb 15, 2021 at 2:10 PM Troels Henriksen wrote:

It is very likely that issue 17386 is the issue. With

{-# OPTIONS_GHC -Wno-overlapping-patterns -Wno-incomplete-patterns -Wno-incomplete-uni-patterns -Wno-incomplete-record-updates #-}

my module(s) compile very quickly. I'll wait and see if GHC 9 does better before I try to create a smaller case (and now I at least have a workaround).

Sebastian Graf writes:

...

Hi,

I'm not sure I see all the context of the conversation, but it is entirely possible that code with many local constraints regresses the pattern-match checker (which is accounted to Desugaring in the profile emitted by -v2), I'm afraid. That simply has to do with the fact that we now actually care about them, previously they were mostly discarded.

I'd be glad if you submitted a relatively isolated reproducer of what is fast with 8.8 and slow with 8.10 (even better 9.0). I hope that things have improved since we fixed https://gitlab.haskell.org/ghc/ghc/-/issues/17836, which is part of 9.0 but not of 8.10.

Cheers, Sebastian

Am Mo., 15. Feb. 2021 um 19:04 Uhr schrieb Troels Henriksen < athas@sigkill.dk>:

...

Carter Schonwald writes:

...

Ccing ghc devs since that’s a better forum perhaps Crazy theory:

this is a regression due the the partial changes to pattern matching coverage checking in 8.10 that finished / landed in ghc 9

Why: Desugaring is when pattern/ case statement translation happens I think? And the only obvious “big thing” is that you have some huge , albeit sane for a compiler, pattern matching

I’d first check if the new ghc 9 release doesn’t have that regression in build time that you experienced. And if it does file a ticket.

I may be totally wrong, but that seems like a decent likelihood !

You may be right! Another module that regressed is also mainly characterised by large-but-not-insane case expressions:

https://github.com/diku-dk/futhark/blob/d0839412bdd11884d75a1494dd5de5191833...

...

I'll try to split these modules up a little bit (I should have done so a while ago anyway) and maybe that will make the picture even clearer.

-- \ Troels /\ Henriksen _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- \ Troels /\ Henriksen

Troels Henriksen writes:

It is very likely that issue 17386 is the issue. With

{-# OPTIONS_GHC -Wno-overlapping-patterns -Wno-incomplete-patterns -Wno-incomplete-uni-patterns -Wno-incomplete-record-updates #-}

my module(s) compile very quickly. I'll wait and see if GHC 9 does better before I try to create a smaller case (and now I at least have a workaround).

I have now tried it with GHC 9, and unfortunately it is still very slow. As time permits, I will see if I can come up with a self-contained module that illustrates the slowdown. I do have an idea for a optimisation: In all of the cases where coverage tracking takes a long time, I have a catch-all case at the bottom. I think that is a fairly common pattern, where a program tries to detect various special cases, before falling back to a general case. Perhaps coverage checking should have a short-circuiting check for whether there is an obvious catch-all case, and if so, not bother looking any closer? -- \ Troels /\ Henriksen

I suspect that to get deeper into this you'll need a debugging build of GHC, which I believe will break down the desugaring pass into its constituent sub-passes. One possibility is that the "gentle optimization" pass is proving not to be so gentle on your code. On Mon, Feb 15, 2021, 12:03 PM Troels Henriksen wrote:

Henning Thielemann writes:

...

On Mon, 15 Feb 2021, Troels Henriksen wrote:

...

I have a program with about 220 modules. One of these modules takes _much_ longer to compile than the others (several minutes longer). I don't understand why. It compiled much faster with GHC 8.8.3, but got orders of magnitude slower when I changed to GHC 8.10.2. I don't understand why, since I don't think there is nothing special about the module - e.g. no type-level programming.

Does the problem persist with GHC-8.10.4?

Yes. I'll try with GHC 9.10.1 once my dependencies catch up.

...

I would try to out-comment parts of the module and watch when compilation becomes faster.

I will try that. I also noticed that other modules are affected as well, although to a lesser degree, and by passing '--ghc-options=-dshow-passes' I obtained this information:

!!! Desugar [Futhark.Pass.ExtractKernels.DistributeNests]: finished in 255448.24 milliseconds, allocated 363811.623 megabytes

So apparently the "desugar" pass takes over four minutes. Apart from lowering Haskell into Core, what exactly does this pass do that is so expensive? The following Simplifier passes take almost no time in comparison.

-- \ Troels /\ Henriksen _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.