#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by schyler): * version: 8.2.1 => 8.0.2 Comment: {{{ $ stack ghc -- --version The Glorious Glasgow Haskell Compilation System, version 8.0.2 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Indeed this program produces some moderately large Core. {{{ *** Desugar [GitHub.Data.Webhooks.Events]: Result size of Desugar (after optimization) = {terms: 18,603, types: 76,783, coercions: 21,668, joins: 0/547} *** Desugar [GitHub.Data.Webhooks.Payload]: Result size of Desugar (after optimization) = {terms: 26,920, types: 195,711, coercions: 41,708, joins: 1/937} }}} Nearly all of this is derived bindings. It would be interesting to study which classes are contributing the most. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: dfeuer Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): Some good news: this appears to build much more quickly (1m13s real time) using the `nightly-2017-12-18` resolver (ghc-8.2.2) as with the 8.0-based `lts-9.18` resolver (which took 2m9s). Of course, those resolvers also differ in other ways. I'm still reducing the test file, and I'll report back when I have a better sense of the real difference. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: dfeuer Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): I've been digging a bit further (with GHC 8.2.2). It appears that the (derived) `Data` instances are responsible for most of the compilation time. I haven't yet tracked down what's taking the most time (no particular compiler phase seems egregiously bad). After peeking at some Core and thinking about the methods of `Data`, it doesn't really seem too surprising that the instances for large products might compile slowly. {{{#!hs gfoldl :: Data a => => (forall d b. Data d => c (d -> b) -> d -> c b) -> (forall g. g -> c g) -> a -> c a gunfold :: Data a => (forall b r. Data b => c (b -> r) -> c r) -> (forall r. r -> c r) -> Constr -> c a }}} Note the `Data` constraints on the function arguments. The implementation of `gfoldl` or `gunfold` needs to provide `Data` dictionaries for each field type. Those will initially be local `let`-bound values. Hopefully the duplicates are eventually eliminated, but if that doesn't happen quickly, we could waste some time. I'm still building HEAD to see if this has changed any lately. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: dfeuer Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): I see also that there's a fair bit of type gunk going on. The signature of `gfoldl` means that the combining function will be applied at O(n) different types, each a bit bigger than the last. Some other methods are similar. I don't know if this has any performance impact in the optimizer, but it might. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: dfeuer Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): I've looked in a little more detail. The thing I've found most surprising and suspicious is that we end up with absolutely ''enormous'' unfoldings for some dictionaries. For `Data (Maybe Int)`, we get this mouthful: {{{#!hs $s$fDataMaybe_s57H [InlPrag=NOUSERINLINE CONLIKE] :: Data (Maybe Int) [LclId, Unf=DFun: \ -> Data.Data.C:Data TYPE: Maybe Int (base-4.11.0.0:Data.Typeable.Internal.mkTrApp @ * @ * @ Maybe @ Int Data.Data.$fDataMaybe5 Data.Data.$fDataInt4) `cast` (Sym (base-4.11.0.0:Data.Typeable.Internal.N:Typeable[0]) <*>_N <Maybe Int>_N :: (Type.Reflection.TypeRep (Maybe Int) :: *) ~R# (Typeable (Maybe Int) :: Constraint)) \ (@ (c_a5c3 :: * -> *)) (k_a5c4 [Occ=Once!] :: forall d b. Data d => c_a5c3 (d -> b) -> d -> c_a5c3 b) (z_a5c7 [Occ=Once*!] :: forall g. g -> c_a5c3 g) (ds_a5c9 [Occ=Once!] :: Maybe Int) -> case ds_a5c9 of { Nothing -> z_a5c7 @ (Maybe Int) (GHC.Base.Nothing @ Int); Just a1_a5ce [Occ=Once] -> k_a5c4 @ Int @ (Maybe Int) Data.Data.$fDataInt (z_a5c7 @ (Int -> Maybe Int) (GHC.Base.Just @ Int)) a1_a5ce } Data.Data.$fDataMaybe_$cgunfold @ Int Data.Data.$fDataInt \ (ds_a51o [Occ=Once!] :: Maybe Int) -> case ds_a51o of { Nothing -> Data.Data.$cNothing; Just _ [Occ=Dead] -> Data.Data.$cJust } \ _ [Occ=Dead] -> Data.Data.$tMaybe ... ... }}} Why are we inlining method definitions into the dictionary unfolding at all? I'd expect us to wait for someone to pluck a method from the dictionary unfolding and then consider whether to inline the method. Now I don't know if this has anything to do with the problem, but it looks weird. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: dfeuer Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): David, can you make a small repro case for comment:11? Is that really a verbatim transcript. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: dfeuer Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): I would think we'd want a general rule against inlining under a constructor in core2core. The context seems inherently too boring. And indeed we ''don't'' inline into the definition of the dictionary itself; we only inline into its unfolding. I don't know why. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: dfeuer Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): There's always a certain amount of non-linearity in the types of multi- field records. For example, given {{{#!hs data Foo = Foo Int Char Integer gfoldlFoo :: (forall d b. Data d => c (d -> b) -> d -> c b) -> (forall g. g -> c g) -> Foo -> c Foo gfoldlFoo c k (Foo x y z) = k Foo `c` x `c` y `c` z }}} we get {{{ Testy.$wgfoldlFoo = \ (@ (c_s4UQ :: * -> *)) (w_s4UR :: forall d b. Data d => c_s4UQ (d -> b) -> d -> c_s4UQ b) (w1_s4US :: forall g. g -> c_s4UQ g) (ww_s4UW :: Int) (ww1_s4UX :: Char) (ww2_s4UY :: Integer) -> w_s4UR @ Integer @ Foo Data.Data.$fDataInteger (w_s4UR @ Char @ (Integer -> Foo) Data.Data.$fDataChar (w_s4UR @ Int @ (Char -> Integer -> Foo) Data.Data.$fDataInt (w1_s4US @ (Int -> Char -> Integer -> Foo) Testy.Foo) ww_s4UW) ww1_s4UX) ww2_s4UY }}} Note that we get types `Foo`, `Integer -> Foo`, `Char -> Integer -> Foo`, and `Int -> Char -> Integer -> Foo`. But does this affect the simplifier? I don't know. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14594: 2 modules / 2500LOC takes nearly 3 minutes to build -------------------------------------+------------------------------------- Reporter: schyler | Owner: dfeuer Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): I have not as yet been able to find anything non-linear, but we seem to pay an awful lot ''per instance''. Given {{{#!hs data Foo7 = Foo7 (Int) deriving Data data Foo8 = Foo8 (Int) deriving Data -- plus 25 more like this }}} compiling with `-O` takes 5 seconds. The single most expensive part is codegen, which takes about a second, but otherwise no particular pass stands out as being particularly expensive. I experimentally tried deriving more of the `Data` methods to try to reduce simplification time. Unfortunately, I paid for it in typechecking time. I still want to check if ''some'' of those extra methods are worthwhile, but otherwise I really don't know. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14594#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler