#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by hvr): * priority: normal => high * failure: None/Unknown => Compile-time performance bug -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by maoe): * cc: maoe (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by j.waldmann): here are two more instances where compile time does increase substantially. {{{ 7.6 7.8 7.10 tpdb 4 6 13 (minutes of travis build time) obdd 2 2 5 }}} https://travis-ci.org/jwaldmann/haskell-tpdb https://travis-ci.org/jwaldmann/haskell-obdd -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by thomie): Can you try with 7.10.3 instead of 7.10.2. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by rwbarton): {{{ rwbarton@morphism:/tmp/haskell-obdd$ cabal-super2 clean cleaning... rwbarton@morphism:/tmp/haskell-obdd$ cabal-super2 configure -w ghc-7.10.1 --with-hc-pkg ghc-pkg-7.10.1 --enable-tests --enable-benchmarks --disable- shared Resolving dependencies... Configuring obdd-0.3.3... rwbarton@morphism:/tmp/haskell-obdd$ time cabal-super2 build Building obdd-0.3.3... Preprocessing library obdd-0.3.3... [1 of 7] Compiling OBDD.IntIntMap ( src/OBDD/IntIntMap.hs, dist/build/OBDD/IntIntMap.o ) [2 of 7] Compiling OBDD.VarIntIntMap ( src/OBDD/VarIntIntMap.hs, dist/build/OBDD/VarIntIntMap.o ) [3 of 7] Compiling OBDD.Data ( src/OBDD/Data.hs, dist/build/OBDD/Data.o ) [4 of 7] Compiling OBDD.Make ( src/OBDD/Make.hs, dist/build/OBDD/Make.o ) [5 of 7] Compiling OBDD.Operation ( src/OBDD/Operation.hs, dist/build/OBDD/Operation.o ) [6 of 7] Compiling OBDD.Property ( src/OBDD/Property.hs, dist/build/OBDD/Property.o ) [7 of 7] Compiling OBDD ( src/OBDD.hs, dist/build/OBDD.o ) Preprocessing test suite 'obdd-placement' for obdd-0.3.3... [1 of 1] Compiling Main ( examples/Placement.hs, dist/build /obdd-placement/obdd-placement-tmp/Main.o ) Linking dist/build/obdd-placement/obdd-placement ... Preprocessing test suite 'obdd-queens' for obdd-0.3.3... [1 of 1] Compiling Main ( examples/Queens.hs, dist/build/obdd- queens/obdd-queens-tmp/Main.o ) Linking dist/build/obdd-queens/obdd-queens ... real 0m3.187s user 0m2.916s sys 0m0.212s }}} (I have a lot of cabal versions installed.) This is without a sandbox, if that matters somehow. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by rwbarton): Travis shows the time executed for each command at the right. So https ://travis-ci.org/jwaldmann/haskell-obdd/jobs/108860083#L233 shows `cabal build` taking ~150 seconds for example. Though I don't know how to reconcile that with the whole job taking "5 min 16 sec", since the other times reported add up to less than a minute. I'm assuming Travis build machines are under huge load and memory pressure, and it's probably unreasonable to expect anything out of the build times beyond "if you make the build more efficient, the numbers will go down, probably". -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by j.waldmann): ... both! (well, update doc index in the sandbox only) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by j.waldmann): Here is some more timing info for travis: I added "time" measurements around the cabal commands https://travis-ci.org/jwaldmann/haskell-tpdb/builds/109678365 So the timing they are showing is indeed wallclock. Still, sometimes "user time" was larger than "real time". I suspect this is because cabal uses `-j $ncpus` by default. I think for `cabal install --dependencies` this means to compile them in parallel, which is OK, but for `cabal build` this means to pass `-j $ncpus` to ghc, which decreases performance, and sometimes drastically so. So, I put explicit `-j 1` for these commands, https://travis-ci.org/jwaldmann/haskell-tpdb/builds/109683703 and now the timings look at least somewhat reasonable. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): Git bisect indicates that the regression in `Data.IP.Addr` occurred in b9e49d3e9580e13d89efd1f779cb76f610e0d6e0 "Add -fspecialise-aggressively". Compiler allocation with `-O` increased from 1.53GB to 2.25GB and compilation time increased from 1.8s to 2.6s. Is this to be expected? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): I'm still puzzled by the ''quadrupling'' in compile time describe in the Description. Is that reproducible. Incrasing from 1.8s to 2.6s seems much less dramatic. Bisection is great! I would not have suspected that patch. Inspecting it, I think that perhaps dfuns are specialised after the patch but not before. Try `-dshow-passes` before and after to show code size; and `-ddump-rules` to show what specialision rules have been created (at least for top level things). Then worth checking that the extra specialisations make sense. (I'm assuming that this program does not actually say `-fspecialise- aggressively`? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): No, the program doesn't say `-fspecialise-aggressively`. `-dshow-passes` indicates that everything stays the same up until specialization, with `terms: 2,528, types: 2,420, coercions: 214`. In specialization, it previously rose to `terms: 2,601, types: 2,487, coercions: 214`, but after the change instead jumps to `terms: 3,468, types: 3,731, coercions: 214`. Looking at `-ddump-rules`, the difference is pretty dramatic. Before, we had just {{{ "SPEC Control.Monad.when @ (Text.Appar.Parser.MkParser GHC.Base.String)" [ALWAYS] forall ($dMonad :: Monad (MkParser String)). when @ (MkParser String) $dMonad = $swhen }}} After the change, we have many, many rules (I'll attach them). While the specializations all look fairly reasonable to me, I'm not sure we really want quite that many by default. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #9630 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * related: => #9630 Comment: It looks like the same commit that causes the regression in `Data.IP.Addr` here was also responsible for the much more dramatic `Language.Haskell.Extension` regression described in #9630. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10818: GHC 7.10.2 takes much longer to compile some packages -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: dfeuer Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: #9630 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * owner: (none) => dfeuer -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10818#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler