#14974: 2-fold memory usage regression GHC 8.2.2 -> GHC 8.4.1 compiling `mmark` package -------------------------------------+------------------------------------- Reporter: hvr | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Description changed by hvr: Old description:

I haven't had time yet to diagnose where the memory is going to, but here's how to reproduce:

{{{ cabal get mmark-0.0.5.6 && cd mmark-0.0.5.6/

cat > cabal.project <

Then, `cabal new-build -w ghc-8.4.1` and ``cabal new-build -w ghc-8.2.2` will respectively output

{{{ Resolving dependencies... Build profile: -w ghc-8.4.1 -O1 In order, the following will be built (use -v for more details): - mmark-0.0.5.6 (lib) (first run) Configuring library for mmark-0.0.5.6.. <> Preprocessing library for mmark-0.0.5.6.. Building library for mmark-0.0.5.6.. [1 of 9] Compiling Text.MMark.Parser.Internal.Type ( Text/MMark/Parser/Internal/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser/Internal/Type.o ) [2 of 9] Compiling Text.MMark.Parser.Internal ( Text/MMark/Parser/Internal.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser/Internal.o ) [3 of 9] Compiling Text.MMark.Type ( Text/MMark/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Type.o )

Text/MMark/Type.hs:38:1: warning: [-Wunused-imports] The import of ‘Data.Semigroup’ is redundant except perhaps to import instances from ‘Data.Semigroup’ To import instances alone, use: import Data.Semigroup() | 38 | import Data.Semigroup | ^^^^^^^^^^^^^^^^^^^^^ [4 of 9] Compiling Text.MMark.Trans ( Text/MMark/Trans.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Trans.o ) [5 of 9] Compiling Text.MMark.Util ( Text/MMark/Util.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Util.o ) [6 of 9] Compiling Text.MMark.Render ( Text/MMark/Render.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Render.o )

Text/MMark/Render.hs:25:1: warning: [-Wunused-imports] The import of ‘Data.Semigroup’ is redundant except perhaps to import instances from ‘Data.Semigroup’ To import instances alone, use: import Data.Semigroup() | 25 | import Data.Semigroup | ^^^^^^^^^^^^^^^^^^^^^ [7 of 9] Compiling Text.MMark.Parser ( Text/MMark/Parser.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser.o ) [8 of 9] Compiling Text.MMark ( Text/MMark.hs, /tmp/mmark-0.0.5.6 /dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark.o ) [9 of 9] Compiling Text.MMark.Extension ( Text/MMark/Extension.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Extension.o ) <> <> }}}

and

{{{ Resolving dependencies... Build profile: -w ghc-8.4.1 -O1 In order, the following will be built (use -v for more details): - mmark-0.0.5.6 (lib) (first run) Configuring library for mmark-0.0.5.6.. <> Preprocessing library for mmark-0.0.5.6.. Building library for mmark-0.0.5.6.. [1 of 9] Compiling Text.MMark.Parser.Internal.Type ( Text/MMark/Parser/Internal/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser/Internal/Type.o ) [2 of 9] Compiling Text.MMark.Parser.Internal ( Text/MMark/Parser/Internal.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser/Internal.o ) [3 of 9] Compiling Text.MMark.Type ( Text/MMark/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Type.o ) [4 of 9] Compiling Text.MMark.Trans ( Text/MMark/Trans.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Trans.o ) [5 of 9] Compiling Text.MMark.Util ( Text/MMark/Util.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Util.o ) [6 of 9] Compiling Text.MMark.Render ( Text/MMark/Render.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Render.o ) [7 of 9] Compiling Text.MMark.Parser ( Text/MMark/Parser.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser.o ) [8 of 9] Compiling Text.MMark ( Text/MMark.hs, /tmp/mmark-0.0.5.6 /dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark.o ) [9 of 9] Compiling Text.MMark.Extension ( Text/MMark/Extension.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Extension.o ) <> <> hvr@duo:/tmp/mmark-0.0.5.6$ cabal new-build -w ghc-8.2.2 Resolving dependencies... Build profile: -w ghc-8.2.2 -O1 In order, the following will be built (use -v for more details): - mmark-0.0.5.6 (lib) (first run) Configuring library for mmark-0.0.5.6.. <> Preprocessing library for mmark-0.0.5.6.. Building library for mmark-0.0.5.6.. [1 of 9] Compiling Text.MMark.Parser.Internal.Type ( Text/MMark/Parser/Internal/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Parser/Internal/Type.o ) [2 of 9] Compiling Text.MMark.Parser.Internal ( Text/MMark/Parser/Internal.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Parser/Internal.o ) [3 of 9] Compiling Text.MMark.Type ( Text/MMark/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Type.o ) [4 of 9] Compiling Text.MMark.Trans ( Text/MMark/Trans.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Trans.o ) [5 of 9] Compiling Text.MMark.Util ( Text/MMark/Util.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Util.o ) [6 of 9] Compiling Text.MMark.Render ( Text/MMark/Render.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Render.o ) [7 of 9] Compiling Text.MMark.Parser ( Text/MMark/Parser.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Parser.o ) [8 of 9] Compiling Text.MMark ( Text/MMark.hs, /tmp/mmark-0.0.5.6 /dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark.o ) [9 of 9] Compiling Text.MMark.Extension ( Text/MMark/Extension.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Extension.o ) <> <> }}}

Consequently,

- GHC 8.2.2: **2179M in use**, 0.001 INIT (0.000 elapsed), 58.237 MUT (61.472 elapsed), 25.823 GC (25.795 elapsed)

- GHC 8.4.1: **4130M in use**, 0.000 INIT (0.000 elapsed), 58.546 MUT (62.124 elapsed), 31.536 GC (31.505 elapsed)

New description: I haven't had time yet to diagnose where the memory is going to, but here's how to reproduce: {{{ cabal get mmark-0.0.5.6 && cd mmark-0.0.5.6/ cat > cabal.project <> Preprocessing library for mmark-0.0.5.6.. Building library for mmark-0.0.5.6.. [1 of 9] Compiling Text.MMark.Parser.Internal.Type ( Text/MMark/Parser/Internal/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser/Internal/Type.o ) [2 of 9] Compiling Text.MMark.Parser.Internal ( Text/MMark/Parser/Internal.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser/Internal.o ) [3 of 9] Compiling Text.MMark.Type ( Text/MMark/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Type.o ) [4 of 9] Compiling Text.MMark.Trans ( Text/MMark/Trans.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Trans.o ) [5 of 9] Compiling Text.MMark.Util ( Text/MMark/Util.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Util.o ) [6 of 9] Compiling Text.MMark.Render ( Text/MMark/Render.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Render.o ) [7 of 9] Compiling Text.MMark.Parser ( Text/MMark/Parser.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Parser.o ) [8 of 9] Compiling Text.MMark ( Text/MMark.hs, /tmp/mmark-0.0.5.6 /dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark.o ) [9 of 9] Compiling Text.MMark.Extension ( Text/MMark/Extension.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.4.1/mmark-0.0.5.6/build/Text/MMark/Extension.o ) <> <> }}} and {{{ Resolving dependencies... Build profile: -w ghc-8.2.2 -O1 In order, the following will be built (use -v for more details): - mmark-0.0.5.6 (lib) (first run) Configuring library for mmark-0.0.5.6.. <> Preprocessing library for mmark-0.0.5.6.. Building library for mmark-0.0.5.6.. [1 of 9] Compiling Text.MMark.Parser.Internal.Type ( Text/MMark/Parser/Internal/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Parser/Internal/Type.o ) [2 of 9] Compiling Text.MMark.Parser.Internal ( Text/MMark/Parser/Internal.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Parser/Internal.o ) [3 of 9] Compiling Text.MMark.Type ( Text/MMark/Type.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Type.o ) [4 of 9] Compiling Text.MMark.Trans ( Text/MMark/Trans.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Trans.o ) [5 of 9] Compiling Text.MMark.Util ( Text/MMark/Util.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Util.o ) [6 of 9] Compiling Text.MMark.Render ( Text/MMark/Render.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Render.o ) [7 of 9] Compiling Text.MMark.Parser ( Text/MMark/Parser.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Parser.o ) [8 of 9] Compiling Text.MMark ( Text/MMark.hs, /tmp/mmark-0.0.5.6 /dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark.o ) [9 of 9] Compiling Text.MMark.Extension ( Text/MMark/Extension.hs, /tmp/mmark-0.0.5.6/dist- newstyle/build/x86_64-linux/ghc-8.2.2/mmark-0.0.5.6/build/Text/MMark/Extension.o ) <> <> }}} Consequently, - GHC 8.2.2: **2179M in use**, 0.001 INIT (0.000 elapsed), 58.237 MUT (61.472 elapsed), 25.823 GC (25.795 elapsed) - GHC 8.4.1: **4130M in use**, 0.000 INIT (0.000 elapsed), 58.546 MUT (62.124 elapsed), 31.536 GC (31.505 elapsed) -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14974#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14974: 2-fold memory usage regression GHC 8.2.2 -> GHC 8.4.1 compiling `mmark` package -------------------------------------+------------------------------------- Reporter: hvr | Owner: davide Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by davide): * owner: (none) => davide -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14974#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14974: 2-fold memory usage regression GHC 8.2.2 -> GHC 8.4.1 compiling `mmark` package -------------------------------------+------------------------------------- Reporter: hvr | Owner: davide Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Compiler | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by davide): Building mmark With -dShow-passes, I get an increase from 1530MB to 2900MB of allocation for SpecConstr passes. The majority of this is from compiling the Text.MMark.Parser module. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14974#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14974: 2-fold memory usage regression GHC 8.2.2 -> GHC 8.4.1 compiling `mmark` package -------------------------------------+------------------------------------- Reporter: hvr | Owner: davide Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Compiler | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by davide): * Attachment "terms_per_pass.png" added. Line plot of core size (in number of terms) for each pass when compiling Text.MMark.Parser. GHC 8.2.2 vs GHC 8.4.1. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14974 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14974: 2-fold memory usage regression GHC 8.2.2 -> GHC 8.4.1 compiling `mmark` package -------------------------------------+------------------------------------- Reporter: hvr | Owner: davide Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Compiler | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Interesting questions * Do we get more or better specialisations from `SpecConstr`? * Do any of the library benchmarks run faster? Maybe invite the author to give insight, once you have characterised better what is happening. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14974#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14974: 2-fold memory usage regression GHC 8.2.2 -> GHC 8.4.1 compiling `mmark` package -------------------------------------+------------------------------------- Reporter: hvr | Owner: davide Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Compiler | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by davide): * Attachment "bench_8.4.1.txt" added. Benchmark results for GHC 8.4.1 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14974 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14974: 2-fold memory usage regression GHC 8.2.2 -> GHC 8.4.1 compiling `mmark` package -------------------------------------+------------------------------------- Reporter: hvr | Owner: davide Type: bug | Status: new Priority: high | Milestone: 8.8.1 Component: Compiler | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by sgraf): davide: Any news on this? Mind if I look into it? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14974#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler