[GHC] #16017: ghc-8.6.1 and ghc-8.6.2 use a log of memory

#16017: ghc-8.6.1 and ghc-8.6.2 use a log of memory -------------------------------------+------------------------------------- Reporter: newhoggy | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.3 Component: Compiler | Version: 8.6.2 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Currently GHC uses a lot of memory to build a relatively small module and causes my CI to fail due to there being a 4G memory limit. The source code can be found here: https://github.com/haskell-works/hw- json/tree/73368cee21dc72eedd5291ba689f9abf10e7fcd2 The problem module is here: https://github.com/haskell-works/hw- json/blob/73368cee21dc72eedd5291ba689f9abf10e7fcd2/test/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.hs The build output follows: {{{ cabal new-build --enable-tests --enable-benchmarks --project- file="cabal.project" -j${CABAL_THREADS:-4} all Build profile: -w ghc-8.6.2 -O2 In order, the following will be built (use -v for more details): - hw-json-0.9.0.1 (lib) (first run) - hw-json-0.9.0.1 (test:hw-json-test) (first run) - hw-json-0.9.0.1 (exe:hw-json) (first run) - hw-json-0.9.0.1 (bench:bench) (first run) Configuring library for hw-json-0.9.0.1.. Preprocessing library for hw-json-0.9.0.1.. Building library for hw-json-0.9.0.1.. [ 1 of 32] Compiling HaskellWorks.Data.Json.DecodeError ( src/HaskellWorks/Data/Json/DecodeError.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/DecodeError.o ) [ 2 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.MakeIndex ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/MakeIndex.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/MakeIndex.o ) [ 3 of 32] Compiling HaskellWorks.Data.Json.Internal.Index ( src/HaskellWorks/Data/Json/Internal/Index.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Index.o ) [ 4 of 32] Compiling HaskellWorks.Data.Json.Internal.PartialIndex ( src/HaskellWorks/Data/Json/Internal/PartialIndex.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/PartialIndex.o ) [ 5 of 32] Compiling HaskellWorks.Data.Json.Internal.Token.Types ( src/HaskellWorks/Data/Json/Internal/Token/Types.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Token/Types.o ) [ 6 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Token.Tokenize ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Token/Tokenize.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Token/Tokenize.o ) [ 7 of 32] Compiling HaskellWorks.Data.Json.Internal.Token ( src/HaskellWorks/Data/Json/Internal/Token.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Token.o ) [ 8 of 32] Compiling HaskellWorks.Data.Json.Internal.Value ( src/HaskellWorks/Data/Json/Internal/Value.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Value.o ) [ 9 of 32] Compiling HaskellWorks.Data.Json.Internal.Word8 ( src/HaskellWorks/Data/Json/Internal/Word8.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Word8.o ) [10 of 32] Compiling HaskellWorks.Data.Json.Internal.Word64 ( src/HaskellWorks/Data/Json/Internal/Word64.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Word64.o ) [11 of 32] Compiling HaskellWorks.Data.Json.Internal.CharLike ( src/HaskellWorks/Data/Json/Internal/CharLike.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/CharLike.o ) [12 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Blank ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Blank.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Blank.o ) [13 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.BlankedJson ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/BlankedJson.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/BlankedJson.o ) [14 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.ToInterestBits64 ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/ToInterestBits64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/ToInterestBits64.o ) [15 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.ToBalancedParens64 ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/ToBalancedParens64.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/ToBalancedParens64.o ) [16 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.IbBp ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/IbBp.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/IbBp.o ) [17 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.SemiIndex ( src/HaskellWorks/Data/Json/Backend/Standard/SemiIndex.hs, /root/project /dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/SemiIndex.o ) [18 of 32] Compiling HaskellWorks.Data.Json.Backend.Simple.SemiIndex ( src/HaskellWorks/Data/Json/Backend/Simple/SemiIndex.hs, /root/project /dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Simple/SemiIndex.o ) [19 of 32] Compiling HaskellWorks.Data.Json.LightJson ( src/HaskellWorks/Data/Json/LightJson.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/LightJson.o ) [20 of 32] Compiling HaskellWorks.Data.Json.PartialValue ( src/HaskellWorks/Data/Json/PartialValue.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/PartialValue.o ) [21 of 32] Compiling HaskellWorks.Data.Json.Type ( src/HaskellWorks/Data/Json/Type.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Type.o ) [22 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.Cursor ( src/HaskellWorks/Data/Json/Backend/Standard/Cursor.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/Cursor.o ) [23 of 32] Compiling HaskellWorks.Data.Json.Cursor ( src/HaskellWorks/Data/Json/Cursor.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Cursor.o ) [24 of 32] Compiling HaskellWorks.Data.Json.Internal.Backend.Standard.Cursor.Token ( src/HaskellWorks/Data/Json/Internal/Backend/Standard/Cursor/Token.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Internal/Backend/Standard/Cursor/Token.o ) [25 of 32] Compiling HaskellWorks.Data.Json.Value ( src/HaskellWorks/Data/Json/Value.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Value.o ) [26 of 32] Compiling HaskellWorks.Data.Json.FromValue ( src/HaskellWorks/Data/Json/FromValue.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/FromValue.o ) [27 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.LoadCursor ( src/HaskellWorks/Data/Json/Backend/Standard/LoadCursor.hs, /root/project /dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/LoadCursor.o ) [28 of 32] Compiling HaskellWorks.Data.Json.LoadCursor ( src/HaskellWorks/Data/Json/LoadCursor.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/LoadCursor.o ) [29 of 32] Compiling HaskellWorks.Data.Json.Backend.Standard.Load ( src/HaskellWorks/Data/Json/Backend/Standard/Load.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Backend/Standard/Load.o ) [30 of 32] Compiling HaskellWorks.Data.Json.Load ( src/HaskellWorks/Data/Json/Load.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json/Load.o ) [31 of 32] Compiling HaskellWorks.Data.Json ( src/HaskellWorks/Data/Json.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/HaskellWorks/Data/Json.o ) [32 of 32] Compiling Paths_hw_json ( /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/autogen/Paths_hw_json.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/opt/build/Paths_hw_json.o ) Configuring test suite 'hw-json-test' for hw-json-0.9.0.1.. Configuring benchmark 'bench' for hw-json-0.9.0.1.. Configuring executable 'hw-json' for hw-json-0.9.0.1.. Preprocessing test suite 'hw-json-test' for hw-json-0.9.0.1.. Building test suite 'hw-json-test' for hw-json-0.9.0.1.. Preprocessing executable 'hw-json' for hw-json-0.9.0.1.. Preprocessing benchmark 'bench' for hw-json-0.9.0.1.. Building benchmark 'bench' for hw-json-0.9.0.1.. Building executable 'hw-json' for hw-json-0.9.0.1.. [ 1 of 11] Compiling HaskellWorks.Data.Json.Backend.Standard.Succinct.Cursor.InterestBitsSpec ( test/HaskellWorks/Data/Json/Backend/Standard/Succinct/Cursor/InterestBitsSpec.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/t /hw-json-test/opt/build/hw-json-test/hw-json-test- tmp/HaskellWorks/Data/Json/Backend/Standard/Succinct/Cursor/InterestBitsSpec.o ) [1 of 2] Compiling Main ( bench/Main.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/b/bench/opt/build/bench/bench-tmp/Main.o ) [1 of 7] Compiling App.Commands.Types ( app/App/Commands/Types.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x /hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/Types.o ) [2 of 7] Compiling App.Lens ( app/App/Lens.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build /hw-json/hw-json-tmp/App/Lens.o ) [3 of 7] Compiling App.Commands.Demo ( app/App/Commands/Demo.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x /hw-json/opt/build/hw-json/hw-json-tmp/App/Commands/Demo.o ) [2 of 2] Compiling Paths_hw_json ( /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/b/bench/opt/build/bench/autogen/Paths_hw_json.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/b/bench/opt/build/bench/bench-tmp/Paths_hw_json.o ) Linking /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/b/bench/opt/build/bench/bench ... [4 of 7] Compiling App.Commands.CreateIndex ( app/App/Commands/CreateIndex.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build /hw-json/hw-json-tmp/App/Commands/CreateIndex.o ) [5 of 7] Compiling App.Commands ( app/App/Commands.hs, /root/project /dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw- json/opt/build/hw-json/hw-json-tmp/App/Commands.o ) [ 2 of 11] Compiling HaskellWorks.Data.Json.Backend.Standard.Succinct.CursorSpec ( test/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.hs, /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/t /hw-json-test/opt/build/hw-json-test/hw-json-test- tmp/HaskellWorks/Data/Json/Backend/Standard/Succinct/CursorSpec.o ) [6 of 7] Compiling Main ( app/Main.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build /hw-json/hw-json-tmp/Main.o ) [7 of 7] Compiling Paths_hw_json ( /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build /hw-json/autogen/Paths_hw_json.hs, /root/project/dist- newstyle/build/x86_64-linux/ghc-8.6.2/hw-json-0.9.0.1/x/hw-json/opt/build /hw-json/hw-json-tmp/Paths_hw_json.o ) Linking /root/project/dist-newstyle/build/x86_64-linux/ghc-8.6.2/hw- json-0.9.0.1/x/hw-json/opt/build/hw-json/hw-json ... cabal: Failed to build test:hw-json-test from hw-json-0.9.0.1. The build process was killed (i.e. SIGKILL). The typical reason for this is that there is not enough memory available (e.g. the OS killed a process using lots of memory). Exited with code 1 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16017 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16017: ghc-8.6.1 and ghc-8.6.2 use a log of memory -------------------------------------+------------------------------------- Reporter: newhoggy | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.3 Component: Compiler | Version: 8.6.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by newhoggy): For my own purposes, breaking up the module into two smaller modules was sufficient to avoid triggering the problem, but it would be good if the memory usage could be reduced for this case. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16017#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16017: ghc-8.6.1 and ghc-8.6.2 use a log of memory -------------------------------------+------------------------------------- Reporter: newhoggy | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.3 Component: Compiler | Version: 8.6.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by gidyn): * cc: gidyn (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16017#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16017: ghc-8.6.1 and ghc-8.6.2 use a log of memory -------------------------------------+------------------------------------- Reporter: newhoggy | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.6.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonpj): * failure: None/Unknown => Compile-time performance bug -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16017#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC