[GHC] #10228: Increased memory usage with GHC 7.10.1
#10228: Increased memory usage with GHC 7.10.1 -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- (Repost from ghc-devs) I just uprgaded both of my machines to use GHC 7.10.1. I keep sandboxed installations of GHC and this means I had to rebuild Agda and Idris because the binaries built with GHC 7.8.4 were stored inside deactivated 7.8.4 sandbox. Sadly, I had problems building both Agda and Idris due to GHC taking up all of available memory. With Idris the problematic module was Idris.ElabTerm (~2900LOC). The interesting part of the story is that when I do a clean build of Idris GHC consumes all of memory when compiling that module and I have to kill the build. But when I restart the build after killing GHC the module is compiled using a reasonable amount of memory and within reasonable time. With Agda the problematic module is Agda.TypeChecking.Serialise (~2000LOC). The trick with killing the build and restarting it didn't work in this case. I had to compile Agda with GHC 7.8.4 (which works without problems though the mentioned module still requires a lot of memory) and alter my setup so that Agda binary is not stored inside GHC sandbox. I wonder if any of you came across similar issues with GHC 7.10.1? Do we have any performance data that allows to compare memory usage and performance of GHC 7.10.1 with previous stable releases? All of the above happened on 64bit Debian Wheezy with 2GB of RAM. On my second machine, this time with 4GB of RAM, compiling Agda ran out of memory (again Agda.TypeChecking.Serialise module) and I had to kill the build. But once I restarted the build the module was compiled succesfully in a matter of minutes and using around 50% of memory. This looks like some kind of memory leak in GHC. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10228 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10228: Increased memory usage with GHC 7.10.1 -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by simonpj): It would be extremely helpful if you, or someone else, could boil out a test case that demonstrates the problem. And, better still, use a profiled compiler to isolate where the problem is. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10228#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10228: Increased memory usage with GHC 7.10.1 -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by simonpj): See also [this email thread](https://mail.haskell.org/pipermail/ghc- devs/2015-April/008751.html), which has further responses. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10228#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10228: Increased memory usage with GHC 7.10.1 -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): The type of behavior you describe is not totally unexpected since when GHC compiles multiple modules in a single run, it retains the information read from all the interface files it has encountered in case it needs that information later. So it's a time/space trade-off, not really a memory leak. Of course, it's possible that that interface file information is consuming much more space than it needs to. (Or the cause could be something else entirely.) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10228#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10228: Increased memory usage with GHC 7.10.1 -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): I tried to investigate this, but Agda has a bunch of upper bounds that rule out building with 7.10, and with `--allow-newer`, I got an error about `No instance for (GHC.Generics.Generic (Ptr a))` and gave up. How did you manage to build Agda with 7.10 in the first place? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10228#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10228: Increased memory usage with GHC 7.10.1 -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): Oh, the unreleased version on github seems to be building with 7.10. Maybe GHC failing to build Agda on machines with low memory is a feature, since Agda itself needs so much memory to typecheck anything anyways :) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10228#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10228: Increased memory usage with GHC 7.10.1 -------------------------------------+------------------------------------- Reporter: jstolarek | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): The memory usage is up only 10% over 7.8, so this doesn't seem to be a significant regression. {{{ 7.8 327,943,402,392 bytes allocated in the heap 18,580,963,960 bytes copied during GC 870,373,552 bytes maximum residency (24 sample(s)) 13,857,232 bytes maximum slop 2446 MB total memory in use (0 MB lost due to fragmentation) 7.10 506,171,221,880 bytes allocated in the heap 34,277,256,496 bytes copied during GC 945,447,464 bytes maximum residency (35 sample(s)) 18,326,864 bytes maximum slop 2681 MB total memory in use (0 MB lost due to fragmentation) }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10228#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC