#13110: GHC API allocates memory which is never GC'd -------------------------------------+------------------------------------- Reporter: DanielG | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: GHC API | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by DanielG): * version: 8.0.1 => 8.0.2 Old description:
I've run into a very strange sort of memory leak it seems. GHC seems to manage to allocate memory which I cannot get rid of no matter what I try even after all references have disappeared. The allocated memory doesn't show up in heap profiles but is still visibly allocated (as seen in ps or pmap).
I discovered this while investigating reports of large memory usage in ghc-mod where this issue is forcing us to dump completion information in a separate process to avoid the allocated memory sticking around for the rest of the user's session.
The attached test case demonstrates this problem by first calling `getModuleInfo` for all modules in all visible packages and then just looping forever in `main`. I would expect all memory GHC allocated to be GC'd at this point but this does not happen.
When run as
{{{ $ ./Leaky -hide-all-packages }}}
the test case consumes around 30M of memory on my system, if we however load some large package, say GHC itself
{{{ $ ./Leaky -hide-all-packages -package ghc }}}
the program will consume around 2-300M of memory which are never deallocated. The behaviour is not related to the loaded package though I tried it with `Cabal` too with the same result though smaller memory usage.
At first I thought this might be related to large CAFs with IORefs inside not being GCd for some reason so I tried calling `resetCAFs` from the RTS before entering the loop. This however makes no difference whatsoever. I also tried forcing a major GC just to be save to no avail.
Next I speculated that maybe the allocated memory is beyond the GC's control (malloc()ed or something) but looking at the memory map of the process with `pmap $(pgrep Leaky)` I can see that the majority of the memory usage I'm seeing comes from address `0x0000000200000000` which is where the RTS allocates memory AFAIK.
New description: I've run into a very strange sort of memory leak it seems. GHC seems to manage to allocate memory which I cannot get rid of no matter what I try even after all references have disappeared. The allocated memory doesn't show up in heap profiles but is still visibly allocated (as seen in ps or pmap). I discovered this while investigating reports of large memory usage in ghc-mod where this issue is forcing us to dump completion information in a separate process to avoid the allocated memory sticking around for the rest of the user's session. The attached test case demonstrates this problem by first calling `getModuleInfo` for all modules in all visible packages and then just looping forever in `main`. I would expect all memory GHC allocated to be GC'd at this point but this does not happen. When run as {{{ $ ./Leaky -hide-all-packages }}} the test case consumes around 30M of memory on my system, if we however load some large package, say GHC itself {{{ $ ./Leaky -hide-all-packages -package ghc }}} the program will consume around 2-300M of memory which is never deallocated. The behaviour is not related to the loaded package though I tried it with `Cabal` too with the same result though smaller memory usage. At first I thought this might be related to large CAFs with IORefs inside not being GCd for some reason so I tried calling `resetCAFs` from the RTS before entering the loop. This however makes no difference whatsoever. I also tried forcing a major GC just to be save to no avail. Next I speculated that maybe the allocated memory is beyond the GC's control (malloc()ed or something) but looking at the memory map of the process with `pmap $(pgrep Leaky)` I can see that the majority of the memory usage I'm seeing comes from address `0x0000000200000000` which is where the RTS allocates memory AFAIK. -- Comment: I can still reproduce this with 8.0.2. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13110#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler