[GHC] #7702: Memory Leak in CoreM (CoreWriter)
#7702: Memory Leak in CoreM (CoreWriter) -----------------------------------------+---------------------------------- Reporter: afarmer | Owner: Type: bug | Status: new Priority: normal | Component: Compiler Version: 7.6.2 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- While running HERMIT on extended examples, we noticed that the CoreWriter component of CoreM is building up unevaluated SimplCount expressions. Attached is a one line patch to make the field in the CoreWriter record strict, which fixes the leak. Also attached are heap profiles before and after to verify the fix. (You'll notice we have other memory leaks in HERMIT to address, the important bit being the big CoreWriter peak is gone in after.pdf.) These were generated by running a large number of HERMIT transformations in a batch, with HERMIT calling ghc with +RTS -hT -RTS. I'm not sure how this affects compilation time or allocation when SimplCount is not the VerySimplCount constructor. Would the nofib suite answer these questions? -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -----------------------------------------+---------------------------------- Reporter: afarmer | Owner: Type: bug | Status: patch Priority: normal | Component: Compiler Version: 7.6.2 | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Blockedby: Blocking: | Related: -----------------------------------------+---------------------------------- Changes (by afarmer): * status: new => patch -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter)
-----------------------------------------+----------------------------------
Reporter: afarmer | Owner:
Type: bug | Status: patch
Priority: normal | Component: Compiler
Version: 7.6.2 | Keywords:
Os: Unknown/Multiple | Architecture: Unknown/Multiple
Failure: Compile-time performance bug | Blockedby:
Blocking: | Related:
-----------------------------------------+----------------------------------
Comment(by anfarmer@…):
commit 8bac590554087adb89d863dd55501c5e7aa25c2c
{{{
Author: Andrew Farmer
#7702: Memory Leak in CoreM (CoreWriter)
-----------------------------------------+----------------------------------
Reporter: afarmer | Owner:
Type: bug | Status: patch
Priority: normal | Component: Compiler
Version: 7.6.2 | Keywords:
Os: Unknown/Multiple | Architecture: Unknown/Multiple
Failure: Compile-time performance bug | Blockedby:
Blocking: | Related:
-----------------------------------------+----------------------------------
Comment(by simonpj@…):
commit 458c653a795ea06e7cbd24872e9961711f7044e8
{{{
Author: Simon Peyton Jones
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: closed Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: fixed | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Changes (by simonpj): * status: patch => closed * difficulty: => Unknown * resolution: => fixed Comment: Thanks very much for the fix! I've just committed it. Simon -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Changes (by afarmer): * status: closed => new * resolution: fixed => Comment: Turns out that the thunk leak in after.pdf (and before-second-patch.pdf) was also due to CoreM. Attached is another patch to fix this. Running the same test as before, you can see that the thunk leak disappears in after- second-patch.pdf. I also tested with _only_ the second patch, and the leak is still present, so both are necessary. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Changes (by afarmer): * status: new => patch -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Comment(by ezyang): In the second patch, you should either use {{{liftIO $ evaluate w3}}}, or move the seq inside the {{{return}}}, on the principle of #5129. I think both would eliminate the space leak but I'm not sure which is more semantically correct in this case. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Comment(by simonpj): Would you like to update the patch in the light of Edward's suggestion, check that it works, and re-submit? Also if either or both of you (or someone else) would like to improve [http://www.haskell.org/haskellwiki/Performance], notably the "space leak" sub-page (deserves a top level heading of its own), that would be great. Specifically, the lore of #5129 belongs there, not buried in a closed ticket. I'm sure there are other articles about looking for space leaks that could usefully be linked to from that space leak page, like [http://neilmitchell.blogspot.co.uk/2013/02/chasing-space-leak-in- shake.html Neil Mitchell's recent blog post]. Thanks! Simon -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Comment(by afarmer): Both: {{{ let w3 = w1 `plusWriter` w2 return $ w3 `seq` (y, s'', w3) }}} and: {{{ let w3 = w1 `plusWriter` w2 liftIO $ evaluate w3 return (y, s'', w3) }}} will fix the leak. I'm inclined to the former, as the latter requires adding an extra import for evaluate itself. Let me know which is preferable and I'll upload a new patch. Can the testsuite check for things like maximum residency? If so, I can also contribute a minimal test case to avoid future regressions. Edward: could you address the documentation that Simon mentioned? I admit I found this mostly with lots of trial and error, and feel like you may be better at explaining #5129 than me. ;-) -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Comment(by simonpj): I don't mind which. Re testsuite, yes... look in `tests/perf`. Look under "Performance tests" on [http://hackage.haskell.org/trac/ghc/wiki/Building/RunningTests/Adding]. Simon -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Comment(by afarmer): Sorry for the delay. Here is the patch with the seq inside the return, as recommended by ezyang. This is confirmed to fix the leak (in conjunction with the first patch you already merged) in our HERMIT use cases. I'm working on a minimal performance test for the testsuite... I'll post here when I have something. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Comment(by afarmer): Note about the testsuite patch: I developed this on a 64-bit machine and picked the max allocation numbers accordingly. If someone has access to a 32-bit machine, the baseline/deviation in all.T should probably be altered for the 32-bit case. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: igloo Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Changes (by simonpj): * owner: => igloo Comment: Great, thanks. Ian can you check and commit? -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: igloo Type: bug | Status: patch Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Changes (by igloo): * milestone: => 7.8.1 -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: igloo Type: bug | Status: closed Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.6.2 Resolution: fixed | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Changes (by igloo): * status: patch => closed * resolution: => fixed Comment: Applied, thanks! -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: new Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.6.2 Resolution: | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Changes (by afarmer): * owner: igloo => * status: closed => new * resolution: fixed => Comment: It doesn't look like all of the performance test got committed. Only the changes to tests/simplCore/should_compile/all.T appear in the commit in the testsuite. None of the new files (like those in the T7702plugin directory) appear. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#7702: Memory Leak in CoreM (CoreWriter) -------------------------------------------+-------------------------------- Reporter: afarmer | Owner: Type: bug | Status: closed Priority: normal | Milestone: 7.8.1 Component: Compiler | Version: 7.6.2 Resolution: fixed | Keywords: Os: Unknown/Multiple | Architecture: Unknown/Multiple Failure: Compile-time performance bug | Difficulty: Unknown Testcase: | Blockedby: Blocking: | Related: -------------------------------------------+-------------------------------- Changes (by igloo): * status: new => closed * resolution: => fixed Comment: Thanks; they should all be added now. -- Ticket URL: http://hackage.haskell.org/trac/ghc/ticket/7702#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (2)
-
GHC -
GHC