[GHC] #12996: Memory leak in recursion when switching from -O1 to -O2
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Keywords: Memory leak, | Operating System: Unknown/Multiple optimization | Architecture: | Type of failure: Runtime Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- For example code see attachment. When compiling with -O1 the code behaves as expected with each loop taking about the same amount of time. At -O2 each iteration takes (nonlinear) more time than the last and leaks memory. It gets slow enough that I never bothered to let it run past 40 iterations since the slowdown seems to be exponential. I initially hit the bug in code using a Set instead of list but turns out the issue can be reduced to an example with lists. Removing either filter, pattern matching or changing the loop fixes the issue. I tested it under the following conditions: * Compiler flags: -fno-full-laziness/No flags * Compiler versions: GHC 8.0.1 and 7.10.3e * Operating Systems: Windows 10, Linux Mint In every case -O1 works as expected with -O2 showing exponential slowdown with each iteration. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by AndreasK): * Attachment "Main.hs" added. Example Code -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by AndreasK): * Attachment "Main.verbose-stg2stg.O2" added. STG output at -O2 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by AndreasK): * Attachment "Main.verbose-stg2stg.O1" added. STG output at -O1 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Description changed by AndreasK: @@ -24,0 +24,4 @@ + + The stg output seems to be the same for both cases as well. I attached STG + dumps created with " -fforce-recomp -ddump-to-file -dverbose-core2core + -dsuppress-uniques -dsuppress-all Main.hs -ddump-simpl -dverbose-stg2stg" New description: For example code see attachment. When compiling with -O1 the code behaves as expected with each loop taking about the same amount of time. At -O2 each iteration takes (nonlinear) more time than the last and leaks memory. It gets slow enough that I never bothered to let it run past 40 iterations since the slowdown seems to be exponential. I initially hit the bug in code using a Set instead of list but turns out the issue can be reduced to an example with lists. Removing either filter, pattern matching or changing the loop fixes the issue. I tested it under the following conditions: * Compiler flags: -fno-full-laziness/No flags * Compiler versions: GHC 8.0.1 and 7.10.3e * Operating Systems: Windows 10, Linux Mint In every case -O1 works as expected with -O2 showing exponential slowdown with each iteration. The stg output seems to be the same for both cases as well. I attached STG dumps created with " -fforce-recomp -ddump-to-file -dverbose-core2core -dsuppress-uniques -dsuppress-all Main.hs -ddump-simpl -dverbose-stg2stg" -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by AndreasK): * Attachment "O1.log" added. Peformance reference for -O1 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by AndreasK): * Attachment "O2.log" added. Performance reference for -O2 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Description changed by AndreasK: @@ -24,4 +24,0 @@ - - The stg output seems to be the same for both cases as well. I attached STG - dumps created with " -fforce-recomp -ddump-to-file -dverbose-core2core - -dsuppress-uniques -dsuppress-all Main.hs -ddump-simpl -dverbose-stg2stg" New description: For example code see attachment. When compiling with -O1 the code behaves as expected with each loop taking about the same amount of time. At -O2 each iteration takes (nonlinear) more time than the last and leaks memory. It gets slow enough that I never bothered to let it run past 40 iterations since the slowdown seems to be exponential. I initially hit the bug in code using a Set instead of list but turns out the issue can be reduced to an example with lists. Removing either filter, pattern matching or changing the loop fixes the issue. I tested it under the following conditions: * Compiler flags: -fno-full-laziness/No flags * Compiler versions: GHC 8.0.1 and 7.10.3e * Operating Systems: Windows 10, Linux Mint In every case -O1 works as expected with -O2 showing exponential slowdown with each iteration. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by AndreasK): * Attachment "Main.verbose-stg2stg.O1" added. STG output at -O1 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by AndreasK): * Attachment "Main.verbose-stg2stg.O2" added. STG output at -O2 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by osa1): I analyzed this a little bit, here's my findings: - The function that causes this slowness is generated by `SpecConstr`. If you compile with `-O2 -fno-spec-constr` it works fine. - When `appLoop` function is defined like this: {{{#!haskell appLoop :: AppState -> IO () appLoop s = do let AppState state = s print state appLoop $ AppState (cycleState state) }}} Demand analyzer thinks `appLoop` is not strict on `s`. If we define it like this: {{{#!haskell appLoop :: AppState -> IO () appLoop (AppState state) = do print state appLoop $ AppState (cycleState state) }}} Worker/wrapper runs instead of SpecConstr and this version compiles to fast code. I don't understand why `-O2` version is getting slower in each iteration, but I suspect it has do to with Cmm rather than STG. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: Type of failure: Runtime | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): This turns out to be an example of #11731. It turns out that the thunk for `cycleState state` is wrongly marked "used-once", and that's why there's a massive slow-down. (Each iteration evaluates it twice, so it goes exponential.) The patch for #11731 fixes it, but it never made it into GHC 8.0. I'll mark it thus in case we ship 8.0.3. It really is quite a serious bug. I'll make this bug into a test case. Meanwhile:
Demand analyzer thinks appLoop is not strict on s.
It looks strict, but the demand analyser has a hack for I/O: earlier operations migth throw an exception, and so we treat them as lazy in their continuation. See `Note [IO hack in the demand analyser]` in `DmdAnal`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#12996: Memory leak in recursion when switching from -O1 to -O2
-------------------------------------+-------------------------------------
Reporter: AndreasK | Owner:
Type: bug | Status: new
Priority: normal | Milestone:
Component: Compiler | Version: 8.0.1
Resolution: | Keywords: Memory leak,
| optimization
Operating System: Unknown/Multiple | Architecture:
Type of failure: Runtime | Unknown/Multiple
performance bug | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Simon Peyton Jones
#12996: Memory leak in recursion when switching from -O1 to -O2 -------------------------------------+------------------------------------- Reporter: AndreasK | Owner: Type: bug | Status: closed Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 Resolution: fixed | Keywords: Memory leak, | optimization Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime | Test Case: performance bug | testsuite/tests/perf/should_run/T12996 Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonpj): * status: new => closed * testcase: => testsuite/tests/perf/should_run/T12996 * resolution: => fixed Comment: Thank you AndreasK. It's hard to tickle this bug, but when tickled it can make things exponentially expensive as you discovered. Thanks for the nice test case. I'll close this, leaving #11731 open so that we remember to merge to 8.0.3 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12996#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC