[GHC] #12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (CodeGen) | Keywords: | Operating System: Linux Architecture: x86_64 | Type of failure: None/Unknown (amd64) | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Summary: Unlike GHC 7.10.2, when GHC 8.0.1 tries to compile a certain complex INLINE function with -O2 -ddump-hi, it runs out of memory. Dropping either -O2 or -ddump-hi enables successful compilation. Oddly enough, triggering the issue requires using the function argument indirectly through a variable bound by a "where" clause, even though it is obvious to a human reader that the indirection is trivial. (Even after simplifying the original code to work around the out of memory issue, that indirection greatly increases the size of the ".dump-hi" file. It might be that the "out of memory" would not happen if one had sufficient RAM.) Using INLINABLE instead of INLINE seems to result in a much smaller RHS being recorded for the function in question, and does not trigger the problem. (Perhaps INLINABLE records a more optimized RHS than does INLINE, though that is not obvious to me from the GHC User's Guide.) The attached files contain a reduced version of the original code that still triggers the problem. (Of course, the reduced version is rather artificial and pointless, other than to demonstrate the problem.) Probably someone with sufficient understanding could shrink the example further. Please note that I have worked around the issue, and so I am not blocked by this problem, at least not currently. Attachments: - crash.bash - Crash.hs Observed behavior: On 64-bit x86 Ubuntu 16.04 LTS, with the attached files in the current directory, GHC 7.10.2 succeeds: {{{ $ ./crash.bash The Glorious Glasgow Haskell Compilation System, version 7.10.2 [1 of 1] Compiling Crash ( Crash.hs, Crash.o ) }}} whereas GHC 8.0.1 runs out of memory (or wedges my system if I comment out both "ulimit" commands in the script): {{{ $ ./crash.bash The Glorious Glasgow Haskell Compilation System, version 8.0.1 [1 of 1] Compiling Crash ( Crash.hs, Crash.o ) ghc: out of memory }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by j6carey): * Attachment "crash.bash" added. Script to compile Crash.hs with arguments that trigger the problem. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by j6carey): * Attachment "Crash.hs" added. Source code to be compiled in order to trigger the problem. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by j6carey): Let me add that the build tool "stack" seems to pass "-ddump-hi" to GHC, which is why I noticed this issue. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): I've learned a bit about this example. 1. It builds big, deeply-nested coercions. 2. The actual memory exhaustion comes from the pretty printer. Here's a dump from `-ticky` of GHC itself, with the `-ddump-hi` on: {{{ Entries Alloc Alloc'd Non-void Arguments STG Name -------------------------------------------------------------------------------- 24320676 1066014032 0 4 MEiM $waboveNest{v rb81} (ghc-8.1:Pretty) (fun) 19400250 621778256 0 3 MEM beside{v r16q} (ghc-8.1:Pretty) (fun) 2773145 143762232 0 4 EMiL ghc-8.1:Pretty.$wsep1{v rb7d} (fun) 3466978 117540632 0 2 >L base-4.9.0.0:GHC.Base.map{v 01X} (fun) 4143110 114461032 0 1 M oneLiner{v r16K} (ghc-8.1:Pretty) (fun) 1939310 97813720 0 1 M ghc-8.1:Coercion.coercionKind_go{v rrbv} (fun) 1475028 66514184 0 3 i.M containers-0.5.7.1@containers-0.5.7.1:Data.IntMap.Base.$winsert{v rnBj} (fun) 902398 65925152 0 2 LL ghc-8.1:Util.$wsplitAtList{v rhOj} (fun) }}} So clearly the pretty printer is behaving very badly when given deeply nested things to print. 3. Aside from that, a tremendous amount of time is spent in `coercionKind`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by thomie): The pretty pinter issue might be the same issue as https://github.com/haskell/pretty/issues/32. I can have a look later this week. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Sounds very similar. Thanks for taking a look. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: None/Unknown | Test Case: | perf/compiler/T12227 Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2397 Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * status: new => patch * testcase: => perf/compiler/T12227 * differential: => Phab:D2397 Comment: The [http://git.haskell.org/packages/pretty.git/commit/bbe9270c5f849a5bb74c9166a5... fix] for that `pretty` issue also fixes this bug. I don't understand why exactly, but I'm happy to apply it. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: None/Unknown | Test Case: | perf/compiler/T12227 Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2397 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Well caught, thomie! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function
-------------------------------------+-------------------------------------
Reporter: j6carey | Owner:
Type: bug | Status: patch
Priority: normal | Milestone:
Component: Compiler | Version: 8.0.1
(CodeGen) |
Resolution: | Keywords:
Operating System: Linux | Architecture: x86_64
| (amd64)
Type of failure: None/Unknown | Test Case:
| perf/compiler/T12227
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D2397
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: merge Priority: normal | Milestone: 8.2.1 Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: Compile-time | Test Case: performance bug | perf/compiler/T12227 Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2397 Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * status: patch => merge * failure: None/Unknown => Compile-time performance bug * milestone: => 8.2.1 Comment: To merge this to the 8.0 branch, you'll need 372dbc4e78abfb6b5d72c0fea27a1c858c5cd797 first. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: merge Priority: normal | Milestone: 8.0.2 Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: Compile-time | Test Case: performance bug | perf/compiler/T12227 Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2397 Wiki Page: | -------------------------------------+------------------------------------- Changes (by thomie): * milestone: 8.2.1 => 8.0.2 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function
-------------------------------------+-------------------------------------
Reporter: j6carey | Owner:
Type: bug | Status: merge
Priority: normal | Milestone: 8.0.2
Component: Compiler | Version: 8.0.1
(CodeGen) |
Resolution: | Keywords:
Operating System: Linux | Architecture: x86_64
| (amd64)
Type of failure: Compile-time | Test Case:
performance bug | perf/compiler/T12227
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D2397
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari

#12227: regression: out of memory with -O2 -ddump-hi on a complex INLINE function -------------------------------------+------------------------------------- Reporter: j6carey | Owner: Type: bug | Status: closed Priority: normal | Milestone: 8.0.2 Component: Compiler | Version: 8.0.1 (CodeGen) | Resolution: fixed | Keywords: Operating System: Linux | Architecture: x86_64 | (amd64) Type of failure: Compile-time | Test Case: performance bug | perf/compiler/T12227 Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D2397 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/12227#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC