[GHC] #14468: Why does alanz's branch blow up GHC's heap?
#14468: Why does alanz's branch blow up GHC's heap? -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: task | Status: new Priority: high | Milestone: 8.4.1 Component: Compiler | Version: 8.2.1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- alanz mentioned that moving all of GHC's `Data` instances [[https://git.haskell.org/ghc.git/blob/12028239b597e17fb3f3734c6b494e127be58e0c:/compiler/hsSyn/HsInstances.hs|instances]] to a new module causes the stage1 compile to run out of memory during compilation (he has 8 GB of memory). This is rather suspicious as this module isn't terribly large. IIRC typechecking time may be non-linear in program size, but I don't see why it would be non-linear in space. Investigate. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14468 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14468: Why does alanz's branch blow up GHC's heap? -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: task | Status: new Priority: high | Milestone: 8.4.1 Component: Compiler | Version: 8.2.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Perhaps unsurprisingly, the dominant contribution here is typechecking, {{{ *** Checking old interface for HsInstances (use -ddump-hi-diffs for more details): *** Parser [HsInstances]: !!! Parser [HsInstances]: finished in 4.90 milliseconds, allocated 9.166 megabytes *** Renamer/typechecker [HsInstances]: !!! Renamer/typechecker [HsInstances]: finished in 48430.88 milliseconds, allocated 59998.701 megabytes *** Desugar [HsInstances]: Result size of Desugar (after optimization) = {terms: 39,619, types: 309,762, coercions: 4,274,106, joins: 0/3,281} !!! Desugar [HsInstances]: finished in 17066.61 milliseconds, allocated 21449.018 megabytes *** Simplifier [HsInstances]: Result size of Simplifier iteration=1 = {terms: 61,148, types: 351,421, coercions: 238,159, joins: 0/3,212} Result size of Simplifier iteration=2 = {terms: 59,333, types: 350,264, coercions: 242,742, joins: 0/2,475} Result size of Simplifier iteration=3 = {terms: 58,354, types: 345,841, coercions: 241,068, joins: 0/2,446} Result size of Simplifier iteration=4 = {terms: 58,338, types: 345,742, coercions: 240,892, joins: 0/2,444} Result size of Simplifier = {terms: 58,338, types: 345,742, coercions: 240,892, joins: 0/2,444} !!! Simplifier [HsInstances]: finished in 3464.16 milliseconds, allocated 5298.698 megabytes *** Specialise [HsInstances]: Result size of Specialise = {terms: 63,565, types: 358,072, coercions: 242,946, joins: 0/2,422} !!! Specialise [HsInstances]: finished in 366.65 milliseconds, allocated 806.110 megabytes *** Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = False}) [HsInstances]: Result size of Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = False}) = {terms: 72,114, types: 387,785, coercions: 242,946, joins: 0/2,790} !!! Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = False}) [HsInstances]: finished in 665.25 milliseconds, allocated 927.697 megabytes *** Simplifier [HsInstances]: Result size of Simplifier iteration=1 = {terms: 72,811, types: 394,019, coercions: 243,374, joins: 0/2,887} Result size of Simplifier iteration=2 = {terms: 70,244, types: 394,237, coercions: 243,092, joins: 0/2,782} Result size of Simplifier iteration=3 = {terms: 70,207, types: 395,136, coercions: 243,092, joins: 0/2,760} Result size of Simplifier = {terms: 70,207, types: 395,136, coercions: 243,092, joins: 0/2,760} !!! Simplifier [HsInstances]: finished in 2800.81 milliseconds, allocated 3389.853 megabytes *** Simplifier [HsInstances]: Result size of Simplifier iteration=1 = {terms: 69,365, types: 393,057, coercions: 243,092, joins: 9/2,769} Result size of Simplifier iteration=2 = {terms: 69,317, types: 393,038, coercions: 243,092, joins: 0/2,760} Result size of Simplifier iteration=3 = {terms: 69,310, types: 393,039, coercions: 243,092, joins: 0/2,770} Result size of Simplifier = {terms: 69,290, types: 393,029, coercions: 243,092, joins: 0/2,760} !!! Simplifier [HsInstances]: finished in 2498.60 milliseconds, allocated 3181.271 megabytes *** Simplifier [HsInstances]: Result size of Simplifier iteration=1 = {terms: 86,688, types: 407,268, coercions: 243,089, joins: 292/3,052} lt size of Simplifier = {terms: 86,688, types: 407,268, coercions: 243,089, joins: 292/3,052} !!! Simplifier [HsInstances]: finished in 1482.55 milliseconds, allocated 1822.554 megabytes *** Float inwards [HsInstances]: Result size of Float inwards = {terms: 86,688, types: 407,268, coercions: 243,089, joins: 292/3,052} !!! Float inwards [HsInstances]: finished in 257.39 milliseconds, allocated 492.612 megabytes *** Called arity analysis [HsInstances]: Result size of Called arity analysis = {terms: 86,688, types: 407,268, coercions: 243,089, joins: 292/3,052} !!! Called arity analysis [HsInstances]: finished in 857.47 milliseconds, allocated 1821.950 megabytes *** Simplifier [HsInstances]: Result size of Simplifier iteration=1 = {terms: 86,688, types: 407,268, coercions: 243,089, joins: 292/3,052} Result size of Simplifier = {terms: 86,688, types: 407,268, coercions: 243,089, joins: 292/3,052} !!! Simplifier [HsInstances]: finished in 6407.60 milliseconds, allocated 1816.619 megabytes *** Demand analysis [HsInstances]: Result size of Demand analysis = {terms: 86,688, types: 407,268, coercions: 243,089, joins: 292/3,052} !!! Demand analysis [HsInstances]: finished in 7202.07 milliseconds, allocated 9270.744 megabytes *** Worker Wrapper binds [HsInstances]: Result size of Worker Wrapper binds = {terms: 90,308, types: 415,860, coercions: 243,120, joins: 292/3,610} !!! Worker Wrapper binds [HsInstances]: finished in 91.28 milliseconds, allocated 42.636 megabytes *** Simplifier [HsInstances]: Result size of Simplifier iteration=1 = {terms: 90,034, types: 416,359, coercions: 243,643, joins: 314/3,097} Result size of Simplifier iteration=2 = {terms: 88,163, types: 411,674, coercions: 243,143, joins: 292/3,055} Result size of Simplifier iteration=3 = {terms: 88,115, types: 411,518, coercions: 243,081, joins: 292/3,052} Result size of Simplifier = {terms: 88,115, types: 411,518, coercions: 243,081, joins: 292/3,052} !!! Simplifier [HsInstances]: finished in 3032.09 milliseconds, allocated 3908.092 megabytes *** Exitification transformation [HsInstances]: Result size of Exitification transformation = {terms: 88,115, types: 411,518, coercions: 243,081, joins: 292/3,052} !!! Exitification transformation [HsInstances]: finished in 43.61 milliseconds, allocated 33.863 megabytes *** Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = True}) [HsInstances]: Result size of Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = True}) = {terms: 90,071, types: 413,216, coercions: 243,081, joins: 0/2,760} !!! Float out(FOS {Lam = Just 0, Consts = True, OverSatApps = True}) [HsInstances]: finished in 801.06 milliseconds, allocated 1044.681 megabytes *** Common sub-expression [HsInstances]: Result size of Common sub-expression = {terms: 83,216, types: 403,886, coercions: 242,872, joins: 0/2,760} !!! Common sub-expression [HsInstances]: finished in 415.19 milliseconds, allocated 768.373 megabytes *** Float inwards [HsInstances]: Result size of Float inwards = {terms: 83,216, types: 403,886, coercions: 242,872, joins: 0/2,760} !!! Float inwards [HsInstances]: finished in 289.80 milliseconds, allocated 470.782 megabytes *** Liberate case [HsInstances]: Result size of Liberate case = {terms: 83,216, types: 403,886, coercions: 242,872, joins: 0/2,760} !!! Liberate case [HsInstances]: finished in 73.12 milliseconds, allocated 103.518 megabytes *** Simplifier [HsInstances]: Result size of Simplifier iteration=1 = {terms: 79,540, types: 395,132, coercions: 242,872, joins: 0/2,760} Result size of Simplifier = {terms: 79,540, types: 395,132, coercions: 242,872, joins: 0/2,760} !!! Simplifier [HsInstances]: finished in 1477.16 milliseconds, allocated 1796.773 megabytes *** SpecConstr [HsInstances]: Result size of SpecConstr = {terms: 79,540, types: 395,132, coercions: 242,872, joins: 0/2,760} !!! SpecConstr [HsInstances]: finished in 433.32 milliseconds, allocated 658.733 megabytes *** Simplifier [HsInstances]: Result size of Simplifier = {terms: 79,540, types: 395,132, coercions: 242,872, joins: 0/2,760} !!! Simplifier [HsInstances]: finished in 731.25 milliseconds, allocated 872.946 megabytes *** Demand analysis [HsInstances]: Result size of Demand analysis = {terms: 79,540, types: 395,132, coercions: 242,872, joins: 0/2,760} !!! Demand analysis [HsInstances]: finished in 2900.86 milliseconds, allocated 3463.096 megabytes *** CoreTidy [HsInstances]: Result size of Tidy Core = {terms: 78,322, types: 391,734, coercions: 242,749, joins: 0/2,760} !!! CoreTidy [HsInstances]: finished in 268.39 milliseconds, allocated 337.202 megabytes writeBinIface: 3171 Names writeBinIface: 3487 dict entries writeBinIface: 3171 Names writeBinIface: 3487 dict entries Created temporary directory: /tmp/ghc7250_0 *** CorePrep [HsInstances]: Result size of CorePrep = {terms: 100,011, types: 474,602, coercions: 242,749, joins: 0/10,157} !!! CorePrep [HsInstances]: finished in 237.39 milliseconds, allocated 349.483 megabytes *** Stg2Stg: *** CodeGen [HsInstances]: !!! CodeGen [HsInstances]: finished in 5228.65 milliseconds, allocated 8228.633 megabytes *** Assembler: gcc -fno-stack-protector -DTABLES_NEXT_TO_CODE -Icompiler/hsSyn -Icompiler/stage3/build -Iincludes -Iincludes/dist -Iincludes/dist- derivedconstants/header -Iincludes/dist-ghcconstants/header -Icompiler/stage3/build -Icompiler/stage3/build/./autogen -Icompiler/. -Icompiler/parser -Icompiler/utils -Icompiler/../rts/dist/build -Icompiler/stage2 -no-pie -x assembler -c /tmp/ghc7250_0/ghc_1.s -o compiler/stage3/build/HsInstances.o *** CorePrep [HsInstances]: Result size of CorePrep = {terms: 100,011, types: 474,602, coercions: 242,749, joins: 0/10,157} !!! CorePrep [HsInstances]: finished in 283.31 milliseconds, allocated 348.963 megabytes *** Stg2Stg: *** CodeGen [HsInstances]: !!! CodeGen [HsInstances]: finished in 8840.65 milliseconds, allocated 8731.219 megabytes *** Assembler: }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14468#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14468: Why does alanz's branch blow up GHC's heap? -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: task | Status: new Priority: high | Milestone: 8.4.1 Component: Compiler | Version: 8.2.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): The dominant entries in the profile are, {{{ tcRnModule' HscMain compiler/main/HscMain.hs:(460,1)-(499,72) 1549 1 0.0 0.0 40.5 44.2 tcRnSrcDecls TcRnDriver compiler/typecheck/TcRnDriver.hs:259:25-65 1562 1 0.0 0.0 40.1 43.8 simplifyTop TcRnDriver compiler/typecheck/TcRnDriver.hs:413:25-39 1582 1 1.3 1.2 16.8 18.1 solveSimples TcInteract compiler/typecheck/TcInteract.hs:(240,5)-(241,21) 1583 3215 0.0 0.0 15.5 16.9 solve_loop TcInteract compiler/typecheck/TcInteract.hs:(245,9)-(249,44) 1584 0 7.8 9.1 15.4 16.9 zonkTopDecls TcRnDriver compiler/typecheck/TcRnDriver.hs:(446,16)-(447,43) 1588 1 0.6 0.8 17.1 18.7 zonkEvBinds TcHsSyn compiler/typecheck/TcHsSyn.hs:(1497,5)-(1500,35) 1589 5630 12.0 13.1 16.5 17.9 }}} Clearly more cost centres are needed. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14468#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14468: Why does alanz's branch blow up GHC's heap? -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: task | Status: new Priority: high | Milestone: 8.6.1 Component: Compiler | Version: 8.2.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.4.1 => 8.6.1 Comment: At this point alanz's branch won't be making it for 8.4, sadly. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14468#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#14468: Why does alanz's branch blow up GHC's heap? -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: task | Status: closed Priority: high | Milestone: 8.6.1 Component: Compiler | Version: 8.2.1 Resolution: invalid | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #8095, #13386 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => closed * resolution: => invalid * related: => #8095, #13386 Comment: I'm going to close this as it has served it's purpose. Alanz suspects that #8095 and #13386 may be relevant. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14468#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC