[GHC] #8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|>

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------+------------------------------------------------- Reporter: | Owner: joelteon | Status: new Type: bug | Milestone: Priority: | Version: 7.8.1-rc2 normal | Operating System: Unknown/Multiple Component: | Type of failure: Compile-time performance bug Compiler | Test Case: Keywords: | Blocking: Architecture: | Unknown/Multiple | Difficulty: | Unknown | Blocked By: | Related Tickets: | -------------------------+------------------------------------------------- To reproduce, install a pre-`0.11.2.1` version of attoparsec. This bug was worked around in 0.11.2.1 by removing the `INLINE` on `plus` in attoparsec. With this test program: {{{#!haskell {-# LANGUAGE OverloadedStrings #-} import Control.Applicative import Data.Attoparsec.Text import Data.Text (Text) parser :: Parser Text parser = string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" <|> string "a" main :: IO () main = parseTest parser "a" }}} and using GHC 7.8.1.rc2: Compiling using `-O2`, GHC tops out at ~1GB of RAM and takes 25s. Using `-O0`, GHC takes 0.47s and uses <150MB of RAM. Compare this with GHC 7.6.3: Compiling using `-O2`, GHC uses <150MB and takes 3.7s. Memory usage is similar with `-O0` although compile time goes down to 0.36s. An extreme version of this bug can be found in the `thyme` package here: https://github.com/liyang/thyme/blob/master/src/Data/Thyme/Format.hs#L589-L6.... Compiling that module with an unfixed attoparsec makes GHC use all available memory and stall out, forcing kill -9. Replacing the function body with `undefined` makes the package compile as expected. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------------------------------+------------------------- Reporter: joelteon | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: Resolution: | 7.8.1-rc2 Operating System: Unknown/Multiple | Keywords: Type of failure: Compile-time performance bug | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Changes (by ihameed): * cc: idhameed@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------------------------------+------------------------- Reporter: joelteon | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: Resolution: | 7.8.1-rc2 Operating System: Unknown/Multiple | Keywords: Type of failure: Compile-time performance bug | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: | Unknown | Blocked By: | Related Tickets: -------------------------------------------------+------------------------- Comment (by rezb1t): I'd like to note that this problem occurs when compiling darcs 2.8.4, happy, alex, network 2.5.0.0, and quite a few other libraries. Something is very wrong here, GHC either uses a tiny amount of RAM or it uses gigabytes. When compiling darcs, GHC used 4GB of RAM on one source file! Weirder yet, I've noticed that when used in tandem with cabal-install, the RAM isn't freed in between source files.. whether this is intended or not, I couldn't tell, but running ghc in make mode doesn't do this. The behavior is the same whether it be GHC 7.8.2 or HEAD of ghc-7.8 currently -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------------------+------------------------------------- Reporter: joelteon | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): This is the dreaded `SpecConstr` blowup again (see #8980, #8941 (possibly), #8960, #7898, #7068, #7944, #5550, #8836). Compiling with `-fno-spec-constr` makes it go through fine, so that's a workaround. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------------------+------------------------------------- Reporter: joelteon | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): I made a little progress here. First, `SpecConstr` is not generating lots of code! In fact it is doing no specialisation whatsoever. BUT it is nevertheless taking a very long time to do, well, nothing. So something is afoot. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------------------+------------------------------------- Reporter: joelteon | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by rwbarton): Somehow I'm unable to reproduce this with ghc 7.8.1, 7.8.3 or HEAD and attoparsec 0.11.1.0. Anyone have specific instructions? Maybe I am using the wrong version of some other dependency? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|>
-------------------------------------+-------------------------------------
Reporter: joelteon | Owner:
Type: bug | Status: new
Priority: normal | Milestone:
Component: Compiler | Version: 7.8.1-rc2
Resolution: | Keywords:
Operating System: | Architecture: Unknown/Multiple
Unknown/Multiple | Difficulty: Unknown
Type of failure: Compile- | Blocked By:
time performance bug | Related Tickets:
Test Case: |
Blocking: |
Differential Revisions: |
-------------------------------------+-------------------------------------
Comment (by Simon Peyton Jones

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------------------+------------------------------------- Reporter: joelteon | Owner: Type: bug | Status: closed Priority: normal | Milestone: Component: Compiler | Version: 7.8.1-rc2 Resolution: fixed | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by simonpj): * status: new => closed * resolution: => fixed Comment: OK I have finally fixed this bug. I think there is a good chance that doing so has also fixed #8852, #8980, #8941 (possibly), #8960, #7898, #7068, #7944, #8836. I have turned these tickets into info-needed status, because they are hard to reproduce, but this ticked #8852 is definitely fine now. It's kind of hard to make a regression test without depending on an (out of date version of) attoparsec, and its dependencies, so I'm not adding a regression test at all for now. Phew Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------------------+------------------------------------- Reporter: joelteon | Owner: Type: bug | Status: closed Priority: normal | Milestone: Component: Compiler | Version: 7.8.1-rc2 Resolution: fixed | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by tibbe): This might also be the cause of https://github.com/kolmodin/binary/issues/60. Could we have a 7.8.4 release? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------------------+------------------------------------- Reporter: joelteon | Owner: Type: bug | Status: closed Priority: normal | Milestone: Component: Compiler | Version: 7.8.1-rc2 Resolution: fixed | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by simonpj): Can you not just switch off `SpecConstr`? But in principle, yes, if there is user pressure, we could push out 7.8.4 Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8852: 7.8.1 uses a lot of memory when compiling attoparsec programs using <|> -------------------------------------+------------------------------------- Reporter: joelteon | Owner: Type: bug | Status: closed Priority: normal | Milestone: 7.8.4 Component: Compiler | Version: 7.8.1-rc2 Resolution: fixed | Keywords: Operating System: | Architecture: Unknown/Multiple Unknown/Multiple | Difficulty: Unknown Type of failure: Compile- | Blocked By: time performance bug | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Changes (by tibbe): * milestone: => 7.8.4 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8852#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC