#14163: Stack Overflow with ApplicativeDo -------------------------------------+------------------------------------- Reporter: lippling | Owner: simonmar Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 Resolution: | Keywords: ApplicativeDo Operating System: MacOS X | Architecture: x86_64 Type of failure: Compile-time | (amd64) crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonpj): * keywords: => ApplicativeDo * owner: (none) => simonmar Comment: OK Simon M is the expert here. But I did find that if I add the following `traceRn` in `rearrangeForApplicativeDo`: {{{ rearrangeForApplicativeDo ctxt stmts0 = do optimal_ado <- goptM Opt_OptimalApplicativeDo let stmt_tree | optimal_ado = mkStmtTreeOptimal stmts | otherwise = mkStmtTreeHeuristic stmts traceRn "rearrangeForADo" (ppr stmt_tree) <------------------- NEW return_name <- lookupSyntaxName' returnMName pure_name <- lookupSyntaxName' pureAName }}} then I get this output {{{ rearrangeForADo (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind (StmtTreeBind }}} That is, it seems that `mkStmtTreeHeuristic`goes into a loop. If you use `-foptimal-applicative-do` it works fine! Over to you, Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14163#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14163: Stack Overflow with ApplicativeDo -------------------------------------+------------------------------------- Reporter: lippling | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Compiler | Version: 8.2.1 Resolution: | Keywords: ApplicativeDo Operating System: MacOS X | Architecture: x86_64 Type of failure: Compile-time | (amd64) crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * priority: normal => high * milestone: => 8.2.2 Old description:

I tried to compile one of our server applications with 8.2.1 (which compiles fine with 8.0.2).

The compilation runs smoothly, but when it reaches a specific file, the RAM usage goes up to > 20GB pretty fast on my 16GB machine and the GHC process gets terminated with a stack overflow error.

I tried to find a minimal example that causes this behavior:

{{{ #!/usr/bin/env stack -- stack --resolver nightly-2017-08-25 script

{-# LANGUAGE ApplicativeDo #-}

x = do (a, _) <- undefined (b, _) <- undefined (c, _) <- undefined undefined

main = undefined }}}

It only happens with at least 3 of these pattern matches.

New description: I tried to compile one of our server applications with 8.2.1 (which compiles fine with 8.0.2). The compilation runs smoothly, but when it reaches a specific file, the RAM usage goes up to > 20GB pretty fast on my 16GB machine and the GHC process gets terminated with a stack overflow error. I tried to find a minimal example that causes this behavior: {{{#!hs #!/usr/bin/env stack -- stack --resolver nightly-2017-08-25 script {-# LANGUAGE ApplicativeDo #-} x = do (a, _) <- undefined (b, _) <- undefined (c, _) <- undefined undefined main = undefined }}} It only happens with at least 3 of these pattern matches. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14163#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14163: Stack Overflow with ApplicativeDo -------------------------------------+------------------------------------- Reporter: lippling | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Compiler | Version: 8.2.1 Resolution: | Keywords: ApplicativeDo Operating System: MacOS X | Architecture: x86_64 Type of failure: Compile-time | (amd64) crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): I have a pretty decent guess about the problem. We have {{{#!hs mkStmtTreeHeuristic :: [(ExprLStmt GhcRn, FreeVars)] -> ExprStmtTree mkStmtTreeHeuristic [one] = StmtTreeOne one mkStmtTreeHeuristic stmts = case segments stmts of [one] -> split one segs -> StmtTreeApplicative (map split segs) where split [one] = StmtTreeOne one split stmts = StmtTreeBind (mkStmtTreeHeuristic before) (mkStmtTreeHeuristic after) where (before, after) = splitSegment stmts }}} The `do` block in question can't actually be split at all (all the pattern matches are strict). So I ''think'' the trick is probably to make sure `before` is non-empty before producing `StmtTreeBind`. I'll give this a whirl. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14163#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14163: Stack Overflow with ApplicativeDo -------------------------------------+------------------------------------- Reporter: lippling | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Compiler | Version: 8.2.1 Resolution: | Keywords: ApplicativeDo Operating System: MacOS X | Architecture: x86_64 Type of failure: Compile-time | (amd64) crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3900 Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * differential: => Phab:D3900 Comment: I have a workaround: if the splitter doesn't actually split, just pull off the first statement and the rest and form a `StmtTreeBind`. But I don't yet understand enough to see why we have this problem with strict patterns and not with other dependencies. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14163#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14163: Stack Overflow with ApplicativeDo -------------------------------------+------------------------------------- Reporter: lippling | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Compiler | Version: 8.2.1 Resolution: | Keywords: ApplicativeDo Operating System: MacOS X | Architecture: x86_64 Type of failure: Compile-time | (amd64) crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3900 Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonmar): @lippling: there's one other workaround, which is to add a ~ to your patterns, like so: {{{ x = do ~(a, _) <- undefined ~(b, _) <- undefined ~(c, _) <- undefined undefined }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14163#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14163: Stack Overflow with ApplicativeDo -------------------------------------+------------------------------------- Reporter: lippling | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Compiler | Version: 8.2.1 Resolution: | Keywords: ApplicativeDo Operating System: MacOS X | Architecture: x86_64 Type of failure: Compile-time | (amd64) crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3900 Wiki Page: | -------------------------------------+------------------------------------- Comment (by George): Is this a duplicate of #14034 ? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14163#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14163: Stack Overflow with ApplicativeDo -------------------------------------+------------------------------------- Reporter: lippling | Owner: simonmar Type: bug | Status: new Priority: high | Milestone: 8.2.2 Component: Compiler | Version: 8.2.1 Resolution: | Keywords: ApplicativeDo Operating System: MacOS X | Architecture: x86_64 Type of failure: Compile-time | (amd64) crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3900 Wiki Page: | -------------------------------------+------------------------------------- Comment (by David Feuer ): In [changeset:"011e15aa2d6949fc56126f1028ea25d5497196d9/ghc" 011e15aa/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="011e15aa2d6949fc56126f1028ea25d5497196d9" Deal with unbreakable blocks in Applicative Do The renamer wasn't able to deal with more than a couple strict patterns in a row with `ApplicativeDo` when using the heuristic splitter. Update it to work with them properly. Reviewers: simonmar, austin, bgamari, hvr Reviewed By: simonmar Subscribers: RyanGlScott, lippling, rwbarton, thomie GHC Trac Issues: #14163 Differential Revision: https://phabricator.haskell.org/D3900 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14163#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14163: Stack Overflow with ApplicativeDo -------------------------------------+------------------------------------- Reporter: lippling | Owner: simonmar Type: bug | Status: closed Priority: high | Milestone: 8.2.2 Component: Compiler | Version: 8.2.1 Resolution: fixed | Keywords: ApplicativeDo Operating System: MacOS X | Architecture: x86_64 Type of failure: Compile-time | (amd64) crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3900 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed Comment: comment:14 merged to `ghc-8.2` as 55b27a3231d6c25bc257006d59b329dd43ac4118. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14163#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler