[GHC] #15002: Panic: collectNBinders
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.4.2 Component: Compiler | Version: 8.4.2-rc1 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | crash or panic Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Steps to reproduce: Download concurrent-extra-0.7.0.12 and add the following stack.yaml file: resolver: nightly-2018-04-04 system-ghc: true compiler-check: newer-minor {{{
ghc --version The Glorious Glasgow Haskell Compilation System, version 8.4.1.20180329
stack build --library-profiling concurrent-extra-0.7.0.12: build (lib) Preprocessing library for concurrent-extra-0.7.0.12.. Building library for concurrent-extra-0.7.0.12.. [7 of 8] Compiling Control.Concurrent.Broadcast ( Control/Concurrent/Broadcast.hs, .stack- work/dist/x86_64-linux/Cabal-2.2.0.1/build/Control/Concurrent/Broadcast.p_o ) ghc: panic! (the 'impossible' happened) (GHC version 8.4.1.20180329 for x86_64-unknown-linux): collectNBinders 1 Call stack: CallStack (from HasCallStack): callStackDoc, called at compiler/utils/Outputable.hs:1150:37 in ghc:Outputable pprPanic, called at compiler/coreSyn/CoreSyn.hs:2189:39 in ghc:CoreSyn
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug -- While building custom Setup.hs for package concurrent-extra-0.7.0.12 using: /home/local/ANT/ericcro/.stack/setup-exe-cache/x86_64-linux/Cabal- simple_mPHDZzAJ_2.2.0.1_ghc-8.4.1.20180329 --builddir=.stack- work/dist/x86_64-linux/Cabal-2.2.0.1 build lib:concurrent-extra --ghc- options " -ddump-hi -ddump-to-file -fdiagnostics-color=always" Process exited with code: ExitFailure 1 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.4.2 Component: Compiler | Version: 8.4.2-rc1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Description changed by crockeea: Old description:
Steps to reproduce:
Download concurrent-extra-0.7.0.12 and add the following stack.yaml file: resolver: nightly-2018-04-04 system-ghc: true compiler-check: newer-minor
{{{
ghc --version The Glorious Glasgow Haskell Compilation System, version 8.4.1.20180329
stack build --library-profiling concurrent-extra-0.7.0.12: build (lib) Preprocessing library for concurrent-extra-0.7.0.12.. Building library for concurrent-extra-0.7.0.12.. [7 of 8] Compiling Control.Concurrent.Broadcast ( Control/Concurrent/Broadcast.hs, .stack- work/dist/x86_64-linux/Cabal-2.2.0.1/build/Control/Concurrent/Broadcast.p_o ) ghc: panic! (the 'impossible' happened) (GHC version 8.4.1.20180329 for x86_64-unknown-linux): collectNBinders 1 Call stack: CallStack (from HasCallStack): callStackDoc, called at compiler/utils/Outputable.hs:1150:37 in ghc:Outputable pprPanic, called at compiler/coreSyn/CoreSyn.hs:2189:39 in ghc:CoreSyn
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
-- While building custom Setup.hs for package concurrent-extra-0.7.0.12 using: /home/local/ANT/ericcro/.stack/setup-exe-cache/x86_64-linux/Cabal- simple_mPHDZzAJ_2.2.0.1_ghc-8.4.1.20180329 --builddir=.stack- work/dist/x86_64-linux/Cabal-2.2.0.1 build lib:concurrent-extra --ghc- options " -ddump-hi -ddump-to-file -fdiagnostics-color=always" Process exited with code: ExitFailure 1 }}}
New description: Steps to reproduce: Download concurrent-extra-0.7.0.12 and add the following stack.yaml file: {{{ resolver: nightly-2018-04-04 system-ghc: true compiler-check: newer-minor }}} Then run: {{{
ghc --version The Glorious Glasgow Haskell Compilation System, version 8.4.1.20180329
stack build --library-profiling concurrent-extra-0.7.0.12: build (lib) Preprocessing library for concurrent-extra-0.7.0.12.. Building library for concurrent-extra-0.7.0.12.. [7 of 8] Compiling Control.Concurrent.Broadcast ( Control/Concurrent/Broadcast.hs, .stack- work/dist/x86_64-linux/Cabal-2.2.0.1/build/Control/Concurrent/Broadcast.p_o ) ghc: panic! (the 'impossible' happened) (GHC version 8.4.1.20180329 for x86_64-unknown-linux): collectNBinders 1 Call stack: CallStack (from HasCallStack): callStackDoc, called at compiler/utils/Outputable.hs:1150:37 in ghc:Outputable pprPanic, called at compiler/coreSyn/CoreSyn.hs:2189:39 in ghc:CoreSyn
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug -- While building custom Setup.hs for package concurrent-extra-0.7.0.12 using: /home/local/ANT/ericcro/.stack/setup-exe-cache/x86_64-linux/Cabal- simple_mPHDZzAJ_2.2.0.1_ghc-8.4.1.20180329 --builddir=.stack- work/dist/x86_64-linux/Cabal-2.2.0.1 build lib:concurrent-extra --ghc- options " -ddump-hi -ddump-to-file -fdiagnostics-color=always" Process exited with code: ExitFailure 1 }}} -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * priority: normal => highest * version: 8.4.2-rc1 => 8.4.1 * component: Compiler => Profiling Comment: Thanks for the bug report. Note that one can also reproduce this with GHC 8.4.1 as well. Here is a minimal example: {{{#!hs module Bug where import Control.Concurrent.MVar (MVar, modifyMVar_, putMVar) import Data.Foldable (for_) broadcastThen :: Either [MVar a] a -> MVar (Either [MVar a] a) -> a -> IO () broadcastThen finalState mv x = modifyMVar_ mv $ \mx -> do case mx of Left ls -> do for_ ls (`putMVar` x) return finalState Right _ -> return finalState }}} {{{ $ ~/Software/ghc-8.4.1/bin/ghc -fprof-auto -prof -fforce-recomp -O1 Bug.hs [1 of 1] Compiling Bug ( Bug.hs, Bug.o ) ghc: panic! (the 'impossible' happened) (GHC version 8.4.1 for x86_64-unknown-linux): collectNBinders 1 Call stack: CallStack (from HasCallStack): callStackDoc, called at compiler/utils/Outputable.hs:1150:37 in ghc:Outputable pprPanic, called at compiler/coreSyn/CoreSyn.hs:2189:39 in ghc:CoreSyn }}} Note that this bug goes away if one removes the `-O1` or `-fprof-auto` flags. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * cc: nomeata (added) Comment: This regression was introduced in 0e953da147c405648356f75ee67eda044fffad49 (Implement a dedicated exitfication pass #14152). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): Thanks for the bisecting and minimization! Exemplary bug reporting! Judging from the report and the fact that profiling is necessary, it looks like profiling breaks the invariant that a joint point has `Lam`s as their right-hand-side. This can be confirmed by extending the `pprPanic` in `collectNBinders` to also show `orig_expr`. It might just be a case of looking through `Tick`s … but I am no expert on how profiled GHC is supposed to be handled. I am building the current `master` to look into it, but I don’t expect the build to finish in time to actually do that today. Unless someone beats me to it, I’ll have a lock tomorrow. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Very good. I suggest that a join-point binding of arity N should always have its lambdas at the top. If there are any ticks involved, they should be inside the N binders. (There may be more than, in which case the normal rules should apply after that.) Moreover, Lint should check this invariant. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: new Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by nomeata): * owner: (none) => nomeata Comment:
Moreover, Lint should check this invariant.
It already does! (`lintRhs`) which is confusing me. Might be a bug in Exitification after all. Will investigate. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: new Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): Ah, I see, when you run this with `-dcore-lint`, then the real culprit is found: CSE. Exitification is just where the bug surfaces without lint (phew :-)). Ryan, if it is just a matter of running a script again, would you mind bisecting again with `-dcore-lint`? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: new Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): nomeata, FWIW, I cannot reproduce either a panic or a Core Lint error on the commit prior to 0e953da147c405648356f75ee67eda044fffad49. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: new Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): Ok, this means that Exitification is creating the (probably legal) code that CSE then stumbles over. Strangely, CSE is not supposed to CSE join points… Anyways, I am on it! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: patch Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4572 Wiki Page: | -------------------------------------+------------------------------------- Changes (by nomeata): * status: new => patch * differential: => Phab:D4572 Comment: I found a way of fixing it, submitted for review at Phab:D4572, and also pushed to `wip/T15002` to see if it has performance impact. This simply does less CSE around join points, and I don’t know all the implications, so it may be deemed too crude. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: patch Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4572 Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): I have commented, but you won't be notified because the Phab upgrade has made Phab worse. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: patch Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4572 Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): I have updated Phab:D4572 and commented, but you won't be notified. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: patch Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4572 Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): Ah, things are never easy. Yes, adding `noCSE` in the right places is the right thing to do, and it fixes this bug. But as often when fixing a “bug” in an optimization, it makes it less aggressive, and we get regressions: {{{ Nofib allocations nofib/allocs/rewrite 16255960 + 5.52% 17153208 bytes nofib/allocs/scs 902408848 + 5.22% 949533296 bytes Nofib instruction counts nofib/instr/rewrite 55292296 + 4.73% 57910048 Testsuite allocations tests/alloc/T8766 16828080 + 348.94% 75548336 bytes }}} Did not investigate anything yet (it’s 1am here). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: patch Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4572 Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): Using `noCSE` was the wrong way of fixing this. I have fixed this bug now more properly, less invasively, and with a Note (still at Phab:D4572). The solution is to simply use `collectNBinders` when traversing into a join point. I am fairly confident, so will push if Harbormaster is happy with it, but review is of course still appreciated. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders
-------------------------------------+-------------------------------------
Reporter: crockeea | Owner: nomeata
Type: bug | Status: patch
Priority: highest | Milestone: 8.4.2
Component: Profiling | Version: 8.4.1
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
Type of failure: Compile-time | Unknown/Multiple
crash or panic | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D4572
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Joachim Breitner
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: merge Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4572 Wiki Page: | -------------------------------------+------------------------------------- Changes (by nomeata): * status: patch => merge Comment: Fixed in ae0cff0a1834d8b041b06d0e1ab6ce969aac44c8. I guess this is the kind of stuff we want in the stable release as well? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15002: Panic: collectNBinders -------------------------------------+------------------------------------- Reporter: crockeea | Owner: nomeata Type: bug | Status: closed Priority: highest | Milestone: 8.4.2 Component: Profiling | Version: 8.4.1 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D4572 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed Comment: Merged to `ghc-8.4` for 8.4.2. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15002#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC