[GHC] #13429: Optimizer produces Core with an infinite <<loop>>

#13429: Optimizer produces Core with an infinite <<loop>>
-------------------------------------+-------------------------------------
Reporter: lehins | Owner: (none)
Type: bug | Status: new
Priority: normal | Milestone:
Component: Compiler | Version: 8.0.2
Keywords: | Operating System: Unknown/Multiple
Architecture: | Type of failure: Incorrect result
Unknown/Multiple | at runtime
Test Case: | Blocked By:
Blocking: | Related Tickets:
Differential Rev(s): | Wiki Page:
-------------------------------------+-------------------------------------
While using vector package to implement convolution and supplying `-O1` or
`-02` to ghc compilation results in core with an infinite loop. In order
to trigger this behavior at least two modules is required. Attached is the
minimal setup that I could come up with, that demonstrates the issue. Here
is the stack trace:
{{{
$ stack --install-ghc --resolver lts-8.3 exec -- ghc -O1 -prof -fprof-auto
main.hs && ./main +RTS -xc
[1 of 2] Compiling Loop ( Loop.hs, Loop.o )
[2 of 2] Compiling Main ( main.hs, main.o )
Linking main ...
*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace:
Main.CAF
--> evaluated by: Main.main,
called from Main.CAF
--> evaluated by: Loop.toKernel.\,
called from Data.Vector.Fusion.Util.>>=,
called from Loop.toKernel,
called from Main.main,
called from Main.CAF
--> evaluated by: Data.Vector.Fusion.Util.>>=,
called from Loop.toKernel,
called from Main.main,
called from Main.CAF
*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace:
Main.CAF
main: <<loop>>
}}}
At first I though that it might be a bug in a vector package, which is
still a possibility, but since I was not able to observe that issue with
ghc-7.8.4, I decided to open an ticket here. I tested attached code with
all of subsequent released ghc versions (7.10.1 - 8.0.2), which resulted
in the infinite loop.
Worth noting that sometimes, when recompilation of only the `main.hs` file
is enforced, program starts to work as expected. Here is an example:
{{{
$ stack --resolver ghc-7.10.3 exec --package vector-0.11.0.0 --package
primitive-0.6.1.0 -- ghc -O1 main.hs && ./main
[1 of 2] Compiling Loop ( Loop.hs, Loop.o )
[2 of 2] Compiling Main ( main.hs, main.o )
Linking main ...
main: <<loop>>
$ touch main.hs
$ stack --resolver ghc-7.10.3 exec --package vector-0.11.0.0 --package
primitive-0.6.1.0 -- ghc -O1 main.hs && ./main
[2 of 2] Compiling Main ( main.hs, main.o )
Linking main ...

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by lehins): * Attachment "Loop.hs" added. Minimal implementation that triggers the issue. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by lehins): * Attachment "main.hs" added. correlation has to be invoked from a separate module. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by lehins): * Attachment "RUNS.md" added. Various versions that compilation was attempted with. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: merge Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => merge * milestone: => 7.10.4 Comment: This appears to be fixed in 8.2 with `vector-0.12.0.0`. Unfortunately, it's quite unclear what fixed it and working this our will likely be a significant investment of effort. Given that there is unlikely to be an 8.0.3, I think it's pretty unlikely that we will fix this in the 8.0 series but I'm marking this as `merge` regardless. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: merge Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by lehins): I was able to reproduce this issue using lists instead of vector package, confirming that it is indeed a bug in ghc. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

I was able to reproduce this issue using lists instead of vector
#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: merge Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): Replying to [comment:2 lehins]: package, confirming that it is indeed a bug in ghc. Could you attach that reproduction here? I suspect this might be worth investigating a bit more, despite the pain, to make sure that the underlying problem was actually fixed and not just buried. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: merge Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by lehins): * Attachment "Loop.2.hs" added. Same implementation triggering a bug using List instead of vector. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: merge Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by rwbarton): Thanks. I confirmed that the bug was present in HEAD as of Feb 1 but is not present any more. Something rather curious is that the bug cannot be reproduced when there are `.hi` and `.o` files present from a previous compilation with the same version of ghc, even when passing `-fforce-recomp`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: merge Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by lehins): It's good to hear that it is fixed. It would be nice to know what is the actual cause and a reliable way to avoid this bug with ghc-8.0.2 and older, so anybody who stumbles upon it has a workaround. It feels like it has something to do with class constraints and inlining, but what exactly is still a mystery to me. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>>
-------------------------------------+-------------------------------------
Reporter: lehins | Owner: (none)
Type: bug | Status: merge
Priority: normal | Milestone: 7.10.4
Component: Compiler | Version: 8.0.2
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
Type of failure: Incorrect result | Unknown/Multiple
at runtime | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => new Comment: While this appears to be fixed, Simon would like to know what precisely fixed it. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): I've verified that commit 2effe18ab51d66474724d38b20e49cc1b8738f60 (The Early Inline Patch) was what fixed this. Somewhat related: the [https://ghc.haskell.org/trac/ghc/changeset/be8122ab72aeec509b5ce4b4f05fbc5cd... commit] which added this program as a regression test isn't actually //running// the compiled program, so it's not verifying if it's looping or not. Surely this isn't intended? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Yes, the test should run. We still need to know why it looped before. The early-inline patch should not, of itself, have changed that! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by lehins): For what it's worth, I figured out a reliable way to avoid the bug, namely adding an extra constraint `Array X e`. So changing the type signature of `correlate` to: `correlate :: (Array X e, Array cs e) => Image X e -> Image cs e -> Image cs e` can be used as a trick for working around the bug. Not sure if it is relevant, but maybe it will help in narrowing down a possible cause. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * Attachment "Loop.hs-reduced" added. Reduced version of Loop -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * Attachment "main.hs-reduced" added. main to go with reduced Loop -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): I am getting the very strong feeling that the problem has a lot to do with `UndecidableInstances`, constraint simplification, or something in that general area. Simplifying constraints by hand seems to make the bug go away. Removing the `INLINE` annotation seems to do so as well. I created a reduced test case, which hopefully will be simpler to work on. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * cc: dfeuer (added) * failure: Incorrect result at runtime => Runtime crash * priority: normal => high -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * Attachment "LoopReduced2.hs" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * Attachment "mainReduced2.hs" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * Attachment "dumpbefore.tgz" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * Attachment "dumpafter.tgz" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): I reduced the test case again. See `LoopReduced2.hs` and `mainReduced2.hs` (you'll have to rename the `LoopReduced2.hs` file to `Loop.hs` to compile). I attached `-dverbose-core2core` results before and after the early inlining patch as `dumpbefore.tgz` and `dumpafter.tgz`. I haven't figured out where things go wrong just yet, but if you look at `dumpbefore/main.dump-simpl`, you'll see {{{#!hs Rec { -- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0} Main.$s$fArrayXe_$dNum [Occ=LoopBreaker] :: Num Word8 [GblId, Str=b] Main.$s$fArrayXe_$dNum = Main.$s$fArrayXe_$dNum end Rec } }}} which is very obviously very bad. For some reason we're "optimizing" a dictionary into a reference to itself! My vague guess is that GHC sees that it can get a `Num Word8` dictionary from an `Array` or `ColorSpace` dictionary, and that it can get `ColorSpace` and `Array` dictionaries from `Num` ones, and makes some very bad decisions. I have no idea why the early inline patch fixes this, but I'm rather nervous about it being a proper fix. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * Attachment "LoopReduced3.hs" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * Attachment "dump-before3.tgz" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): I've reduced it once more. I believe the fundamental problem here happens in specialization. We get {{{ Rec { -- RHS size: {terms: 2, types: 1, coercions: 0, joins: 0/0} $dArray_s1RC :: Array X Word8 [LclId] $dArray_s1RC = Loop.$fArrayXe @ Word8 GHC.Word.$fNumWord8 -- RHS size: {terms: 2, types: 1, coercions: 0, joins: 0/0} $dArray1_s1RG [Occ=OnceL] :: Array X Word8 [LclId] $dArray1_s1RG = Loop.$fArrayXe @ Word8 $dNum_s1RH -- RHS size: {terms: 2, types: 2, coercions: 0, joins: 0/0} $dNum_s1RH :: Num Word8 [LclId] $dNum_s1RH = Loop.$p1Array @ X @ Word8 $dArray_s1RC -- RHS size: {terms: 7, types: 5, coercions: 0, joins: 0/0} $s$fArrayXe_s1RS [InlPrag=CONLIKE] :: Array X Word8 [LclId, Unf=DFun: \ -> Loop.C:Array TYPE: X TYPE: Word8 Loop.$fArrayXe_$cp1Array @ Word8 $dNum_s1RH Loop.$fArrayXe_$cpromote @ Word8 $dNum_s1RH Loop.$fArrayXe_$cmakeImage @ Word8 $dNum_s1RH] $s$fArrayXe_s1RS = Loop.C:Array @ X @ Word8 (Loop.$fArrayXe_$cp1Array @ Word8 $dNum_s1RH) (Loop.$fArrayXe_$cpromote @ Word8 $dNum_s1RH) (Loop.$fArrayXe_$cmakeImage @ Word8 $dNum_s1RH) ... }}} which doesn't seem ''inherently'' bad (although it's not very clean), but then we also get this rather awful specialization rule: {{{ "SPEC/Main $fArrayXe @ Word8" forall (v_s1RJ :: Num Word8). Loop.$fArrayXe @ Word8 v_s1RJ = $s$fArrayXe_s1RS }}} If this rule fires in the RHS of `$dArray_s1RC`, then we get {{{ $dArray_s1RC = $s$fArrayXe_s1RS }}} Inlining that into `$dNum_s1RH` gives {{{ $dNum_s1RH = Loop.$p1Array @ X @ Word8 $s$fArrayXe_s1RS }}} But `$dNum_s1RH` is used in the definition of `$s$fArrayXe_s1RS`, so we have a loop. I don't know if I've told exactly the right story here, but I think the real story is probably pretty similar. The results will be excruciatingly fragile, depending on whether the right or wrong specialization rule fires. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by dfeuer): * Attachment "LoopReduced3.hs" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): So David and I discussed this a bit and I suspect that the loop arises as early as typechecking. David summarized quite well above, but I thought I might try to recap to make sure I also understand. Recall that the minimized example has the following, {{{#!hs module Loop where data X class Num e => Array cs e where ... instance Num e => Array X e where ... module Main where import Loop {- some bindings requiring an Array X Word8 instance -} }}} The output from the desugarer is nothing special, containing a local dictionary binding for `Array X Word8`, {{{#!hs -- main.dump-ds $dArray_a1zv :: Array X Word8 $dArray_a1zv = Loop.$fArrayXe @Word8 GHC.Word.$fNumWord8 }}} The specialiser then takes this and sees the `Loop.$fArrayXe` DFun from the `Array X e` instance and tries to specialise it. This results in, {{{#!hs $dArray_s1BE :: Array X Word8 $dArray_s1BE = Loop.$fArrayXe @Word8 GHC.Word.$fNumWord8 $s$fArrayXe_s1BU [InlPrag=[ALWAYS] CONLIKE] :: Array X Word8 $s$fArrayXe_s1BU = Loop.C:Array @ X @ Word8 (Loop.$fArrayXe_$cp1Array @Word8 $dNum_s1BJ) (Loop.$fArrayXe_$cpromote @Word8 $dNum_s1BJ) (Loop.$fArrayXe_$cmakeImage @Word8 $dNum_s1BJ) $dNum_s1BJ :: Num Word8 $dNum_s1BJ = Loop.$p1Array @X @Word8 $dArray_s1BE -- N.B. $p1Array is a selector for Array's Num superclass {- Rules: "SPEC/Main $fArrayXe @Word8" [ALWAYS] forall ($dNum_s1BL :: Num Word8). Loop.$fArrayXe @Word8 $dNum_s1BL = $s$fArrayXe_s1BU -} }}} As David pointed out above, this introduces a rather problematic rule. It's problematic because it will rewrite `$dArray_s1BE`, which introduces the concrete `fNumWord8` dictionary into our local `Array X Word8` dictionary. It will rewrite `$dArray_s1BE` in terms of `$s$fArrayXe_s1BU`, which has no mention of any concrete `Num` dictionary, instead pulling it out of `$dArray_s1BE` via a superclass selector. This is clearly a bad situation: the specialiser has introduced a loop where there was none previously. It's not entirely obvious how to pin down what is "bad" about this rule, but the fact that it throws away a dictionary argument is certainly relevant. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): Alberto Valverde [https://mail.haskell.org/pipermail/ghc- devs/2017-May/014222.html discovered] another occurrence of this bug: the `fingertree-0.1.1.0` [http://hackage.haskell.org/package/fingertree-0.1.1.0 test suite]: {{{ $ ~/Software/ghc-8.0.2/bin/ghc -O -fforce-recomp FingerTreeTestMain.hs [1 of 2] Compiling FingerTree ( FingerTree.hs, FingerTree.o ) [2 of 2] Compiling Main ( FingerTreeTestMain.hs, FingerTreeTestMain.o ) Linking FingerTreeTestMain ... $ ./FingerTreeTestMain True $ ~/Software/ghc-8.2.0.20170507/bin/ghc -O -fforce-recomp FingerTreeTestMain.hs [1 of 2] Compiling FingerTree ( FingerTree.hs, FingerTree.o ) [2 of 2] Compiling Main ( FingerTreeTestMain.hs, FingerTreeTestMain.o ) Linking FingerTreeTestMain ... $ ./FingerTreeTestMain FingerTreeTestMain: <<loop>> }}} In case it's useful, I'll attach a minimized version of the test suite. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * Attachment "FingerTree.hs" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * Attachment "FingerTreeTestMain.hs" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Thanks, that's helpful. Does the `fingertree` example of the bug happen in HEAD? (The other example does not.) Are you sure it's the same bug? Just for info, I have investigated this a bit, and I know what's going on. The tricky bit is that we want to specialise dictionary functions (which arise from instance decls); but they produce dictionaries, and it's rather easy to mess up. I think I have a reasonably simple solution, but it's a solition I can't say is wrong, rather than one that obviously right. So I've been ruminating, so far to little avail. Is it actually a show-stopper for anyone? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): Simon, I can't speak for absolute HEAD, but I tried it with a very recent build and the bug happens there. I tried modifying the code to use type families and equality constraints to eliminate `FunctionalDependencies`, `FlexibleContexts`, and `UndecidableInstances`, but the same bug shows up. I've not yet been able to make the bug happen using single-parameter type classes, but that certainly doesn't mean it's impossible. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:18 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by dfeuer): As for your other question, Simon, I think we likely ''should'' consider this a show-stopper. This is not test code designed to break GHC; it's test code designed to break a library, using absolutely standard methodology. I don't think it does anything that would be surprising to see in production code. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): By the way, commit a920404fb12fb52a59e4f728cce4d662a418c5f8 (Fix TcSimplify.decideQuantification for kind variables) is the first commit in which the `fingertree` looping behavior can be observed. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #13750 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * related: => #13750 Comment: See also #13750, in which the same commit that caused this ticket also caused a program to start //segfaulting//. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #13750 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Further to comment:17 I have made no further progress, and now I'm away for a week. Hmm. I think that disabling specialisation (by `specImports`) for imported dictinoary functions would probably cure it, at some modest cost in optimisation. Might be worth a try as a temporary expedient. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #13750 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by lehins): * Attachment "Array.hs" added. Another minimal example of the bug. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 7.10.4 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #13750 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by lehins): * Attachment "Main.hs" added. Main file that goes with Array.hs -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.2.1 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #13750 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonpj): * priority: high => highest * milestone: 7.10.4 => 8.2.1 Comment: This is a bad bug, and the `FingerTree` example is still happening in HEAD, so I assume also in 8.2. (Previously it was only happening in 8.0.) I know what is going on but have been struggling to find time to fix it. Hopefully this week. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: simonpj Type: bug | Status: new Priority: highest | Milestone: 8.2.1 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: #13750 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonpj): * owner: (none) => simonpj -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>>
-------------------------------------+-------------------------------------
Reporter: lehins | Owner: simonpj
Type: bug | Status: new
Priority: highest | Milestone: 8.2.1
Component: Compiler | Version: 8.0.2
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
| Unknown/Multiple
Type of failure: Runtime crash | Test Case:
Blocked By: | Blocking:
Related Tickets: #13750 | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Simon Peyton Jones

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: simonpj Type: bug | Status: merge Priority: highest | Milestone: 8.2.1 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: | simplCore/should_run/T13429, | T13429_2 Blocked By: | Blocking: Related Tickets: #13750 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonpj): * testcase: => simplCore/should_run/T13429, T13429_2 * status: new => merge Comment: Worth merging I think! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:26 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: simonpj Type: bug | Status: merge Priority: highest | Milestone: 8.2.1 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: | simplCore/should_run/T13429, | T13429_2 Blocked By: | Blocking: Related Tickets: #13750 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by nh2): I've backported the above fix to GHC 8.0.2 and created bindist for Linux: https://github.com/nh2/ghc/releases/tag/ghc-8.0.2-bugfix-12545-backport-2017... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:27 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: simonpj Type: bug | Status: merge Priority: highest | Milestone: 8.2.1 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: | simplCore/should_run/T13429, | T13429_2 Blocked By: | Blocking: Related Tickets: #13750, #12545 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by nh2): * cc: nh2 (added) * related: #13750 => #13750, #12545 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:28 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: simonpj Type: bug | Status: merge Priority: highest | Milestone: 8.2.1 Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: | simplCore/should_run/T13429, | T13429_2 Blocked By: | Blocking: Related Tickets: #13750, #12545 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Remarkably, this patch also apparently fixed #13750 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:29 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#13429: Optimizer produces Core with an infinite <<loop>> -------------------------------------+------------------------------------- Reporter: lehins | Owner: simonpj Type: bug | Status: closed Priority: highest | Milestone: 8.2.1 Component: Compiler | Version: 8.0.2 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: Runtime crash | Test Case: | simplCore/should_run/T13429, | T13429_2 Blocked By: | Blocking: Related Tickets: #13750, #12545 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed Comment: Merged to `ghc-8.2` in a7a1d7fdcd3e34787f28be4eb24645c2e14a9f3d. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/13429#comment:30 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC