#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by thomie): I can not reproduce this bug with HEAD. To reproduce with ghc-7.10.2, `cabal install repa`, and compile the above example with `-O -funfolding-use-threshold=90` (`-funfolding-use- threshold=80` makes the bug go away). Maybe it is fixed. But I can't be certain: change the example ever so slightly, and the bug is not reproducible with ghc-7.10.2 either. Adding this as a test is difficult, because it depends on `repa`, and we don't want to add that to the testsuite. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Can anyone reproduce this on HEAD or 8.0.1? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by jscholl): I have a theory what is happening here: The important values here are {{{arr2}}} and some value called {{{get}}}. In the original definition {{{get}}} evaluates {{{arr2}}} and uses some component of it. Now {{{arr2}}} just is some tuple type ({{{ADelayed}}}) with two lazy fields. It is constructed by evaluating another value {{{arr}}} and then building this tuple. The second field of the tuple is just a thunk. So the simplifier inlines {{{arr2}}} in {{{get}}} and thus avoids building the tuple thing of {{{arr2}}}. Now the last reference to {{{arr2}}} was dropped. In the next run the simplifier replaces {{{arr2}}} with {{{absentError}}}. Shortly after that a reference to {{{arr2}}} appears again in the code. So what went wrong? {{{get}}} had an unfolding! In this unfolding it mentions {{{arr2}}} and the simplifier spots a position where it wants to inline {{{get}}}. Now we have a reference to {{{arr2}}} again because the unoptimized version of {{{get}}} is inlined. But this time we will optimize in a different way as {{{arr2}}} was incorrectly changed to {{{absentError}}}. So the main question would be: Does the compiler consider unfoldings when determining whether a value is unused and thus can be replaced by {{{absentError}}}? If not this could be the culprit. Sadly the bug relies on a very specific order of events in the compiler, so it will be hard to find a smaller test case for this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by jscholl): * Attachment "Unfold.hs" added. program with wrong intermediate usage annotation -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: Type: bug | Status: new Priority: highest | Milestone: 8.0.1 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj):

So the main question would be: Does the compiler consider unfoldings when determining whether a value is unused and thus can be replaced by `absentError`?

I'm very impressed. You've waded though the swamp and indeed I think you have nailed a culprit. Whether it is ''the'' culprit in the original program I don't know, but this does look wrong. Just to lay it out, we have {{{ f x = let arr2 = ... let getIt :: Int -> Int [Unf = { Src=InlineStable, Tmpl=..arr2... }] getIt n = ...(no arr2)... in ... }}} So `arr2` is not used in `getIt`'s RHS (presumably because `arr2` has already been inlined into it, but it ''is'' used in `getIt`'s ''unfolding''. Moreover, it's a "stable" unfolding, meaning that it stays unaffected by transformations in `getIt`'s RHS. So the following sequence could happen * Strictness analysis thinks that `arr2` is unused * Worker-wrapper replaces its binding with `arr2 = absentError "blah"` * `getIt` is inlined at some call site, and lo! `arr2` is resurrected. Hmm. Now you have exposed this so well, it's clear that the demand analyser should take account of the free vars of the unfolding, at least so far as absence analysis is concerned. I'll look at how to do that. Most helpful. Please apply your forensic powers to other bugs :-). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: Type: bug | Status: new Priority: high | Milestone: 8.0.2 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.0.1 => 8.0.2 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: Type: bug | Status: new Priority: high | Milestone: 8.2.1 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Simon says, * It's rather rare so we just want to always account for unfolding's free variables in absence analysis * "When doing strictness analysis we don't want to make it appear that it will be used lazily if it's used strictness" -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: bgamari Type: bug | Status: patch Priority: high | Milestone: 8.2.1 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3221 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * owner: (none) => bgamari -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: bgamari Type: bug | Status: patch Priority: high | Milestone: 8.6.1 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3221 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * milestone: 8.4.1 => 8.6.1 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: (none) Type: bug | Status: patch Priority: high | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3221 Wiki Page: | -------------------------------------+------------------------------------- Comment (by sgraf): This is an error that came up repeatedly when I worked on my own usage analysis (which includes absence analysis). It was caused by RULEs (orphans and associated ones) and/or unfoldings in module `Lib` referencing otherwise absent/dead bindings, so that some binding (e.g. specialisations of an exported binding) that appeared to be dead could suddenly become alive when a specialisation fired in some client module `Main`. That lead to such an absent error, spread over multiple modules. FWIW, this won't frequently occur with the current demand analyser, because it assumes that *all* top-level bindings are alive instead of trying hard to find the minimal set, like the occurence analyser does. But this may pop up again e.g. for Call Arity, where the assumption is that only exported ids are such 'usage roots'. The disregard for RULEs and Unfoldings could lead, in theory, to a situation where we eta-expand a binding from arity 2 to arity 3 because every considered live call allows that, but some call in a RULE or Unfolding might still only have arity 2. Like in #10176, this could make bindings too lazy for their own good. The (naive) fix of treating Unfoldings/RULEs as an additional RHS of a binding (`if ? then orig_rhs else unfolding`) had detrimental effect on the precision of my usage analysis. I suspect this is the case here, too? In the end, I decided not to include the fix in order to be comparable to Call Arity and demand analysis, which is quite a shame. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: bgamari Type: bug | Status: patch Priority: high | Milestone: 8.8.1 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3221 Wiki Page: | -------------------------------------+------------------------------------- Comment (by Krzysztof Gogolewski ): In [changeset:"448b77b93b369745e9bfbc8b46a5b87bb73dd379/ghc" 448b77b9/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="448b77b93b369745e9bfbc8b46a5b87bb73dd379" Add RubbishLit for absent bindings of UnliftedRep Summary: Trac #9279 reminded us that the worker wrapper transformation copes really badly with absent unlifted boxed bindings. As `Note [Absent errors]` in WwLib.hs points out, we can't just use `absentError` for unlifted bindings because there is no bottom to hide the error in. So instead, we synthesise a new `RubbishLit` of type `forall (a :: TYPE 'UnliftedRep). a`, which code-gen may subsitute for any boxed value. We choose `()`, so that there is a good chance that the program crashes instead instead of leading to corrupt data, should absence analysis have been too optimistic (#11126). Reviewers: simonpj, hvr, goldfire, bgamari, simonmar Reviewed By: simonpj Subscribers: osa1, rwbarton, carter GHC Trac Issues: #15627, #9279, #4306, #11126 Differential Revision: https://phabricator.haskell.org/D5153 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#11126: Entered absent arg in a Repa program -------------------------------------+------------------------------------- Reporter: tuplanolla | Owner: bgamari Type: bug | Status: patch Priority: high | Milestone: 8.8.1 Component: Compiler | Version: 7.10.2 Resolution: | Keywords: | DemandAnalysis Operating System: Unknown/Multiple | Architecture: Type of failure: Incorrect result | Unknown/Multiple at runtime | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3221 Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonpj): * keywords: => DemandAnalysis -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/11126#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler