[GHC] #14128: Possible bug in Renamer when dealing with orphans

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.2.1 (Type checker) | Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- In a [https://github.com/shayan-najd/GrowableGHC branch] of GHC with [https://ghc.haskell.org/trac/ghc/wiki/ImplementingTreesThatGrow Growable ASTs], the code compiles with `devel2` configuration, but fails with `./validate`. It passes all the test cases, except two that already fail in HEAD (see [https://ghc.haskell.org/trac/ghc/ticket/14126 #14126]). Especifically, the compiler returns the following error: {{{ ghc-stage1: panic! (the 'impossible' happened) (GHC version 8.3.20170806 for x86_64-unknown-linux): ASSERT failed! HsPat [GHC.Base, GHC.Float, Data.Binary.Generic, Data.ByteString.Builder, HsBinds, HsDoc, HsExpr, HsLit, HsPat, HsTypes, PprCore, GHC.LanguageExtensions, Data.Time.Calendar.Gregorian, Data.Time.Format.Parse, Data.Time.LocalTime.Internal.LocalTime, Data.Time.LocalTime.Internal.ZonedTime] Call stack: CallStack (from HasCallStack): prettyCurrentCallStack, called at compiler/utils/Outputable.hs:1133:58 in ghc:Outputable callStackDoc, called at compiler/utils/Outputable.hs:1188:22 in ghc:Outputable assertPprPanic, called at compiler/rename/RnNames.hs:364:99 in ghc:RnNames Call stack: CallStack (from HasCallStack): prettyCurrentCallStack, called at compiler/utils/Outputable.hs:1133:58 in ghc:Outputable callStackDoc, called at compiler/utils/Outputable.hs:1137:37 in ghc:Outputable pprPanic, called at compiler/utils/Outputable.hs:1186:5 in ghc:Outputable assertPprPanic, called at compiler/rename/RnNames.hs:364:99 in ghc:RnNames Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug }}} I suspect (as hinted above) it has to do with a bug in the renamer when dealing with orphan instances (and probably in conjunction with boot files). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): I'm confused. If it passes all test cases, precisely when does the ASSERT failure happen? Does it happen in HEAD? How can someone else reproduce your problem? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Shayan-Najd): About my branch - It builds with `make` (with below steps) - It passes all tests (except the two mentioned in #14126) by using `make test`. - But, it does NOT build with `./validate`(with below steps). About HEAD, - It builds with `make` (with below steps) - It passes all the tests (except the two mentioned in #14126) by using `make test`. - It also builds with `./validate`. As I said earlier,
I suspect (as hinted above) it has to do with a bug in the renamer when dealing with orphan instances (and probably in conjunction with boot files).
The easiest way (that I can think of now) to reproduce my problem is to clone my branch, and run `./validate`. @alanz could also reproduce this problem. Here are my exact steps for validating: 1. cloning my [https://github.com/shayan-najd/GrowableGHC branch] 2. `cd ghc` 3. ./validate Here are my exact steps for building and testing: 1. cloning my [https://github.com/shayan-najd/GrowableGHC branch] 2. `cd ghc` 3. `cp mk/build.mk.sample mk/build.mk` 4. set `BuildFlavour = devel2` and then 5. ./boot 6. ./configure 7. make -j4 8. THREADS=4 make test" -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonpj): Ben could you possible look at this? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Of course. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: new Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * owner: (none) => bgamari * milestone: => 8.2.2 Comment: Surprisingly, I'm having trouble even getting as far as Shayan reports. Instead I'm seeing renaming failures while building stage1, namely during `HsExpr.hs`: {{{ ghc: panic! (the 'impossible' happened) (GHC version 8.2.0.20170704 for x86_64-unknown-linux): tcIfaceGlobal (local): not found You are in a maze of twisty little passages, all alike. While forcing the thunk for TyThing SyntaxExpr which was lazily initialized by initIfaceTcRn, I tried to tie the knot, but I couldn't find SyntaxExpr in the current type environment. If you are developing GHC, please read Note [Tying the knot] and Note [Type-checking inside the knot]. Consider rebuilding GHC with profiling for a better stack trace. Contents of current type environment: [] Call stack: CallStack (from HasCallStack): prettyCurrentCallStack, called at compiler/utils/Outputable.hs:1133:58 in ghc:Outputable callStackDoc, called at compiler/utils/Outputable.hs:1137:37 in ghc:Outputable pprPanic, called at compiler/iface/TcIface.hs:1689:23 in ghc:TcIface }}} I am able to reproduce this with both 8.0.2 and 8.2.1-rc3. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: new Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Strangely enough I'm unable to reproduce the issue I reported in comment:5 in another environment. Instead I reproduce the issue reported by Shayan. I'll continue investigation there. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: new Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): It seems like there is a bit of inconsistency regarding whether or not a module can be in its own `dep_orphs`. `TcRnDriver.tcRnImports` seems to suggest that `imp_orphs` (from which `DsUsage.mkDependencies` derives `dep_orphs`) can contain the current module due to `hs-boot`s, {{{#!hs -- Load any orphan-module (including orphan family -- instance-module) interfaces, so that their rules and -- instance decls will be found. But filter out a -- self hs-boot: these instances will be checked when -- we define them locally. -- (We don't need to load non-orphan family instance -- modules until we either try to use the instances they -- define, or define our own family instances, at which -- point we need to check them for consistency.) ; loadModuleInterfaces (text "Loading orphan modules") (filter (/= this_mod) (imp_orphs imports)) }}} However, the assertion in `RnNames.calculateAvails` (which is what Shayan reports) obvious disagrees: {{{#!hs orphans | orph_iface = ASSERT2( not (imp_sem_mod `elem` dep_orphs deps), ppr imp_sem_mod <+> ppr (dep_orphs deps) ) imp_sem_mod : dep_orphs deps }}} Looking at `mkDependendies`, it seems that we explicitly filter the currently module out of `imp_dep_mods`, {{{#!hs let dep_mods = modDepsElts (delFromUFM (imp_dep_mods imports) (moduleName mod)) -- M.hi-boot can be in the imp_dep_mods, but we must remove -- it before recording the modules on which this one depends! -- (We want to retain M.hi-boot in imp_dep_mods so that -- loadHiBootInterface can see if M's direct imports depend -- on M.hi-boot, and hence that we should do the hi-boot consistency -- check.) }}} We do not do the same thing for `dep_orphs`; I wonder if this should change. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: new Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Shayan-Najd): Thank you for looking into this. I am still puzzled on why it builds with `make` (with `devel2`) but fails with `validate`. Is it because `ASSERT`s are all ignored using `make`? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: new Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Replying to [comment:8 Shayan-Najd]:
Thank you for looking into this. I am still puzzled on why it builds with `make` (with `devel2`) but fails with `validate`. Is it because `ASSERT`s are all ignored using `make`?
The `devel2` build flavour builds only the stage 2 compile with assertions enable (specifically by defining the `DEBUG` macro). `validate`, on the other hand, enables assertions when building stage 1. Here, however, we are seeing the assertions fail in a run of the stage1 compiler. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: new Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I believe the testcase in Phab:D3890 should demonstrate this issue. There is also the question of whether `mkDependencies` also needs to remove the current module from `dep_finsts`. I suspect so as `calculateAvails` asserts the same invariant there. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: patch Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3892 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => patch * differential: => Phab:D3892 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: patch Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3892 Wiki Page: | -------------------------------------+------------------------------------- Changes (by simonpj): * cc: ezyang (added) Comment: Edward is the expert here. Edward: might you look at Ben's proposed fix? If correct, it needs a careful Note along the lines of the Summary of Phab:D3892. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans
-------------------------------------+-------------------------------------
Reporter: Shayan-Najd | Owner: bgamari
Type: bug | Status: patch
Priority: normal | Milestone: 8.2.2
Component: Compiler (Type | Version: 8.2.1
checker) |
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D3892
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari

#14128: Possible bug in Renamer when dealing with orphans
-------------------------------------+-------------------------------------
Reporter: Shayan-Najd | Owner: bgamari
Type: bug | Status: patch
Priority: normal | Milestone: 8.2.2
Component: Compiler (Type | Version: 8.2.1
checker) |
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D3892
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari

#14128: Possible bug in Renamer when dealing with orphans
-------------------------------------+-------------------------------------
Reporter: Shayan-Najd | Owner: bgamari
Type: bug | Status: patch
Priority: normal | Milestone: 8.2.2
Component: Compiler (Type | Version: 8.2.1
checker) |
Resolution: | Keywords:
Operating System: Unknown/Multiple | Architecture:
| Unknown/Multiple
Type of failure: None/Unknown | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s): Phab:D3892
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: closed Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3892 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => closed * resolution: => fixed Comment: Merged to `ghc-8.2` as well. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#14128: Possible bug in Renamer when dealing with orphans -------------------------------------+------------------------------------- Reporter: Shayan-Najd | Owner: bgamari Type: bug | Status: closed Priority: normal | Milestone: 8.2.2 Component: Compiler (Type | Version: 8.2.1 checker) | Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Phab:D3892 Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I briefly looked at doing the treatment in comment:14 to `dep_finsts` but it seems unnecessary as you cannot define type family instances in `hs- boot` files. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/14128#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC