[GHC] #15105: `typecheckModule` from GHC API crashes on MacOS for files with TH
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------------+---------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Keywords: | Operating System: MacOS X Architecture: Unknown/Multiple | Type of failure: Runtime crash Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: ----------------------------------------+---------------------------------- I believe this is the same issue that is causing manually built `haddock` and `doctest` to crash on MacOS when fed TH (originally reported https://github.com/haskell/haddock/issues/767 and https://github.com/sol/doctest/issues/199). I've attached a minimal program that uses the GHC API and exhibits the same problem. {{{ $ ghc-8.4.2 -package ghc -package containers -package ghc-paths Prog.hs [1 of 1] Compiling Main ( Prog.hs, Prog.o ) Linking Prog ... $ ./Prog Main-no-TH.hs -package template-haskell $ ./Prog Main-TH.hs -package template-haskell Prog: lookupSymbol failed in relocateSection (RELOC_GOT) /usr/local/lib/ghc-8.4.2/integer-gmp-1.0.2.0/HSinteger-gmp-1.0.2.0.o: unknown symbol `___gmp_rands' Prog: Prog: unable to load package `integer-gmp-1.0.2.0' }}} In case it isn't clear, I do not expect `Main-TH.hs` to crash `Prog`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Changes (by harpocrates): * Attachment "Main-no-TH.hs" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Changes (by harpocrates): * Attachment "Main-TH.hs" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Changes (by harpocrates): * Attachment "Prog.hs" added. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Comment (by harpocrates): The problem doesn't manifest itself with an inplace GHC. {{{ $ diff Prog.hs InplaceProg.hs 4d3 < import qualified GHC.Paths as GhcPaths 9c8 < libDir <- pure GhcPaths.libdir ---
let libDir = "./inplace/lib" }}}
Then, {{{ $ ./inplace/bin/ghc-stage2 -package ghc -package containers InplaceProg.hs [1 of 1] Compiling Main ( InplaceProg.hs, InplaceProg.o ) Linking InplaceProg ... $ ./InplaceProg Main-no-TH.hs -package template-haskell $ ./InplaceProg Main-TH.hs -package template-haskell -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Comment (by darchon): We have some code in the Clash compiler to do with dynamic linking, which I expect is TH-related: https://github.com/clash-lang/clash- compiler/blame/d7dfef9d89b30d096370887899be24b9027914ac/clash-ghc/src- ghc/Clash/GHC/LoadModules.hs#L174-L179 {{{ let ghcDynamic = case lookup "GHC Dynamic" (DynFlags.compilerInfo dflags) of Just "YES" -> True _ -> False let dflags3 = if ghcDynamic then DynFlags.gopt_set dflags2 DynFlags.Opt_BuildDynamicToo else dflags2 _ <- GHC.setSessionDynFlags dflags3 }}} I don't have access to an OS X machine, so could you check if setting the {{{Opt_BuildDynamicToo}}} flag fixes the TH issue? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
We have some code in the Clash compiler to do with dynamic linking, which I expect is TH-related: https://github.com/clash-lang/clash- compiler/blame/d7dfef9d89b30d096370887899be24b9027914ac/clash-ghc/src- ghc/Clash/GHC/LoadModules.hs#L174-L179
{{{ let ghcDynamic = case lookup "GHC Dynamic" (DynFlags.compilerInfo dflags) of Just "YES" -> True _ -> False let dflags3 = if ghcDynamic then DynFlags.gopt_set dflags2 DynFlags.Opt_BuildDynamicToo else dflags2 _ <- GHC.setSessionDynFlags dflags3 }}}
I don't have access to an OS X machine, so could you check if setting
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Comment (by harpocrates): Replying to [comment:2 darchon]: the {{{Opt_BuildDynamicToo}}} flag fixes the TH issue? I just tried this on OS X and it unfortunately makes no difference: in either case, `lookup "GHC Dynamic" (DynFlags.compilerInfo dynflags')` produced `Just "NO"` (and then manually enabling `DynFlags.Opt_BuildDynamicToo` also didn't change anything). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Changes (by lerkok): * cc: lerkok (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Comment (by bgamari): Any updates on this harpocrates? Judging by the haddock ticket it sounds like you are pretty close. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Comment (by bgamari): On https://github.com/haskell/haddock/issues/767 harpocrates said,
Alright, I think I'm close to nailing this. I think there is something suspicious about the `HSinteger-gmp-1.0.2.0.o` file from GHC's `integer- gmp` lib folder. I think all that we need to do to fix this issue is remove this file. When I do that, everything starts working again.
Note that the symbols provided by this object file are already in the static library of the same name. Furthermore, when I manually install GHC (after building from source), this troublesome file is not present.
I'm obviously still investigating, but I figured I'd ask if anyone knows any reason `HSinteger-gmp-1.0.2.0.o` should not be deleted...
This object is a monolithic relocatable object used by statically-linked GHCi. Are you seeing it being loaded? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Changes (by chak): * cc: chak (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Comment (by harpocrates): After removing `HSinteger-gmp-1.0.2.0.o` more than two weeks ago, I stopped having this issue. Since then, I haven't encountered any other issues/side-effects (isn't a statically linked GHCi the default? GHCi is still working for me...). Everything has continued to function just fine and as expected. I confess I haven't really spent any time on this since that last comment on the Haddock ticket. I'd be glad to help (since I can replicate the bug quite easily), but I'm not sure what to investigate next. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Comment (by lerkok): I can confirm that the workaround @harpocrates is describing works for me. (In particular, the doctest package on Mac.) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: 8.6.1 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Comment (by simonmic): Also confirmed, I can run doctests again and don't yet see the ill effects from {{{ ~$ locate HSinteger-gmp-1.0.2.0.o /Users/simon/.stack/programs/x86_64-osx/ghc-8.4.2/lib/ghc-8.4.2/integer- gmp-1.0.2.0/HSinteger-gmp-1.0.2.0.o ~$ mv /Users/simon/.stack/programs/x86_64-osx/ghc-8.4.2/lib/ghc-8.4.2 /integer-gmp-1.0.2.0/HSinteger-gmp-1.0.2.0.o{,_DISABLE_GHC_ISSUE_15105} }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.3 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Changes (by bgamari): * priority: normal => high * milestone: 8.8.1 => 8.6.3 Comment: Apparently [[https://github.com/commercialhaskell/stack/issues/3990|several others]] have encountered this. Bumping in priority. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.3 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Changes (by berdario): * Attachment "symboltable.xz" added. Symbol table for HSinteger-gmp-1.0.2.0.o -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.3 Component: GHC API | Version: 8.4.2 Resolution: | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Comment (by bgamari): I have been looking into this with Dario Bertini at Munihac. It appears that `___gmp_rands` is a "common" symbol. With linker debugging enabled I see the following output, {{{ relocateSection: making jump island for ___stack_chk_guard, extern = 1, X86_64_RELOC_GOT lookupSymbol: looking up ___stack_chk_guard lookupSymbol: symbol not found lookupSymbol: looking up ___stack_chk_guard with dlsym relocateSection: looked up ___stack_chk_guard, external X86_64_RELOC_GOT or X86_64_RELOC_GOT_LOAD : addr = 0x7fffabebd070 relocateSection: value = 0x10e3e1d50 relocateSection: relocation 8481 : type = 3 : address = 283153 : symbolnum = 4303 : pcrel = 1 : length = 2 : extern = 1 : type = 3 relocateSection: length = 2, thing = 0, baseValue = 0x10e3125f5 relocateSection: making jump island for ___gmp_rands, extern = 1, X86_64_RELOC_GOT lookupSymbol: looking up ___gmp_rands lookupSymbol: symbol not found lookupSymbol: looking up ___gmp_rands with dlsym relocateSection: looked up ___gmp_rands, external X86_64_RELOC_GOT or X86_64_RELOC_GOT_LOAD : addr = 0x0 ghc-stage2: lookupSymbol failed in relocateSection (RELOC_GOT) /Users/berdario/Projects/ghc/libraries/integer-gmp/dist-install/build /HSinteger-gmp-1.0.2.0.o: unknown symbol `___gmp_rands' removeLibrarySearchPath: ptr = `0x0' }}} In principle common symbols do appear to be handled by `ocGetNames_MachO`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH
----------------------------------+----------------------------------------
Reporter: harpocrates | Owner: (none)
Type: bug | Status: new
Priority: high | Milestone: 8.6.3
Component: GHC API | Version: 8.4.2
Resolution: | Keywords:
Operating System: MacOS X | Architecture: Unknown/Multiple
Type of failure: Runtime crash | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
----------------------------------+----------------------------------------
Comment (by Ben Gamari
#15105: `typecheckModule` from GHC API crashes on MacOS for files with TH ----------------------------------+---------------------------------------- Reporter: harpocrates | Owner: (none) Type: bug | Status: closed Priority: high | Milestone: 8.6.3 Component: GHC API | Version: 8.4.2 Resolution: fixed | Keywords: Operating System: MacOS X | Architecture: Unknown/Multiple Type of failure: Runtime crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | ----------------------------------+---------------------------------------- Changes (by bgamari): * status: new => closed * resolution: => fixed Comment: Merged with 11fd7df565897fb98cda9273dab28ccc654a6d95. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/15105#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC