[GHC] #13531: GHC fails with "Dynamic linker not initialised" when -j is on and trying to load nonexistent .so file
#13531: GHC fails with "Dynamic linker not initialised" when -j is on and trying to load nonexistent .so file -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: Compile-time Unknown/Multiple | crash or panic Test Case: | Blocked By: Blocking: | Related Tickets: #13137, #9868, | #10355 Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- For ghc 8.0.2 (on Linux, in nixpkgs `0c041520c3` for exact reproducibility), when TH is used in a `ghc --make -j1` invocation, and thus ghc does the whole {{{ cc -fno-stack-protector -DTABLES_NEXT_TO_CODE '-std=c++14' -Wno- deprecated-declarations --print-file-name liblibglog.so Loading object (dynamic) glog ... done }}} business, and the `.so` file in question does not exist, then usually GHC prints {{{ <command line>: user specified .o/.so/.DLL could not be loaded (libglog.so: cannot open shared object file: No such file or directory) }}} However, I found that when parallel compilation is enabled, (e.g. `ghc -j4`), I can get this instead: {{{ [ 23 of 130] Compiling Mymodule ( Mymodule.hs, dist/build/Mymodule.o ) <no location info>: error: ghc: panic! (the 'impossible' happened) (GHC version 8.0.2 for x86_64-unknown-linux): Dynamic linker not initialised Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13531> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13531: GHC fails with "Dynamic linker not initialised" when -j is on and trying to load nonexistent .so file -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #13137, #9868, | Differential Rev(s): #10355 | Wiki Page: | -------------------------------------+------------------------------------- Comment (by nh2): {{{ (13:52:43) Phyx-: it seems like a race condition, something is calling a linker function which requires the persistent state without first calling initDynLinker (13:52:52) Phyx-: would make sense if it goes away with -j1 (13:53:26) nh2: Phyx-: that sounds plausible to me; would a `ghc -v` dump help here (does it show when it initialises the linker)? (13:54:46) Phyx-: no, that part is mostly completely silent. you might get lucky with a debug build and +RTS -Dl (13:56:57) Phyx-: all the top level exposed functions in Linker.hs call initDynLinker though. (13:58:11) Phyx-: except for extendLoadedPkgs, extendLinkEnv and deleteFromLinkEnv (13:59:17) Phyx-: If you can reproduce it consistently, easiest would be just to add some print statements and see what calls what (14:00:55) Phyx-: nh2: or, better yet, try a profiling build and see if you get a stack trace from the panic (15:16:05) nh2: Phyx-: the race condition seems to happen only when the linking actually _fails_, e.g. when you would get "no such file or directory" when GHC -j1 tries to load something (15:19:40) Phyx-: nh2: then my completely wild speculation is that the function that failed was supposed to initialise the linker but didn't because of the error. (15:20:17) Phyx-: I think one of the tickets RyanGlScott said it also happens for Windows. I'll see if I can repo }}} Note I can reproduce this 100% in nixpkgs but haven't extracted it into a publicly available reproduction yet. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13531#comment:1> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13531: GHC fails with "Dynamic linker not initialised" when -j is on and trying to load nonexistent .so file -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #13137, #9868, | Differential Rev(s): #10355, #13607 | Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * related: #13137, #9868, #10355 => #13137, #9868, #10355, #13607 Comment: See also #13607, which experiences the same panic with `-j2` (but using a profiled compiler). -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13531#comment:2> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13531: GHC fails with "Dynamic linker not initialised" when -j is on and trying to load nonexistent .so file -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #13137, #9868, | Differential Rev(s): #10355, #13607 | Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): Scratch that, #13607 doesn't require a profiled compiler at all. So at this point, I'd say that #13607 and #13531 are symptoms of the same issue (I'll leave this ticket open though, since I don't know if fixing #13531 would necessarily fix this issue as well, given the complexity of the code involved.) -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13531#comment:3> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13531: GHC fails with "Dynamic linker not initialised" when -j is on and trying to load nonexistent .so file -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #13137, #9868, | Differential Rev(s): #10355, #13607 | Wiki Page: | -------------------------------------+------------------------------------- Comment (by nh2): For people who google with quotes and don't use British English: dynamic linker not initialized -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13531#comment:4> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13531: GHC fails with "Dynamic linker not initialised" when -j is on and trying to load nonexistent .so file -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #13137, #9868, | Differential Rev(s): Phab:D5012 #10355, #13607, #10919 | Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * status: new => patch * differential: => Phab:D5012 * related: #13137, #9868, #10355, #13607 => #13137, #9868, #10355, #13607, #10919 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13531#comment:5> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13531: GHC fails with "Dynamic linker not initialised" when -j is on and trying to load nonexistent .so file -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: patch Priority: normal | Milestone: Component: Compiler | Version: 8.0.2 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #13137, #9868, | Differential Rev(s): Phab:D5012 #10355, #13607, #10919 | Wiki Page: | -------------------------------------+------------------------------------- Comment (by Ben Gamari <ben@…>): In [changeset:"4fc6524a2a4a0003495a96c8b84783286f65c198/ghc" 4fc6524/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="4fc6524a2a4a0003495a96c8b84783286f65c198" Stop the linker panic If we fail to initialize the liker properly, we still set the `initLinkerDone`. In fact we even set that prior to actually initializing the linker. However if the linker initialization fails, we the `Done` state is still true. As such we run into the `Dynamic Linker not initialised` error. Which while technically corret is confusing as it pulls the attation away from the real issue. This change puts the Done state into an MVar, and as such ensureing that all parallel access needs to wait for the linker to be actually initialized, or try to re-initilize if it fails. Reviewers: bgamari, RyanGlScott, simonmar, hvr Reviewed By: bgamari Subscribers: rwbarton, thomie, carter GHC Trac Issues: #9868, #10355, #13137, #13607, #13531 Differential Revision: https://phabricator.haskell.org/D5012 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13531#comment:6> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#13531: GHC fails with "Dynamic linker not initialised" when -j is on and trying to load nonexistent .so file -------------------------------------+------------------------------------- Reporter: nh2 | Owner: (none) Type: bug | Status: closed Priority: normal | Milestone: 8.8.1 Component: Compiler | Version: 8.0.2 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #13137, #9868, | Differential Rev(s): Phab:D5012 #10355, #13607, #10919 | Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => closed * resolution: => fixed * milestone: => 8.8.1 Comment: Fixed. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/13531#comment:7> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
participants (1)
-
GHC