#10322: In ghci object code loader, linking against the previous temp dylib is not enough on OS X -------------------------------------+------------------------------------- Reporter: rwbarton | Owner: Type: bug | Status: new Priority: high | Milestone: 7.10.2 Component: Compiler | Version: 7.10.1 Keywords: | Operating System: MacOS X Architecture: | Type of failure: None/Unknown Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- joelteon encountered this issue in a rather complicated program, and we worked out the cause over IRC. Suppose that ghci needs to sequentially load three modules A, B and C, where B refers to symbols from A and C refers to symbols from both A and B. (For example modules B, C and another module D all contain Template Haskell that refers to the previous module(s).) The object code linker currently works like this: - Link the module `A.o` into a dylib `ghc_1.dylib` - Link the module `B.o` against `ghc_1.dylib` into a new dylib `ghc_2.dylib` - Link the module `C.o` against `ghc_2.dylib` into a new dylib `ghc_3.dylib` As a result, `ghc_2.dylib` ends up with a NEEDED (or whatever it's called in Mach-O) entry for `ghc_1.dylib`, and `ghc_3.dylib` ends up with a NEEDED entry for `ghc_2.dylib`. However, this apparently does not satisfy the OS X `dlopen` implementation, which complains about a missing symbol `_A_foo` referenced by `ghc_3.dylib` and which is defined in `ghc_1.dylib`. Apparently the dynamic loader only checks the direct dependencies when trying to resolve undefined symbols. (The Linux dynamic loader seems to be perfectly happy to use an indirect dependency to resolve an undefined symbol. But I found out that the linker [https://sourceware.org/bugzilla/show_bug.cgi?id=10238 gold] has the same sort of behavior as the OS X dynamic loader. I don't know whether there is any standard here, but it seems that we cannot rely on the Linux dynamic loader's behavior.) Presumably the fix is to keep track of all the previous temporary dylibs (rather than just one in `last_temp_so`) and link against all of them when building a new temporary dylib. I'm slightly worried about quadratic behavior here, but I think it's unlikely to be an issue in practice. I have a reproducer at http://lpaste.net/808723564239781888 (which I'll add to the test suite next) and oherrala ran the test on an OS X system with the following results: {{{ =====> Last(ghci) 1167 of 4480 [0, 0, 0] cd ./ghci/scripts && HC="/Users/oherrala/gits/ghc/inplace/bin/ghc-stage2" HC_OPTS="-dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci-history " "/Users/oherrala/gits/ghc/inplace/bin/ghc-stage2" -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci- history --interactive -v0 -ignore-dot-ghci +RTS -I0.1 -RTS <Last.script > Last.run.stdout 2> Last.run.stderr Actual stderr output differs from expected: --- /dev/null 2015-04-18 17:23:26.000000000 +0300 +++ ./ghci/scripts/Last.run.stderr 2015-04-18 17:23:34.000000000 +0300 @@ -0,0 +1,9 @@ +ghc-stage2: panic! (the 'impossible' happened) + (GHC version 7.11.20150418 for x86_64-apple-darwin): + Loading temp shared object failed: dlopen(/var/folders/64/90jfy8lj65bcm1k02syxz_l80000gn/T/ghc18812_0/libghc18812_12.dylib, 5): Symbol not found: _LastA_a_closure + Referenced from: /var/folders/64/90jfy8lj65bcm1k02syxz_l80000gn/T/ghc18812_0/libghc18812_12.dylib + Expected in: flat namespace + in /var/folders/64/90jfy8lj65bcm1k02syxz_l80000gn/T/ghc18812_0/libghc18812_12.dylib + +Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug + Actual stdout output differs from expected: --- ./ghci/scripts/Last.stdout 2015-04-18 16:26:55.000000000 +0300 +++ ./ghci/scripts/Last.run.stdout 2015-04-18 17:23:34.000000000 +0300 @@ -1,3 +1,2 @@ 3 4 -7 *** unexpected failure for Last(ghci) }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10322> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler