[GHC] #8266: Dynamic linking on Mac

#8266: Dynamic linking on Mac ------------------------------+-------------------------------------------- Reporter: kazu- | Owner: yamamoto | Status: new Type: bug | Milestone: Priority: highest | Version: 7.7 Component: Build | Operating System: MacOS X System | Type of failure: GHC doesn't work at all Keywords: | Test Case: Architecture: | Blocking: Unknown/Multiple | Difficulty: Unknown | Blocked By: | Related Tickets: | ------------------------------+-------------------------------------------- Many dynamic libraries refer to the build tree. (1) Every ".dylib" installed with GHC head refers to itself in its build directory. E.g. {{{ % otool -L libHSbase-4.7.0.0-ghc7.7.20130909.dylib | grep base libHSbase-4.7.0.0-ghc7.7.20130909.dylib: /Users/kazu/work/ghc/libraries/base/dist- install/build/libHSbase-4.7.0.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) }}} (2) Some ".dylib" installed with GHC head refers to other libraries in their build directories. E.g. {{{ % otool -L libHSvector-0.9.1-ghc7.7.20130909.dylib | grep /Users /Users/kazu/work/ghc/libraries/vector/dist- install/build/libHSvector-0.9.1-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/primitive/dist- install/build/libHSprimitive-0.4.0.1-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) }}} (3) All user installed libraries by cabal-install refer to other libraries in their build directories. E.g. {{{ % otool -L libHSghc-paths-0.1.0.9-ghc7.7.20130909.dylib| grep Users /Users/kazu/Library/Haskell/ghc-7.7.20130909/lib/ghc- paths-0.1.0.9/lib/libHSghc-paths-0.1.0.9-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/base/dist- install/build/libHSbase-4.7.0.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/integer-gmp/dist-install/build /libHSinteger-gmp-0.5.1.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/ghc-prim/dist-install/build /libHSghc-prim-0.3.1.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): This means that GHCi does not work well if we execute "make clean": {{{ % ghci GHCi, version 7.7.20130909: http://www.haskell.org/ghc/ :? for help Loading package ghc-prim ... linking ... done. Loading package integer-gmp ... linking ... done. Loading package base ... linking ... done.
:m System.SetEnv setEnv "TEST" "TEST" Loading package array-0.4.0.2 ... linking ... done. Loading package deepseq-1.3.0.2 ... linking ... done. Loading package bytestring-0.10.3.0 ... linking ... done. Loading package old-locale-1.0.0.5 ... linking ... done. Loading package time-1.4.1 ... linking ... done. Loading package unix-2.7.0.0 ... linking ... done. Loading package setenv-0.1.0 ... can't load .so/.DLL for: /Users/kazu/Library/Haskell/ghc-7.7.20130909/lib/setenv-0.1.0/lib/libHSsetenv-0.1.0-ghc7.7.20130909.dylib (dlopen(/Users/kazu/Library/Haskell/ghc-7.7.20130909/lib/setenv-0.1.0/lib/libHSsetenv-0.1.0-ghc7.7.20130909.dylib, 9): Library not loaded: /Users/kazu/work/ghc/libraries/unix/dist- install/build/libHSunix-2.7.0.0-ghc7.7.20130909.dylib Referenced from: /Users/kazu/Library/Haskell/ghc-7.7.20130909/lib/setenv-0.1.0/lib/libHSsetenv-0.1.0-ghc7.7.20130909.dylib Reason: image not found)
}}} Here is what "setenv" refers to: {{{ % otool -L libHSsetenv-0.1.0-ghc7.7.20130909.dylib | grep /Users /Users/kazu/Library/Haskell/ghc-7.7.20130909/lib/setenv-0.1.0/lib/libHSsetenv-0.1.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/unix/dist- install/build/libHSunix-2.7.0.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/time/dist- install/build/libHStime-1.4.1-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/old-locale/dist-install/build /libHSold-locale-1.0.0.5-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/bytestring/dist- install/build/libHSbytestring-0.10.3.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/deepseq/dist- install/build/libHSdeepseq-1.3.0.2-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/array/dist- install/build/libHSarray-0.4.0.2-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/base/dist- install/build/libHSbase-4.7.0.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/integer-gmp/dist-install/build /libHSinteger-gmp-0.5.1.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) /Users/kazu/work/ghc/libraries/ghc-prim/dist-install/build /libHSghc-prim-0.3.1.0-ghc7.7.20130909.dylib (compatibility version 0.0.0, current version 0.0.0) }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): The provided patch makes all library references relative. Dynamic libraries now refer to @rpath instead of @loader_path for the location of other installed dynlibs. All executables, hence also the ghc binary, set an @rpath link/anchor to: "@loader_path/..", meaning that all package directories must be in the same directory as the parent directory of the executable. If the ghc binary is ever moved from <TOP>/lib/bin, the @rpath link should me updated accordingly. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): Great! I confirmed that this patch fixes the problem. Please merge it. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): In my previous patch, `8226_fix.path`, I forgot to update the identification name of libffi.dynlib to a relative path. The new patch, `8226_proper_fix.patch`, does remove this last remaining reference to the build directory. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): I did not notice your previous comment. Sorry for the delay. I tried 8266_proper_fix.patch and confirmed that it works well. 8266_fix.patch: {{{ % otool -L libffi.dylib libffi.dylib: /Users/kazu/work/ghc/libffi/build/inst/lib/libffi.6.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0) }}} 8266_proper_fix.patch: {{{ % otool -L libffi.dylib libffi.dylib: @rpath/rts-1.0/libffi.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0) }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): I hope this patch will be merged quickly. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): Wait. A program compiled with dynamic linking cannot search its libraries: {{{ % cat test.hs main = putStrLn "Hello" % ghc -fforce-recomp -dynamic test.hs % ./test dyld: Library not loaded: @rpath/base-4.7.0.0/libHSbase-4.7.0.0-ghc7.7.20131001.dylib Referenced from: /Users/kazu/./test Reason: image not found zsh: trace trap ./test }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): GHC should call GCC with "-Wl,-rpath,<path>": {{{ % cat test.hs main = putStrLn "Hello" % ghc -fforce-recomp -dynamic test.hs -optl "-Wl,-rpath,/ghc- head/lib/ghc-7.7.20131001/" % ./test Hello }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): "8226_fix.patch" should be merged. And GHC on Mac should also specify "-Wl,-rpath,<path>" to GCC when the -dynamic option is specified. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by simonpj): * status: new => patch * milestone: => 7.8.1 Comment: I am not following the details of this patch (I'm afraid I'm clueless about dynamic linking), but I'll milestone this for 7.8.1 and change it to `patch`, so that Austin is sure to see it. Austin: I'm not expressing an opinion about whether this is "ready", just wanting to be sure you review it. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

Wait. A program compiled with dynamic linking cannot search its
#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): Replying to [comment:7 kazu-yamamoto]: libraries:
{{{ % cat test.hs main = putStrLn "Hello" % ghc -fforce-recomp -dynamic test.hs % ./test dyld: Library not loaded:
@rpath/base-4.7.0.0/libHSbase-4.7.0.0-ghc7.7.20131001.dylib
Referenced from: /Users/kazu/./test Reason: image not found zsh: trace trap ./test }}} Indeed, my patch does not resolve that problem, as I considered it as something separate from the reported bug. My patch only touches the built/make system, and only removes references to the built tree in the created libraries. I want to note that the problem you describe in your comment also occurs in the current, unpatched, version of ghc.
-- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

GHC should call GCC with "-Wl,-rpath,<path>":
{{{ % cat test.hs main = putStrLn "Hello" % ghc -fforce-recomp -dynamic test.hs -optl "-Wl,-rpath,/ghc-
#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): Replying to [comment:8 kazu-yamamoto]: head/lib/ghc-7.7.20131001/"
% ./test Hello }}} There is the `-fuse-rpaths` flag, perhaps this can be enabled by default on OS X when building an executable? I haven't tested the flag though.
-- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): Thanks. I'm now building GHC with such a hack. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): I've updated my patch so that executables built by ghc now get an rpath link to ghc's `topDir`: https://gist.github.com/christiaanb/6822676 I think the behaviour on OS X and Linux is now almost the same: * Linux: multiple absolute rpath's to all the dependent package directories * OS X: single rpath; relative, rpath-referencing, locations for GHC- installed packages; absolute locations for user-installed packages Perhaps a future version of GHC could have an even more desirable situation on OS X: * two rpaths: user/cabal location and ghc location * All libraries locations are relative, rpath-referencing This would make distribution of a dynamically-linked OS X binary easier, as you would only need to add extra rpath(s) for the location of the dependent packages. This would require an update in Cabal however, as it needs to add the second rpath to the executable. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): Great! https://gist.github.com/christiaanb/6822676 fixes everything! Please merge. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: patch Component: Build System | Milestone: 7.8.1 Resolution: | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by thoughtpolice): * owner: => thoughtpolice -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: patch Component: Build System | Milestone: 7.8.1 Resolution: | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by AndreasVoellmy): * cc: andreas.voellmy@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac
--------------------------------------------+------------------------------
Reporter: kazu-yamamoto | Owner:
Type: bug | thoughtpolice
Priority: highest | Status: patch
Component: Build System | Milestone: 7.8.1
Resolution: | Version: 7.7
Operating System: MacOS X | Keywords:
Type of failure: GHC doesn't work at all | Architecture:
Test Case: | Unknown/Multiple
Blocking: | Difficulty: Unknown
| Blocked By:
| Related Tickets:
--------------------------------------------+------------------------------
Comment (by Austin Seipp

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: patch Component: Build System | Milestone: 7.8.1 Resolution: | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by thoughtpolice): Merged, thanks! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by thoughtpolice): * status: patch => closed * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by amosrobinson): It looks like this is still an issue: {{{ % ghc Main.hs -dynamic % ./Main dyld: Library not loaded: @rpath/base-4.7.0.0/libHSbase-4.7.0.0-ghc7.7.20131125.dylib Referenced from: /Users/amos/ghc/ghc-head/libraries/dph/dph- examples/t/./Main Reason: image not found Trace/BPT trap: 5 }}} I don't have the libraries in `ghc-head/inplace/lib` or `ghc-head/lib` - only in `ghc-head/bindisttest/install\ \ \ dir/lib/`. This is the incantation I needed to get a dynamic to work: {{{ ghc Main.hs -dynamic -optl "-Wl,-rpath,/Users/amos/ghc/ghc- head/bindisttest/install dir/lib/ghc-7.7.20131125/" }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by amosrobinson): I don't know if this helps, but I managed to hack around this for now with something like {{{ cd inplace/lib for lib in ../../bindisttest/install\ \ \ dir/lib/ghc-*/* do if [ -d "$lib" ] then ln -s "$lib" . fi done }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): The current design indeed only works for one, let's call it the //default//, situation; which is: * When GHC is installed in a //normal// directory using e.g. `make install` All other situations, including using the GHC binary from the build-tree, are either not working, or are working by accident. amosrobinson: could you link or include as attachment the output of `otool -l Main`? Of course of the version that doesn't have the extra rpath. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by amosrobinson): Ok, fair enough. It would be nice if we could get it working without having to `make install` - I don't know about other people, but this is my method of having multiple versions of ghc to test against. Here's the output of otool. Not sure if it's of much help: {{{ kallisti:t amos$ otool -L Main Main: @rpath/base-4.7.0.0/libHSbase-4.7.0.0-ghc7.7.20131125.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/integer-gmp-0.5.1.0/libHSinteger- gmp-0.5.1.0-ghc7.7.20131125.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/ghc-prim-0.3.1.0/libHSghc- prim-0.3.1.0-ghc7.7.20131125.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/rts-1.0/libHSrts-ghc7.7.20131125.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/rts-1.0/libffi.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/local/lib/libgmp.10.dylib (compatibility version 12.0.0, current version 12.1.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0) }}} The other thing is, the only reason I started compiling with `-dynamic` was because I was getting errors about missing `.dyn_o`. Something about template haskell? I don't really understand. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): Actually, I wanted the output of `otool -l`, so lower-case 'l', more specifically I wanted to know the value(s) of `LC_RPATH`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:25 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by amosrobinson): Ah, I misread! I've attached it, and the value of `LC_RPATH` is `/Users/amos/ghc/ghc- head/inplace/lib`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:26 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): The `LC_RPATH` value tells me that you're using the GHC binary from `/inplace/`. But how, and why, are you using the GHC binary from `/inplace/`, while using the libraries from `/bindisttest/install dir/`? If you're using the libraries from `/bindisttest/install dir/`, then why wouldn't you be using the GHC binary from `/bindisttest/install dir/`? I feel that this ticket should stay closed, as the original problem (references to the build directory) is fixed; //normal// users of GHC are no longer affected. However, I agree that the current design could be improved for //advanced// users and GHC developers. So perhaps you can file a feature request that specifically states your needs/use-case that is not met by the current design? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:27 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by amosrobinson): I'm using the binary from `inplace/bin/` because that's how I've always done it - I thought that //was// the //normal// way. The only reason I was using libraries from `bindisttest/install dir/lib/` is because that's how I managed to get it to work. OK! So, if I set my path to `bindisttest/install dir/bin/` instead of `inplace/bin/` it works without the dodgy symlinks. That's good enough for now. I feel that this is a bug: it was my understanding `inplace` is supposed to work the same as an installed copy, but I'll file a separate ticket. Thanks! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:28 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by thoughtpolice): Not being able to use the built inplace compiler as if it was a regular compiler is absolutely, 100% a bug. This is a crucial way to test GHC builds IMO without throwing installations everywhere on your `$PATH`, and is particularly useful for stress testing packages, since you can `cabal install -w ~/ghc-src/inplace/bin/ghc-stage2` to have it act normally. This also works on all other platforms. I missed this when I merged that patch. I'll investigate it ASAP. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:29 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): I currently don't have my OS X machine, so I can't make the patch right now. But the patch should be "straightforward": just make OS X (MACH-O) behave the same way as Linux (ELF). I guess this whole mess started in the first place because MACH-O only has absolute paths to dynamic libraries. Currently on OS X there's a single point of reference, an `@rpath`, that's pointing to a `lib/` directory, and expecting all libraries bundled with GHC to be there. This should be changed to multiple `@rpath`s. So the current situation looks like this: {{{ -- Installed by GHC install @rpath/package1-package1version/package1.dylib @rpath/package2-package1version/package2.dylib @rpath/package3-package1version/package3.dylib -- Installed by Cabal /absolute_path_to_library/package4.dylib /absolute_path_to_library/package5.dylib }}} Where a executable gets an `@rpath` pointing to the same location as the value of `ghc --print-libdir`. With the exception of the GHC binary, which has an `@rpath` pointing to `@loader_path/..`. The new situation should be: {{{ -- Installed by GHC install @rpath/package1.dylib @rpath/package2.dylib @rpath/package3.dylib -- Installed by Cabal @rpath/package4.dylib @rpath/package5.dylib }}} Where: * Libraries bundled with GHC should get, for all transitively depending libraries, an `@rpath` pointing to `@loader_path/../dependentpackage/` (similar to the situation on Linux, which uses `$ORIGIN/../dependentpackage/`) * The GHC executable should get, for all transitively depending libraries, an `@rpath` pointing to `@loader_path/../dependentpackage/` (again similar to Linux) * Libraries installed by Cabal should get, for all transitively depending library, an `@rpath` pointing to `/absolute_path_to_package` (similar to linux) * Executables should get, for all transitively depending libraries, an `@rpath` pointing to `/absolute_path_to_package` (similar to linux) This patch would involve adding a code-path for OS-X/MACH-O similar to lines 1757-1776 in `compiler/main/DriverPipeline.hs`, and of course changing `rules/relative-dynlib-references.mk`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:30 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): The latest patch makes Mach-O (currently just Darwin, not iOS) behave the same as ELF in terms of dynamic linking. That is, all dynamic library locations are now relative to `@rpath`, and GHC adds `LC_RPATH` commands for every directory containing the dependent libraries. This enables the use-case of using `cabal -w inplace/bin/ghc-stage2`. Cabal will however need an update as well, as libraries installed by Cabal will still use absolute paths: {{{ -- Installed by GHC install @rpath/package1.dylib @rpath/package2.dylib @rpath/package3.dylib -- Installed by Cabal /absolute_path_to_library/package4.dylib /absolute_path_to_library/package5.dylib }}} By applying the following patch to Cabal: {{{ diff --git a/Cabal/Distribution/Simple/GHC.hs b/Cabal/Distribution/Simple/GHC.hs index c7ea633..78cdcbb 100644 --- a/Cabal/Distribution/Simple/GHC.hs +++ b/Cabal/Distribution/Simple/GHC.hs @@ -867,11 +867,6 @@ buildOrReplLib forRepl verbosity pkg_descr lbi lib clbi = do ghcOptDynLinkMode = toFlag GhcDynamicOnly, ghcOptInputFiles = dynamicObjectFiles, ghcOptOutputFile = toFlag sharedLibFilePath, - -- For dynamic libs, Mac OS/X needs to know the install location - -- at build time. - ghcOptDylibName = if buildOS == OSX - then toFlag sharedLibInstallPath - else mempty, ghcOptPackageName = toFlag pkgid, ghcOptNoAutoLinkPackages = toFlag True, ghcOptPackageDBs = withPackageDB lbi, }}} The situation will change to: {{{ -- Installed by GHC install @rpath/package1.dylib @rpath/package2.dylib @rpath/package3.dylib -- Installed by Cabal @rpath/package4.dylib @rpath/package5.dylib }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:31 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by thoughtpolice): * cc: dcoutts (added) Comment: This isn't a blocker for the release, but Duncan should be looped in here if a cabal patch is needed. Duncan - any thoughts on this going upstream? Not immediate but input appreciated. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:32 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | thoughtpolice Priority: highest | Status: closed Component: Build System | Milestone: 7.8.1 Resolution: fixed | Version: 7.7 Operating System: MacOS X | Keywords: Type of failure: GHC doesn't work at all | Architecture: Test Case: | Unknown/Multiple Blocking: | Difficulty: Unknown | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by thoughtpolice): * cc: dcoutts (removed) * cc: duncan (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:33 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by thoughtpolice): * owner: thoughtpolice => * status: closed => new * resolution: fixed => -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:34 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by thoughtpolice): * status: new => patch -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:35 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): I confirmed that @rpath is not used at this moment. Here is an example: {{{ % otool -L libHSsyb-0.4.1-ghc7.7.20140108.dylib libHSsyb-0.4.1-ghc7.7.20140108.dylib: /Users/kazu/work/ghc-mod/.cabal-sandbox/lib/x86_64-osx- ghc-7.7.20140108/syb-0.4.1/libHSsyb-0.4.1-ghc7.7.20140108.dylib (compatibility version 0.0.0, current version 0.0.0) }}} But note that this works without any problems. I applied darchon's patch, fully built GHC, and re-install cabal-install with the new GHC and Cabal. Unfortunately, this new cabal-install does now work well. For instance, syb becames: {{{ % otool -L libHSsyb-0.4.1-ghc7.7.20140108.dylib libHSsyb-0.4.1-ghc7.7.20140108.dylib: /private/var/folders/k0/548g5xg90jjfbrj5j09nvwv80000gq/T/syb-0.4.1-34816/syb-0.4.1/dist /dist-sandbox-a0e7ff68/build/libHSsyb-0.4.1-ghc7.7.20140108.dylib (compatibility version 0.0.0, current version 0.0.0) }}} So, libraries depending on syb cannot be installed. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:36 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): I will have a better look at Cabal... I really thought removing those lines would make sure that Cabal doesn't pass along an `install_name` to GHC; hence GHC would default to choosing an `@rpath`-based `install_name`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:37 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): The patches for GHC and Cabal work fine on my machine: {{{ % otool -L ~/.cabal/lib/x86_64-osx- ghc-7.7.20140103/parsec-3.1.5/libHSparsec-3.1.5-ghc7.7.20140103.dylib /Users/christiaan/.cabal/lib/x86_64-osx- ghc-7.7.20140103/parsec-3.1.5/libHSparsec-3.1.5-ghc7.7.20140103.dylib: @rpath/libHSparsec-3.1.5-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libHStext-1.1.0.0-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libHSmtl-2.1.2-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libHStransformers-0.3.0.0-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libHSbytestring-0.10.4.0-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libHSdeepseq-1.3.0.2-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libHSarray-0.5.0.0-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libHSbase-4.7.0.0-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libHSinteger-gmp-0.5.1.0-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) @rpath/libHSghc-prim-0.3.1.0-ghc7.7.20140103.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/local/lib/libgmp.10.dylib (compatibility version 12.0.0, current version 12.1.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0) }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:38 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): @darchon Can you guess which instruction of mine is wrong? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:39 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): It seems as if you _only_ applied the Cabal patch. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:40 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): I made another try. I applied your patch to Cabal-1.18.1.1 and built cabal-install 1.18.0.2 with GHC 7.6.3. I used the cabal-install again with GHC head but the result is the same. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:41 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by duncan): I have to say this makes me very nervous. Cabal does not currently support relocatable packages at all, and a little patch like this is not enough to make it do so. There were at the time good reasons to specify the final install location and so we will need to see if those reasons are no longer valid (e.g. OSX linker features have improved and no longer need to support older OSX releases). Am I correct in thinking that this only affects people using ghc inplace in the build tree? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:42 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): I guess I should have articulated better that the patch for Cabal is _not_ needed for everything to work. The reason for the Cabal patch is simply: * Uniformity between `install_name`s of dynamic libraries supplied by GHC and those installed with Cabal. The new GHC patch adds an `LC_RPATH` command for every dependent library, pointing to the directory that the dependent library is installed in. So if a library is linked against, e.g., `@rpath/libHSbase-4.7.0.0-ghc7.7.20140103.dylib`, it gets an `LC_RPATH` command that points to e.g. `/usr/local/ghc/lib/base-4.7.0.0/`. Now, _without_ the patch to Cabal, a library like `text` would get an `install_name` of `/Users/darchon/.cabal/lib/x86_64-osx- ghc-7.7.20140101/text-1.1.0.0/libHStext-1.1.0.0-ghc7.7.20140103.dylib`. Under the new patch, a library like `parsec` would be linked against `/Users/darchon/.cabal/lib/x86_64-osx- ghc-7.7.20140101/text-1.1.0.0/libHStext-1.1.0.0-ghc7.7.20140103.dylib`, but GHC would still add an `LC_RPATH` pointing to: `/Users/darchon/.cabal/lib/x86_64-osx-ghc-7.7.20140101/text-1.1.0.0`. Of course this `LC_RPATH` command is superfluous/bogus, as there isn't be any `@rpath` referenced dynamic library to be found there. I believe these superfluous `LC_PATH`s to be harmless though. Note that neither the GHC patch nor the Cabal patch intended to make packages/libraries relocatable. It essentially traded one direct/full path for another. In the pre-patch situation the `install_name` is a direct path, in the post-patch situation a library has an `LC_RPATH` that is a direct/full path. So, to sum up: * The new GHC patch solves the original issue of having the GHC build-dir referenced in built/distributed libraries, while also supporting the use- case of building/installing packages using `cabal-install` and the `inplace/bin/ghc-stage2`. * The Cabal patch only add uniformity between `install_names`, but is not necessary for a correctly working GHC/Cabal system. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:43 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by carter): so am I correct in thinking that this issue only matters when someone is building ghc from source and using it without installing it? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:44 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by darchon): Could you elaborate 'this issue'? Do you mean, does the current situation (with my previous patch, f213e48447050bf468bc4d91fc4d810402c23b85/ghc, already committed) result in a non-working GHC install for 'normal' users? The answer to that would be _no_, the current situation is just fine for the largest part of the user- base of GHC. If you mean, that you cannot properly use `inplace/bin/ghc-stage2` in combination with `cabal-install`, which matters to both GHC developers/hackers and those wanting bleeding-edge GHC? Then the answer would be _yes_, the way dynamic libraries are set up with my previous patch do not work well with `cabal -w inplace/bin/ghc-stage2`. My _new_ patch, attached as `8266_MachO_same_as_ELF.patch`, is just there to solve the latter issue. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:45 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): Ah! I did not notice 8266_MachO_same_as_ELF.patch. I applied it as well as Cabal's patch and started building GHC head again. It take a long time. Please wait. If Cabal's patch should not be applied, please interupt me. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:46 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): OK. I verified that if two patches are applied, "cabal install" works as expected. That is, @rpath is properly used. One patch is not good enough. @darchon sorry for wasting your time. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:47 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by edsko): * cc: edsko@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:48 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): @austin Would you merge the two patches? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:49 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac
--------------------------------------------+------------------------------
Reporter: kazu-yamamoto | Owner:
Type: bug | Status: patch
Priority: highest | Milestone: 7.8.1
Component: Build System | Version: 7.7
Resolution: | Keywords:
Operating System: MacOS X | Architecture:
Type of failure: GHC doesn't work at all | Unknown/Multiple
Test Case: | Difficulty: Unknown
Blocking: | Blocked By:
| Related Tickets:
--------------------------------------------+------------------------------
Comment (by Austin Seipp

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by thoughtpolice): Thank you Christiaan. The GHC patch is merged and it seems to work well. I have not merged the Cabal patch, as that's more under the authority of Duncan. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:51 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by kazu-yamamoto): Duncan, would you merge the patch ASAP? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:52 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by duncan): Since the Cabal patch is apparently not necessary, and that I don't yet understand it, then we'll hold off. I do not yet understand the OSX @rpath stuff well enough to see what the consequences would be. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:53 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by duncan): Replying to [comment:53 duncan]:
Since the Cabal patch is apparently not necessary, and that I don't yet understand it, then we'll hold off. I do not yet understand the OSX @rpath stuff well enough to see what the consequences would be.
I've changed my mind (now that I think I understand it). See https://github.com/haskell/cabal/issues/1660#issuecomment-33701508 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:54 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac
--------------------------------------------+------------------------------
Reporter: kazu-yamamoto | Owner:
Type: bug | Status: patch
Priority: highest | Milestone: 7.8.1
Component: Build System | Version: 7.7
Resolution: | Keywords:
Operating System: MacOS X | Architecture:
Type of failure: GHC doesn't work at all | Unknown/Multiple
Test Case: | Difficulty: Unknown
Blocking: | Blocked By:
| Related Tickets:
--------------------------------------------+------------------------------
Comment (by Herbert Valerio Riedel

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: merge Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by hvr): * status: patch => merge -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:56 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: closed Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: fixed | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Changes (by thoughtpolice): * status: merge => closed * resolution: => fixed Comment: Merged. We'll need to update Cabal submodule in HEAD and 7.8 when we do the actual release. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:57 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

Merged. We'll need to update Cabal submodule in HEAD and 7.8 when we do
#8266: Dynamic linking on Mac --------------------------------------------+------------------------------ Reporter: kazu-yamamoto | Owner: Type: bug | Status: closed Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: fixed | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work at all | Unknown/Multiple Test Case: | Difficulty: Unknown Blocking: | Blocked By: | Related Tickets: --------------------------------------------+------------------------------ Comment (by hvr): Replying to [comment:57 thoughtpolice]: the actual release. For the record; there's nothing left to do for this ticket in particular, as `ghc-7.8` already contains the fix as of 37d6e2c54f117f0a65f3032b6f30b6878b3f6f58 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:58 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8266: Dynamic linking on Mac -------------------------------------+------------------------------------- Reporter: kazu-yamamoto | Owner: Type: bug | Status: closed Priority: highest | Milestone: 7.8.1 Component: Build System | Version: 7.7 Resolution: fixed | Keywords: Operating System: MacOS X | Architecture: Type of failure: GHC doesn't work | Unknown/Multiple at all | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by PHO): * cc: PHO (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8266#comment:59 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC