[GHC] #9498: GHC links against unversioned .so files
#9498: GHC links against unversioned .so files ------------------------------------+-------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: low | Milestone: Component: Compiler (FFI) | Version: 7.6.3 Keywords: Debian | Operating System: Linux Architecture: Unknown/Multiple | Type of failure: Other Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: ------------------------------------+-------------------------------- Greetings, GHC tries to load unversioned dynamic libraries instead of versioned (i.e. libfoo.so instead of libfoo.so.1.2.3). This is a problem, since Distributions like Debian (I don't know about other distributions) don't include unversioned .SOs in their runtime packages and the unversioned are only available in the -dev packages as a symlink to the verioned ones. This means FFI libraries depend on the -dev packages, even though they don't really need them. It would be nice if GHC would try to load the versioned libraries as well. Regards Sven -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature | Status: new request | Milestone: Priority: low | Version: 7.6.3 Component: Compiler | Keywords: Debian (FFI) | Architecture: Unknown/Multiple Resolution: | Difficulty: Unknown Operating System: Linux | Blocked By: Type of failure: Other | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by thomie): Kritzefitz: thank you for the report. I am adding an example here for later reference. Please let me me know if there are any mistakes in the text below, I just learned this stuff tonight. On Ubuntu 14.04, `libbsd.so` (unversioned) is a symbolic link to the file `libbsd.so.0.6.0` (versioned). That symbolic link is created by the `libbsd-dev` (dev) package, while the library file is installed by the `libbsd0` (runtime) package. {{{ $ ll /usr/lib/x86_64-linux-gnu/libbsd.so lrwxrwxrwx 1 root root 37 Mar 20 2014 /usr/lib/x86_64-linux-gnu/libbsd.so -> /lib/x86_64-linux-gnu/libbsd.so.0.6.0 $ dpkg -S /usr/lib/x86_64-linux-gnu/libbsd.so libbsd-dev: /usr/lib/x86_64-linux-gnu/libbsd.so $ dpkg -S /lib/x86_64-linux-gnu/libbsd.so.0.6.0 libbsd0:amd64: /lib/x86_64-linux-gnu/libbsd.so.0.6.0 }}} This is a small program (called `testbsd.hs`) that calls a function in the libbsd library via the foreign function interface: {{{#!haskell module Main where import Foreign.C.Types foreign import ccall "arc4random" version :: CUInt main = print version }}} GHC links this program to `libbsd.so.0`, not `libbsd.so.0.6.0`, so the dev package will be needed to run it: {{{ $ ghc --make -lbsd testbsd.hs [1 of 1] Compiling Main ( testbsd.hs, testbsd.o ) Linking testbsd ... $ ldd testbsd ... libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007faef5c2c000) ... $ ./testbsd 1131902945 }}} Note that some runtime packages do create the link from the unversioned to the versioned library themselves. This seems to be in particular true for all libraries in the `/lib/` directory. For example for `libpng`: {{{ $ ll /lib/x86_64-linux-gnu/libpng12.so.0 lrwxrwxrwx 1 root root 18 Apr 1 2014 /lib/x86_64-linux-gnu/libpng12.so.0 -> libpng12.so.0.50.0 $ dpkg -S /lib/x86_64-linux-gnu/libpng12.so.0.50.0 libpng12-0:amd64: /lib/x86_64-linux-gnu/libpng12.so.0.50.0 $ dpkg -S /lib/x86_64-linux-gnu/libpng12.so.0 libpng12-0:amd64: /lib/x86_64-linux-gnu/libpng12.so.0 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature | Status: new request | Milestone: Priority: low | Version: 7.6.3 Component: Compiler | Keywords: Debian (FFI) | Architecture: Unknown/Multiple Resolution: | Difficulty: Unknown Operating System: Linux | Blocked By: Type of failure: Other | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by thomie): Kritzefitz: thank you once again for the report. My first version of the above text was wrong. This is the corrected version. I don't understand the problem you're proposal is trying to solve. Could you please give an example of the situation on your system? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature | Status: new request | Milestone: Priority: low | Version: 7.6.3 Component: Compiler | Keywords: Debian (FFI) | Architecture: Unknown/Multiple Resolution: | Difficulty: Unknown Operating System: Linux | Blocked By: Type of failure: Other | Related Tickets: Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by Kritzefitz): I will demonstrate with the curl package: Assuming I have installed the libcurl3 package but not the libcurl-dev package. This means I have the following shared libraries belonging to curl installed: /usr/lib/i386-linux-gnu/libcurl-gnutls.so.3 /usr/lib/i386-linux-gnu/libcurl.so.3 (I actually have some curl version 4 libraries installed, but those shouldn't matter now.) Then I start ghci and try to use curl: {{{ $ ghci GHCi, version 7.6.3: http://www.haskell.org/ghc/ :? for help Loading package ghc-prim ... linking ... done. Loading package integer-gmp ... linking ... done. Loading package base ... linking ... done. Prelude> import Network.Curl Prelude Network.Curl> curlGet "http://weltraumschlangen.de/" [] Loading package array-0.4.0.1 ... linking ... done. Loading package deepseq-1.3.0.1 ... linking ... done. Loading package bytestring-0.10.0.2 ... linking ... done. Loading package containers-0.5.0.0 ... linking ... done. Loading package curl-1.3.8 ... can't load .so/.DLL for: libcurl.so (libcurl.so: cannot open shared object file: No such file or directory) Prelude Network.Curl> }}} So it seems it tries to load libcurl.so, which is contained in libcurl- dev, instead of libcurl.so.3. It's not working when compiling a file either: test.hs: {{{ import Network.Curl main = curlGet "http://weltraumschlangen.de/" [] }}} compiling: {{{ $ ghc --make test.hs [1 of 1] Compiling Main ( test.hs, test.o ) Linking test ... /usr/bin/ld: cannot find -lcurl collect2: error: ld returned 1 exit status }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: infoneeded Priority: low | Milestone: Component: Compiler (FFI) | Version: 7.6.3 Resolution: | Keywords: Debian Operating System: Linux | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by thomie): * status: new => infoneeded Comment: Kritzefitz: thank you for your response. I don't think there isn't anything for GHC to do here, and I'm inclined to close as invalid. What is your opinion? Here's my understanding of what's happening. I'm keeping this linux only (`sudo apt-get install libcurl4-openssl-dev` and `cabal install curl` first): * The .cabal file of the curl package specifies `extra-libraries: curl` * After cabal installing curl, this stanza ends up in curl's ghc package file (check with `ghc-pkg describe curl | grep extra-libraries`) * When you make test.hs, ghc figures out that curl is required. It parses curl's ghc package file, and extracts the extra-libraries field. When it's time for linking, it then passes `-lcurl` to the linker. * Since the linker wasn't told which version of curl it should find, it searches for a file `libcurl.so` in any of the system lib directories. On Debian/Ubuntu, as you mentioned, this unversioned file (symbolic link) is only present after installing a curl `-dev` package (1). * The linker then follows the symbolic link, and produces a binary that //is// linked to a versioned .so file (`libcurl.so.4` in my case). This is good, because that means that users of the binary will not need the `-dev` package for curl to be installed, but just the runtime package will do. The linker can be also be told to look for a specific file, by using `-l:filename` instead of `-lname` (see `man ld` and search for `--library=`). This `filename` is necessarily platform specific (e.g. dll on Windows), but cabal and ghc support this format in principal (though there seems to be a bug when doing this via ghci). Here's what I've tried: * cabal get curl && cd curl-1.3.8 * # edit curl.cabal and change `extra-libraries: curl` to `extra- libraries: libcurl.so.4` * cabal install * sudo apt-get remove libcurl4-openssl-dev * # verify that libcurl.so is no longer present * # change to directory with your test.hs * ghc test.hs It works! Responding to your issue, either: 1. Debian's should change its policy (1) 2. or the curl library should be updated 3. or users of the curl library should just install the curl -dev package (they are developers after all, not end users). (1) https://www.debian.org/doc/debian-policy/ch-sharedlibs.html#s -sharedlibs-dev -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: low | Milestone: Component: Compiler (FFI) | Version: 7.6.3 Resolution: | Keywords: Debian Operating System: Linux | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by nomeata): * status: infoneeded => new Comment: It is right that the cabal file specifies only `curl` in the cabal file, after all, no specific version is required here. But after GHC has produced a `.so` file, this `.so` file, as you say, is tied to a specific version of the `curl` library. So the correct thing to do would be to store the precisely used version in the package data base. This way, building the curl bindings requires the `libcurl-dev` package, but using the curl bindings, e.g. when building a Haskell executable or from ghci, would only require the specific runtime package. This would actually make packaging Haskell for Debian a bit simpler... -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: low | Milestone: Component: Compiler (FFI) | Version: 7.6.3 Resolution: | Keywords: Debian Operating System: Linux | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by nomeata): (But maybe this means that this is actually a Cabal bug?) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: low | Milestone: Component: Compiler (FFI) | Version: 7.6.3 Resolution: | Keywords: Debian Operating System: Linux | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by thomie): You're right! curl's cabal file would still contain `extra-libraries: curl`. But when installing curl, whoever decides the parameters for the ghc-pkg file (either cabal or ghc, I don't know at the moment) could ask the linker which version it actually found, and write `extra-libraries: :libcurl.so.<version>` to the ghc-pkg file. That should work (at least on linux). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
You're right!
curl's cabal file would still contain `extra-libraries: curl`. But when installing curl, whoever decides the parameters for the ghc-pkg file (either cabal or ghc, I don't know at the moment) could ask the linker which version it actually found, and write `extra-libraries: :libcurl.so.<version>` to the ghc-pkg file. I think it is sufficient for GHCi to link against `libHScurl<something>.so` (the Haskell library) and omit `-lcurl` (the C
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: trommler Type: feature request | Status: new Priority: low | Milestone: Component: Compiler (FFI) | Version: 7.6.3 Resolution: | Keywords: Debian Operating System: Linux | Architecture: Type of failure: Other | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by trommler): * owner: => trommler Comment: Replying to [comment:7 thomie]: library) from the `ld` command altogether. This would make the linking of dynamic libraries way more efficient as fewer shared libraries need to be opened and fewer symbol tables need to be read. See p 41 of Ulrich Drepper's "How To Write Shared Libraries" http://www.akkadia.org/drepper/dsohowto.pdf. I am going to look into this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: trommler Type: feature request | Status: new Priority: low | Milestone: Component: Compiler (FFI) | Version: 7.6.3 Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): That means we need to record the fact that `-lcurl` is necessary when linking against the static Haskell library but not when linking against the dynamic Haskell library, right? Or can we assume that this is always the case? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
That means we need to record the fact that `-lcurl` is necessary when
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: low | Milestone: Component: Compiler (FFI) | Version: 7.6.3 Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by trommler): * owner: trommler => Comment: Replying to [comment:9 rwbarton]: linking against the static Haskell library but not when linking against the dynamic Haskell library, right? You are right, the dynamic library is linked against the versioned C library and no extra `-lcurl` is required. At build time we need the development package for libcurl so the link editor (ld) can link the dynamic Haskell library. To use the dynamic Haskell library the unversioned library is not needed anymore. To load a static library we use the RTS linker and the RTS linker needs to load dependent C libraries explicitly. Unfortunately we cannot assume that the C library with a specific version is installed. So the RTS linker looks for the unversioned library and tries to load that. So yes, the unversioned C library is currently necessary to load a static Haskell library but not needed to load a dynamic Haskell library. Perhaps we could make cabal record the version information for C libraries in the package database as suggested in comment:8 and change the RTS linker to use that information? I don't have time to work on this now, so I am disowning the ticket. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
To load a static library we use the RTS linker and the RTS linker needs to load dependent C libraries explicitly. Unfortunately we cannot assume
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: low | Milestone: Component: Compiler (FFI) | Version: 7.6.3 Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by nomeata): that the C library with a specific version is installed. So the RTS linker looks for the unversioned library and tries to load that. BTW, why is that? Is it a valid and supported use case to build against one version and run against another version of the library? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: low | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by ezyang): * component: Compiler (FFI) => Compiler (Linking) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by rwbarton): * priority: low => normal -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by trommler): * cc: trommler (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
To load a static library we use the RTS linker and the RTS linker needs to load dependent C libraries explicitly. Unfortunately we cannot assume that the C library with a specific version is installed. So the RTS
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by trommler): Replying to [comment:11 nomeata]: linker looks for the unversioned library and tries to load that.
BTW, why is that? Is it a valid and supported use case to build against
one version and run against another version of the library? You don't have to recompile for say a security fix in a shared library. The official GHC binaries are build against a certain version of the standard C library (libc) but generally work with a newer libc as long as it is binary compatible. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by trommler): * cc: simonmar (added) Comment: How about this for a fix in the static case: 1. Prepare a dummy shared library `libHSC<package>.so` at build time where we use `ld` to deal with finding the right shared libraries. If the package depends on more than one C library still only one dummy shared library needs to be created. 2. Install the dummy shared library with the static library `libHS<package>.a` 3. Teach RTS linker to load the dummy shared library to satisfy C dependencies. I can look into this after #10458. CC'ing @simonmar -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:16 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 10458 | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by trommler): * blockedby: => 10458 Comment: The dynamic case will be fixed by #10458. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 10458 | Blocking: 9237 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonmar): All this scares me a lot. Why does the package database need to record the versioned .so dependency? It is not required, either for linking against the dynamic library (because it is already recorded as a dependency in the .so) or against the static library (because we use -lfoo). We require the `-dev` versions of libraries for GHCi very deliberately, because GHCi is supposed to mimic the linking that would be done for a standalone executable. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
We require the `-dev` versions of libraries for GHCi very deliberately, because GHCi is supposed to mimic the linking that would be done for a standalone executable. OK. So the in the dynamic case GHCi SHOULD NOT (MUST NOT?) require the
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 10458 | Blocking: 9237 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by trommler): Replying to [comment:19 simonmar]: presence of the development versions of C libraries when using an installed Haskell package. This is currently not the case. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:20 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
OK. So the in the dynamic case GHCi SHOULD NOT (MUST NOT?) require the
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 10458 | Blocking: 9237 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonmar): presence of the development versions of C libraries when using an installed Haskell package. This is currently not the case. I think we could make it so that GHCi, if using dynamic libraries, doesn't require the `-dev` version of C library dependencies, but what's the gain from doing that? GHC would still require it when not using `-dynamic`, so you wouldn't be able to omit the dependency on `-dev` in the distro packages, for example. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
OK. So the in the dynamic case GHCi SHOULD NOT (MUST NOT?) require the
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 10458 | Blocking: 9237 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by trommler): Replying to [comment:21 simonmar]: presence of the development versions of C libraries when using an installed Haskell package. This is currently not the case.
I think we could make it so that GHCi, if using dynamic libraries,
doesn't require the `-dev` version of C library dependencies, but what's the gain from doing that? GHC would still require it when not using `-dynamic`, so you wouldn't be able to omit the dependency on `-dev` in the distro packages, for example. No, if we implemented my proposal in comment:16 we would not need the dependency on C library `-dev` packages anymore. The `-dev` package would only be a build requirement. This makes a difference on openSUSE's build service where we build each package in a separate virtual machine and install only packages that are actually needed for the current build. So when we __use__ the such a package in another build we would not have to install the C `-dev` packages and some of those can be fairly large. Moreover, we could get rid of the code chasing C libraries in `Linker.hs` and remove the code that parses linker scripts in `rts/Linker.c`. #9237 fails because the parsing of linker scripts is incomplete. #10046 might be an issue with linker scripts, too. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:22 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 10458 | Blocking: 9237 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonmar):
No, if we implemented my proposal in comment:16 we would not need the dependency on C library -dev packages anymore. The -dev package would only be a build requirement.
I think maybe we're talking about different things, let me try to clarify. * If you install a Haskell package to be used with GHC, you still need the `-dev` version of any C library dependencies. That's independent of your proposal in comment:16, because we need the -dev libraries when building standalone executables. * If you install a package because it is a runtime dependency of an executable or another library, then you don't need the -dev version of C dependencies. So you could, if you wanted, split compiled Haskell packages into dev and non-dev distributions. * The case you seem to be referring to is somewhere between these two, where you're only using a package in GHCi, or only via the GHC API. In those cases you could avoid needing the `-dev` dependency, but I'm not sure how you would distinguish this kind of dependency. How can you tell that the user isn't going to try to use GHC? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 10458 | Blocking: 9237 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by trommler): Replying to [comment:23 simonmar]:
I think maybe we're talking about different things, let me try to
clarify.
* If you install a Haskell package to be used with GHC, you still need
the `-dev` version of any C library dependencies. That's independent of your proposal in comment:16, because we need the -dev libraries when building standalone executables. Is the C library going to be a static (archive) or da dynamic (shared object) library? If it is the latter then I think it would still work with comment:16 and if we pass a linker flag (something along the lines of copy all DT_NEEDED tags for C libraries from the dummy SO into the executable) the performance impact would be small. But in the completely static case you are right, the dev dependency is always required. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:24 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 10458 | Blocking: 9237 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by simonmar):
Is the C library going to be a static (archive) or da dynamic (shared object) library?
If it is the latter then I think it would still work with comment:16 and if we pass a linker flag (something along the lines of copy all DT_NEEDED tags for C libraries from the dummy SO into the executable) the
Usually both, so that you can use `-optl-static` if you want. The `-dev` package of a C library normally includes both the unversioned `.so` and the `.a`. Remember that GHC links static Haskell packages but dynamic C libraries by default. performance impact would be small. Did you know the code in `Linker.lhs` is only used for GHCi and not linking of standalone executables? I have a suspicion that there's some confusion here. There is no "dummy SO" when linking a standalone executable. Since you have a design in mind, maybe it would be good to flesh it out in a wiki page so we can discuss with more precision? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:25 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 10458 | Blocking: 9237 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by trommler): Replying to [comment:25 simonmar]:
Is the C library going to be a static (archive) or a dynamic (shared object) library?
Usually both, so that you can use `-optl-static` if you want. The `-dev` package of a C library normally includes both the unversioned `.so` and the `.a`.
Remember that GHC links static Haskell packages but dynamic C libraries by default. Static Haskell with dynamic C is the issue that I want to address in comment:16
Did you know the code in `Linker.lhs` is only used for GHCi and not
[...] linking of standalone executables? I have a suspicion that there's some confusion here. There is no "dummy SO" when linking a standalone executable. Sorry, this is indeed confusing. This dummy SO is not the same as the dummy SO in #10458. Let me call the #11458 dummy SO "GHCi dummy SO" and the one in comment:16 "package dummy SO". The package dummy SO belongs to a Haskell package and is installed next to the static (`libHS*.a`) and the dynamic (`libHS*.so`) Haskell library. Then the package dummy SO would be used in the static Haskell dynamic C case in GHCi only. We could do this to help user's that are in the situation described in this ticket. On the other hand we could include the essence of your comment:19 "GHCi works like ld" in the user's guide (Section 2.6.2?). Perhaps we can discuss this in [wiki:LinkingHaskell] When linking a static executable a static executable then all C dev dependencies are required. That is no different from linking a C executable.
Since you have a design in mind, maybe it would be good to flesh it out
in a wiki page so we can discuss with more precision? I created [wiki:LinkingHaskell] -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:26 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#9498: GHC links against unversioned .so files -------------------------------------+------------------------------------- Reporter: Kritzefitz | Owner: Type: feature request | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.6.3 (Linking) | Resolution: | Keywords: Debian Operating System: Linux | Architecture: | Unknown/Multiple Type of failure: Other | Test Case: Blocked By: 11238 | Blocking: 9237 Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by nomeata): This, or a related problem, came up in Debian: https://bugs.debian.org /cgi-bin/bugreport.cgi?bug=834026#10 The summary is: Currently, if a Haskell library foo depends on bar, and bar uses a C library libbaz, then creating the shared library libHSfoo.so will be built with -lbaz, and hence link to libbaz.so.42, even though that dependency is already encoded in libHSbar.foo. Now, if libbaz is upgraded and bar is rebuilt the new version of libbaz, then baz’s ABI stays the same (so foo is not going to be rebuilt), but now foo is broken, as it cannot find `libbaz.so.42` any more. @trommler writes on LinkingHaskell (Emphasis mine), which should avoid this problem for us.
All Haskell libraries and all '''directly''' dependent C libraries must be passed to the linker.
-- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9498#comment:28 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC