Hi, Continuing my investigations about my building problem, I think I have found its cause and a way to reproduce it. Briefly, the important point is that the chroot environment must be created with devtools <= 20130408 (the last version that has makechrootpkg that support the "-d" flag). Once created, packages can be built even with newest versions of devtools that doesn't support the "-d" flag (but this requires to adapt the makeahpkg script to remove it). At the opposite, if the chroot is created with a newer version, Haskell packages will build and install, but with a message like: Diff-0.3.0: cannot find libHSDiff-0.3.0.a on library path (use --force to override) To reproduce, run the following commands once with devtools 20130408 and once with 20131020 (and don't forget to remove completely the "habs-temp" folder between the two tests): git clone https://github.com/StreakyCobra/habs.git habs-temp cd habs-temp cblrepo sync cblrepo pkgbuild Diff ./makeahpkg -c -- Diff # sudo rm habs-temp -rf Can anybody confirm this behavior? Or is it just me? Best regards, Fabien Dubosson
Hello again,
With a little help of "git bisect" on the devtools repository (
https://projects.archlinux.org/devtools.git/), I have found the reason of
the following kind of messages:
Diff-0.3.0: cannot find libHSDiff-0.3.0.a on library path (use --force
to override)
The commit 0d16a9135055d0c998cac236608cb630c93f0ac7 (
https://projects.archlinux.org/devtools.git/commit/?id=0d16a9135055d0c998cac...)
make "!staticlibs" default in makepkg.conf. So every chroot environment
containing this version will throw away static libraries when building
packages. The commit was pushed to arch repository 8 days ago, so only
newest chroot environments have this problem.
I see two solutions:
1. Change the "makepkg.conf" when creating the chroot environment to
remove "!staticlibs",
or, better
2. Change cblrepo to add «options=('strip' 'staticlibs')» when generating
the PKGBUILDs.
Because I don't know if all packages have/need these static libraries, I
ask you, experienced arch-haskeller, to point me the right direction?
Best regards,
Fabien Dubosson
2013/10/28 Fabien Dubosson
Hi,
Continuing my investigations about my building problem, I think I have found its cause and a way to reproduce it.
Briefly, the important point is that the chroot environment must be created with devtools <= 20130408 (the last version that has makechrootpkg that support the "-d" flag). Once created, packages can be built even with newest versions of devtools that doesn't support the "-d" flag (but this requires to adapt the makeahpkg script to remove it).
At the opposite, if the chroot is created with a newer version, Haskell packages will build and install, but with a message like:
Diff-0.3.0: cannot find libHSDiff-0.3.0.a on library path (use --force to override)
To reproduce, run the following commands once with devtools 20130408 and once with 20131020 (and don't forget to remove completely the "habs-temp" folder between the two tests):
git clone https://github.com/StreakyCobra/habs.git habs-temp cd habs-temp cblrepo sync cblrepo pkgbuild Diff ./makeahpkg -c -- Diff # sudo rm habs-temp -rf
Can anybody confirm this behavior? Or is it just me?
Best regards, Fabien Dubosson
On Mon, Oct 28, 2013 at 12:39:31PM +0100, Fabien Dubosson wrote:
Hello again,
With a little help of "git bisect" on the devtools repository ( https://projects.archlinux.org/devtools.git/), I have found the reason of the following kind of messages:
Diff-0.3.0: cannot find libHSDiff-0.3.0.a on library path (use --force to override)
The commit 0d16a9135055d0c998cac236608cb630c93f0ac7 ( https://projects.archlinux.org/devtools.git/commit/?id=0d16a9135055d0c998cac...) make "!staticlibs" default in makepkg.conf. So every chroot environment containing this version will throw away static libraries when building packages. The commit was pushed to arch repository 8 days ago, so only newest chroot environments have this problem.
Interesting. I suspect this is related to the discussions on the Arch mailing list about static libs.
I see two solutions:
1. Change the "makepkg.conf" when creating the chroot environment to remove "!staticlibs", or, better 2. Change cblrepo to add «options=('strip' 'staticlibs')» when generating the PKGBUILDs.
Because I don't know if all packages have/need these static libraries, I ask you, experienced arch-haskeller, to point me the right direction?
We absolutely do need the static libs to be included in the generated packages. Ghc 7.6 uses static linking by default and thus all our tool packages rely on static libs in order to build properly. It could of course be changed but it'd require a change to cblrepo to generate PKGBUILDs that uses `depends` rather than `makedepends`. /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus Hard work may not kill me, but why take the chance.
We absolutely do need the static libs to be included in the generated packages. Ghc 7.6 uses static linking by default and thus all our tool packages rely on static libs in order to build properly. It could of course be changed but it'd require a change to cblrepo to generate PKGBUILDs that uses `depends` rather than `makedepends`.
So I made a proposition (pull request to cblrepo) to explicitly include the
"staticlibs" option in generated PKGBUILD since this is the GHC default
linking (and also because it was suggested by Allan here:
https://mailman.archlinux.org/pipermail/arch-dev-public/2013-October/025542....
)
Actually the generated PKGBUILDs seem to already use 'depends' and not
'makedepends', or am I missing the point?
I also noticed two points: First it seems the archlinux default idea is to
use dynamic linking:
https://mailman.archlinux.org/pipermail/arch-general/2013-October/034337.htm....
Secondly GHC offers the possibility to build and use dynamic libraries:
http://www.haskell.org/ghc/docs/7.6.3/html/users_guide/using-shared-libs.htm....
Based on these two facts, it appears possible to build the haskell
ecosystem on a dynamic library principle. It will of course require a large
refactoring, but does it worth it? It is just curiosity ;)
2013/10/28 Magnus Therning
On Mon, Oct 28, 2013 at 12:39:31PM +0100, Fabien Dubosson wrote:
Hello again,
With a little help of "git bisect" on the devtools repository ( https://projects.archlinux.org/devtools.git/), I have found the reason of the following kind of messages:
Diff-0.3.0: cannot find libHSDiff-0.3.0.a on library path (use --force to override)
The commit 0d16a9135055d0c998cac236608cb630c93f0ac7 (
https://projects.archlinux.org/devtools.git/commit/?id=0d16a9135055d0c998cac... )
make "!staticlibs" default in makepkg.conf. So every chroot environment containing this version will throw away static libraries when building packages. The commit was pushed to arch repository 8 days ago, so only newest chroot environments have this problem.
Interesting. I suspect this is related to the discussions on the Arch mailing list about static libs.
I see two solutions:
1. Change the "makepkg.conf" when creating the chroot environment to remove "!staticlibs", or, better 2. Change cblrepo to add «options=('strip' 'staticlibs')» when generating the PKGBUILDs.
Because I don't know if all packages have/need these static libraries, I ask you, experienced arch-haskeller, to point me the right direction?
We absolutely do need the static libs to be included in the generated packages. Ghc 7.6 uses static linking by default and thus all our tool packages rely on static libs in order to build properly. It could of course be changed but it'd require a change to cblrepo to generate PKGBUILDs that uses `depends` rather than `makedepends`.
/M
-- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus
Hard work may not kill me, but why take the chance.
_______________________________________________ arch-haskell mailing list arch-haskell@haskell.org http://www.haskell.org/mailman/listinfo/arch-haskell
On Mon, Oct 28, 2013 at 11:01:21PM +0100, Fabien Dubosson wrote:
We absolutely do need the static libs to be included in the generated packages. Ghc 7.6 uses static linking by default and thus all our tool packages rely on static libs in order to build properly. It could of course be changed but it'd require a change to cblrepo to generate PKGBUILDs that uses `depends` rather than `makedepends`.
So I made a proposition (pull request to cblrepo) to explicitly include the "staticlibs" option in generated PKGBUILD since this is the GHC default linking (and also because it was suggested by Allan here: https://mailman.archlinux.org/pipermail/arch-dev-public/2013-October/025542....)
Actually the generated PKGBUILDs seem to already use 'depends' and not 'makedepends', or am I missing the point?
Look at the generated PKGBUILD for a tool such as `cblrepo` and you'll see what I mean.
I also noticed two points: First it seems the archlinux default idea is to use dynamic linking: https://mailman.archlinux.org/pipermail/arch-general/2013-October/034337.htm....
Indeed, there's been numerous threads on that lately.
Secondly GHC offers the possibility to build and use dynamic libraries: http://www.haskell.org/ghc/docs/7.6.3/html/users_guide/using-shared-libs.htm....
Based on these two facts, it appears possible to build the haskell ecosystem on a dynamic library principle. It will of course require a large refactoring, but does it worth it? It is just curiosity ;)
The default for GHC has always been to link statically, so that's where we started. Then we added building of shared libs at one point, but there was never any push to actually use them. After reading the page you point to above (in the GHC 7.6.3 docs) I'm also wondering what flags to give at build time to make an executable use dynamic libs. I had a look at pandoc, which is a package providing both a lib and an exe. It is configured with `--enable-shared` but the executable isn't dynamically linked against the libs it uses: % ldd /usr/bin/pandoc linux-vdso.so.1 (0x00007fffbbb93000) libz.so.1 => /usr/lib/libz.so.1 (0x00007f23e993a000) libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007f23e96d4000) libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f23e94d0000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f23e92b2000) libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007f23e903b000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f23e8d38000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f23e8b30000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f23e891a000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f23e856f000) /lib64/ld-linux-x86-64.so.2 (0x00007f23e9b50000) /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus "Sendmail" and "make" are two well known programs that are pretty widely regarded as being debugged into existence. That's why their command languages are so poorly thought out and difficult to learn. It's not just you -- everyone finds them troublesome. -- Peter van der Linden, Expert C Programming, p. 220
Look at the generated PKGBUILD for a tool such as `cblrepo` and you'll see what I mean.
Hum, I looked again but they seem to use already `depends`. There is 233 packages in habs, and 218 have empty `makedepends`: $ cd habs $ cblrepo pkgbuild $(cblrepo build base|tail -n +2) $ ls */PKGBUILD | wc -l 233 $ grep -R "makedepends=()" */PKGBUILD | wc -l 218 $ grep -R "makedepends=()" */PKGBUILD haskell-abstract-deque/PKGBUILD:makedepends=() haskell-abstract-par/PKGBUILD:makedepends=() haskell-aeson/PKGBUILD:makedepends=() haskell-anansi/PKGBUILD:makedepends=() [[...]] haskell-xmonad/PKGBUILD:makedepends=() haskell-yaml/PKGBUILD:makedepends=() haskell-zip-archive/PKGBUILD:makedepends=() haskell-zlib/PKGBUILD:makedepends=()
The default for GHC has always been to link statically, so that's where we started. Then we added building of shared libs at one point, but there was never any push to actually use them.
After reading the page you point to above (in the GHC 7.6.3 docs) I'm also wondering what flags to give at build time to make an executable use dynamic libs. I had a look at pandoc, which is a package providing both a lib and an exe. It is configured with `--enable-shared` but the executable isn't dynamically linked against the libs it uses:
% ldd /usr/bin/pandoc linux-vdso.so.1 (0x00007fffbbb93000) libz.so.1 => /usr/lib/libz.so.1 (0x00007f23e993a000) libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007f23e96d4000) libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f23e94d0000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f23e92b2000) libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007f23e903b000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f23e8d38000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f23e8b30000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f23e891a000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f23e856f000) /lib64/ld-linux-x86-64.so.2 (0x00007f23e9b50000)
The flag "--enable-shared" generates the shared libraries. The flag "-dynamic" builds executable against these libraries. I managed to build pandoc with shared libraries (and it worked, with a small hack, see problem at the end of the message): $ ldd /usr/bin/pandoc linux-vdso.so.1 (0x00007fff819fe000) libHSpandoc-1.12.1-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHSpandoc-1.12.1-ghc7.6.3.so (0x00007fd3929ec000) libHSzip-archive-0.1.4-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHSzip-archive-0.1.4-ghc7.6.3.so (0x00007fd39279c000) libHSzlib-0.5.4.1-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHSzlib-0.5.4.1-ghc7.6.3.so (0x00007fd392577000) [[...]] libHStransformers-0.3.0.0-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHStransformers-0.3.0.0-ghc7.6.3.so (0x00007fd3869d6000) libHSinteger-gmp-0.5.0.0-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHSinteger-gmp-0.5.0.0-ghc7.6.3.so (0x00007fd3867c5000) libHSmtl-2.1.2-ghc7.6.3.so => /usr/lib/ghc-7.6.3/site-local/mtl-2.1.2/ libHSmtl-2.1.2-ghc7.6.3.so (0x00007fd386598000) librt.so.1 => /usr/lib/librt.so.1 (0x00007fd386390000) libutil.so.1 => /usr/lib/libutil.so.1 (0x00007fd38618d000) libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fd385f89000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fd385d6b000) libHSdeepseq-1.3.0.1-ghc7.6.3.so => /usr/lib/ghc-7.6.3/deepseq-1.3.0.1/ libHSdeepseq-1.3.0.1-ghc7.6.3.so (0x00007fd385b62000) libHSarray-0.4.0.1-ghc7.6.3.so => /usr/lib/ghc-7.6.3/array-0.4.0.1/ libHSarray-0.4.0.1-ghc7.6.3.so (0x00007fd3858c1000) libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007fd38564a000) libffi.so.6 => /usr/lib/libffi.so.6 (0x00007fd385442000) libm.so.6 => /usr/lib/libm.so.6 (0x00007fd38513f000) /lib64/ld-linux-x86-64.so.2 (0x00007fd3937a3000) I have modified cblrepo to add a flag "--dynamic" to the "pkgbuild" command to generate a PKGBUILD with dynamic linking: $ cblrepo pkgbuild --dynamic pandoc See: https://github.com/StreakyCobra/cblrepo/commit/677fe94c311c9e71f0ad99c1f7ee3... I'll propose that as a pull request when all will be working. The current problem ----------------------------- There is still a small problem: Referencing the shared libraries. As stated at the bottom of [1], there is two ways: 1. By embedding a RPATH into the executable. 2. Using LD_LIBRARY_PATH (or equivalently and better /etc/ld.so.conf.d/*) to specify libraries paths. The first solution is system dependent, and also less elegant from my point of view. So I focused on the second. Sadly the current hierarchy of Haskell packages doesn't place all shared libraries in the same folder, but in separate folders of the form: /usr/lib/ghc-<version>/[site-local]/<package>-<version>/<sharedLibraryName>.so It is unfeasible to reference all these folders (even more with the version in the folder name). So I actually just made a small hack: a "/usr/lib/ghc-7.6.3/shared" folder and soft-links of all "*.so" inside. Then I added a new "/etc/ld.so.conf.d/haskell.conf" file containing the line: /usr/lib/ghc-7.6.3/shared and run `ldconfig` to update the libraries paths. Do you see any better way to reference these libraries? If no I suggest to put (or link) all "*.so" libraries either in "/usr/lib" or in another folder on a specific place on the system. It is also possible to add, if it is needed, a "/etc/ld.so.conf.d/haskell.conf" file to a new package like "haskell-runtime" and make all Haskell package depends on. [1] http://www.haskell.org/ghc/docs/7.6.3/html/users_guide/using-shared-libs.htm...
On Thu, Oct 31, 2013 at 05:49:27PM +0100, Fabien Dubosson wrote:
Look at the generated PKGBUILD for a tool such as `cblrepo` and you'll see what I mean.
Hum, I looked again but they seem to use already `depends`. There is 233 packages in habs, and 218 have empty `makedepends`:
$ cd habs $ cblrepo pkgbuild $(cblrepo build base|tail -n +2) $ ls */PKGBUILD | wc -l 233 $ grep -R "makedepends=()" */PKGBUILD | wc -l 218 $ grep -R "makedepends=()" */PKGBUILD haskell-abstract-deque/PKGBUILD:makedepends=() haskell-abstract-par/PKGBUILD:makedepends=() haskell-aeson/PKGBUILD:makedepends=() haskell-anansi/PKGBUILD:makedepends=() [[...]] haskell-xmonad/PKGBUILD:makedepends=() haskell-yaml/PKGBUILD:makedepends=() haskell-zip-archive/PKGBUILD:makedepends=() haskell-zlib/PKGBUILD:makedepends=()
Exactly, everything named haskell-* are library packages, only a few packages are pure tool packages (e.g. cblrepo, git-annex). Pure tool packages make use of makedepends. The relevant lines in cblrepo are found in Util.Translation, lines 302 and 303 in the current version (https://github.com/magthe/cblrepo/blob/master/src/Util/Translation.hs#L302).
The default for GHC has always been to link statically, so that's where we started. Then we added building of shared libs at one point, but there was never any push to actually use them.
After reading the page you point to above (in the GHC 7.6.3 docs) I'm also wondering what flags to give at build time to make an executable use dynamic libs. I had a look at pandoc, which is a package providing both a lib and an exe. It is configured with `--enable-shared` but the executable isn't dynamically linked against the libs it uses:
% ldd /usr/bin/pandoc linux-vdso.so.1 (0x00007fffbbb93000) libz.so.1 => /usr/lib/libz.so.1 (0x00007f23e993a000) libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007f23e96d4000) libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f23e94d0000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f23e92b2000) libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007f23e903b000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f23e8d38000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f23e8b30000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f23e891a000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f23e856f000) /lib64/ld-linux-x86-64.so.2 (0x00007f23e9b50000)
The flag "--enable-shared" generates the shared libraries. The flag "-dynamic" builds executable against these libraries. I managed to build pandoc with shared libraries (and it worked, with a small hack, see problem at the end of the message):
So, while it's possible to pass --enable-shared to build shared libs when configuring the package, it won't actually *use* shared libs. -dynamic is a flag for GHC, correct?
$ ldd /usr/bin/pandoc linux-vdso.so.1 (0x00007fff819fe000) libHSpandoc-1.12.1-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHSpandoc-1.12.1-ghc7.6.3.so (0x00007fd3929ec000) libHSzip-archive-0.1.4-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHSzip-archive-0.1.4-ghc7.6.3.so (0x00007fd39279c000) libHSzlib-0.5.4.1-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHSzlib-0.5.4.1-ghc7.6.3.so (0x00007fd392577000) [[...]] libHStransformers-0.3.0.0-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHStransformers-0.3.0.0-ghc7.6.3.so (0x00007fd3869d6000) libHSinteger-gmp-0.5.0.0-ghc7.6.3.so => /usr/lib/ghc-7.6.3/shared/ libHSinteger-gmp-0.5.0.0-ghc7.6.3.so (0x00007fd3867c5000) libHSmtl-2.1.2-ghc7.6.3.so => /usr/lib/ghc-7.6.3/site-local/mtl-2.1.2/ libHSmtl-2.1.2-ghc7.6.3.so (0x00007fd386598000) librt.so.1 => /usr/lib/librt.so.1 (0x00007fd386390000) libutil.so.1 => /usr/lib/libutil.so.1 (0x00007fd38618d000) libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fd385f89000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fd385d6b000) libHSdeepseq-1.3.0.1-ghc7.6.3.so => /usr/lib/ghc-7.6.3/deepseq-1.3.0.1/ libHSdeepseq-1.3.0.1-ghc7.6.3.so (0x00007fd385b62000) libHSarray-0.4.0.1-ghc7.6.3.so => /usr/lib/ghc-7.6.3/array-0.4.0.1/ libHSarray-0.4.0.1-ghc7.6.3.so (0x00007fd3858c1000) libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007fd38564a000) libffi.so.6 => /usr/lib/libffi.so.6 (0x00007fd385442000) libm.so.6 => /usr/lib/libm.so.6 (0x00007fd38513f000) /lib64/ld-linux-x86-64.so.2 (0x00007fd3937a3000)
I have modified cblrepo to add a flag "--dynamic" to the "pkgbuild" command to generate a PKGBUILD with dynamic linking:
$ cblrepo pkgbuild --dynamic pandoc
See: https://github.com/StreakyCobra/cblrepo/commit/677fe94c311c9e71f0ad99c1f7ee3...
I'll propose that as a pull request when all will be working.
What effects does passing --ghc-options at `Setup.hs configure` have on any `ghc-options:` field present in the .cabal?
The current problem -----------------------------
There is still a small problem: Referencing the shared libraries. As stated at the bottom of [1], there is two ways:
1. By embedding a RPATH into the executable. 2. Using LD_LIBRARY_PATH (or equivalently and better /etc/ld.so.conf.d/*) to specify libraries paths.
Interestingly the shared libs don't seem to suffer from this problem; run `ldd´ on one of the shared libs in a package to see what I mean. It sure looks like cabal has solved the problem halfway only.
The first solution is system dependent, and also less elegant from my point of view. So I focused on the second. Sadly the current hierarchy of Haskell packages doesn't place all shared libraries in the same folder, but in separate folders of the form:
/usr/lib/ghc-<version>/[site-local]/<package>-<version>/<sharedLibraryName>.so
It is unfeasible to reference all these folders (even more with the version in the folder name).
So I actually just made a small hack: a "/usr/lib/ghc-7.6.3/shared" folder and soft-links of all "*.so" inside. Then I added a new "/etc/ld.so.conf.d/haskell.conf" file containing the line: /usr/lib/ghc-7.6.3/shared and run `ldconfig` to update the libraries paths.
Do you see any better way to reference these libraries?
I'll start with asking on haskell-cafe, because it sure looks like this problem ought to be solved by Cabal and not by us.
If no I suggest to put (or link) all "*.so" libraries either in "/usr/lib" or in another folder on a specific place on the system. It is also possible to add, if it is needed, a "/etc/ld.so.conf.d/haskell.conf" file to a new package like "haskell-runtime" and make all Haskell package depends on.
I think the /usr/lib/ghc-7.6.3/shared folder is as good a place as any. IMHO we shouldn't place the Haskell libs straight into /usr/lib. Then the easiest thing would be to put the haskell-ldconf file into ghc itself, since each lib package depends on ghc anyway. Of course, this highlights one aspect to consider before going down the path of using shared libs for pure tools: installing such a package will bring with it all libs it depends on as well as ghc, while at the moment it is completely free standing. However, I think using shared libs for executables contained in libs packaged, e.g. /usr/bin/pandoc from haskell-pandoc, clearly is a good thing. /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus Good powers of observation are frequently called "cynicism" by those that don't have them.
I'll start with asking on haskell-cafe, because it sure looks like this problem ought to be solved by Cabal and not by us.
Yep I saw your mail, and also the first response. Since I read the answer after I wrote this message, I added some comment in curly braces regarding to this new information. I'll try later to use the `--enable-executable-dynamic` instead of `--ghc-options=-dynamic`.
Exactly, everything named haskell-* are library packages, only a few packages are pure tool packages (e.g. cblrepo, git-annex). Pure tool packages make use of makedepends. The relevant lines in cblrepo are found in Util.Translation, lines 302 and 303 in the current version ( https://github.com/magthe/cblrepo/blob/master/src/Util/Translation.hs#L302).
Ok, and if we want to use shared libraries, it will just require us to modify this part to put all the dependences only in `depends` right? Also will the static libraries still be needed if all packages use shared libraries?
So, while it's possible to pass --enable-shared to build shared libs when configuring the package, it won't actually *use* shared libs. -dynamic is a flag for GHC, correct?
Yes correct. Actually I had to add ghc "--disable-library-profiling" flag too because it was complaining that `base` shared libraries where not built with profiling capabilities (e.g. see the first comment here: http://ghc.haskell.org/trac/ghc/ticket/5021). {Regarding the first answer to your Haskell-Cafe message, using `--enable-executable-dynamic` will maybe solve this issue?!? To be verified...}
What effects does passing --ghc-options at `Setup.hs configure` have on any `ghc-options:` field present in the .cabal?
I don't know, but sure this's a point to verify! {Again using cabal flag `--enable-executable-dynamic` will certainly permit cabal to handle itself this precise situation.}
Interestingly the shared libs don't seem to suffer from this problem; run `ldd´ on one of the shared libs in a package to see what I mean.
It sure looks like cabal has solved the problem halfway only.
Good idea to check the libs! I just had a look at them. The difference imho is they are using the first option with RPATH. This is confirmed by running: $ readelf --dynamic /usr/lib/ghc-7.6.3/site-local/xmonad-0.11/ libHSxmonad-0.11-ghc7.6.3.so [[...]] 0x000000000000000f (RPATH) Library rpath: [/usr/lib/ghc-7.6.3/site-local/utf8-string-0.3.7:/usr/lib/ghc-7.6.3/process-1.1.0.2:/usr/lib/ghc-7.6.3/site-local/mtl-2.1.2:/usr/lib/ghc-7.6.3/site-local/transformers-0.3.0.0:/usr/lib/ghc-7.6.3/site-local/extensible-exceptions-0.1.1.4:/usr/lib/ghc-7.6.3/directory-1.2.0.1:/usr/lib/ghc-7.6.3/unix-2.6.0.1:/usr/lib/ghc-7.6.3/bytestring-0.10.0.2:/usr/lib/ghc-7.6.3/time-1.4.0.1:/usr/lib/ghc-7.6.3/filepath-1.3.0.1:/usr/lib/ghc-7.6.3/site-local/X11-1.6.1.1:/usr/lib/ghc-7.6.3/site-local/data-default-0.5.3:/usr/lib/ghc-7.6.3/site-local/data-default-instances-old-locale-0.0.1:/usr/lib/ghc-7.6.3/old-locale-1.0.0.5:/usr/lib/ghc-7.6.3/site-local/data-default-instances-dlist-0.0.1:/usr/lib/ghc-7.6.3/site-local/dlist-0.5:/usr/lib/ghc-7.6.3/site-local/data-default-instances-containers-0.0.1:/usr/lib/ghc-7.6.3/containers-0.5.0.0:/usr/lib/ghc-7.6.3/deepseq-1.3.0.1:/usr/lib/ghc-7.6.3/array-0.4.0.1:/usr/lib/ghc-7.6.3/site-local/data-default-instances-base-0.0.1:/usr/lib/ghc-7.6.3/site-local/data-default-class-0.0.1:/usr/lib/ghc-7.6.3/base-4.6.0.1:/usr/lib/ghc-7.6.3/integer-gmp-0.5.0.0:/usr/lib/ghc-7.6.3/ghc-prim-0.3.0.0:/usr/lib/ghc-7.6.3] [[...]] All the paths are embedded into the libraries. In fact it is possible to use the same principle to build executable, it is even the default way (I passed `-dynload=deploy` to override and use LD_LIBRARY_PATH instead). I just understand why this was not working with `pandoc` when I was using RPATH the first time: 1. To begin, the library is built in the folder `src/<pkgname>/dist/build/` 2. Then the executable is build on the top of shared libraries. But because the pandoc library is in the `src/<pkgname>/dist/build/` folder (because it is not installed at building time) the RPATH include this path instead of the path where the library will be found when installed. And at run time when the pandoc folder is cleared it doesn't found any longer the library. So I don't know if there is a way to solve this problem, that what I have switched to LD_LIB based solution. {I must also verify if this situation persists with `--enable-executable-dynamic`, but I suspect it will still be present because cabal can not know in advance where the library will be put}.
I think the /usr/lib/ghc-7.6.3/shared folder is as good a place as any. IMHO we shouldn't place the Haskell libs straight into /usr/lib. Then the easiest thing would be to put the haskell-ldconf file into ghc itself, since each lib package depends on ghc anyway. Of course, this highlights one aspect to consider before going down the path of using shared libs for pure tools: installing such a package will bring with it all libs it depends on as well as ghc, while at the moment it is completely free standing.
I see now more clearly the two different options: 1. Use RPATH referencing, this is done at build time, but requires to solve the linking problem cited above. 2. Use LD_LIBRARY_PATH referencing, this requires to regroup all shared libraries into one place. Does someone know how most archlinux packages handle that? LD or RPATH?
However, I think using shared libs for executables contained in libs packaged, e.g. /usr/bin/pandoc from haskell-pandoc, clearly is a good thing.
I agree. All this maybe not brings a lot of advantages (maybe just size?), but I like things done well and in a consistent way :) Regards, Fabien
On Thu, Oct 31, 2013 at 10:55:30PM +0100, Fabien Dubosson wrote:
I'll start with asking on haskell-cafe, because it sure looks like this problem ought to be solved by Cabal and not by us.
Yep I saw your mail, and also the first response. Since I read the answer after I wrote this message, I added some comment in curly braces regarding to this new information. I'll try later to use the `--enable-executable-dynamic` instead of `--ghc-options=-dynamic`.
I had a look at the change you created and I'm not sure a flag to the pkgbuild command is a good way to go. I'm much more in favour of simply making it the way we build, i.e. add '--enable-executable-dynamic' on all builds of lib packages.
Exactly, everything named haskell-* are library packages, only a few packages are pure tool packages (e.g. cblrepo, git-annex). Pure tool packages make use of makedepends. The relevant lines in cblrepo are found in Util.Translation, lines 302 and 303 in the current version (https://github.com/magthe/cblrepo/blob/master/src/Util/Translation.hs#L302).
Ok, and if we want to use shared libraries, it will just require us to modify this part to put all the dependences only in `depends` right? Also will the static libraries still be needed if all packages use shared libraries?
Sure, but I'm not convinced using shared libs is the route to take for pure executable packages (like cblrepo and git-annex). As long as pure executable packages are linked statically we'll need the static libs of course. It's also the case that linking statically is the Ghc default, which speaks in favour of keeping the static libs. /M -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus Some operating systems are called 'user friendly', Linux however is 'expert friendly'.
I had a look at the change you created and I'm not sure a flag to the pkgbuild command is a good way to go. I'm much more in favour of simply making it the way we build, i.e. add '--enable-executable-dynamic' on all builds of lib packages.
Of course, my modifications are for the case where building dynamic executables is not the default way. But obviously if that becomes the default option, adding it on all PKGBUILDs is the simplest thing.
Sure, but I'm not convinced using shared libs is the route to take for pure executable packages (like cblrepo and git-annex). As long as pure executable packages are linked statically we'll need the static libs of course. It's also the case that linking statically is the Ghc default, which speaks in favour of keeping the static libs.
Well, your second argument is pretty unbreakable :) As long as this is the ghc default, static libraries are needed.
participants (2)
-
Fabien Dubosson -
Magnus Therning