[PATCH] fix runtests to set LD_LIBRARY_PATH environment variable

newer
Problem in C Code Generator, with...

Karel Gardas

24 Jan 2013 24 Jan '13

4:28 p.m.

This patch follows Windows and Darwin way of setting environment variable to set the file-system paths to GHC's shared libraries. It does the same thing for any other OS, which should be Unix-like OS presumably. This patch fixes a lot of `dyn' tests failures on Solaris which fail with following error message: Failed to open shared library: ld.so.1: T3807-load: fatal: libffi.so.6: open failed: No such file or directory --- driver/runtests.py | 48 ++++++++++++++++++++++++++---------------------- 1 files changed, 26 insertions(+), 22 deletions(-) diff --git a/driver/runtests.py b/driver/runtests.py index 66e3bf4..16deda6 100644 --- a/driver/runtests.py +++ b/driver/runtests.py @@ -181,28 +181,32 @@ from testlib import * # On Windows we need to set $PATH to include the paths to all the DLLs # in order for the dynamic library tests to work. -if windows or darwin: - pkginfo = getStdout([config.ghc_pkg, 'dump']) - topdir = config.libdir - for line in pkginfo.split('\n'): - if line.startswith('library-dirs:'): - path = line.rstrip() - path = re.sub('^library-dirs: ', '', path) - path = re.sub('\\$topdir', topdir, path) - if path.startswith('"'): - path = re.sub('^"(.*)"$', '\\1', path) - path = re.sub('\\\\(.)', '\\1', path) - if windows: - if config.cygwin: - # On cygwin we can't put "c:\foo" in $PATH, as : is a - # field separator. So convert to /cygdrive/c/foo instead. - # Other pythons use ; as the separator, so no problem. - path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) - path = re.sub('\\\\', '/', path) - os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) - else: - # darwin - os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) +# if windows or darwin: +pkginfo = getStdout([config.ghc_pkg, 'dump']) +topdir = config.libdir +for line in pkginfo.split('\n'): + if line.startswith('library-dirs:'): + path = line.rstrip() + path = re.sub('^library-dirs: ', '', path) + path = re.sub('\\$topdir', topdir, path) + if path.startswith('"'): + path = re.sub('^"(.*)"$', '\\1', path) + path = re.sub('\\\\(.)', '\\1', path) + if windows: + if config.cygwin: + # On cygwin we can't put "c:\foo" in $PATH, as : is a + # field separator. So convert to /cygdrive/c/foo instead. + # Other pythons use ; as the separator, so no problem. + path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) + path = re.sub('\\\\', '/', path) + os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) + elif darwin: + # darwin + os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) + else: + # unix + os.environ['LD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("LD_LIBRARY_PATH", "")]) + global testopts_local testopts_local.x = TestOptions() -- 1.7.3.2

Show replies by date

David Terei

24 Jan 24 Jan

8:33 p.m.

Committed. Thanks! On 24 January 2013 13:28, Karel Gardas wrote:

...

This patch follows Windows and Darwin way of setting environment variable to set the file-system paths to GHC's shared libraries. It does the same thing for any other OS, which should be Unix-like OS presumably. This patch fixes a lot of `dyn' tests failures on Solaris which fail with following error message: Failed to open shared library: ld.so.1: T3807-load: fatal: libffi.so.6: open failed: No such file or directory --- driver/runtests.py | 48 ++++++++++++++++++++++++++---------------------- 1 files changed, 26 insertions(+), 22 deletions(-)

diff --git a/driver/runtests.py b/driver/runtests.py index 66e3bf4..16deda6 100644 --- a/driver/runtests.py +++ b/driver/runtests.py @@ -181,28 +181,32 @@ from testlib import *

# On Windows we need to set $PATH to include the paths to all the DLLs # in order for the dynamic library tests to work. -if windows or darwin: - pkginfo = getStdout([config.ghc_pkg, 'dump']) - topdir = config.libdir - for line in pkginfo.split('\n'): - if line.startswith('library-dirs:'): - path = line.rstrip() - path = re.sub('^library-dirs: ', '', path) - path = re.sub('\\$topdir', topdir, path) - if path.startswith('"'): - path = re.sub('^"(.*)"$', '\\1', path) - path = re.sub('\\\\(.)', '\\1', path) - if windows: - if config.cygwin: - # On cygwin we can't put "c:\foo" in $PATH, as : is a - # field separator. So convert to /cygdrive/c/foo instead. - # Other pythons use ; as the separator, so no problem. - path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) - path = re.sub('\\\\', '/', path) - os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) - else: - # darwin - os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) +# if windows or darwin: +pkginfo = getStdout([config.ghc_pkg, 'dump']) +topdir = config.libdir +for line in pkginfo.split('\n'): + if line.startswith('library-dirs:'): + path = line.rstrip() + path = re.sub('^library-dirs: ', '', path) + path = re.sub('\\$topdir', topdir, path) + if path.startswith('"'): + path = re.sub('^"(.*)"$', '\\1', path) + path = re.sub('\\\\(.)', '\\1', path) + if windows: + if config.cygwin: + # On cygwin we can't put "c:\foo" in $PATH, as : is a + # field separator. So convert to /cygdrive/c/foo instead. + # Other pythons use ; as the separator, so no problem. + path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) + path = re.sub('\\\\', '/', path) + os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) + elif darwin: + # darwin + os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) + else: + # unix + os.environ['LD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("LD_LIBRARY_PATH", "")]) +

global testopts_local testopts_local.x = TestOptions() -- 1.7.3.2

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Simon Marlow

25 Jan 25 Jan

3:02 a.m.

Hold on a minute. Why do you need to set LD_LIBRARY_PATH? It should be unnecessary because the binaries are linked with -rpath options so they can find their libraries. Cheers, Simon On 25/01/13 01:33, David Terei wrote:

...

Committed. Thanks!

On 24 January 2013 13:28, Karel Gardas wrote:

...
This patch follows Windows and Darwin way of setting environment variable to set the file-system paths to GHC's shared libraries. It does the same thing for any other OS, which should be Unix-like OS presumably. This patch fixes a lot of `dyn' tests failures on Solaris which fail with following error message: Failed to open shared library: ld.so.1: T3807-load: fatal: libffi.so.6: open failed: No such file or directory --- driver/runtests.py | 48 ++++++++++++++++++++++++++---------------------- 1 files changed, 26 insertions(+), 22 deletions(-)

diff --git a/driver/runtests.py b/driver/runtests.py index 66e3bf4..16deda6 100644 --- a/driver/runtests.py +++ b/driver/runtests.py @@ -181,28 +181,32 @@ from testlib import *

# On Windows we need to set $PATH to include the paths to all the DLLs # in order for the dynamic library tests to work. -if windows or darwin: - pkginfo = getStdout([config.ghc_pkg, 'dump']) - topdir = config.libdir - for line in pkginfo.split('\n'): - if line.startswith('library-dirs:'): - path = line.rstrip() - path = re.sub('^library-dirs: ', '', path) - path = re.sub('\\$topdir', topdir, path) - if path.startswith('"'): - path = re.sub('^"(.*)"$', '\\1', path) - path = re.sub('\\\\(.)', '\\1', path) - if windows: - if config.cygwin: - # On cygwin we can't put "c:\foo" in $PATH, as : is a - # field separator. So convert to /cygdrive/c/foo instead. - # Other pythons use ; as the separator, so no problem. - path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) - path = re.sub('\\\\', '/', path) - os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) - else: - # darwin - os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) +# if windows or darwin: +pkginfo = getStdout([config.ghc_pkg, 'dump']) +topdir = config.libdir +for line in pkginfo.split('\n'): + if line.startswith('library-dirs:'): + path = line.rstrip() + path = re.sub('^library-dirs: ', '', path) + path = re.sub('\\$topdir', topdir, path) + if path.startswith('"'): + path = re.sub('^"(.*)"$', '\\1', path) + path = re.sub('\\\\(.)', '\\1', path) + if windows: + if config.cygwin: + # On cygwin we can't put "c:\foo" in $PATH, as : is a + # field separator. So convert to /cygdrive/c/foo instead. + # Other pythons use ; as the separator, so no problem. + path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) + path = re.sub('\\\\', '/', path) + os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) + elif darwin: + # darwin + os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) + else: + # unix + os.environ['LD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("LD_LIBRARY_PATH", "")]) +

global testopts_local testopts_local.x = TestOptions() -- 1.7.3.2

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Karel Gardas

2:12 p.m.

Hi Simon, it's simple, every lib is rpath-ed except the libffi. See: End of readFile001 compilation with -v looks: *** Linker: /usr/sfw/bin/gcc -DTABLES_NEXT_TO_CODE -o readFile001 readFile001.o -L/export/home/karel/vcs/ghc-src/ghc-head/libraries/base/dist-install/build -Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/libraries/base/dist-install/build -L/export/home/karel/vcs/ghc-src/ghc-head/libraries/integer-gmp/dist-install/build -Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/libraries/integer-gmp/dist-install/build -L/export/home/karel/vcs/ghc-src/ghc-head/libraries/ghc-prim/dist-install/build -Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/libraries/ghc-prim/dist-install/build -L/export/home/karel/vcs/ghc-src/ghc-head/rts/dist/build -Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/rts/dist/build /tmp/ghc28147_0/ghc28147_0.o -lHSbase-4.7.0.0-ghc7.7.20130125 -lHSinteger-gmp-0.5.1.0-ghc7.7.20130125 -lgmp -lHSghc-prim-0.3.1.0-ghc7.7.20130125 -lHSrts-ghc7.7.20130125 -lm -lrt -ldl -u ghczmprim_GHCziTypes_Izh_static_info -u ghczmprim_GHCziTypes_Czh_static_info -u ghczmprim_GHCziTypes_Fzh_static_info -u ghczmprim_GHCziTypes_Dzh_static_info -u base_GHCziPtr_Ptr_static_info -u ghczmprim_GHCziTypes_Wzh_static_info -u base_GHCziInt_I8zh_static_info -u base_GHCziInt_I16zh_static_info -u base_GHCziInt_I32zh_static_info -u base_GHCziInt_I64zh_static_info -u base_GHCziWord_W8zh_static_info -u base_GHCziWord_W16zh_static_info -u base_GHCziWord_W32zh_static_info -u base_GHCziWord_W64zh_static_info -u base_GHCziStable_StablePtr_static_info -u ghczmprim_GHCziTypes_Izh_con_info -u ghczmprim_GHCziTypes_Czh_con_info -u ghczmprim_GHCziTypes_Fzh_con_info -u ghczmprim_GHCziTypes_Dzh_con_info -u base_GHCziPtr_Ptr_con_info -u base_GHCziPtr_FunPtr_con_info -u base_GHCziStable_StablePtr_con_info -u ghczmprim_GHCziTypes_False_closure -u ghczmprim_GHCziTypes_True_closure -u base_GHCziPack_unpackCString_closure -u base_GHCziIOziException_stackOverflow_closure -u base_GHCziIOziException_heapOverflow_closure -u base_ControlziExceptionziBase_nonTermination_closure -u base_GHCziIOziException_blockedIndefinitelyOnMVar_closure -u base_GHCziIOziException_blockedIndefinitelyOnSTM_closure -u base_ControlziExceptionziBase_nestedAtomically_closure -u base_GHCziWeak_runFinalizzerBatch_closure -u base_GHCziTopHandler_flushStdHandles_closure -u base_GHCziTopHandler_runIO_closure -u base_GHCziTopHandler_runNonIO_closure -u base_GHCziConcziIO_ensureIOManagerIsRunning_closure -u base_GHCziConcziSync_runSparks_closure -u base_GHCziConcziSignal_runHandlers_closure link: done *** Deleting temp files: Deleting: /tmp/ghc28147_0/ghc28147_0.o /tmp/ghc28147_0/ghc28147_0.c *** Deleting temp dirs: Deleting: /tmp/ghc28147_0 there is no libffi linked there as it's linked in RTS lib already. it's probably not rpath-ed there (for obvious reason) so ldd's output on the test looks: $ ldd readFile001 libHSbase-4.7.0.0-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/libraries/base/dist-install/build/libHSbase-4.7.0.0-ghc7.7.20130125.so libHSinteger-gmp-0.5.1.0-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.5.1.0-ghc7.7.20130125.so libgmp.so.3 => /usr/lib/libgmp.so.3 libHSghc-prim-0.3.1.0-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.1.0-ghc7.7.20130125.so libHSrts-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/rts/dist/build/libHSrts-ghc7.7.20130125.so libm.so.2 => /lib/libm.so.2 librt.so.1 => /lib/librt.so.1 libdl.so.1 => /lib/libdl.so.1 libc.so.1 => /lib/libc.so.1 libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 libffi.so.6 => (file not found) And of course for the purpose of the test I've unset LD_LIBRARY_PATH completely: karel@silence:~/vcs/ghc-src/ghc-head/libraries/base/tests/IO$ echo $LD_LIBRARY_PATH karel@silence:~/vcs/ghc-src/ghc-head/libraries/base/tests/IO$ So that's why I've thought my solution of setting LD_LIBRARY_PATH in runtests is the most easiest one but if you prefer something different just let me know. Thanks! Karel On 01/25/13 09:02 AM, Simon Marlow wrote:

...

Hold on a minute. Why do you need to set LD_LIBRARY_PATH? It should be unnecessary because the binaries are linked with -rpath options so they can find their libraries.

Cheers, Simon

On 25/01/13 01:33, David Terei wrote:

...
Committed. Thanks!

On 24 January 2013 13:28, Karel Gardas wrote:

...
This patch follows Windows and Darwin way of setting environment variable to set the file-system paths to GHC's shared libraries. It does the same thing for any other OS, which should be Unix-like OS presumably. This patch fixes a lot of `dyn' tests failures on Solaris which fail with following error message: Failed to open shared library: ld.so.1: T3807-load: fatal: libffi.so.6: open failed: No such file or directory --- driver/runtests.py | 48 ++++++++++++++++++++++++++---------------------- 1 files changed, 26 insertions(+), 22 deletions(-)

diff --git a/driver/runtests.py b/driver/runtests.py index 66e3bf4..16deda6 100644 --- a/driver/runtests.py +++ b/driver/runtests.py @@ -181,28 +181,32 @@ from testlib import *

# On Windows we need to set $PATH to include the paths to all the DLLs # in order for the dynamic library tests to work. -if windows or darwin: - pkginfo = getStdout([config.ghc_pkg, 'dump']) - topdir = config.libdir - for line in pkginfo.split('\n'): - if line.startswith('library-dirs:'): - path = line.rstrip() - path = re.sub('^library-dirs: ', '', path) - path = re.sub('\\$topdir', topdir, path) - if path.startswith('"'): - path = re.sub('^"(.*)"$', '\\1', path) - path = re.sub('\\\\(.)', '\\1', path) - if windows: - if config.cygwin: - # On cygwin we can't put "c:\foo" in $PATH, as : is a - # field separator. So convert to /cygdrive/c/foo instead. - # Other pythons use ; as the separator, so no problem. - path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) - path = re.sub('\\\\', '/', path) - os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) - else: - # darwin - os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) +# if windows or darwin: +pkginfo = getStdout([config.ghc_pkg, 'dump']) +topdir = config.libdir +for line in pkginfo.split('\n'): + if line.startswith('library-dirs:'): + path = line.rstrip() + path = re.sub('^library-dirs: ', '', path) + path = re.sub('\\$topdir', topdir, path) + if path.startswith('"'): + path = re.sub('^"(.*)"$', '\\1', path) + path = re.sub('\\\\(.)', '\\1', path) + if windows: + if config.cygwin: + # On cygwin we can't put "c:\foo" in $PATH, as : is a + # field separator. So convert to /cygdrive/c/foo instead. + # Other pythons use ; as the separator, so no problem. + path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) + path = re.sub('\\\\', '/', path) + os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) + elif darwin: + # darwin + os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) + else: + # unix + os.environ['LD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("LD_LIBRARY_PATH", "")]) +

global testopts_local testopts_local.x = TestOptions() -- 1.7.3.2

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Simon Marlow

3:40 p.m.

On 25/01/13 19:12, Karel Gardas wrote:

...

Hi Simon,

it's simple, every lib is rpath-ed except the libffi. See:

Then that sounds like a bug, no? I'm not up to speed on this rpath stuff, maybe Ian knows what's going on. I'd rather not hide the bug by committing a workaround to the testsuite. Cheers, Simon

...

End of readFile001 compilation with -v looks:

*** Linker: /usr/sfw/bin/gcc -DTABLES_NEXT_TO_CODE -o readFile001 readFile001.o -L/export/home/karel/vcs/ghc-src/ghc-head/libraries/base/dist-install/build -Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/libraries/base/dist-install/build -L/export/home/karel/vcs/ghc-src/ghc-head/libraries/integer-gmp/dist-install/build -Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/libraries/integer-gmp/dist-install/build -L/export/home/karel/vcs/ghc-src/ghc-head/libraries/ghc-prim/dist-install/build -Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/libraries/ghc-prim/dist-install/build -L/export/home/karel/vcs/ghc-src/ghc-head/rts/dist/build -Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/rts/dist/build /tmp/ghc28147_0/ghc28147_0.o -lHSbase-4.7.0.0-ghc7.7.20130125 -lHSinteger-gmp-0.5.1.0-ghc7.7.20130125 -lgmp -lHSghc-prim-0.3.1.0-ghc7.7.20130125 -lHSrts-ghc7.7.20130125 -lm -lrt -ldl -u ghczmprim_GHCziTypes_Izh_static_info -u ghczmprim_GHCziTypes_Czh_static_info -u ghczmprim_GHCziTypes_Fzh_static_info -u ghczmprim_GHCziTypes_Dzh_static_info -u base_GHCziPtr_Ptr_static_info -u ghczmprim_GHCziTypes_Wzh_static_info -u base_GHCziInt_I8zh_static_info -u base_GHCziInt_I16zh_static_info -u base_GHCziInt_I32zh_static_info -u base_GHCziInt_I64zh_static_info -u base_GHCziWord_W8zh_static_info -u base_GHCziWord_W16zh_static_info -u base_GHCziWord_W32zh_static_info -u base_GHCziWord_W64zh_static_info -u base_GHCziStable_StablePtr_static_info -u ghczmprim_GHCziTypes_Izh_con_info -u ghczmprim_GHCziTypes_Czh_con_info -u ghczmprim_GHCziTypes_Fzh_con_info -u ghczmprim_GHCziTypes_Dzh_con_info -u base_GHCziPtr_Ptr_con_info -u base_GHCziPtr_FunPtr_con_info -u base_GHCziStable_StablePtr_con_info -u ghczmprim_GHCziTypes_False_closure -u ghczmprim_GHCziTypes_True_closure -u base_GHCziPack_unpackCString_closure -u base_GHCziIOziException_stackOverflow_closure -u base_GHCziIOziException_heapOverflow_closure -u base_ControlziExceptionziBase_nonTermination_closure -u base_GHCziIOziException_blockedIndefinitelyOnMVar_closure -u base_GHCziIOziException_blockedIndefinitelyOnSTM_closure -u base_ControlziExceptionziBase_nestedAtomically_closure -u base_GHCziWeak_runFinalizzerBatch_closure -u base_GHCziTopHandler_flushStdHandles_closure -u base_GHCziTopHandler_runIO_closure -u base_GHCziTopHandler_runNonIO_closure -u base_GHCziConcziIO_ensureIOManagerIsRunning_closure -u base_GHCziConcziSync_runSparks_closure -u base_GHCziConcziSignal_runHandlers_closure link: done *** Deleting temp files: Deleting: /tmp/ghc28147_0/ghc28147_0.o /tmp/ghc28147_0/ghc28147_0.c *** Deleting temp dirs: Deleting: /tmp/ghc28147_0

there is no libffi linked there as it's linked in RTS lib already. it's probably not rpath-ed there (for obvious reason) so ldd's output on the test looks:

$ ldd readFile001 libHSbase-4.7.0.0-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/libraries/base/dist-install/build/libHSbase-4.7.0.0-ghc7.7.20130125.so

libHSinteger-gmp-0.5.1.0-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.5.1.0-ghc7.7.20130125.so

libgmp.so.3 => /usr/lib/libgmp.so.3 libHSghc-prim-0.3.1.0-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.1.0-ghc7.7.20130125.so

libHSrts-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/rts/dist/build/libHSrts-ghc7.7.20130125.so

libm.so.2 => /lib/libm.so.2 librt.so.1 => /lib/librt.so.1 libdl.so.1 => /lib/libdl.so.1 libc.so.1 => /lib/libc.so.1 libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 libffi.so.6 => (file not found)

And of course for the purpose of the test I've unset LD_LIBRARY_PATH completely: karel@silence:~/vcs/ghc-src/ghc-head/libraries/base/tests/IO$ echo $LD_LIBRARY_PATH

karel@silence:~/vcs/ghc-src/ghc-head/libraries/base/tests/IO$

So that's why I've thought my solution of setting LD_LIBRARY_PATH in runtests is the most easiest one but if you prefer something different just let me know.

Thanks! Karel

On 01/25/13 09:02 AM, Simon Marlow wrote:

...
Hold on a minute. Why do you need to set LD_LIBRARY_PATH? It should be unnecessary because the binaries are linked with -rpath options so they can find their libraries.

Cheers, Simon

On 25/01/13 01:33, David Terei wrote:

...
Committed. Thanks!

On 24 January 2013 13:28, Karel Gardas wrote:

...
This patch follows Windows and Darwin way of setting environment variable to set the file-system paths to GHC's shared libraries. It does the same thing for any other OS, which should be Unix-like OS presumably. This patch fixes a lot of `dyn' tests failures on Solaris which fail with following error message: Failed to open shared library: ld.so.1: T3807-load: fatal: libffi.so.6: open failed: No such file or directory --- driver/runtests.py | 48 ++++++++++++++++++++++++++---------------------- 1 files changed, 26 insertions(+), 22 deletions(-)

diff --git a/driver/runtests.py b/driver/runtests.py index 66e3bf4..16deda6 100644 --- a/driver/runtests.py +++ b/driver/runtests.py @@ -181,28 +181,32 @@ from testlib import *

# On Windows we need to set $PATH to include the paths to all the DLLs # in order for the dynamic library tests to work. -if windows or darwin: - pkginfo = getStdout([config.ghc_pkg, 'dump']) - topdir = config.libdir - for line in pkginfo.split('\n'): - if line.startswith('library-dirs:'): - path = line.rstrip() - path = re.sub('^library-dirs: ', '', path) - path = re.sub('\\$topdir', topdir, path) - if path.startswith('"'): - path = re.sub('^"(.*)"$', '\\1', path) - path = re.sub('\\\\(.)', '\\1', path) - if windows: - if config.cygwin: - # On cygwin we can't put "c:\foo" in $PATH, as : is a - # field separator. So convert to /cygdrive/c/foo instead. - # Other pythons use ; as the separator, so no problem. - path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) - path = re.sub('\\\\', '/', path) - os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) - else: - # darwin - os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) +# if windows or darwin: +pkginfo = getStdout([config.ghc_pkg, 'dump']) +topdir = config.libdir +for line in pkginfo.split('\n'): + if line.startswith('library-dirs:'): + path = line.rstrip() + path = re.sub('^library-dirs: ', '', path) + path = re.sub('\\$topdir', topdir, path) + if path.startswith('"'): + path = re.sub('^"(.*)"$', '\\1', path) + path = re.sub('\\\\(.)', '\\1', path) + if windows: + if config.cygwin: + # On cygwin we can't put "c:\foo" in $PATH, as : is a + # field separator. So convert to /cygdrive/c/foo instead. + # Other pythons use ; as the separator, so no problem. + path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) + path = re.sub('\\\\', '/', path) + os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) + elif darwin: + # darwin + os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) + else: + # unix + os.environ['LD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("LD_LIBRARY_PATH", "")]) +

global testopts_local testopts_local.x = TestOptions() -- 1.7.3.2

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Karel Gardas

4:09 p.m.

On 01/25/13 09:40 PM, Simon Marlow wrote:

...

On 25/01/13 19:12, Karel Gardas wrote:

...
Hi Simon,

it's simple, every lib is rpath-ed except the libffi. See:

Then that sounds like a bug, no?

IMHO it's more an expected behavior than a bug, but as Ian implemented it anyway, let's leave him a last word about it. From my point of view it looks like libffi is provided by any linux distro when you install GHC. (e.g. on ubuntu/debian you install ghc and libffi/libffi-dev is installed too). So this means on linux libffi is in /usr/lib and so available w/o any issue, it means the issue will not hit you on Linux. The problem I see is on Solaris. There is no system libffi and so ghc brings its own. The decision that libffi is linked only to rts lib seems to be also good as you don't need to decide while linking application if libffi is used or not. Simply link rts and be done with it. Another decision that libffi is linked into rts without rpath is also good IMHO as it allows you to move rts library around -- for example for installation. So well, that's at least my understanding of this and that's also the reason why I consider this to be more of expected behavior than of a bug... Anyway, thanks a lot for your very careful eyes following what's going into GHC and what not! Karel

...

I'm not up to speed on this rpath stuff, maybe Ian knows what's going on. I'd rather not hide the bug by committing a workaround to the testsuite.

Cheers, Simon

...
End of readFile001 compilation with -v looks:

*** Linker: /usr/sfw/bin/gcc -DTABLES_NEXT_TO_CODE -o readFile001 readFile001.o -L/export/home/karel/vcs/ghc-src/ghc-head/libraries/base/dist-install/build

-Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/libraries/base/dist-install/build

-L/export/home/karel/vcs/ghc-src/ghc-head/libraries/integer-gmp/dist-install/build

-Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/libraries/integer-gmp/dist-install/build

-L/export/home/karel/vcs/ghc-src/ghc-head/libraries/ghc-prim/dist-install/build

-Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/libraries/ghc-prim/dist-install/build

-L/export/home/karel/vcs/ghc-src/ghc-head/rts/dist/build -Wl,-rpath -Wl,/export/home/karel/vcs/ghc-src/ghc-head/rts/dist/build /tmp/ghc28147_0/ghc28147_0.o -lHSbase-4.7.0.0-ghc7.7.20130125 -lHSinteger-gmp-0.5.1.0-ghc7.7.20130125 -lgmp -lHSghc-prim-0.3.1.0-ghc7.7.20130125 -lHSrts-ghc7.7.20130125 -lm -lrt -ldl -u ghczmprim_GHCziTypes_Izh_static_info -u ghczmprim_GHCziTypes_Czh_static_info -u ghczmprim_GHCziTypes_Fzh_static_info -u ghczmprim_GHCziTypes_Dzh_static_info -u base_GHCziPtr_Ptr_static_info -u ghczmprim_GHCziTypes_Wzh_static_info -u base_GHCziInt_I8zh_static_info -u base_GHCziInt_I16zh_static_info -u base_GHCziInt_I32zh_static_info -u base_GHCziInt_I64zh_static_info -u base_GHCziWord_W8zh_static_info -u base_GHCziWord_W16zh_static_info -u base_GHCziWord_W32zh_static_info -u base_GHCziWord_W64zh_static_info -u base_GHCziStable_StablePtr_static_info -u ghczmprim_GHCziTypes_Izh_con_info -u ghczmprim_GHCziTypes_Czh_con_info -u ghczmprim_GHCziTypes_Fzh_con_info -u ghczmprim_GHCziTypes_Dzh_con_info -u base_GHCziPtr_Ptr_con_info -u base_GHCziPtr_FunPtr_con_info -u base_GHCziStable_StablePtr_con_info -u ghczmprim_GHCziTypes_False_closure -u ghczmprim_GHCziTypes_True_closure -u base_GHCziPack_unpackCString_closure -u base_GHCziIOziException_stackOverflow_closure -u base_GHCziIOziException_heapOverflow_closure -u base_ControlziExceptionziBase_nonTermination_closure -u base_GHCziIOziException_blockedIndefinitelyOnMVar_closure -u base_GHCziIOziException_blockedIndefinitelyOnSTM_closure -u base_ControlziExceptionziBase_nestedAtomically_closure -u base_GHCziWeak_runFinalizzerBatch_closure -u base_GHCziTopHandler_flushStdHandles_closure -u base_GHCziTopHandler_runIO_closure -u base_GHCziTopHandler_runNonIO_closure -u base_GHCziConcziIO_ensureIOManagerIsRunning_closure -u base_GHCziConcziSync_runSparks_closure -u base_GHCziConcziSignal_runHandlers_closure link: done *** Deleting temp files: Deleting: /tmp/ghc28147_0/ghc28147_0.o /tmp/ghc28147_0/ghc28147_0.c *** Deleting temp dirs: Deleting: /tmp/ghc28147_0

there is no libffi linked there as it's linked in RTS lib already. it's probably not rpath-ed there (for obvious reason) so ldd's output on the test looks:

$ ldd readFile001 libHSbase-4.7.0.0-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/libraries/base/dist-install/build/libHSbase-4.7.0.0-ghc7.7.20130125.so

libHSinteger-gmp-0.5.1.0-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.5.1.0-ghc7.7.20130125.so

libgmp.so.3 => /usr/lib/libgmp.so.3 libHSghc-prim-0.3.1.0-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.1.0-ghc7.7.20130125.so

libHSrts-ghc7.7.20130125.so => /export/home/karel/vcs/ghc-src/ghc-head/rts/dist/build/libHSrts-ghc7.7.20130125.so

libm.so.2 => /lib/libm.so.2 librt.so.1 => /lib/librt.so.1 libdl.so.1 => /lib/libdl.so.1 libc.so.1 => /lib/libc.so.1 libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 libffi.so.6 => (file not found)

And of course for the purpose of the test I've unset LD_LIBRARY_PATH completely: karel@silence:~/vcs/ghc-src/ghc-head/libraries/base/tests/IO$ echo $LD_LIBRARY_PATH

karel@silence:~/vcs/ghc-src/ghc-head/libraries/base/tests/IO$

So that's why I've thought my solution of setting LD_LIBRARY_PATH in runtests is the most easiest one but if you prefer something different just let me know.

Thanks! Karel

On 01/25/13 09:02 AM, Simon Marlow wrote:

...
Hold on a minute. Why do you need to set LD_LIBRARY_PATH? It should be unnecessary because the binaries are linked with -rpath options so they can find their libraries.

Cheers, Simon

On 25/01/13 01:33, David Terei wrote:

...
Committed. Thanks!

On 24 January 2013 13:28, Karel Gardas wrote:

...
This patch follows Windows and Darwin way of setting environment variable to set the file-system paths to GHC's shared libraries. It does the same thing for any other OS, which should be Unix-like OS presumably. This patch fixes a lot of `dyn' tests failures on Solaris which fail with following error message: Failed to open shared library: ld.so.1: T3807-load: fatal: libffi.so.6: open failed: No such file or directory --- driver/runtests.py | 48 ++++++++++++++++++++++++++---------------------- 1 files changed, 26 insertions(+), 22 deletions(-)

diff --git a/driver/runtests.py b/driver/runtests.py index 66e3bf4..16deda6 100644 --- a/driver/runtests.py +++ b/driver/runtests.py @@ -181,28 +181,32 @@ from testlib import *

# On Windows we need to set $PATH to include the paths to all the DLLs # in order for the dynamic library tests to work. -if windows or darwin: - pkginfo = getStdout([config.ghc_pkg, 'dump']) - topdir = config.libdir - for line in pkginfo.split('\n'): - if line.startswith('library-dirs:'): - path = line.rstrip() - path = re.sub('^library-dirs: ', '', path) - path = re.sub('\\$topdir', topdir, path) - if path.startswith('"'): - path = re.sub('^"(.*)"$', '\\1', path) - path = re.sub('\\\\(.)', '\\1', path) - if windows: - if config.cygwin: - # On cygwin we can't put "c:\foo" in $PATH, as : is a - # field separator. So convert to /cygdrive/c/foo instead. - # Other pythons use ; as the separator, so no problem. - path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) - path = re.sub('\\\\', '/', path) - os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) - else: - # darwin - os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) +# if windows or darwin: +pkginfo = getStdout([config.ghc_pkg, 'dump']) +topdir = config.libdir +for line in pkginfo.split('\n'): + if line.startswith('library-dirs:'): + path = line.rstrip() + path = re.sub('^library-dirs: ', '', path) + path = re.sub('\\$topdir', topdir, path) + if path.startswith('"'): + path = re.sub('^"(.*)"$', '\\1', path) + path = re.sub('\\\\(.)', '\\1', path) + if windows: + if config.cygwin: + # On cygwin we can't put "c:\foo" in $PATH, as : is a + # field separator. So convert to /cygdrive/c/foo instead. + # Other pythons use ; as the separator, so no problem. + path = re.sub('([a-zA-Z]):', '/cygdrive/\\1', path) + path = re.sub('\\\\', '/', path) + os.environ['PATH'] = os.pathsep.join([path, os.environ.get("PATH", "")]) + elif darwin: + # darwin + os.environ['DYLD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("DYLD_LIBRARY_PATH", "")]) + else: + # unix + os.environ['LD_LIBRARY_PATH'] = os.pathsep.join([path, os.environ.get("LD_LIBRARY_PATH", "")]) +

global testopts_local testopts_local.x = TestOptions() -- 1.7.3.2

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

Ian Lynagh

7:35 p.m.

Hi Karel, On Fri, Jan 25, 2013 at 08:12:53PM +0100, Karel Gardas wrote:

...

On 01/25/13 09:02 AM, Simon Marlow wrote:

...
Hold on a minute. Why do you need to set LD_LIBRARY_PATH? It should be unnecessary because the binaries are linked with -rpath options so they can find their libraries.

it's simple, every lib is rpath-ed except the libffi. See:

I think the right thing to do is to use rpath to find libffi too. I've reverted the testsuite patch. I'm not sure why it doesn't work already; possibly something isn't right around the getPackageLibraryPath call in DriverPipeline.linkBinary? Thanks Ian

Karel Gardas

26 Jan 26 Jan

1:46 p.m.

Hi Ian, On 01/26/13 01:35 AM, Ian Lynagh wrote:

...

Hi Karel,

On Fri, Jan 25, 2013 at 08:12:53PM +0100, Karel Gardas wrote:

...
On 01/25/13 09:02 AM, Simon Marlow wrote:

...
Hold on a minute. Why do you need to set LD_LIBRARY_PATH? It should be unnecessary because the binaries are linked with -rpath options so they can find their libraries.

it's simple, every lib is rpath-ed except the libffi. See:

I think the right thing to do is to use rpath to find libffi too. I've reverted the testsuite patch.

OK! So before going into DriverPipeline.linkBinary business I'd like to ask you intended behavior of this. The current behavior is: 1) libffi is linked to rts lib, i.e. libHSrts-ghc7.7.*.so 2) libffi is not linked into application binary Now, are both of those points right? Or better do you agree that the implementation will behave in this way? If so, then DriverPipeline.linkBinary does not need to deal with libffi at all probably, but we will need to change a way how we link libHSrts-ghc7.7.*.so (add rpath to libffi there) and we will also need to relink it during the installation step (to add target dir rpath of libffi again). Do you agree with this? Another option might be to link libffi into target application binary with all the rpath machinery in place like is used in case of other haskell/rts libraries. This way, we do not need to change the installation to relink rts lib, but will need to fix DriverPipeline.linkBinary probably to include libffi there. What's your opinion on this? Thanks a lot! Karel

Karel Gardas

2:06 p.m.

One note: I've tested manually relink libHSrts-ghc7.7.20130126.so and added -v -optl=-v -optl=-Wl,-rpath -optl=-Wl,$HOME/vcs/ghc-src/ghc-head/rts/dist/build to its params and as expected this works fine. RTS lib knows where to find libffi and testsuite is happy. Thanks, Karel On 01/26/13 07:46 PM, Karel Gardas wrote:

...

Hi Ian,

On 01/26/13 01:35 AM, Ian Lynagh wrote:

...
Hi Karel,

On Fri, Jan 25, 2013 at 08:12:53PM +0100, Karel Gardas wrote:

...
On 01/25/13 09:02 AM, Simon Marlow wrote:

...
Hold on a minute. Why do you need to set LD_LIBRARY_PATH? It should be unnecessary because the binaries are linked with -rpath options so they can find their libraries.

it's simple, every lib is rpath-ed except the libffi. See:

I think the right thing to do is to use rpath to find libffi too. I've reverted the testsuite patch.

OK! So before going into DriverPipeline.linkBinary business I'd like to ask you intended behavior of this. The current behavior is:

1) libffi is linked to rts lib, i.e. libHSrts-ghc7.7.*.so

2) libffi is not linked into application binary

Now, are both of those points right? Or better do you agree that the implementation will behave in this way? If so, then DriverPipeline.linkBinary does not need to deal with libffi at all probably, but we will need to change a way how we link libHSrts-ghc7.7.*.so (add rpath to libffi there) and we will also need to relink it during the installation step (to add target dir rpath of libffi again). Do you agree with this?

Another option might be to link libffi into target application binary with all the rpath machinery in place like is used in case of other haskell/rts libraries. This way, we do not need to change the installation to relink rts lib, but will need to fix DriverPipeline.linkBinary probably to include libffi there.

What's your opinion on this?

Thanks a lot! Karel

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

4497

Age (days ago)

4499

Last active (days ago)

List overview

Download

8 comments

4 participants

participants (4)

David Terei
Ian Lynagh
Karel Gardas
Simon Marlow