Rich Neswold wrote:

The NetBSD Project has a different opinion and strongly encourages the use of RPATHs.

NetBSD is of course entitled to its opinions. Indeed, this reinforces my point that use of RPATH should not be cooked into portable tools, because it's a policy that varies across ELF-using platforms.

On Wed, Jun 13, 2007 at 03:00:04PM +0200, Clemens Fruhwirth wrote:

...

...

3) Running programs

...

Let's see how libtool handles this situation.

I would recommend against following libtool's lead in this area. Libtool's fondness for cooking RPATH into binaries makes it very difficult to deal with, because it's quite common for those binaries to get installed and distributed, RPATH and all. RPATH should only be used by a user who knows they have a large-calibre weapon pointed at their foot.

Did I understand that correctly that you don't want to see binaries with rpath's pointing to install directories such as /usr/lib/gcc-6.6? So, this forces us to use a wrapper in all cases.

Please think seriously about mangling the names of Haskell libraries to include version information and dropping them in $PREFIX/lib with every other language's libraries. Haskell is not special, and users expect libraries to be in /usr/lib. No wrapper needed, no RPATH needed, as far as I can see no fanciness at all. Stefan

Rich Neswold wrote:

Actually, Haskell libraries ought to be placed in /usr/local/lib (or /usr/pkg/lib for systems that use the Package System: http://www.pkgsrc.org).

They should go in $(libdir), as Simon suggests. If a particular OS wants to override libdir to put them somewhere else, that's then easy to arrange. Stefan is also correct that the version of GHC used should be cooked into the name of the shared library. So a library named foo would be installed as libfoo-ghc661.so.1, or something similar (using the soname would work too). This will allow system package managers to automatically resolve dependencies safely, and to keep copies of a shared library built by different versions of GHC around without them clashing.

On Wed, Jun 13, 2007 at 11:39:27AM -0700, Bryan O'Sullivan wrote:

Stefan is also correct that the version of GHC used should be cooked into the name of the shared library. So a library named foo would be installed as libfoo-ghc661.so.1, or something similar (using the soname

A minor point, but libghc661-foo.so.1 is probably better so they all clump together in directory listings etc. Thanks Ian

Clemens Fruhwirth wrote:

...

Libtool's fondness for cooking RPATH into binaries makes it very difficult to deal with, because it's quite common for those binaries to get installed and distributed, RPATH and all. RPATH should only be used by a user who knows they have a large-calibre weapon pointed at their foot.

Did I understand that correctly that you don't want to see binaries with rpath's pointing to install directories such as /usr/lib/gcc-6.6?

That's right.

So, this forces us to use a wrapper in all cases.

Not necessarily. Many systems provide a global mechanism to manage the paths that ld.so searches for shared objects. This is the standard on Linux, for example (/etc/ld.so.conf). For systems that don't provide this mechanism, one possibility would be to provide an option that cooks the rpath in.

On Wed, 2007-06-13 at 15:24 -0700, Stefan O'Rear wrote:

On Wed, Jun 13, 2007 at 11:33:36AM -0700, Bryan O'Sullivan wrote:

Or better yet, put them directly in one of the LD_LIBRARY_PATH dirs. $PREFIX/lib/ghc-$VERSION is a relic of the static library system and IMO shouldn't be duplicated.

Hmm.. well what Felix does is: (a) The bulk of the system lives in $PREFIX/lib/felix$VERSION (b) Libraries are called *.flx or *.h or *.a or *.so and have NO version encoding (c) Executables are compiled and run using a bash script The script is the only program installed in /usr/bin This script selects the install to use, defaulting to the most recent. You can override the install directory on the script command line or with an environment variable. The bash script uses LD_LIBRARY_PATH, etc, and command line switches to the executables, to ensure a coherent 'assembly' of components are used together. If you opt for static linkage, the resulting executable is of course independent of the version. All the standalone binaries require non-defaulted switches to select where components other than load-time shared libs live. This system supports multiple installations and also multiple cross compilation. IMHO: $PREFIX/lib/ghc-$VERSION style isn't a relic. It is the current ridiculous versioning of shared libraries which is a relic of even older faulty design which unfortunately Debian and other package installers copied. In particular, package components should always live in the same directory tree, related by package/versions, and NEVER get split up into lib/bin/include etc. In fact that is a relic of archaic dog slow Unix directory searching, and was a performance hack. It was never the correct design to split packages into directories by function, and the Unix directory tree is only capable of supporting C programs .. and it isn't really suitable for C either. The rules for package managers like Debian are that a component library split into /usr/include, /usr/lib MUST have a man page, and it MUST be a separately usable C/C++-callable component. If the only user of these libraries is the Haskell or other system, and the interfaces aren't documented, then the libraries must NOT be placed in /usr/lib. In other words, unless there is a distinct separable Debian package for the library it must NOT go in /usr/lib. The way most modern languages work, each version has a number of utterly incompatible components. When you use Felix, Ocaml, Mlton, or Haskell version xx.yy you can't use any other version of any of the libraries .. and probably can't use any user libraries either. It's likely the ABI (application binary interface) changes with each version, the set of library entry points changes in some detail, or whatever .. if not, why is a new version being released?? This doesn't happen (usually) with C/C++, but it does sometimes: the C++ ABI changed for Linux recently and it broke Debian for months and months while everything got upgraded. SO in my view, the only thing you might consider sharing between (advanced language of your choice) versions is the source code of software written in the standardised language -- but never, NEVER the support libraries, run time, or compilers. FYI: this is a particularly nasty problem for Ocaml, since the ABI changes with every patch. The debian-ocaml team has to rebuild and upload binaries to the package server every time the compiler is changed .. and end users have to recompile every program they wrote (for bytecode .. for native code there's no dynamic loading anyhow), and every library (for both bytecode and native code). This isn't necessary with Felix because it *defines* execution in terms of source code, not in terms of binaries, which are regarded as mere cached values, and managed automatically (i.e. rebuilt whenever there's a version mismatch). So for something like Haskell I'd recommend the system be split in two parts: (a) the 'system' which provides a distinct installation for every version all in a single directory tree (b) libraries written in standardised Haskell 98 or whatever are separate packages of source code, and are separately maintained and installed Any caching of partial compilations of the standard source libraries should be automatic. Option (b) is rather hard to organise without redesiging your tool chain to work entirely in terms of source code .. but I recommend that anyhow -- Haskell semantics are defined in terms of sources and the 'average' user should know about anything else: it's a basic principle of abstraction. -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net

On Fri, 2007-06-15 at 17:28 +0100, Ian Lynagh wrote:

Hi John,

On Thu, Jun 14, 2007 at 09:17:16AM +1000, skaller wrote:

...

The rules for package managers like Debian are that a component library split into /usr/include, /usr/lib MUST have a man page,

I'm not sure where you're getting that from? Debian policy: http://www.debian.org/doc/debian-policy/ch-docs.html#s12.1 says Each [...] function *should* have an associated manual page (my emphasis), and doesn't say anything about where the function is in the file system.

(it would be great if someone were to tweak haddock to generate suitable manpages, incidentally).

...

and it MUST be a separately usable C/C++-callable component.

Have you got a reference for that? I can't see it in a quick skim of http://www.debian.org/doc/debian-policy/ch-sharedlibs.html and the FHS.

No, it's the concept, not the letter of the Debian Law. When you install a shared library, it is installed as a publically accessible component to be shared. That implies the interface is both documented and callable. A library with undocumented symbols which are private to one application such as GHC, or even a set of applications generated by GHC, has no business in a directory intended for everyone to be able to use it .. especially if the interface and ABI change due to internal restructuring. One way to measure this is: if you removed GHC and applications, and there are (necessarily) no users of the remaining library package .. the library package shouldn't be in the global public place (/usr/lib+include etc). In that case, the lib is an intrinsic part of GHC, and should be in a GHC master package (sub)directory. The problem with Debian is FSH standard doesn't really account for packages like programming languages. If you look you see for example Python lives in /usr/lib/python and ocaml lives in /usr/lib/ocaml. That's an abuse forced by a grossly inadequate directory model. Ocaml and Python aren't libraries. Solaris does this more correctly IMHO: packages live in a vendor specific place, part of the /opt directory tree. Here 'opt' means 'optional', that is, not part of the core operating system and utilities required to maintain it. Anyhow, system like GHC, Ocaml, Python, Felix, etc, just do NOT fit into the C model: bin/ lib/ include/ with or without any versioning. They have bytecode, configuration, and many other kinds of 'object' and 'development source' and 'library' files, some of which may happen to be 'shared libraries' but that's irrelevant. All those things need to be managed in a (compiler-)system specific way. The 'specific' way for C and Unix OS tools is the Debian FSH ... it shouldn't be used for anything else. Note that the Ocaml team is currently grappling with a second nasty problem: Ocaml 3.10 uses a source incompatible pre-processor (camlp4). So now, not only will a non-Debian installed Ocaml library or bytecode executable be broken by an upgrade until the user recompiles from source (the usual situation for Ocaml) but now all sources using camlp4 macros are broken until the end user edits them. So the team is almost forced to support installation of non-conflicting separate versions now, just to allow migration. Once you start versioning stuff .. splitting files into subdirectories like bin/ lib/ include/ is a nightmare. Note that the 'include' part of that utterly fails for C code anyhow.. so even the basic model is flawed for the very kind of code it was designed to support. The bottom line is Debian FSH is archaic and work should be done to eliminate all user level packages from it: only core OS packages should be allowed in /usr. IMHO the best workaround for this problem is to use a thunk/ driver script in /usr/bin and put all the real stuff in a single version specific compile install directory whose location is entirely irrelevant. This is more or less what gcc does, and the gcc model works very well I think. Multiple gcc versions, including cross compiler, can be installed and 'just work' with all the right bits glued together. -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net

On Fri, 2007-06-15 at 19:40 -0500, Spencer Janssen wrote:

On Sat, 16 Jun 2007 08:21:50 +1000 skaller wrote:

...

One way to measure this is: if you removed GHC and applications, and there are (necessarily) no users of the remaining library package .. the library package shouldn't be in the global public place (/usr/lib+include etc).

As I understand it, the entire point of this effort (shared libraries in GHC) is to allow dynamically linked Haskell executables. In this case, applications outside the GHC toolchain will in fact depend on these shared objects. As a concrete case, a binary darcs package could be a user of libghc66-base.so and libghc66-mtl.so -- with no dependencies on the GHC compiler package itself.

Does this pass your litmus test?

Yes, it passes the separability test. My darcs wouldn't run otherwise! And versioning the library filename as above is a good idea too. Felix adds _dynamic for shared libs and _static for static link archives to ensure the Linux linker doesn't get confused. However, the libs still aren't fully public if the interfaces are only private details of the GHC tool chain. Hmmm. -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net