Question about binary distributions

Hi all, GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll focus the dicussion below on these three platforms. The binary distributions for Linux and macOS are designed to be unpacked, then the user types ./configure && make install. This is not the case for Windows. On all platforms it's possible to create "relocatable" installations, such that GHC doesn't really care where it's installed, and commands will still work if the install directory changes location on the filesystem. So my question is, why do we have a ./configure step on Linux and macOS? Why could we not have bindists for all platforms that work like the Windows one? I.e. a binary distribution that you just unpack, in any directory of your choice, without any configuration or installation step.

Hi Mathieu,
you can! See http://hackage.mobilehaskell.org/; it's been one of the
design goals I had when I was hacking on hadrian. The whole configure
&& make install shenanigans were just too much.
Initially I wanted to drop that outright, but was convinced that
configure and make install is something distributions want (that
install into different locations) as well as some people who prefer to
install
into ghc into custom locations.
GHC has for a while now had relocatable support on our major
platforms, which means you don't even need that wrapper script anymore
as long as the bin and lib folder are next to each other and you
operating system can find the path of the executable. I'm told AIX
can't if the executable is a symlink.
I'm all for having "unpack and run" bindists with an optional
configure && make install phase for those who want it. As we are
moving over to hadrian as the primary build system I think this should
work,
but might have regressed?
Cheers,
Moritz
On Fri, Aug 7, 2020 at 9:50 PM Mathieu Boespflug
Hi all,
GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll focus the dicussion below on these three platforms. The binary distributions for Linux and macOS are designed to be unpacked, then the user types ./configure && make install. This is not the case for Windows.
On all platforms it's possible to create "relocatable" installations, such that GHC doesn't really care where it's installed, and commands will still work if the install directory changes location on the filesystem. So my question is, why do we have a ./configure step on Linux and macOS? Why could we not have bindists for all platforms that work like the Windows one? I.e. a binary distribution that you just unpack, in any directory of your choice, without any configuration or installation step.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

"Mathieu Boespflug"
Hi all,
GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll focus the dicussion below on these three platforms. The binary distributions for Linux and macOS are designed to be unpacked, then the user types ./configure && make install. This is not the case for Windows.
On all platforms it's possible to create "relocatable" installations, such that GHC doesn't really care where it's installed, and commands will still work if the install directory changes location on the filesystem. So my question is, why do we have a ./configure step on Linux and macOS? Why could we not have bindists for all platforms that work like the Windows one? I.e. a binary distribution that you just unpack, in any directory of your choice, without any configuration or installation step.
There are a few reasons: * Relocatable GHC builds have only been supported for only a few releases now and only under the Hadrian build system, which is not currently used to produce our binary distributions (hopefully this will change for 9.2). * On Windows we have the luxury of having a very well-controlled environment as we rely on essentially nothing from the host system. We provide our own mingw toolchain, statically link against libc, and have no additional dynamic dependencies. By contrast, on Linux we have to deal with a much larger configuration space: * several linkers, each with their own bugs * several C compilers, supporting various subsets of functionality and quirks (e.g. some distributions enable -pie by default, others do not) * various LLVM packaging schemes Since it would be quite expensive to probe the toolchain characteristics on every compiler invocation, we rather do this once in the configure script during bindist installation and package the result in the installed `settings` file. * On Linux we may have additional dynamic dependencies (e.g. libdw, numactl) which we check for during configuration time, lest the user be faced with an unsightly linker error if they happen to be missing a library. In principle we could perhaps avoid the need for many of these checks by creating one binary distribution per operating system distribution. However, we will first need to move to Hadrian to build our binary distributions. Cheers, - Ben

Per https://gitlab.haskell.org/ghc/ghc/issues/17191 I do hope to break up our configure script soon.[1] Then the bindist will need not ship the "entire" configure script, but just what is necessary to fill in the settings file(s) which have that information Ben mentions. I think that will improve the optics of the situation a bit; for example, I don't think the reduced bindist configure script should need to worry about directories at all since GHC is relocatable (when built by Hadrian). John [1]: I will be able to resume work on that once I get to the bottom of https://gitlab.haskell.org/ghc/ghc/merge_requests/1102. All help greatly appreciated! On 8/7/20 11:15 AM, Ben Gamari wrote:
"Mathieu Boespflug"
writes: Hi all,
GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll focus the dicussion below on these three platforms. The binary distributions for Linux and macOS are designed to be unpacked, then the user types ./configure && make install. This is not the case for Windows.
On all platforms it's possible to create "relocatable" installations, such that GHC doesn't really care where it's installed, and commands will still work if the install directory changes location on the filesystem. So my question is, why do we have a ./configure step on Linux and macOS? Why could we not have bindists for all platforms that work like the Windows one? I.e. a binary distribution that you just unpack, in any directory of your choice, without any configuration or installation step. There are a few reasons:
* Relocatable GHC builds have only been supported for only a few releases now and only under the Hadrian build system, which is not currently used to produce our binary distributions (hopefully this will change for 9.2).
* On Windows we have the luxury of having a very well-controlled environment as we rely on essentially nothing from the host system. We provide our own mingw toolchain, statically link against libc, and have no additional dynamic dependencies.
By contrast, on Linux we have to deal with a much larger configuration space:
* several linkers, each with their own bugs
* several C compilers, supporting various subsets of functionality and quirks (e.g. some distributions enable -pie by default, others do not)
* various LLVM packaging schemes
Since it would be quite expensive to probe the toolchain characteristics on every compiler invocation, we rather do this once in the configure script during bindist installation and package the result in the installed `settings` file.
* On Linux we may have additional dynamic dependencies (e.g. libdw, numactl) which we check for during configuration time, lest the user be faced with an unsightly linker error if they happen to be missing a library.
In principle we could perhaps avoid the need for many of these checks by creating one binary distribution per operating system distribution. However, we will first need to move to Hadrian to build our binary distributions.
Cheers,
- Ben
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

It's only relocatable given some assumptions which are violated by
various distributions (AIX was already mentioned; and the bin and lib
directories may not be next to each other with some distributions'
preferred configurations). Basically the configure mechanism gives us
some flexibility not otherwise available, without requiring a full
build from source (which is fairly involved and requires an existing
ghc).
On 8/7/20, John Cotton Ericson
Per https://gitlab.haskell.org/ghc/ghc/issues/17191 I do hope to break up our configure script soon.[1] Then the bindist will need not ship the "entire" configure script, but just what is necessary to fill in the settings file(s) which have that information Ben mentions.
I think that will improve the optics of the situation a bit; for example, I don't think the reduced bindist configure script should need to worry about directories at all since GHC is relocatable (when built by Hadrian).
John
[1]: I will be able to resume work on that once I get to the bottom of https://gitlab.haskell.org/ghc/ghc/merge_requests/1102. All help greatly appreciated!
On 8/7/20 11:15 AM, Ben Gamari wrote:
"Mathieu Boespflug"
writes: Hi all,
GHC currently has 3 tier-1 platforms: Linux, macOS and Windows. I'll focus the dicussion below on these three platforms. The binary distributions for Linux and macOS are designed to be unpacked, then the user types ./configure && make install. This is not the case for Windows.
On all platforms it's possible to create "relocatable" installations, such that GHC doesn't really care where it's installed, and commands will still work if the install directory changes location on the filesystem. So my question is, why do we have a ./configure step on Linux and macOS? Why could we not have bindists for all platforms that work like the Windows one? I.e. a binary distribution that you just unpack, in any directory of your choice, without any configuration or installation step. There are a few reasons:
* Relocatable GHC builds have only been supported for only a few releases now and only under the Hadrian build system, which is not currently used to produce our binary distributions (hopefully this will change for 9.2).
* On Windows we have the luxury of having a very well-controlled environment as we rely on essentially nothing from the host system. We provide our own mingw toolchain, statically link against libc, and have no additional dynamic dependencies.
By contrast, on Linux we have to deal with a much larger configuration space:
* several linkers, each with their own bugs
* several C compilers, supporting various subsets of functionality and quirks (e.g. some distributions enable -pie by default, others do not)
* various LLVM packaging schemes
Since it would be quite expensive to probe the toolchain characteristics on every compiler invocation, we rather do this once in the configure script during bindist installation and package the result in the installed `settings` file.
* On Linux we may have additional dynamic dependencies (e.g. libdw, numactl) which we check for during configuration time, lest the user be faced with an unsightly linker error if they happen to be missing a library.
In principle we could perhaps avoid the need for many of these checks by creating one binary distribution per operating system distribution. However, we will first need to move to Hadrian to build our binary distributions.
Cheers,
- Ben
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
-- brandon s allbery kf8nh allbery.b@gmail.com
participants (5)
-
Ben Gamari
-
Brandon Allbery
-
John Cotton Ericson
-
Mathieu Boespflug
-
Moritz Angermann