Hi Manuel, thanks for the link! Ahh yes lovely RPATH (this reminds me, I should look to verify if I can make GHC link recursively.) The conf files came up on #ghc yesterday as well. Maybe we need some $home, $sysroot or similar marker to make them relocatable. I'll probably take a crack at this today. Not sure how far I get though. The primary motivation is that packaging the cross compilers in a neat way looks like layering patched upon patches onto the configure script. And in light of Hadrian, we might just want to package up a directory and not deal with the binary-dist logic where we can? Cheers, Moritz

On Oct 24, 2017, at 9:45 AM, Manuel M T Chakravarty wrote:

Hi Moritz,

Haskell for Mac is fully relocatable (as every Mac users expects this of a Mac app).

Unfortunately, it is not just the libdir in the wrapper scripts that’s in the way of a relocatable GHC. Additional problems are (1) the (at least, as of 8.0) still not macOS-friendly RPATHs in dynamic libraries and (2) the absolute paths in package .conf files.

In Haskell for Mac, I use two arguably hacky solutions to those two issues. I rewrite all RPATHs in .dylibs to be relative and I have got a little tool that relocates .conf files. I’d certainly prefer a cleaner solution!

Those aspects of Haskell for Mac are actually in a separate framework (GHC.framework) that wraps GHC and some other tools. This is all in a public repo if you like to have a look

https://github.com/haskellformac/GHCframework

I haven’t put a license on it, but I’d be happy to make it all BSD3. It is of course all highly Mac specific and requires Xcode to build for all the code signing, sandboxing, etc.

Cheers, Manuel

...

Am 23.10.2017 um 17:04 schrieb Moritz Angermann :

Hi,

while trying to make the binary-distribution logic work for cross compilers. I've come wonder, how hard it could be to make GHC relocatable. (e.g. just unpack the distribution, and use the compiler from the `bin` folder within).

Right now this does not work due to the need for the path to package.conf, and this is hardcoded in the wrapper script to provide the proper libdir to ghc via -B[1]. Supposedly this is not an issue on Windows, as a relative path is common on windows and finding the location of the executable can be done safely. Or that's at least how I understand[1].

For macOS there is the haskell-platform, and ghc-dot-app[2]

From [3], it sounds like automake is a build, and not a packaging system, and the binary dist usecase with configure is not really a standard use case.

So why do I bring this up NOW? Apart from me trying to make cross compiler binary distributions working? The reason is that we are also trying to move towards hadrian, and by "starting from scratch", maybe we have a chance to reconsider how we do things?

I must admit, I'm quite happy to use packages like llvm, by just downloading a package and adding the `bin` path to my $PATH.

There is however one thing that the current configure appraoch does, which I think is quite noteworthy (apart from setting $prefix). That is, it does check for compilers and sets them accordingly, which might be desirable.

Cheers, Moritz

-- [1]: https://ghc.haskell.org/trac/ghc/wiki/Building/Installing [2]: https://ghc.haskell.org/trac/ghc/wiki/Building/MacOSX/Installer [3]: https://lists.gnu.org/archive/html/automake/2006-10/msg00005.html _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

On Tue, Oct 24, 2017 at 5:02 AM, Moritz Angermann < moritz.angermann@gmail.com> wrote:

However, I am now again at the point where I start hacking on the build system, while Hadrian is imminent. And this is quite depressing

Realistically, while Hadrian going into the tree may be imminent, as I understand it Hadrian becoming the primary build system --- or even a viable alternative build system --- is not. It's more of a repo logistics speed bump. Just let it go for now. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Discussion I've been seeing is Hadrian will not be ready for production use for 8.4. Pulling it in tree is scheduled for 8.4 mostly to make it easier to keep up to date with other changes and to make it easier to work on and test the various missing pieces: as I understand it, Hadrian can't yet deal with all of the various platform special cases, etc. that an actual release requires. On Tue, Oct 24, 2017 at 8:02 PM, Manuel M T Chakravarty < chak@justtesting.org> wrote:

Why are you saying that?

I think, Moritz is right. Hadrian is supposed to be the build system for 8.4. Adding new functionality for cross-compilation to the old build system is frustrating.

Manuel

Brandon Allbery :

On Tue, Oct 24, 2017 at 5:02 AM, Moritz Angermann < moritz.angermann@gmail.com> wrote:

...

However, I am now again at the point where I start hacking on the build system, while Hadrian is imminent. And this is quite depressing

Realistically, while Hadrian going into the tree may be imminent, as I understand it Hadrian becoming the primary build system --- or even a viable alternative build system --- is not. It's more of a repo logistics speed bump. Just let it go for now.

-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Hi, just to elaborate a bit. We *can* build cross compiler, and cross compile ghc with the current make based build system. I'm not certain how well packaging binary distributions of cross compiled ghc's work. I've only run into several when trying to build binary distributions for cross compilers (compiler host != compiler target). This is also where the relocatable topic came up. If the build system would build ghc in such a way that I could just take the stage1/stage2/stageN folder and relocate it into it's final place. Building binary distributions would be a simple as packaging up that folder. If we want to do some additional configuration on the install host, we could add an automake script, but it wouldn't be strictly necessary. For the case of cross compilers, I'm pretty much sure, I do want almost none of the current distrib configure script. My toolchain is locked down pretty hard, so I know the tools I *need* to use. Anything that the configure script at install time could mess up is just going to result in more issues down the line. Now as mentioned I was able to get most of the relocatable logic sorted by dropping the wrapper script, and have ghc determine it's topdir on it's own (for mac and linux) by reusing the codepaths already in place for windows. The package db *does* understand $topdir and ${pkgroot}. Again reusing the $topdir logic, present for windows, allows relocatable package databases. Now the final issue that came up is dynamic libraries, which on macOS to some degree encode their location. However at build time the layout of libraries is sufficiently different from the layout of libraries once installed. Which would require to do some additional path manipulation with the insall_name_tool at install time. To make this easier, it would be preferable to have the same layout at build time as it will be at install time. Another issue I have with the current build system is, that everything is built *in tree*. This means that a) if cleaning doesn't work properly, I might have some stale data lurking around and b) I am unable to build multiple configurations from the same source tree (or modifications thereof) without resorting to some commit/push/pull on local copies of the source tree, or wiping out the source tree for each build. Therefore, after trying to relocate build artifacts in the current build system such that build and install layouts are more similar. I hereby declare defeat. Not only would this change the way the current build system works quite a bit, it's als pretty hard to refactor the current aged build system in that way, and I believe it would result in a number of additional bugs. And finally, even if that change would make it into GHC, Hadrian would have to be adapted to it as well. I have now started trying to graft this different layout ontop of Hadrian. If this works out as I hope. I believe it would drastically simplify the installation rules as well as binary distribution rules in Hadrian. It might also provide me with some knowledge about Hadrian, and how much I like/dislike it over the current make system. Maybe this is the direction we need to take to make Hadrian a viable option for the build system. Otherwise I fear Hadrian will never make it into ghc, if the current build system keeps evolving and Hadrian fails to keep up. Ideally we'd probably have features in Hadrian that the current build system is lacking, which make Hadrian the attractive alternative. E.g. moving Hadrian to "can do X better" instead of "can also build GHC, but doesn't have all the features that the current build system has". Cheers, Moritz

On Oct 25, 2017, at 8:12 AM, Manuel M T Chakravarty wrote:

That doesn’t mean it can’t be used for cross-compilation once it is in the tree.

...

Am 25.10.2017 um 11:06 schrieb Brandon Allbery :

Discussion I've been seeing is Hadrian will not be ready for production use for 8.4. Pulling it in tree is scheduled for 8.4 mostly to make it easier to keep up to date with other changes and to make it easier to work on and test the various missing pieces: as I understand it, Hadrian can't yet deal with all of the various platform special cases, etc. that an actual release requires.

On Tue, Oct 24, 2017 at 8:02 PM, Manuel M T Chakravarty wrote: Why are you saying that?

I think, Moritz is right. Hadrian is supposed to be the build system for 8.4. Adding new functionality for cross-compilation to the old build system is frustrating.

Manuel

...

Brandon Allbery :

On Tue, Oct 24, 2017 at 5:02 AM, Moritz Angermann wrote: However, I am now again at the point where I start hacking on the build system, while Hadrian is imminent. And this is quite depressing

Realistically, while Hadrian going into the tree may be imminent, as I understand it Hadrian becoming the primary build system --- or even a viable alternative build system --- is not. It's more of a repo logistics speed bump. Just let it go for now.

-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

-- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Am 24.10.2017 um 20:02 schrieb Moritz Angermann :

Hi *,

so I've given this a stab[1]. While getting the wrapper requirement, and the .conf files sorted was rather easy. (We already have getExecutablePath, and the relocatable logic for windows. As well as $topdir and ${pkgroot} support for paths in .conf files.)

Awesome!

Now as Manuel laid out the dynamic libraries situation is a bit annoying. Specifically as the ghc-stage2 we build *in-tree*, and the libraries have a completely different layout from the final install location. And as such setting @rpath, and @loader_path, is quite tricky, as essentially the installation is not a relocation of the stage1 or stage2 compiler.

Yes, that’s why I took the hacky short-cut of running the normal install procedure and then patching the libs up in place with ’install_name_tool’. (BTW, a not too widely known tool that makes messing with dylibs, RPATHs, and signatures way nicer than otool is <http://newosxbook.com/tools/jtool.html http://newosxbook.com/tools/jtool.html> — the output is so much more readable.)

However, I am now again at the point where I start hacking on the build system, while Hadrian is imminent. And this is quite depressing.

I hear you. Cheers, Manuel

Cheers, Moritz

-- [1]: https://phabricator.haskell.org/D4121

...

On Oct 24, 2017, at 9:54 AM, Moritz Angermann wrote:

Hi Manuel,

thanks for the link! Ahh yes lovely RPATH (this reminds me, I should look to verify if I can make GHC link recursively.) The conf files came up on #ghc yesterday as well. Maybe we need some $home, $sysroot or similar marker to make them relocatable.

I'll probably take a crack at this today. Not sure how far I get though. The primary motivation is that packaging the cross compilers in a neat way looks like layering patched upon patches onto the configure script. And in light of Hadrian, we might just want to package up a directory and not deal with the binary-dist logic where we can?

Cheers, Moritz

...

On Oct 24, 2017, at 9:45 AM, Manuel M T Chakravarty wrote:

Hi Moritz,

Haskell for Mac is fully relocatable (as every Mac users expects this of a Mac app).

Unfortunately, it is not just the libdir in the wrapper scripts that’s in the way of a relocatable GHC. Additional problems are (1) the (at least, as of 8.0) still not macOS-friendly RPATHs in dynamic libraries and (2) the absolute paths in package .conf files.

In Haskell for Mac, I use two arguably hacky solutions to those two issues. I rewrite all RPATHs in .dylibs to be relative and I have got a little tool that relocates .conf files. I’d certainly prefer a cleaner solution!

Those aspects of Haskell for Mac are actually in a separate framework (GHC.framework) that wraps GHC and some other tools. This is all in a public repo if you like to have a look

https://github.com/haskellformac/GHCframework

I haven’t put a license on it, but I’d be happy to make it all BSD3. It is of course all highly Mac specific and requires Xcode to build for all the code signing, sandboxing, etc.

Cheers, Manuel

...

Am 23.10.2017 um 17:04 schrieb Moritz Angermann :

Hi,

while trying to make the binary-distribution logic work for cross compilers. I've come wonder, how hard it could be to make GHC relocatable. (e.g. just unpack the distribution, and use the compiler from the `bin` folder within).

Right now this does not work due to the need for the path to package.conf, and this is hardcoded in the wrapper script to provide the proper libdir to ghc via -B[1]. Supposedly this is not an issue on Windows, as a relative path is common on windows and finding the location of the executable can be done safely. Or that's at least how I understand[1].

For macOS there is the haskell-platform, and ghc-dot-app[2]

From [3], it sounds like automake is a build, and not a packaging system, and the binary dist usecase with configure is not really a standard use case.

So why do I bring this up NOW? Apart from me trying to make cross compiler binary distributions working? The reason is that we are also trying to move towards hadrian, and by "starting from scratch", maybe we have a chance to reconsider how we do things?

I must admit, I'm quite happy to use packages like llvm, by just downloading a package and adding the `bin` path to my $PATH.

There is however one thing that the current configure appraoch does, which I think is quite noteworthy (apart from setting $prefix). That is, it does check for compilers and sets them accordingly, which might be desirable.

Cheers, Moritz

-- [1]: https://ghc.haskell.org/trac/ghc/wiki/Building/Installing [2]: https://ghc.haskell.org/trac/ghc/wiki/Building/MacOSX/Installer [3]: https://lists.gnu.org/archive/html/automake/2006-10/msg00005.html _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs