Hello all, I've been spending some of my winter break trying my hand at compiling GHC, with a mind to hopefully contributing down the line. I've got it working, but I ran into a few things along the way that I figure might be worth fixing and/or documenting. In the approximate order I encountered them: - The first pacman mirror on the list bundled with MSYS2 is down, with the result that every download pacman makes takes ~10sec longer than it should. It downloads a lot, so that really adds up - but it's easy to fix, just "pacman -Sy pacman-mirrors" before doing anything else with it. Is that worth mentioning on the wiki? I was thinking a line on https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows could be helpful. - That page mentions "If you see errors related to fork(), try closing and reopening the shell" - I've determined that you can reliably avoid that problem by following the instructions at http://sourceforge.net/p/msys2/wiki/MSYS2%20installation/#iii-updating-packa..., ie by running "pacman --needed -S bash pacman msys2-runtime", then closing & re-opening the MSYS shell, before you tell pacman to install the GHC prerequisite packages. - A minor point: I found it helpful to include "man-db" in the list of packages to install - without it, "git help" breaks down with " failed to exec 'man'". - I note "./sync-all --help" says, under "Flags", that "--windows also clones the ghc-tarballs repository (enabled by default on Windows)", and I've confirmed that default behaviour experimentally - but https://ghc.haskell.org/trac/ghc/wiki/Building/GettingTheSources tells you to manually clone ghc-tarballs when on Windows. Is that line on the wiki obsolete, or am I overlooking something? - And finally, the big one: cabal and/or ghc-pkg put some files outside the MSYS root directory, and caused me no end of trouble in doing so... I made a bit of a mess at one point, and tried to fix it by starting over completely from scratch. I expected uninstalling & reinstalling MSYS to achieve this (it deletes its root directory when you uninstall it), but that left me with a huge pile of errors when I tried to run "cabal install -j --prefix=/usr/local alex happy", of the form "Could not find module `...': There are files missing in the `...' package". I noticed that the cabal output made reference to "C:\Users\Martin\AppData\Roaming\cabal\", so tried moving that out of the way, but it only made the problem worse. I did figure it out eventually: in addition to that directory, "%APPDATA%\cabal", there were also files left over in "%APPDATA%\ghc". Once I removed that directory as well, things started working again - but it took me a lot of time & frustration to get there. I'm not entirely sure, but I think the copy of Cabal I already had from installing the Platform may also have been storing files in those directories, in which case this process completely mangled them - which isn't great. It seems to me that, ideally, the "build GHC inside MSYS" procedure would keep itself entirely inside the MSYS directory structure: if it were wholly self-contained, you'd know where everything is, and it couldn't break anything outside. As far as I can tell, the only breach is those two directories courtesy of Cabal, so I didn't think it would be too difficult - but none of the things I've tried (the --package-db cabal flag, a custom cabal --config-file, setting the GHC_PACKAGE_PATH environment variable, maybe some others I've forgotten) had the desired effect. Is it possible? Is it even a good idea? If that's just how it has to be, I feel like there should be an obvious note somewhere for the sake of the next person to trip over it. I'd be happy to amend the wiki for any/all of the first four points, if people think it'd be appropriate; I'm not sure at all what to do about the last one. Any thoughts? - Martin
Hello Martin, Here's just some minor additional context information... On 2015-01-01 at 19:01:53 +0100, Martin Foster wrote: [...]
- I note "./sync-all --help" says, under "Flags", that "--windows also clones the ghc-tarballs repository (enabled by default on Windows)", and I've confirmed that default behaviour experimentally - but https://ghc.haskell.org/trac/ghc/wiki/Building/GettingTheSources tells you to manually clone ghc-tarballs when on Windows. Is that line on the wiki obsolete, or am I overlooking something?
Somewhat related: when https://phabricator.haskell.org/D339 is landed, we can finally forget about having to clone that objectionable ghc-tarballs repo... [...]
I noticed that the cabal output made reference to "C:\Users\Martin\AppData\Roaming\cabal\", so tried moving that out of the way, but it only made the problem worse. I did figure it out eventually: in addition to that directory, "%APPDATA%\cabal", there were also files left over in "%APPDATA%\ghc". Once I removed that directory as well, things started working again - but it took me a lot of time & frustration to get there.
That's btw because Cabal/GHC uses `getAppUserDataDirectory "cabal"` and `getAppUserDataDirectory "ghc"` respectively... Cheers, hvr
On Thu, Jan 1, 2015 at 6:42 PM, Herbert Valerio Riedel
I noticed that the cabal output made reference to "C:\Users\Martin\AppData\Roaming\cabal\", so tried moving that out of the way, but it only made the problem worse. I did figure it out eventually: in addition to that directory, "%APPDATA%\cabal", there were also files left over in "%APPDATA%\ghc". Once I removed that directory as well, things started working again - but it took me a lot of time & frustration to get there.
That's btw because Cabal/GHC uses `getAppUserDataDirectory "cabal"` and `getAppUserDataDirectory "ghc"` respectively...
Hrm. It seems the behaviour of getAppUserDataDirectory is decided at compile-time, by way of `#if defined(mingw32_HOST_OS)`: http://hackage.haskell.org/package/directory/docs/src/System-Directory.html#... So, given the "build GHC on Windows" procedure involves downloading Cabal as a binary, it could only be changed by using a different binary. And much as I might be able to argue it'd make sense for getAppUserDataDirectory to use $HOME instead of %APPDATA% when in MSYS, changing it would be a backwards-compatibility-breaking change to System.Directory. Well, I suppose, theoretically, it could do something like... use the value of some special environment variable if it's set at runtime, otherwise retain the existing behaviour. But that doesn't feel like it'd be good practice to me - too hacky. So I guess that'll be staying as it is, then. Good to know why it's like that, though - it makes a lot more sense now. Thanks for the pointer! - Martin
On 1. 1. 2015 19:01, Martin Foster wrote:
Hello all,
I've been spending some of my winter break trying my hand at compiling GHC, with a mind to hopefully contributing down the line.
I've got it working, but I ran into a few things along the way that I figure might be worth fixing and/or documenting. In the approximate order I encountered them:
* The first pacman mirror on the list bundled with MSYS2 is down, with the result that every download pacman makes takes ~10sec longer than it should. It downloads a lot, so that really adds up - but it's easy to fix, just "pacman -Sy pacman-mirrors" before doing anything else with it. Is that worth mentioning on the wiki? I was thinking a line on https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows could be helpful.
This is an unfortunate, but temporary situation. The next MSYS2 installer will come with updated mirror lists. I don't know what's the policy on including this kind of information on the wiki.
* That page mentions "If you see errors related to fork(), try closing and reopening the shell" - I've determined that you can reliably avoid that problem by following the instructions at http://sourceforge.net/p/msys2/wiki/MSYS2%20installation/#iii-updating-packa..., ie by running "pacman --needed -S bash pacman msys2-runtime", then closing & re-opening the MSYS shell, before you tell pacman to install the GHC prerequisite packages.
This may be true for the GHC guide, but AFAIK if you decide to install other packages, you may still encounter fork errors. The installation process is taken care of by updating bash, pacman and the runtime separately, but subsequent invocations of MSYS2 programs could still fail due to newly installed MSYS2 libraries (if any).
* And finally, the big one: cabal and/or ghc-pkg put some files outside the MSYS root directory, and caused me no end of trouble in doing so...
I made a bit of a mess at one point, and tried to fix it by starting over completely from scratch. I expected uninstalling & reinstalling MSYS to achieve this (it deletes its root directory when you uninstall it), but that left me with a huge pile of errors when I tried to run "cabal install -j --prefix=/usr/local alex happy", of the form "Could not find module `...': There are files missing in the `...' package".
I noticed that the cabal output made reference to "C:\Users\Martin\AppData\Roaming\cabal\", so tried moving that out of the way, but it only made the problem worse. I did figure it out eventually: in addition to that directory, "%APPDATA%\cabal", there were also files left over in "%APPDATA%\ghc". Once I removed that directory as well, things started working again - but it took me a lot of time & frustration to get there.
I'm not entirely sure, but I think the copy of Cabal I already had from installing the Platform may also have been storing files in those directories, in which case this process completely mangled them - which isn't great.
It seems to me that, ideally, the "build GHC inside MSYS" procedure would keep itself entirely inside the MSYS directory structure: if it were wholly self-contained, you'd know where everything is, and it couldn't break anything outside. As far as I can tell, the only breach is those two directories courtesy of Cabal, so I didn't think it would be too difficult - but none of the things I've tried (the --package-db cabal flag, a custom cabal --config-file, setting the GHC_PACKAGE_PATH environment variable, maybe some others I've forgotten) had the desired effect. Is it possible? Is it even a good idea?
If that's just how it has to be, I feel like there should be an obvious note somewhere for the sake of the next person to trip over it.
I had problems with this also, so I definitely support mentioning these two on the wiki. If we ever get to having a ghc package for MSYS2, it will use $HOME instead of $APPDATA, but that won't actually help with the problem of MSYS2 re-install not cleaning everything the build left behind. -- David Macek
participants (3)
-
David Macek -
Herbert Valerio Riedel -
Martin Foster