Hello! I can only address a small subset of your comments as I don't usually develop on Windows and hence lack any significant Windows experience... On 2014-09-27 at 23:04:38 +0200, Gintautas Miliauskas wrote: [...]

2. Since the msys2 setup instructions are so simple and linear, perhaps it would be even better to put them in a shellscript and check that in? Then the wikipage would turn into a one-liner.

Fwiw, I recently tried msys2, but after wasting several hours I simply couldn't get it to work in a way where I could ssh (tried Freesshd and Winsshd) into an MSYS2 environemnt. Then I simply installed Cygwin against the recommendation, and then I finally got a working ssh-able environment with a few steps, which roughly where OTTOMH: - Install cygwin (make sure 'wget' and 'openssh' gets installed), - grab https://github.com/transcode-open/apt-cyg and place into /usr/local/bin/ - "cygrunsrv" in order to install sshd service - allow 'sshd.exe' in Windows-firewall settings from now on the graphical Windows console/deskopt isn't required anymore (which was my primary goal). Everything else can be done headless/remote via ssh: - `wget` ghc-7.6.3 windows bindist and unpack to /opt/ghc-7.6.3 - prepend /opt/ghc-7.6.3/bin/ and /opt/ghc-7.6.3/mingw/bin/ to user's PATH via ~/.bash_profile or similiar - follow similiar steps to grab cabal and install alex/happy like for MSYS2; you need to pass `--configure-option=--build=x86_64-w64-mingw32` to cabal, though. - apt-cyg install automake git - clone and build ghc as if it was MSYS2 Long-story short, if it wasn't for Cygwin, I wouldn't have been able to clean up after some of the Windows breakage I caused recently. It'd be interesting though if there was a way to `ssh` directly (with a sane terminal-emulation) into an MSYS2 environment capable of building GHC.

3. Why is ghc-tarballs a git repository? That does not seem very wise. [...] Could we have a stable folder under haskell.org/ to put the files in, to make sure that they never go away, and just wget/curl them from there?

http://thread.gmane.org/gmane.comp.lang.haskell.ghc.devel/4883/focus=4887

8. A broader question: what general approach to ghc on Windows shall we take? [...] I think it would make sense to be consistent with the Linux builds; we don't bundle compilers with those. In that sense msys2 would be like another distribution.

I'd like that...

9. If I understand correctly, one other thing to consider before dropping ghc-tarballs is that Windows ghc still needs GCC utilities (like cpp) to function properly, [...] Fwiw, at least `cpp` could in theory be replaced by `cpphs` [...]

10. Following the idea in (8), I tried to build ghc using the mingw gcc provided by msys2 instead of the one in ghc-tarballs. It was a bit weird. I had to hack configure.ac to disable use of ghc-tarballs and try to use system tools. How about a configure option to enable/disable use of ghc-tarballs?

Fwiw, Cygwin also provides mingw-gcc (cross)compiler packages to install. So the same could work on Cygwin...

11. A build with the host gcc failed. I think the cause is that it is too new (4.9.1, significantly newer than 4.6.3 in ghc-tarballs). The build of the currently checked in GMP (libraries/integer-gmp) fails because a utility used in the build process segfaults. I tried upgrading gmp from 5.0.3 to 6.0.0, and 6.0.0 builds fine by itself but the ghc-specific patch used for 5.0.3 no longer applies (is it still necessary?). Oh brother. One of the advantages of tracking msys2's gcc would be that we would notice such breakage earlier. Shall I open an issue?

The patched in-tree GMP lib isn't necessary anymore with the rewritten integer-gmp2 package (which was one of the motivations to do the rewrite in the first place). [...]

Might be a good idea to put in a guard in the configure script to warn if a cygwin gcc is detected (or add explicit support for it). Actually, looks like there's already a related issue open, although I'm not quite sure what the scope is there (#8842 https://ghc.haskell.org/trac/ghc/ticket/8842, thanks Thomas).

It'd be great if one couldn't only use Cygwin as a `build`-environment, but also as the `host`-environment you compile for.

14. The test runner assumes native Windows Python, but it's only a few small changes away from working fine on the python2 provided by msys2, which would cut another external build dependency. Could someone review and merge my patches (#9604 https://ghc.haskell.org/trac/ghc/ticket/9604, #9626 https://ghc.haskell.org/trac/ghc/ticket/9626)? Thanks.

Fwiw, the test-runner seems to work fine with the Cygwin-provided Python interpreter.

...

3. Why is ghc-tarballs a git repository? That does not seem very wise. [...]

...

Could we have a stable folder under haskell.org/ to put the files in, to make sure that they never go away, and just wget/curl them from there?

http://thread.gmane.org/gmane.comp.lang.haskell.ghc.devel/4883/focus=4887

Hmm, that was a while ago. Whom should I contact to get the files deployed under haskell.org?

Here's a different solution to the 'big binary blobs' problem: * Keep the ghc-tarballs git repository, and add it as a submodule * Make sure it doesn't get cloned by default git config -f .gitmodules submodule.ghc-tarballs.update none * Windows developers run (after initial clone --recursive of the ghc repository, one time): git config submodule.ghc-tarballs.update checkout git submodule update --depth=1 * After that, windows developers run the normal: git submodule update The advantages are: * only the most recent ghc-tarballs commit gets cloned initially * subsequent 'git submodule update' runs will make sure always the most recent version of ghc-tarballs is available * full history of ghc-tarballs is tracked, easier bisecting * no extra scripts needed I don't know how much space overhead git adds. wget-ting just the files themselves might still be faster.

I'm sure we could make git handle the tarballs, but it just seems like the wrong tool for the job. We'd have to use multiple advanced features of git where a simple wget/curl would do. Versioning is also a moot point, since we would embed versions in filenames. In fact, versioning would be easier and nicer when the filenames with versions are in a file on the main repository rather than in a submodule. I was thinking of performing the wget (if necessary) in the Makefile, to further bring down the number of steps that users have to execute for a working build. Any strong objections? Whom should I contact to get some static files deployed in a folder under haskell.org? On Mon, Sep 29, 2014 at 11:40 AM, Thomas Miedema wrote:

...

...

3. Why is ghc-tarballs a git repository? That does not seem very wise. [...]

...

Could we have a stable folder under haskell.org/ to put the files in, to make sure that they never go away, and just wget/curl them from there?

http://thread.gmane.org/gmane.comp.lang.haskell.ghc.devel/4883/focus=4887

Hmm, that was a while ago. Whom should I contact to get the files deployed under haskell.org?

Here's a different solution to the 'big binary blobs' problem:

* Keep the ghc-tarballs git repository, and add it as a submodule * Make sure it doesn't get cloned by default git config -f .gitmodules submodule.ghc-tarballs.update none * Windows developers run (after initial clone --recursive of the ghc repository, one time): git config submodule.ghc-tarballs.update checkout git submodule update --depth=1 * After that, windows developers run the normal: git submodule update

The advantages are: * only the most recent ghc-tarballs commit gets cloned initially * subsequent 'git submodule update' runs will make sure always the most recent version of ghc-tarballs is available * full history of ghc-tarballs is tracked, easier bisecting * no extra scripts needed

I don't know how much space overhead git adds. wget-ting just the files themselves might still be faster.

-- Gintautas Miliauskas

However, overall (not GHC use cases) gcc 4.9.1 still looks more buggy on Windows than 4.8.3.

'Mingw-builds' project (which is now a part of mingw-w64 project and is considered to be an "official" mingw-w64 gcc distribution and is maintained by a man close to Msys2 project) has very nice and complete build of 4.8.3 (64-bit build, for example, is here: http://sourceforge.net/

I also urge to not use "msys2's gcc" phrase because it's possible to install gcc which uses msys2 runtime and generates binaries which use msys2 runtime, but it is absolutely wrong thing. What do we want in fact is to use mingw64 (x86_64-w64-mingw32) or mingw32 (i686-w64-mingw32) toolchains

Hmm, that sounds like an argument against trusting msys2 to provide a sane mingw gcc compiler... Bummer. What kind of bugs did you have in mind? projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/ mingw-builds/4.8.3/threads-posix/seh/). Sounds good. Should we switch to this using this package instead of the rubenvb build? based on and targeting mingw-w64 runtime. Yes, I know (now, after wasting some time on that...). The suggestion was exactly to warn the user if such a type of gcc is detected, because that's almost never what you want. On Sun, Sep 28, 2014 at 9:08 AM, kyra wrote:

On 9/28/2014 1:04 AM, Gintautas Miliauskas wrote:

...

11. A build with the host gcc failed. I think the cause is that it is too new (4.9.1, significantly newer than 4.6.3 in ghc-tarballs). The build of the currently checked in GMP (libraries/integer-gmp) fails because a utility used in the build process segfaults. I tried upgrading gmp from 5.0.3 to 6.0.0, and 6.0.0 builds fine by itself but the ghc-specific patch used for 5.0.3 no longer applies (is it still necessary?). Oh brother. One of the advantages of tracking msys2's gcc would be that we would notice such breakage earlier. Shall I open an issue?

I believe this is not a segfault, it's a GHC RTS' dynamic linker barfing it can't handle duplicated symbols. Since 4.9 gcc (on Windows at least) started to generate .linkonce section for external symbols. I have a patch for this and I successfully built GHC HEAD with the most fresh stock msys2 mingw64 components.

However, overall (not GHC use cases) gcc 4.9.1 still looks more buggy on Windows than 4.8.3. 'Mingw-builds' project (which is now a part of mingw-w64 project and is considered to be an "official" mingw-w64 gcc distribution and is maintained by a man close to Msys2 project) has very nice and complete build of 4.8.3 (64-bit build, for example, is here: http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting% 20Win64/Personal%20Builds/mingw-builds/4.8.3/threads-posix/seh/).

I also urge to not use "msys2's gcc" phrase because it's possible to install gcc which uses msys2 runtime and generates binaries which use msys2 runtime, but it is absolutely wrong thing. What do we want in fact is to use mingw64 (x86_64-w64-mingw32) or mingw32 (i686-w64-mingw32) toolchains based on and targeting mingw-w64 runtime.

Cheers, Kyra

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

-- Gintautas Miliauskas

However, overall (not GHC use cases) gcc 4.9.1 still looks more buggy on Windows than 4.8.3. 'Mingw-builds' project (which is now a part of mingw-w64 project and is considered to be an "official" mingw-w64 gcc distribution and is maintained by a man close to Msys2 project) has very nice and complete build of 4.8.3 (64-bit build, for example, is here: http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting% 20Win64/Personal%20Builds/mingw-builds/4.8.3/threads-posix/seh/).

I am looking into migrating ghc to the newer gcc package linked above. There is also the associated question of what to do with ghc-tarballs. Here's an idea: how about downloading the mingw package directly from sourceforge at configure time with curl/wget? That should work pretty well. I see two potential issues: 1. URL stability. The sourceforge repo is not in our control and files in it could go away any time. The repo seems stable though, with some files from 2011. If we're concerned about this, copying the file to a domain under our control should not be a problem. I did not get any responses about who to contact about that though... 2. Download failing due to internet connectivity issues or missing proxy configuration. This could easily be addressed by printing a message with a URL and a filesystem location to put the file in the case that the download fails. If there are no objections, I'll proceed with whipping up a patch. -- Gintautas Miliauskas

On 2014-10-13 at 10:57:10 +0200, Simon Peyton Jones wrote:

I think the potential difficulty is (1). Maybe they take it down (e.g. they move on to version X so they take down old version Y).

An alternative would be to stash a copy somewhere on GHC’s main web server, and wget that. I’d be more comfortable doing that; less dependence on others.

I guess storing a copy somewhere on https://ghc.haskell.org/ should be ok (I'm hoping Austin may weigh in wrt CDN-related considerations). I'd suggest using it as a fallback location though. I.e. try downloading from the official upstream location, and if that fails (either due to I/O errors and/or unexpected checksum), fallback to using our locally mirrored copy. However, we may need to take into account license issues, such as hosting the source-code as well, if we host binary distributions depending on the licenses involved (I'm not sure if this was ever considered for ghc-tarballs.git to begin with) Cheers, hvr

On 27/09/2014 22:04, Gintautas Miliauskas wrote:

8. A broader question: what general approach to ghc on Windows shall we take? The prebuilt packages currently provided by ghc-tarballs are also covered by msys2's package manager. Why not offload that pain to msys2 then? The advantage here is less maintenance (and automatic upgrades of the toolchain), the disadvantage is that the distribution becomes less stable and msys2 updates could break ghc builds more easily. I think it would make sense to be consistent with the Linux builds; we don't bundle compilers with those. In that sense msys2 would be like another distribution. Of course, we need to also consider if msys2 can be trusted to stick around and stay up to date in the long run. It looks like a relatively new project, so there's some risk.

9. If I understand correctly, one other thing to consider before dropping ghc-tarballs is that Windows ghc still needs GCC utilities (like cpp) to function properly, and so we need to have a prepackaged bundle of binary GCC utilities (and maybe hardcoded paths? not sure) to make that work. On the other hand, a custom-built ghc should work just fine in the msys2 environment which does provide cpp et al., and the additional GCC bundles would perhaps best be owned by, for example, the Haskell Platform project rather than be part of core ghc?

10. Following the idea in (8), I tried to build ghc using the mingw gcc provided by msys2 instead of the one in ghc-tarballs. It was a bit weird. I had to hack configure.ac http://configure.ac to disable use of ghc-tarballs and try to use system tools. How about a configure option to enable/disable use of ghc-tarballs? I also ran into some weird issues, for example, the system ld and nm would not get detected by the configure script correctly. They were found when I explicitly set LD=ld and NM=nm. Weird. Will look into that later. Other than that, there were no major problems, except...

11. A build with the host gcc failed. I think the cause is that it is too new (4.9.1, significantly newer than 4.6.3 in ghc-tarballs). The build of the currently checked in GMP (libraries/integer-gmp) fails because a utility used in the build process segfaults. I tried upgrading gmp from 5.0.3 to 6.0.0, and 6.0.0 builds fine by itself but the ghc-specific patch used for 5.0.3 no longer applies (is it still necessary?). Oh brother. One of the advantages of tracking msys2's gcc would be that we would notice such breakage earlier. Shall I open an issue?

We created ghc-tarballs for stability reasons. In the past, some versions of mingw were broken, so we wanted to ensure that everyone building GHC on Windows was using the same gcc, and that a given build of GHC will ship with a predictable gcc, rather than grabbing whatever is installed. I think it's pretty important that GHC can be installed independently of mingw. That dependency used to be huge source of pain when we had it. Windows is unlike Linux, in that on Linux it's easy to install a working gcc. Many distributions already ship it, and even when they don't, the package manager makes it easy to add gcc as a dependency of GHC so it gets installed automatically. Cheers, Simon

Hi,

All we need is someone to act as convenor/coordinator and we are good to go. Would any of you be willing to play that role?

Indeed, the next thing I was going to ask was about expediting the decision process. I would be happy to try and coordinate a push in Windows matters. There is a caveat though: I don't have any skin in the GHC-on-Windows game, so I will want to move on to other things afterwards. An advantage of having a working group is that you can *decide* things. At

the moment people often wait for GHC HQ to make a decision, and end up waiting a long time. It would be better if a working group was responsible for the GHC-on-Windows build and then if (say) you want to mandate msys2, you can go ahead and mandate it. Well, obviously consult ghc-devs for advice, but you are in the lead. Does that make sense?

Sounds great. The question still remains about making changes to code: is there a particular person with commit rights that we could lean on for code reviews and committing changes to the main repository?

I think an early task is to replace what Neil Mitchell encountered: FIVE different wiki pages describing how to build GHC on Windows. We want just one! (Others can perhaps be marked “out of date/archive” rather than deleted, but it should be clear which is the main choice.)

Indeed, it's a bit of a mess. I intended to shape up the msys2 page to serve as the default, but wanted to see more testing done before before dropping the other pages.

I agree with using msys2 as the main choice. (I’m using it myself.) It may be that Gintautas’s page https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows/MSYS2 is already sufficient. Although I’d like to see it tested by others. For example, I found that it was CRUCIAL to set MSYSYSTEM=MINGW whereas Gintautas’s page says nothing about that.

Are you sure that is a problem? The page specifically instructs to use the msys64_shell.bat script (through a shortcut) that is included in msys2, and that script takes care of setting MSYSTEM=MINGW64, among other important things. Other small thoughts:

· We started including the ghc-tarball stuff because when we relied directly on the gcc that came with msys, we kept getting build failures because the gcc that some random person happened to be using did not work (e..g. they had a too-old or too-new version of msys). By using a single, fixed gcc, we avoided all this pain.

Makes sense. Just curious: why is this less of a problem on GNU/Linux distros compared to msys2? Does msys2 see comparatively less testing, or is it generally more bleeding edge?

· I don’t know what a “rubenvb” build is, but I think you can go ahead and say “use X and Y in this way”. The important thing is that it should be reproducible, and not dependent on the particular Cygwin or gcc or whatever the that user happens to have installed.

A "rubenvb" build is one of the available types of prebuilt binary packages of mingw for Windows. Let's figure out if there is something more mainstream and if we can migrate to that. -- Gintautas Miliauskas

I think I’m fairly behind on the current build process of GHC, but as I do use GHC mainly on Windows, at such a time as you would like to move on to other things, I would certainly throw my hat In the ring. That sounds helpful, thank you. Are we at the point where we could form a GHC-on-Windows Task Force? With its own wiki page on the GHC Trac, and with named participants. (Of course you can drop off again.) But it would be really helpful to have an explicit group who feels a sense of ownership about making sure GHC works well on Windows. At the moment we are reduced to folk memory “I recall that Gintautas did something like that a few months ago”. It sounds as if Tamar would be a willing member. Would anyone else be willing? I’d say that being a member indicates a positive willingness to help others, along with some level of expertise, NOT a promise to drop everything to attend to someone else’s problem. Simon From: lonetiger@gmail.com [mailto:lonetiger@gmail.com] Sent: 09 October 2014 06:04 To: Gintautas Miliauskas; Simon Peyton Jones Cc: Randy Polen; kyra; Marek Wawrzos; Roman Kuznetsov; Neil Mitchell; ghc-devs@haskell.org Subject: Re: Building ghc on Windows with msys2 Hi Gintautas,

Indeed, the next thing I was going to ask was about expediting the decision process. I would be happy to try and coordinate a push in Windows matters. There is a caveat though: I don't have any skin in the GHC-on-Windows game, so I will want to move on to other things afterwards.

I think I’m fairly behind on the current build process of GHC, but as I do use GHC mainly on Windows, at such a time as you would like to move on to other things, I would certainly throw my hat In the ring. Cheers, Tamar From: Gintautas Miliauskasmailto:gintautas.miliauskas@gmail.com Sent: ‎Thursday‎, ‎October‎ ‎2‎, ‎2014 ‎22‎:‎32 To: Simon Peyton Jonesmailto:simonpj@microsoft.com Cc: Randy Polenmailto:randyhaskell@outlook.com, kyramailto:kyrab@mail.ru, Marek Wawrzosmailto:marek.28.93@gmail.com, Tamar Christinamailto:lonetiger@gmail.com, Roman Kuznetsovmailto:kuznero@hotmail.com, Neil Mitchellmailto:ndmitchell@gmail.com, ghc-devs@haskell.orgmailto:ghc-devs@haskell.org Hi,

All we need is someone to act as convenor/coordinator and we are good to go. Would any of you be willing to play that role?

Indeed, the next thing I was going to ask was about expediting the decision process. I would be happy to try and coordinate a push in Windows matters. There is a caveat though: I don't have any skin in the GHC-on-Windows game, so I will want to move on to other things afterwards. An advantage of having a working group is that you can decide things. At the moment people often wait for GHC HQ to make a decision, and end up waiting a long time. It would be better if a working group was responsible for the GHC-on-Windows build and then if (say) you want to mandate msys2, you can go ahead and mandate it. Well, obviously consult ghc-devs for advice, but you are in the lead. Does that make sense? Sounds great. The question still remains about making changes to code: is there a particular person with commit rights that we could lean on for code reviews and committing changes to the main repository? I think an early task is to replace what Neil Mitchell encountered: FIVE different wiki pages describing how to build GHC on Windows. We want just one! (Others can perhaps be marked “out of date/archive” rather than deleted, but it should be clear which is the main choice.) Indeed, it's a bit of a mess. I intended to shape up the msys2 page to serve as the default, but wanted to see more testing done before before dropping the other pages. I agree with using msys2 as the main choice. (I’m using it myself.) It may be that Gintautas’s page https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows/MSYS2 is already sufficient. Although I’d like to see it tested by others. For example, I found that it was CRUCIAL to set MSYSYSTEM=MINGW whereas Gintautas’s page says nothing about that. Are you sure that is a problem? The page specifically instructs to use the msys64_shell.bat script (through a shortcut) that is included in msys2, and that script takes care of setting MSYSTEM=MINGW64, among other important things. Other small thoughts: • We started including the ghc-tarball stuff because when we relied directly on the gcc that came with msys, we kept getting build failures because the gcc that some random person happened to be using did not work (e..g. they had a too-old or too-new version of msys). By using a single, fixed gcc, we avoided all this pain. Makes sense. Just curious: why is this less of a problem on GNU/Linux distros compared to msys2? Does msys2 see comparatively less testing, or is it generally more bleeding edge? • I don’t know what a “rubenvb” build is, but I think you can go ahead and say “use X and Y in this way”. The important thing is that it should be reproducible, and not dependent on the particular Cygwin or gcc or whatever the that user happens to have installed. A "rubenvb" build is one of the available types of prebuilt binary packages of mingw for Windows. Let's figure out if there is something more mainstream and if we can migrate to that. -- Gintautas Miliauskas

Better to continue on ghc-devs; that way others have visibility of what is going on. Simon From: Gintautas Miliauskas [mailto:gintautas.miliauskas@gmail.com] Sent: 09 October 2014 10:29 To: Simon Peyton Jones Cc: Neil Mitchell; Randy Polen; Roman Kuznetsov; lonetiger@gmail.com; ghc-devs@haskell.org; kyra; Marek Wawrzos Subject: RE: Building ghc on Windows with msys2 I'll set up a wikipage this evening. Should we get a mailing list of our own too, or do you think it's best to continue on ghc-devs@? -- Gintautas Miliauskas On Oct 9, 2014 9:52 AM, "Simon Peyton Jones" mailto:simonpj@microsoft.com> wrote: I think I’m fairly behind on the current build process of GHC, but as I do use GHC mainly on Windows, at such a time as you would like to move on to other things, I would certainly throw my hat In the ring. That sounds helpful, thank you. Are we at the point where we could form a GHC-on-Windows Task Force? With its own wiki page on the GHC Trac, and with named participants. (Of course you can drop off again.) But it would be really helpful to have an explicit group who feels a sense of ownership about making sure GHC works well on Windows. At the moment we are reduced to folk memory “I recall that Gintautas did something like that a few months ago”. It sounds as if Tamar would be a willing member. Would anyone else be willing? I’d say that being a member indicates a positive willingness to help others, along with some level of expertise, NOT a promise to drop everything to attend to someone else’s problem. Simon From: lonetiger@gmail.commailto:lonetiger@gmail.com [mailto:lonetiger@gmail.commailto:lonetiger@gmail.com] Sent: 09 October 2014 06:04 To: Gintautas Miliauskas; Simon Peyton Jones Cc: Randy Polen; kyra; Marek Wawrzos; Roman Kuznetsov; Neil Mitchell; ghc-devs@haskell.orgmailto:ghc-devs@haskell.org Subject: Re: Building ghc on Windows with msys2 Hi Gintautas,

Indeed, the next thing I was going to ask was about expediting the decision process. I would be happy to try and coordinate a push in Windows matters. There is a caveat though: I don't have any skin in the GHC-on-Windows game, so I will want to move on to other things afterwards.

I think I’m fairly behind on the current build process of GHC, but as I do use GHC mainly on Windows, at such a time as you would like to move on to other things, I would certainly throw my hat In the ring. Cheers, Tamar From: Gintautas Miliauskasmailto:gintautas.miliauskas@gmail.com Sent: ‎Thursday‎, ‎October‎ ‎2‎, ‎2014 ‎22‎:‎32 To: Simon Peyton Jonesmailto:simonpj@microsoft.com Cc: Randy Polenmailto:randyhaskell@outlook.com, kyramailto:kyrab@mail.ru, Marek Wawrzosmailto:marek.28.93@gmail.com, Tamar Christinamailto:lonetiger@gmail.com, Roman Kuznetsovmailto:kuznero@hotmail.com, Neil Mitchellmailto:ndmitchell@gmail.com, ghc-devs@haskell.orgmailto:ghc-devs@haskell.org Hi,

All we need is someone to act as convenor/coordinator and we are good to go. Would any of you be willing to play that role?

Indeed, the next thing I was going to ask was about expediting the decision process. I would be happy to try and coordinate a push in Windows matters. There is a caveat though: I don't have any skin in the GHC-on-Windows game, so I will want to move on to other things afterwards. An advantage of having a working group is that you can decide things. At the moment people often wait for GHC HQ to make a decision, and end up waiting a long time. It would be better if a working group was responsible for the GHC-on-Windows build and then if (say) you want to mandate msys2, you can go ahead and mandate it. Well, obviously consult ghc-devs for advice, but you are in the lead. Does that make sense? Sounds great. The question still remains about making changes to code: is there a particular person with commit rights that we could lean on for code reviews and committing changes to the main repository? I think an early task is to replace what Neil Mitchell encountered: FIVE different wiki pages describing how to build GHC on Windows. We want just one! (Others can perhaps be marked “out of date/archive” rather than deleted, but it should be clear which is the main choice.) Indeed, it's a bit of a mess. I intended to shape up the msys2 page to serve as the default, but wanted to see more testing done before before dropping the other pages. I agree with using msys2 as the main choice. (I’m using it myself.) It may be that Gintautas’s page https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows/MSYS2 is already sufficient. Although I’d like to see it tested by others. For example, I found that it was CRUCIAL to set MSYSYSTEM=MINGW whereas Gintautas’s page says nothing about that. Are you sure that is a problem? The page specifically instructs to use the msys64_shell.bat script (through a shortcut) that is included in msys2, and that script takes care of setting MSYSTEM=MINGW64, among other important things. Other small thoughts: • We started including the ghc-tarball stuff because when we relied directly on the gcc that came with msys, we kept getting build failures because the gcc that some random person happened to be using did not work (e..g. they had a too-old or too-new version of msys). By using a single, fixed gcc, we avoided all this pain. Makes sense. Just curious: why is this less of a problem on GNU/Linux distros compared to msys2? Does msys2 see comparatively less testing, or is it generally more bleeding edge? • I don’t know what a “rubenvb” build is, but I think you can go ahead and say “use X and Y in this way”. The important thing is that it should be reproducible, and not dependent on the particular Cygwin or gcc or whatever the that user happens to have installed. A "rubenvb" build is one of the available types of prebuilt binary packages of mingw for Windows. Let's figure out if there is something more mainstream and if we can migrate to that. -- Gintautas Miliauskas

Thank you, that is brilliant. Simon From: Gintautas Miliauskas [mailto:gintautas.miliauskas@gmail.com] Sent: 10 October 2014 21:01 To: Simon Peyton Jones Cc: lonetiger@gmail.com; Randy Polen; kyra; Marek Wawrzos; Roman Kuznetsov; Neil Mitchell; ghc-devs@haskell.org Subject: Re: Building ghc on Windows with msys2 Hey, I have created https://ghc.haskell.org/trac/ghc/wiki/WindowsTaskForce, and added the two people from whom I heard a confirmation that they want to be on the list. Please edit the page and add yourself if you should be on that list. Feel free to hack the page up and add additional info as you see fit. On Thu, Oct 9, 2014 at 9:51 AM, Simon Peyton Jones mailto:simonpj@microsoft.com> wrote: I think I’m fairly behind on the current build process of GHC, but as I do use GHC mainly on Windows, at such a time as you would like to move on to other things, I would certainly throw my hat In the ring. That sounds helpful, thank you. Are we at the point where we could form a GHC-on-Windows Task Force? With its own wiki page on the GHC Trac, and with named participants. (Of course you can drop off again.) But it would be really helpful to have an explicit group who feels a sense of ownership about making sure GHC works well on Windows. At the moment we are reduced to folk memory “I recall that Gintautas did something like that a few months ago”. It sounds as if Tamar would be a willing member. Would anyone else be willing? I’d say that being a member indicates a positive willingness to help others, along with some level of expertise, NOT a promise to drop everything to attend to someone else’s problem. Simon From: lonetiger@gmail.commailto:lonetiger@gmail.com [mailto:lonetiger@gmail.commailto:lonetiger@gmail.com] Sent: 09 October 2014 06:04 To: Gintautas Miliauskas; Simon Peyton Jones Cc: Randy Polen; kyra; Marek Wawrzos; Roman Kuznetsov; Neil Mitchell; ghc-devs@haskell.orgmailto:ghc-devs@haskell.org Subject: Re: Building ghc on Windows with msys2 Hi Gintautas,

Indeed, the next thing I was going to ask was about expediting the decision process. I would be happy to try and coordinate a push in Windows matters. There is a caveat though: I don't have any skin in the GHC-on-Windows game, so I will want to move on to other things afterwards.

I think I’m fairly behind on the current build process of GHC, but as I do use GHC mainly on Windows, at such a time as you would like to move on to other things, I would certainly throw my hat In the ring. Cheers, Tamar From: Gintautas Miliauskasmailto:gintautas.miliauskas@gmail.com Sent: ‎Thursday‎, ‎October‎ ‎2‎, ‎2014 ‎22‎:‎32 To: Simon Peyton Jonesmailto:simonpj@microsoft.com Cc: Randy Polenmailto:randyhaskell@outlook.com, kyramailto:kyrab@mail.ru, Marek Wawrzosmailto:marek.28.93@gmail.com, Tamar Christinamailto:lonetiger@gmail.com, Roman Kuznetsovmailto:kuznero@hotmail.com, Neil Mitchellmailto:ndmitchell@gmail.com, ghc-devs@haskell.orgmailto:ghc-devs@haskell.org Hi,

All we need is someone to act as convenor/coordinator and we are good to go. Would any of you be willing to play that role?

Indeed, the next thing I was going to ask was about expediting the decision process. I would be happy to try and coordinate a push in Windows matters. There is a caveat though: I don't have any skin in the GHC-on-Windows game, so I will want to move on to other things afterwards. An advantage of having a working group is that you can decide things. At the moment people often wait for GHC HQ to make a decision, and end up waiting a long time. It would be better if a working group was responsible for the GHC-on-Windows build and then if (say) you want to mandate msys2, you can go ahead and mandate it. Well, obviously consult ghc-devs for advice, but you are in the lead. Does that make sense? Sounds great. The question still remains about making changes to code: is there a particular person with commit rights that we could lean on for code reviews and committing changes to the main repository? I think an early task is to replace what Neil Mitchell encountered: FIVE different wiki pages describing how to build GHC on Windows. We want just one! (Others can perhaps be marked “out of date/archive” rather than deleted, but it should be clear which is the main choice.) Indeed, it's a bit of a mess. I intended to shape up the msys2 page to serve as the default, but wanted to see more testing done before before dropping the other pages. I agree with using msys2 as the main choice. (I’m using it myself.) It may be that Gintautas’s page https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows/MSYS2 is already sufficient. Although I’d like to see it tested by others. For example, I found that it was CRUCIAL to set MSYSYSTEM=MINGW whereas Gintautas’s page says nothing about that. Are you sure that is a problem? The page specifically instructs to use the msys64_shell.bat script (through a shortcut) that is included in msys2, and that script takes care of setting MSYSTEM=MINGW64, among other important things. Other small thoughts: • We started including the ghc-tarball stuff because when we relied directly on the gcc that came with msys, we kept getting build failures because the gcc that some random person happened to be using did not work (e..g. they had a too-old or too-new version of msys). By using a single, fixed gcc, we avoided all this pain. Makes sense. Just curious: why is this less of a problem on GNU/Linux distros compared to msys2? Does msys2 see comparatively less testing, or is it generally more bleeding edge? • I don’t know what a “rubenvb” build is, but I think you can go ahead and say “use X and Y in this way”. The important thing is that it should be reproducible, and not dependent on the particular Cygwin or gcc or whatever the that user happens to have installed. A "rubenvb" build is one of the available types of prebuilt binary packages of mingw for Windows. Let's figure out if there is something more mainstream and if we can migrate to that. -- Gintautas Miliauskas -- Gintautas Miliauskas

On 2014-10-07 at 15:04:18 +0200, cg wrote: [...]

I guess the current two build server are all Cygwin based, they are failing at the same permission issue at early building stage, it prevents checking out the real problem. It seems msys2 (or msys) seldom has such issue.

Btw, while I finally managed to get a pure MSYS2 environment to work with a manually started sshd.exe, it'd be great if somebody could point me to instructions on how to setup e.g. cygrunsrv+sshd in order to have sshd.exe startup automatically on boot-up *and* have sshd.exe be able to log into more than just one single account (currently if I start sshd, I can only log into the very same account which started sshd) Having such a setup would be really useful to provide the GHC development improve its infrastructure as well to help reduce the regular windows-arch failures we're seeing so often. Cheers, hvr

2014-10-07 17:02 GMT+02:00 Páli Gábor János :

2014-10-07 15:04 GMT+02:00 cg :

...

I guess the current two build server are all Cygwin based, they are failing at the same permission issue at early building stage, it prevents checking out the real problem. It seems msys2 (or msys) seldom has such issue.

For what it is worth, I have been witnessing those permission issues with msys2 on my Windows builders. They worked (more or the less) fine until September 24, but suddenly, something has changed (not on my side) and all I got those errors since.

Looks like the commit with the Cabal submodule update causes this [1]. The revision before that commit still builds fine on my system, while everything else after that commit dies early at build [2]. Is this only me, has anybody else experienced the problem? Perhaps I am doing something wrong? I do not remember to see any related "heads-up" message on the list, like I should update any of the build-time dependencies. [1] http://git.haskell.org/ghc.git/commit/4b648be19c75e6c6a8e6f9f93fa12c7a4176f0... [2] http://haskell.inf.elte.hu/builders/windows-x86-head/56/10.html

2014-10-09 9:12 GMT+02:00 Herbert Valerio Riedel :

I didn't see this issue on a newly setup MSYS2 environment either. How old is your MSYS environment? (And what filesystem & windows version are you running?)

I use a 64-bit Windows 7 SP1 (6.1.7601) with both the 32-bit (i686) and 64-bit (x86_64) msys2 installed on a regular NTFS partition. The i686 instance is of July 4, 2014, and the x86_64 instance is of February 16, 2014. I have GHC 7.6.3 and GCC 4.5.2 (32-bit) and GCC 4.6.3 (64-bit) installed, respectively.

By the way, I've noticed that ghc occasionally segfaults during Windows builds, like this: "inplace/bin/ghc-stage1.exe" -hisuf hi -osuf o -hcsuf hc -static -H64m -O0 -fasm -hide-all-packages -i -iutils/hsc2hs/. -iutils/hsc2hs/dist-install/build -iutils/hsc2hs/dist-install/build/autogen -Iutils/hsc2hs/dist-install/build -Iutils/hsc2hs/dist-install/build/autogen -optP-include -optPutils/hsc2hs/dist-install/build/autogen/cabal_macros.h -package-key base_ESD4aQEEWwsHtYJVc1BwtJ -package-key conta_ChF4XLXB9JmByIycPzerow -package-key direc_HU5aFxMIQNwGQFzisjuinu -package-key filep_34DFDFT9FVD9pRLLgh8IdQ -package-key proce_7ZlAbRkwiRO8qgXx3NNP0G -XHaskell98 -XCPP -XForeignFunctionInterface -no-user-package-db -rtsopts -odir utils/hsc2hs/dist-install/build -hidir utils/hsc2hs/dist-install/build -stubdir utils/hsc2hs/dist-install/build -c utils/hsc2hs/./HSCParser.hs -o utils/hsc2hs/dist-install/build/HSCParser.o utils/hsc2hs/ghc.mk:16: recipe for target 'utils/hsc2hs/dist-install/build/HSCParser.o' failed make[1]: *** [utils/hsc2hs/dist-install/build/HSCParser.o] Segmentation fault make[1]: *** Deleting file 'utils/hsc2hs/dist-install/build/HSCParser.o' The errors are not deterministic at all. Any idea what's happening? Any suggestions for debugging this? On Wed, Oct 8, 2014 at 1:21 AM, Gintautas Miliauskas < gintautas.miliauskas@gmail.com> wrote:

I've cleaned up the main Windows build https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows page, moved MSYS2 instructions there, moved out legacy instructions and added backlinks / warnings / redirects. It would be great if someone could go through, verify the instructions and make sure there are no loose ends or misleading wikipages.

On Wed, Oct 1, 2014 at 12:25 AM, Simon Peyton Jones

...

wrote:

...

Gintautas, and other folk building GHC on Windows,

There has been some activity on the “GHC on Windows” front, which is great.

Some time ago I wrote:

I would love it if you guys formed a GHC-on-Windows Task Force, who tried to make sure that the Windows experience was always good. At the moment we have lots of Windows users but very few who are willing to help make it work, the recipients of this email being honourable exceptions.

but nothing really happened. Maybe this time it can! Possible member of such a task force are:

· Gintautas Miliauskas gintautas.miliauskas@gmail.com

· kyra kyrab@mail.ru

· Marek Wawrzos marek.28.93@gmail.com

· Tamar Christina

· Roman Kuznetsov

· Randy Polen

All we need is someone to act as convenor/coordinator and we are good to go. Would any of you be willing to play that role?

An advantage of having a working group is that you can *decide* things. At the moment people often wait for GHC HQ to make a decision, and end up waiting a long time. It would be better if a working group was responsible for the GHC-on-Windows build and then if (say) you want to mandate msys2, you can go ahead and mandate it. Well, obviously consult ghc-devs for advice, but you are in the lead. Does that make sense?

I think an early task is to replace what Neil Mitchell encountered: FIVE different wiki pages describing how to build GHC on Windows. We want just one! (Others can perhaps be marked “out of date/archive” rather than deleted, but it should be clear which is the main choice.)

I agree with using msys2 as the main choice. (I’m using it myself.) It may be that Gintautas’s page https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows/MSYS2 is already sufficient. Although I’d like to see it tested by others. For example, I found that it was CRUCIAL to set MSYSYSTEM=MINGW whereas Gintautas’s page says nothing about that.

Other small thoughts:

· We started including the ghc-tarball stuff because when we relied directly on the gcc that came with msys, we kept getting build failures because the gcc that some random person happened to be using did not work (e..g. they had a too-old or too-new version of msys). By using a single, fixed gcc, we avoided all this pain.

· I don’t know what a “rubenvb” build is, but I think you can go ahead and say “use X and Y in this way”. The important thing is that it should be reproducible, and not dependent on the particular Cygwin or gcc or whatever the that user happens to have installed.

Simon

*From:* ghc-devs [mailto:ghc-devs-bounces@haskell.org] *On Behalf Of *Gintautas Miliauskas *Sent:* 15 September 2014 21:58 *To:* ghc-devs@haskell.org *Subject:* Building ghc on Windows with msys2

Hello,

I have been messing around a little bit with building GHC from source on Windows, and found the msys2 wikipage https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows/MSYS2 quite useful, but somewhat outdated. Quite a few steps in those instructions are no longer necessary and can be omitted. I am working on cleaning up that wikipage right now and should be done in a day or two.

I've found a recent email http://www.haskell.org/pipermail/ghc-devs/2014-September/006251.html in the middle of updating the wikipage about other people planning to do the same, so I thought I'd shoot an email to make sure that work is not being duplicated.

msys2 seems to be in good shape and should probably be promoted to the primary suggested method to build ghc on Windows. Let's look into that once the new build instructions have been proofread and verified.

Best regards,

-- Gintautas Miliauskas

-- Gintautas Miliauskas

-- Gintautas Miliauskas

On Tue, Oct 7, 2014 at 11:25 AM, cg wrote:

On 9/16/2014 4:57 AM, Gintautas Miliauskas wrote:

...

msys2 seems to be in good shape and should probably be promoted to the primary suggested method to build ghc on Windows. Let's look into that once the new build instructions have been proofread and verified.

I am trying to build the latest code but without success.

First, I encounter twe 'Ambiguous occurrence' issues:

libraries\binary\src\Data\Binary\Builder\Base.hs:110:15: Ambiguous occurrence "empty" It could refer to either "Data.Binary.Builder.Base.empty", defined at libraries\binary\src\Data\Binary\Builder\Base.hs:124:1 or "GHC.Base.empty", imported from "GHC.Base" at libraries\binary\src\Data\Binary\Builder\Base.hs:84:1-15

and the same with Prelude.foldr in this Base.hs.

I hide 'empty' and 'foldr' at importing point and the code compiles.

Has anyone see the same issues?

Ugh, this is some fallout I thought we had fixed, but apparently not. I'll fix it shortly, thanks.

Then, I get this 'out of memory' error at 'building final phase' "inplace/bin/ghc-stage1.exe" ... -c libraries/Cabal/Cabal/./Language/Haskell/Extension.hs ghc-stage1.exe: out of memory

I am building on Windows 8.1 with 16G ram, with 'BuildFlavour = quick'. And I see ghc-stage1 is using 16G+ memory in Windows Task Manager.

Why does ghc-stage1.exe use so much memory?

Wow, I thought we fixed this one too! Please see this bug: https://ghc.haskell.org/trac/ghc/ticket/9630 What GHC commit are you using? Are your submodules all up to date? In particular, if 'binary' is not up to date, even if the rest of your tree is, you'll see this problem.

-- cg

_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

On 2014-10-08 at 04:28:50 +0200, cg wrote: [...]

After cloning ghc repository, I switch every sub-module to Master (it is usually HEAD) branch.

Why are you doing that? :-) [...]

Now after cloning ghc repository, if I don't switch to any branch -- 'git branch' will show all submodules are detached -- the build will succeed.

Well, that's how you're supposed to work with submodules in ghc.git[1]

So why the Master/HEAD branches don't have the latest code?

Generally, the "master" branch refers to the latest upstream code, which is not always supposed to work with GHC HEAD (yet). And if the package is not owned by GHC HQ, you are not allowed to push changes to "master" anyway (as it'd be automatically overwritten by the automatic Git mirror job) Take Cabal for example, we have an automatic mirror-job that keeps Cabal's "master" branch synced to the state of the github.com/ However, we only update the gitlink for Cabal in ghc.git every couple of weeks to Cabal's latest "master" tip commit, as it has the potential to affect performance numbers or simply be temporarily in a broken state wrt GHC HEAD. Then there's Haddock for which it was recently decided to let upstream development progress decoupled from GHC HEAD's API changes, and have GHC HEAD simply use its own branch 'ghc-head' to diverge from upstream until shortly before a GHC release is close (at which point Haddock will converge again). Fwiw, 'git submodule update --remote utils/haddock' will track the "ghc-head" branch in this case. Finally, "binary" is a case where we needed a patch merged into binary, but couldn't wait for the "binary" upstream to merge the pull-request, as it was blocking GHC HEAD development. So that's why we temporarily are on a "ghc-head" branch, which will be switched away from again as soon as "binary"'s upstream "master" branch can be used again with GHC HEAD. And then there's also the potential case when we need to temporarily rollback a submodule update; then we don't necessarily need to 'git revert' commits inside that submodule, but we simply just reset the pointed-to submodule commit to an older commit. I hope this sheds a bit of light on the situation. Then there's also [1] which may provide further pointers. [1]: https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Git/Submodules Cheers, hvr