Hi Roman, Yes, what Adam said is accurate: we're working on integrating Windows into our build pipeline on Phabricator. As well as that, Gabor's bots (at http://haskell.inf.elte.hu/builders) also build on Windows. You can see the build results on the ghc-builds@haskell.org mailing list. But we can always use more help! GHC on Windows has a small amount of hackers, so any amount of help will make a small difference. On Tue, Oct 21, 2014 at 1:38 PM, Adam Sandberg Eriksson wrote:

Hello,

There is a lot of building infrastructure and I believe most of it is listed on [1]. For example the nightly builders [2] where there are indeed 2 windows buildmachines working (but they seem to fail to build currently).

I believe there is ongoing work on adding windows machines to Harbormaster [3] for validating each commit as well as patches submitted to Phabricator.

Regards, Adam Sandberg Eriksson

[1]: https://ghc.haskell.org/trac/ghc/wiki/Infrastructure [2]: http://haskell.inf.elte.hu/builders/ [3]: https://ghc.haskell.org/trac/ghc/wiki/Phabricator/Harbormaster

On Tue, Oct 21, 2014, at 07:58 PM, Roman Kuznetsov wrote:

...

Hello *,

As I am still new in this, I will ask.

Were there any attempt to automate GHC build process on Windows with some kind of CI engine, like Hudson, Jenkins, etc.?

It seems to be rather helpful to publish build results if not after each commit, but at least once every day.

WDYT?

-- Sincerely yours, Roman Kuznetsov _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

---

ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

-- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/

Hi Páli, that error is reminiscent of issues when trying to move a file that is open (and implicitly locked) by another process. I have seen some sporadic issues like that in the build unfortunately (never tracked them down to the root cause though), Maybe it's something silly like one process not having enough time to clean up and close the file before the next command tries to move it? I'd add a sleep before the mv and see if that helps. Does the build proceed if you try "make" without cleaning the repository, or does it hang again at the same spot? On Sat, Oct 25, 2014 at 1:26 PM, Páli Gábor János wrote:

2014-10-25 2:03 GMT+02:00 Austin Seipp :

...

As well as that, Gabor's bots (at http://haskell.inf.elte.hu/builders) also build on Windows.

Unfortunately, my builders are still suffering from the sudden breakage induced by a Cabal library update on September 24 [1]. That is, they are unable to complete the build due to some ghc-cabal failure right after bootstrapping [2]. As a result, I basically suspended the builders until I could do something with this. Yes, I have just checked it, the situation is still the same.

Curiously, I do not know about others who are experiencing the same problem, however, the revisions before the referenced commit (mostly) build just fine.

As of yet, I have not had the time and chance to even attempt to fix this. I have already answered Herbert the version of the build environment (Windows 7 SP1, MinGW from July 4 (32 bit) and February 16 (64 bit), with GHC 7.6.3) -- I do not even know if that was considered old or problematic. I am not also sure if the developers involved in the aforementioned change are aware of this issue and what their opinion on this is. I admit that I have not submitted a ticket on this, perhaps I shall.

Probably I shall also experiment with moving to a newer version of the toolchain per the recently revamped Windows build instructions in the meantime.

[1] http://git.haskell.org/ghc.git/commit/4b648be19c75e6c6a8e6f9f93fa12c7a4176f0... [2] http://haskell.inf.elte.hu/builders/windows-x86-head/56/10.html _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

-- Gintautas Miliauskas

Can you try running the offending command with -v to see which step breaks? I tried running it locally under strace but did not see any file renames either. On Tue, Oct 28, 2014 at 6:37 PM, Páli Gábor János wrote:

2014-10-26 18:17 GMT+01:00 Gintautas Miliauskas :

...

Maybe it's something silly like one process not having enough time to clean up and close the file before the next command tries to move it? I'd add a sleep before the mv and see if that helps.

I am far from understanding all the details of the problem, but I believe the move operation originates from the ghc-cabal binary. For what it is worth, I could not even find the place in the sources where the referenced MoveFileEx is called:

"inplace/bin/ghc-cabal.exe" configure libraries/binary dist-boot "" --with-ghc="/ghc-7.6.3/bin/ghc.exe" --with-ghc-pkg="/ghc-7.6.3/bin/ghc-pkg"

--package-db=C:/msys32/home/ghc-builder/work/builder/tempbuild/build/libraries/bootstrapping.conf --disable-library-for-ghci --enable-library-vanilla --enable-library-for-ghci --disable-library-profiling --disable-shared --with-hscolour="/c/Users/ghc-builder/AppData/Roaming/cabal/bin/HsColour" --configure-option=CFLAGS=" -U__i686 -march=i686 -fno-stack-protector " --configure-option=LDFLAGS=" " --configure-option=CPPFLAGS=" " --gcc-options=" -U__i686 -march=i686 -fno-stack-protector " --constraint "binary == 0.7.1.0" --constraint "Cabal == 1.21.1.0" --constraint "hpc == 0.6.0.2" --constraint "bin-package-db == 0.0.0.0" --constraint "hoopl == 3.10.0.2" --constraint "transformers == 0.4.1.0" --with-gcc="C:/msys32/ghc-7.6.3/lib/../mingw/bin/gcc.exe" --configure-option=--with-cc="C:/msys32/ghc-7.6.3/lib/../mingw/bin/gcc.exe" --with-ar="C:/msys32/ghc-7.6.3/lib/../mingw/bin/ar.exe" --with-alex="/c/Users/ghc-builder/AppData/Roaming/cabal/bin/alex" --with-happy="/c/Users/ghc-builder/AppData/Roaming/cabal/bin/happy" Configuring binary-0.7.1.0... ghc-cabal.exe: dist-boot\setup-config3736.tmp: MoveFileEx "dist-boot\\setup-config3736.tmp" "dist-boot\\setup-config": permission denied (Access is denied.) libraries/binary/ghc.mk:3: recipe for target 'libraries/binary/dist-boot/package-data.mk' failed

...

Does the build proceed if you try "make" without cleaning the repository, or does it hang again at the same spot?

It is fully reproducible by the subsequent make(1) invocations as well, that is, yes, it hangs at the same spot.

2014-10-28 17:18 GMT+01:00 Gintautas Miliauskas :

...

Any luck with getting the Windows continuous builds back up and running?

No, not yet. But I will keep posted on the results.

...

I'd be willing to try to help out if remote access is available

I think this could be arranged.

...

although the build environment should be updated first in case that's the source of the trouble.

Right, I will contact you off-list about this once I moved to toolchain versions to the ones recommended at the Windows build page.

-- Gintautas Miliauskas

Thanks for pushing this forward. I wonder what's going on with DeleteFile. What is the step that's failing? Can you post the log? I also wonder why this issue is not arising on other Windows machines... On Thu, Oct 30, 2014 at 12:18 AM, Páli Gábor János wrote:

2014-10-28 21:49 GMT+01:00 Gintautas Miliauskas :

...

Can you try running the offending command with -v to see which step breaks?

I have tried it, even together with building the GHC sources with a recent toolchain, but I did not get much forward.

...

I tried running it locally under strace but did not see any file renames either.

Although, I think I managed to find the place where some renaming happens. That is `writeFileAtomic` in Cabal's Distribution.Simple.Utils module [1]. The bin-package-db library has a patched version of this function [2] that has a workaround for Windows. After incorporating this change in Cabal, I was able to pass the previously problematic point in the build. Unfortunately, this was not enough for the complete build, as a similar error (with DeleteFile that time) was raised.

[1] https://github.com/haskell/cabal/blob/master/Cabal/Distribution/Simple/Utils... [2] https://github.com/ghc/ghc/blob/master/libraries/bin-package-db/GHC/PackageD...

-- Gintautas Miliauskas

Without knowing much about the surrounding code, I would try two things here. 1. Is it possible that the file was opened by for writing and not closed because of lazy I/O? 2. Since the error is completely deterministic, you could try "freezing" evaluation right before the crash (either by a breakpoint in ghci or just adding a time delay / console read on the code), and then using a utility app http://serverfault.com/questions/1966/how-do-you-find-what-process-is-holdin... to check which process is keeping the file open, whether it's the same process or something else. If any of the involved paths is a directory, this could be #8482 https://ghc.haskell.org/trac/ghc/ticket/8482, although that does not seem to be the case. On Thu, Oct 30, 2014 at 7:13 PM, Páli Gábor János wrote:

2014-10-30 16:24 GMT+01:00 Gintautas Miliauskas :

...

I wonder what's going on with DeleteFile. What is the step that's failing?

Basically it happens at the same point, that is, at the "configure" phase but at the ghc-prim package. Note that the previously mentioned workaround has a "removeFile" action [1], I guess the failure of that triggers the DeleteFile exception.

...

Can you post the log?

"inplace/bin/ghc-cabal.exe" configure libraries/ghc-prim dist-install "" --with-ghc="C:/msys64/home/ghc-builder/ghc/inplace/bin/ghc-stage1.exe" --with-ghc-pkg="C:/msys64/home/ghc-builder/ghc/inplace/bin/ghc-pkg.exe" --flag=include-ghc-prim --disable-library-for-ghci --enable-library-vanilla --enable-library-for-ghci --enable-library-profiling --disable-shared --configure-option=CFLAGS=" -U__i686 -march=i686 -fno-stack-protector " --configure-option=LDFLAGS=" " --configure-option=CPPFLAGS=" " --gcc-options=" -U__i686 -march=i686 -fno-stack-protector " --with-gcc="C:/msys64/home/ghc-builder/ghc/inplace/mingw/bin/gcc.exe" --with-ld="C:/msys64/home/ghc-builder/ghc/inplace/mingw/bin/ld.exe"

--configure-option=--with-cc="C:/msys64/home/ghc-builder/ghc/inplace/mingw/bin/gcc.exe" --with-ar="/usr/bin/ar" --with-alex="/usr/local/bin/alex" --with-happy="/usr/local/bin/happy" Configuring ghc-prim-0.3.1.0... ghc-cabal.exe: DeleteFile "dist-install\\setup-config": permission denied (The process cannot access the file because it is being used by another process.) libraries/ghc-prim/ghc.mk:4: recipe for target 'libraries/ghc-prim/dist-install/package-data.mk' failed make[1]: *** [libraries/ghc-prim/dist-install/package-data.mk] Error 1 Makefile:71: recipe for target 'all' failed

...

I also wonder why this issue is not arising on other Windows machines...

As the comment in the workaround goes, it has a "Big fat hairy race condition". Therefore I am inclined to believe that it may be a problem for other systems as well, but I am the most unfortunate one who hits this error with 100% probability :-)

[1] https://github.com/ghc/ghc/blob/master/libraries/bin-package-db/GHC/PackageD...

-- Gintautas Miliauskas

2014-11-03 1:24 GMT+01:00 Gintautas Miliauskas :

Without knowing much about the surrounding code, I would try two things here. [..] 2. Since the error is completely deterministic, you could try "freezing" evaluation right before the crash ([..] just adding a time delay / console read on the code), and then using a utility app to check which process is keeping the file open, whether it's the same process or something else.

I tried this approach and it seems I do not even need to check the environment further: when ghc-cabal.exe stops in writeFileAtomic (waiting for a keypress) and displays the parameters it was called with, it says that the same dist-boot\setup-configXXXX.tmp file is about to renamed (to dist-boot\setup-config) twice after each other (perhaps from different threads?)! Note that I do not experience the same when the writeFileAtomic function is called for other files. Well, I guess this can easily cause the problems I have seen so far... I can also imagine that this bug is present on other systems as well, but does not cause any harm (for some reason) so it is basically invisible. However, first it would be nice if somebody else could confirm that it is indeed the case.

Any progress on the Windows builder? The last build on the status page is back from October, so I take it the answer is in the negative :( Sorry for completely disappearing from the radar. I started my new job in December, and have had very little free time so far. I also don't have access to a proper Windows machine since I have been traveling quite a bit. Not sure when that is going to change :( On Tue, Nov 4, 2014 at 11:42 PM, Páli Gábor János wrote:

2014-11-04 23:16 GMT+01:00 Páli Gábor János :

...

it says that the same dist-boot\setup-configXXXX.tmp file is about to renamed (to dist-boot\setup-config) twice after each other (perhaps from different threads?)! [..]

Well, I guess this can easily cause the problems I have seen so far... I can also imagine that this bug is present on other systems as well, but does not cause any harm (for some reason) so it is basically invisible.

However, first it would be nice if somebody else could confirm that it is indeed the case.

Yeah, for what it is worth, this also happens on my FreeBSD builder too, but it works fine despite of that.

-- Gintautas Miliauskas

Any luck with getting the Windows continuous builds back up and running? I'd be willing to try to help out if remote access is available, although the build environment should be updated first in case that's the source of the trouble. On Sat, Oct 25, 2014 at 1:26 PM, Páli Gábor János wrote:

2014-10-25 2:03 GMT+02:00 Austin Seipp :

...

As well as that, Gabor's bots (at http://haskell.inf.elte.hu/builders) also build on Windows.

Unfortunately, my builders are still suffering from the sudden breakage induced by a Cabal library update on September 24 [1]. That is, they are unable to complete the build due to some ghc-cabal failure right after bootstrapping [2]. As a result, I basically suspended the builders until I could do something with this. Yes, I have just checked it, the situation is still the same.

Curiously, I do not know about others who are experiencing the same problem, however, the revisions before the referenced commit (mostly) build just fine.

As of yet, I have not had the time and chance to even attempt to fix this. I have already answered Herbert the version of the build environment (Windows 7 SP1, MinGW from July 4 (32 bit) and February 16 (64 bit), with GHC 7.6.3) -- I do not even know if that was considered old or problematic. I am not also sure if the developers involved in the aforementioned change are aware of this issue and what their opinion on this is. I admit that I have not submitted a ticket on this, perhaps I shall.

Probably I shall also experiment with moving to a newer version of the toolchain per the recently revamped Windows build instructions in the meantime.

[1] http://git.haskell.org/ghc.git/commit/4b648be19c75e6c6a8e6f9f93fa12c7a4176f0... [2] http://haskell.inf.elte.hu/builders/windows-x86-head/56/10.html _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

-- Gintautas Miliauskas http://www.haskell.org/mailman/listinfo/ghc-devs