Windows build failing in a new way

My windows build is more broken than usual. I can't even build a GHC. Please, could someone fix this? I'm getting desperate. Simon libtool: compile: C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -fexceptions -MT src/x86/ffi64.lo -MMD -MP -MF src/x86/.deps/ffi64.Tpo -c ../src/x86/ffi64.c -o src/x86/ffi64.o depbase=`echo src/x86/unix64.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\ /bin/sh ./libtool --mode=compile C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -MT src/x86/unix64.lo -MMD -MP -MF $depbase.Tpo -c -o src/x86/unix64.lo ../src/x86/unix64.S &&\ mv -f $depbase.Tpo $depbase.Plo libtool: compile: C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -MT src/x86/unix64.lo -MMD -MP -MF src/x86/.deps/unix64.Tpo -c ../src/x86/unix64.S -o src/x86/unix64.o ../src/x86/unix64.S: Assembler messages: ../src/x86/unix64.S:45: Error: junk at end of line, first unrecognized character is `f' ../src/x86/unix64.S:204: Error: junk at end of line, first unrecognized character is `f' ../src/x86/unix64.S:208: Error: junk at end of line, first unrecognized character is `f' ../src/x86/unix64.S:326: Error: junk at end of line, first unrecognized character is `f' ../src/x86/unix64.S:334: Error: junk at end of line, first unrecognized character is `,' make[5]: *** [Makefile:1335: src/x86/unix64.lo] Error 1 make[5]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys' make[4]: *** [Makefile:1596: all-recursive] Error 1 make[4]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys' make[3]: *** [Makefile:730: all] Error 2 make[3]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys' make[2]: *** [Makefile:608: all-all] Error 2 rts/ghc.mk:494: rts/dist/build/.depend-v-l-debug-thr-thr_debug-thr_l.c_asm: No such file or directory make[1]: *** [libffi/ghc.mk:115: libffi/stamp.ffi.static.build] Error 2 make: *** [Makefile:127: all] Error 2 /c/code/HEAD$

Hi Simon, As of this morning the Windows build was working fine https://github.com/Mistuke/GhcWindowsBuild/commit/aa6906b2535224721d8b049cee... those are my nightly logs for last night at commit a02b80f That error seems to be coming from gcc and not ghc. We did update the crt and maybe the scripts didn't do the right thing. Could you try nuking ghc-tarballs and trying again? Of course running with - -enable-tarballs-autodownloads on. Tamar On Thu, 9 Mar 2017, 16:28 Simon Peyton Jones via ghc-devs, < ghc-devs@haskell.org> wrote:
My windows build is more broken than usual. I can’t even build a GHC.
Please, could someone fix this? I’m getting desperate.
Simon
libtool: compile: C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -fexceptions -MT src/x86/ffi64.lo -MMD -MP -MF src/x86/.deps/ffi64.Tpo -c ../src/x86/ffi64.c -o src/x86/ffi64.o
depbase=`echo src/x86/unix64.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/sh ./libtool --mode=compile C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -MT src/x86/unix64.lo -MMD -MP -MF $depbase.Tpo -c -o src/x86/unix64.lo ../src/x86/unix64.S &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -MT src/x86/unix64.lo -MMD -MP -MF src/x86/.deps/unix64.Tpo -c ../src/x86/unix64.S -o src/x86/unix64.o
../src/x86/unix64.S: Assembler messages:
../src/x86/unix64.S:45: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:204: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:208: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:326: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:334: Error: junk at end of line, first unrecognized character is `,'
make[5]: *** [Makefile:1335: src/x86/unix64.lo] Error 1
make[5]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys'
make[4]: *** [Makefile:1596: all-recursive] Error 1
make[4]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys'
make[3]: *** [Makefile:730: all] Error 2
make[3]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys'
make[2]: *** [Makefile:608: all-all] Error 2
rts/ghc.mk:494: rts/dist/build/.depend-v-l-debug-thr-thr_debug-thr_l.c_asm: No such file or directory
make[1]: *** [libffi/ghc.mk:115: libffi/stamp.ffi.static.build] Error 2
make: *** [Makefile:127: all] Error 2
/c/code/HEAD$ _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

That said, running the build on HEAD it seems that the template haskell
stuff committed today has broken the build again. I'd suggest checking out
the commit in my previous email which is just a few hours old. And cleaning
ghc-tarballs. As the error you seem to be getting is local.
On Thu, 9 Mar 2017, 16:38 Phyx,
Hi Simon,
As of this morning the Windows build was working fine https://github.com/Mistuke/GhcWindowsBuild/commit/aa6906b2535224721d8b049cee... those are my nightly logs for last night at commit a02b80f
That error seems to be coming from gcc and not ghc. We did update the crt and maybe the scripts didn't do the right thing. Could you try nuking ghc-tarballs and trying again? Of course running with - -enable-tarballs-autodownloads on.
Tamar
On Thu, 9 Mar 2017, 16:28 Simon Peyton Jones via ghc-devs, < ghc-devs@haskell.org> wrote:
My windows build is more broken than usual. I can’t even build a GHC.
Please, could someone fix this? I’m getting desperate.
Simon
libtool: compile: C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -fexceptions -MT src/x86/ffi64.lo -MMD -MP -MF src/x86/.deps/ffi64.Tpo -c ../src/x86/ffi64.c -o src/x86/ffi64.o
depbase=`echo src/x86/unix64.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/sh ./libtool --mode=compile C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -MT src/x86/unix64.lo -MMD -MP -MF $depbase.Tpo -c -o src/x86/unix64.lo ../src/x86/unix64.S &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -MT src/x86/unix64.lo -MMD -MP -MF src/x86/.deps/unix64.Tpo -c ../src/x86/unix64.S -o src/x86/unix64.o
../src/x86/unix64.S: Assembler messages:
../src/x86/unix64.S:45: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:204: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:208: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:326: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:334: Error: junk at end of line, first unrecognized character is `,'
make[5]: *** [Makefile:1335: src/x86/unix64.lo] Error 1
make[5]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys'
make[4]: *** [Makefile:1596: all-recursive] Error 1
make[4]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys'
make[3]: *** [Makefile:730: all] Error 2
make[3]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys'
make[2]: *** [Makefile:608: all-all] Error 2
rts/ghc.mk:494: rts/dist/build/.depend-v-l-debug-thr-thr_debug-thr_l.c_asm: No such file or directory
make[1]: *** [libffi/ghc.mk:115: libffi/stamp.ffi.static.build] Error 2
make: *** [Makefile:127: all] Error 2
/c/code/HEAD$ _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Checking again, I think something else is wrong I'll have to check later.
Sorry, but that commit above is still good. If you are really stuck use
that one.
Tamar
On Thu, 9 Mar 2017, 17:11 Phyx,
That said, running the build on HEAD it seems that the template haskell stuff committed today has broken the build again. I'd suggest checking out the commit in my previous email which is just a few hours old. And cleaning ghc-tarballs. As the error you seem to be getting is local.
On Thu, 9 Mar 2017, 16:38 Phyx,
wrote: Hi Simon,
As of this morning the Windows build was working fine https://github.com/Mistuke/GhcWindowsBuild/commit/aa6906b2535224721d8b049cee... those are my nightly logs for last night at commit a02b80f
That error seems to be coming from gcc and not ghc. We did update the crt and maybe the scripts didn't do the right thing. Could you try nuking ghc-tarballs and trying again? Of course running with - -enable-tarballs-autodownloads on.
Tamar
On Thu, 9 Mar 2017, 16:28 Simon Peyton Jones via ghc-devs, < ghc-devs@haskell.org> wrote:
My windows build is more broken than usual. I can’t even build a GHC.
Please, could someone fix this? I’m getting desperate.
Simon
libtool: compile: C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -fexceptions -MT src/x86/ffi64.lo -MMD -MP -MF src/x86/.deps/ffi64.Tpo -c ../src/x86/ffi64.c -o src/x86/ffi64.o
depbase=`echo src/x86/unix64.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/sh ./libtool --mode=compile C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -MT src/x86/unix64.lo -MMD -MP -MF $depbase.Tpo -c -o src/x86/unix64.lo ../src/x86/unix64.S &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: C:/code/HEAD/inplace/mingw/bin/gcc.exe -DHAVE_CONFIG_H -I. -I.. -I. -I../include -Iinclude -I../src -I. -I../include -Iinclude -I../src -Wall -Werror -fno-stack-protector -w -MT src/x86/unix64.lo -MMD -MP -MF src/x86/.deps/unix64.Tpo -c ../src/x86/unix64.S -o src/x86/unix64.o
../src/x86/unix64.S: Assembler messages:
../src/x86/unix64.S:45: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:204: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:208: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:326: Error: junk at end of line, first unrecognized character is `f'
../src/x86/unix64.S:334: Error: junk at end of line, first unrecognized character is `,'
make[5]: *** [Makefile:1335: src/x86/unix64.lo] Error 1
make[5]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys'
make[4]: *** [Makefile:1596: all-recursive] Error 1
make[4]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys'
make[3]: *** [Makefile:730: all] Error 2
make[3]: Leaving directory '/c/code/HEAD/libffi/build/x86_64-pc-msys'
make[2]: *** [Makefile:608: all-all] Error 2
rts/ghc.mk:494: rts/dist/build/.depend-v-l-debug-thr-thr_debug-thr_l.c_asm: No such file or directory
make[1]: *** [libffi/ghc.mk:115: libffi/stamp.ffi.static.build] Error 2
make: *** [Makefile:127: all] Error 2
/c/code/HEAD$ _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Actually it asked me thus:
Error:
Needed msys2 tarballs are missing. You have a few options to get them,
* run configure with the --enable-tarballs-autodownload option
So I did as requested and ran configure with that flag. Success:
checking for path to top of build tree... C:/code/HEAD
configure: Checking for Windows toolchain tarballs...
######################################################################## 100.0%
configure: Extracting Windows toolchain from archives (may take a while)...
configure: In-tree MingW-w64 tree created
configure: Making in-tree perl tree
configure: In-tree perl tree created
checking whether the C compiler works... yes
checking for C compiler default output file name... a.exe
Then I validated from scratch
sh validate --fast
and got the error I reported.
I suppose I could delete ghc-tarballs and try again. I’ll do that when I’m next connected.
In case it helps, the lines in unix64.S that are being rejected are
.type ffi_call_unix64,@function
…
.size ffi_call_unix64,.-ffi_call_unix64
..
.type ffi_closure_unix64,@function
…
.size ffi_closure_unix64,.-ffi_closure_unix64
…
.section .eh_frame,"a",@progbits
Simon
From: Phyx [mailto:lonetiger@gmail.com]
Sent: 09 March 2017 16:38
To: Simon Peyton Jones

Simon Peyton Jones via ghc-devs
My windows build is more broken than usual. I can't even build a GHC. Please, could someone fix this? I'm getting desperate.
This is very odd; Harbormaster is also seeing it but I've been unable to reproduce locally. It seems that the libffi build is failing but it's quite unclear why it would fail for you yet succeed for me. I'll try to reproduce on the Harbormaster machine. Cheers, - Ben

My CI server is also reproducing it while I also haven't been able to
locally. Weird indeed.
On Thu, 9 Mar 2017, 17:36 Ben Gamari,
Simon Peyton Jones via ghc-devs
writes: My windows build is more broken than usual. I can't even build a GHC. Please, could someone fix this? I'm getting desperate.
This is very odd; Harbormaster is also seeing it but I've been unable to reproduce locally. It seems that the libffi build is failing but it's quite unclear why it would fail for you yet succeed for me. I'll try to reproduce on the Harbormaster machine.
Cheers,
- Ben
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Phyx
My CI server is also reproducing it while I also haven't been able to locally. Weird indeed.
Ahhh, I just noticed that Reid stumbled upon the culprit yesterday. See [1]. Cheers, - Ben [1] https://phabricator.haskell.org/rGHCe901ed1c5d66

Ah great,
Triples again.. I still wonder why it is suddenly an issue. We haven’t touched the .m4 file in a while and no one changed libffi either right? This is just like last time the normalization bit us. Causing days of broken builds on different targets while everyone fixed the one they were interested in.
Why do we do the normalization? It doesn’t seem to give us any benefits at all.. Maybe we should stop doing it after the branch for 8.4?
From: Ben Gamari
Sent: Thursday, March 9, 2017 18:51
To: Phyx; Simon Peyton Jones; ghc-devs@haskell.org
Subject: Re: Windows build failing in a new way
Phyx
My CI server is also reproducing it while I also haven't been able to locally. Weird indeed.
Ahhh, I just noticed that Reid stumbled upon the culprit yesterday. See [1]. Cheers, - Ben [1] https://phabricator.haskell.org/rGHCe901ed1c5d66

lonetiger@gmail.com writes:
Ah great,
Triples again.. I still wonder why it is suddenly an issue. We haven’t touched the .m4 file in a while and no one changed libffi either right? This is just like last time the normalization bit us. Causing days of broken builds on different targets while everyone fixed the one they were interested in.
Well, the patch that Reid points out indeed does change the triple which we pass to subproject configures. However, I have been utterly unable to reproduce this locally nor on the Harbormaster machine (both with ./validate). Nevertheless, I have a hypothesis for the cause and a proposed fix in D3304.
Why do we do the normalization? It doesn’t seem to give us any benefits at all.. Maybe we should stop doing it after the branch for 8.4?
Well, there are a few different normalizations which you might mean here. The patch in question affects autoconf's canonicalization. The patch in question actually removes what may be the last reference to the autoconf-canonicalized triple. However, my proposed fix then reintroduces the need for it, since I suspect the cause is that we are passing an empty triple to subproject configures. There is also the matter of GHC's own triple normalization (e.g. GHC_CONVERT_OS and friends, defined in aclocal.m4). I'm frankly not entirely sure this is doing much harm and replacing it with autoconf's canonicalized triple would be a non-trivial amount of work. However, if you want to try I wouldn't be opposed. Cheers, - Ben

Ben Gamari
lonetiger@gmail.com writes:
Ah great,
Triples again.. I still wonder why it is suddenly an issue. We haven’t touched the .m4 file in a while and no one changed libffi either right? This is just like last time the normalization bit us. Causing days of broken builds on different targets while everyone fixed the one they were interested in.
Well, the patch that Reid points out indeed does change the triple which we pass to subproject configures. However, I have been utterly unable to reproduce this locally nor on the Harbormaster machine (both with ./validate).
Nevertheless, I have a hypothesis for the cause and a proposed fix in D3304.
I believe at this point Harbormaster has demonstrated [1] that the fix is effective. I'll go ahead and merge. Cheers, - Ben [1] https://phabricator.haskell.org/harbormaster/build/23078/
participants (4)
-
Ben Gamari
-
lonetiger@gmail.com
-
Phyx
-
Simon Peyton Jones