I have a suspicion that the segmentation faults are related to fork errors, which appear in logs from time to time. After a few builds fork() errors become very frequent, and segfaults seem to occur more often. I just ran the build (validate.sh) in a loop, and after a while got a bunch of segfaults in Makefile rules as basic as rm invocations in the initial "make clean", e.g.:

"rm" -rf driver/split/dist
driver/split/ghc.mk:19: recipe for target 'clean_driver/split_dist' failed
make[1]: *** [clean_driver/split_dist] Segmentation fault
Makefile:94: recipe for target 'maintainer-clean' failed

Closing and reopening the msys2 console seems to help - for some time.




On Tue, Nov 4, 2014 at 1:30 AM, Ray Donnelly <mingw.android@gmail.com> wrote:
On Mon, Nov 3, 2014 at 11:07 PM, Gintautas Miliauskas
<gintautas.miliauskas@gmail.com> wrote:
> +ghc-devs
>
> Hi Ray,
>
> thanks for the feedback. ghc-stage1 is a native application, it is built
> using a host ghc and a mingw gcc bundled with the ghc distribution (in order
> to keep Windows builds more predictable). The thing is, the same builds
> (using make) that were stable on cygwin seem to be crashing on msys2, even
> though the (bundled) gcc used for the build is the same. It could be that
> msys2 is triggering a bug in ghc somehow, but it could be something going on
> in msys2 itself.
>
> The tricky part is that the crashes are rare, one in thousands of ghc
> invocations within a make build, and I'm having trouble pinning one down to
> investigate. I'll try some basic tracing first, but ideas for something more
> sophisticated would be very helpful.

Ah, ok. I skim read your initial email and applied totally incorrect
weighting to the "not very hard to reproduce" bit, apologies!

It *should* be possible to setup post-mortem debugging using either Qt
Creator (Tools->Options->Debugger, tick "Use Qt Creator for
post-mortem debugging") or Dr. Mingw
(https://github.com/jrfonseca/drmingw). I briefly tested both options:

Qt Creator: It seems there's a bug in its handling of the
-wincrashevent command line which I've just added a note about to the
MINGW-packages todo list since I'd find this feature more than very
useful:
https://github.com/Alexpux/MINGW-packages/blob/master/TODO.md

Dr Mingw: Launching the crashing executable from Windows Explorer, it
launches it and gives me useful information.

Unfortunately, regardless of the debugger used, when invoking the
crashing executable from the mintty commandline, something inhibits
all post mortem debugging. I will ask the mingw-w64 guys if they know
what this is. I guess intrusive dialogs would be annoying, but I'd
like an env. var override for that I think.

There is another possibility, and that's that bash is crashing, which
is an MSYS2 program. We do have a way of hooking into post-mortem
debugging there since Cygwin provided a way and we improved it. If you
check your msys2_shell.bat you'll see:
rem set MSYS=error_start:%WD%../../mingw64/bin/qtcreator.exe|-debug|<process-id>

If you remove the rem and make sure the path is correct (it may have
rotted some since I last used it) then hopefully you'll get something
useful. To be really useful, you should rebuild two packages,
MSYS2-packages/{msys2-runtime,bash}/PKGBUILD with
options=('debug' '!strip')
and then install them.

Finally, can anyone else confirm the problem?

There may be some edge cases when e.g. PATH is around 1024 characters,
I've seen some hardcoded limits in the msys2-runtime (nee Cygwin)
code base for things like that. I'd advise cutting your Windows PATH
down to just the essentials.

Cheers,

Ray.

>
> A PKGBUILD for ghc should be feasible, although the bootstrapping is a bit
> tricky (some Haskell tools need to be preinstalled). I'm not sure how useful
> it would be since for Windows there is already a nicely packaged native
> Haskell Platform installer.
>
>
> On Mon, Nov 3, 2014 at 3:30 AM, Ray Donnelly <mingw.android@gmail.com>
> wrote:
>>
>> On Sun, Nov 2, 2014 at 11:45 PM, Gintautas Miliauskas
>> <gintautas.miliauskas@gmail.com> wrote:
>> > Hello,
>> >
>> > I have been working on building GHC, the Glasgow Haskell Compiler, on
>> > Windows using msys2 [1]. We have been having some strange trouble with
>> > ghc
>> > segfaulting during the bootstrapping process (i.e., during make). The
>> > crashes are not very hard to reproduce, but they are not predictable
>> > either.
>> > This is almost certainly a bug in ghc itself and not with msys2
>> > (although
>> > the crashes do not seem to occur in other environments), but I've been
>> > having some trouble pinning it down.
>> >
>>
>> Hi Gintuatas,
>>
>> Great. I spotted that MSYS2 was the recommended env. for GHC on
>> Windows, hopefully it will remain so!
>>
>> > What's a good way to debug such crashes? Is it possible to somehow stop
>> > the
>> > ghc process when it segfaults and attach gdb, or to dump core somehow
>> > for
>> > later inspection?
>> >
>> > [1] https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows
>> >
>> > Here's one example crash:
>> >
>> > "inplace/bin/ghc-stage1.exe" -hisuf hi -osuf  o -hcsuf hc -static  -H32m
>> > -O
>> > -Werror -Wall -H64m -O0    -this-package-key ghc_4ugNArSu5ba0Z1uHXrbTlU
>> > -hide-all-packages -i -icompiler/basicTypes -icompiler/cmm
>> > -icompiler/codeGen -icompiler/coreSyn -icompiler/deSugar -icompiler/ghci
>> > -icompiler/hsSyn -icompiler/iface -icompiler/llvmGen -icompiler/main
>> > -icompiler/nativeGen -icompiler/parser -icompiler/prelude
>> > -icompiler/profiling -icompiler/rename -icompiler/simplCore
>> > -icompiler/simplStg -icompiler/specialise -icompiler/stgSyn
>> > -icompiler/stranal -icompiler/typecheck -icompiler/types
>> > -icompiler/utils
>> > -icompiler/vectorise -icompiler/stage2/build
>> > -icompiler/stage2/build/autogen
>> > -Icompiler/stage2/build -Icompiler/stage2/build/autogen -Icompiler/.
>> > -Icompiler/parser -Icompiler/utils -Icompiler/../rts/dist/build
>> > -Icompiler/stage2   -optP-DGHCI -optP-include
>> > -optPcompiler/stage2/build/autogen/cabal_macros.h -package-key
>> > Win32_43THQMouBnk2wpnouztX1X -package-key array_GX4NwjS8xZkC2ZPtjgwhnz
>> > -package-key base_ESD4aQEEWwsHtYJVc1BwtJ -package-key
>> > binpa_17GphrLqCXt1H1lm4Kse1p -package-key bytes_Kc0PyaputnzDnBdZW0y2Gv
>> > -package-key conta_ChF4XLXB9JmByIycPzerow -package-key
>> > direc_HU5aFxMIQNwGQFzisjuinu -package-key filep_34DFDFT9FVD9pRLLgh8IdQ
>> > -package-key hoopl_IZAd44CED5NCOlpg8p2Kaj -package-key
>> > hpc_1QTsfQSN40FHN9p3mydARY -package-key proce_7ZlAbRkwiRO8qgXx3NNP0G
>> > -package-key templ_F6UJgDtBcDIFWuHmMGEFzy -package-key
>> > time_HGs4JcQCd4wF6U8vJQ5fNH -package-key trans_5jw4w9yTgmZ89ByuixDAKP
>> > -Wall
>> > -fno-warn-name-shadowing -this-package-key ghc -XHaskell2010
>> > -optc-DTHREADED_RTS -DGHCI_TABLES_NEXT_TO_CODE -DSTAGE=2 -Rghc-timing
>> > -O2
>> > -fwarn-tabs -O -dcore-lint  -no-user-package-db -rtsopts      -odir
>> > compiler/stage2/build -hidir compiler/stage2/build -stubdir
>> > compiler/stage2/build   -c compiler/basicTypes/UniqSupply.lhs -o
>> > compiler/stage2/build/UniqSupply.o
>> >
>> > compiler/ghc.mk:657: recipe for target
>> > 'compiler/stage2/build/UniqSupply.o'
>> > failed
>> > make[1]: *** [compiler/stage2/build/UniqSupply.o] Segmentation fault
>> >
>>
>> Well, it's pretty much the same story as with Linux. Build the
>> executable and as many libraries as you can with (something like)
>> "-ggdb -O0" then use gdb command line or some IDE of choice.
>> Personally I use Qt Creator (we have packages for this) as it can
>> debug external programs and, even though I don't dislike command
>> lines, I find debugging from them a bit too masochistic.
>>
>> Is ghc-stage1.exe an MSYS2 program or a native one? Which compiler is
>> it built with exactly?
>>
>> What would be involved in creating a PKGBUILD for ghc? We'd love to
>> have one, and it'd certainly make me a lot more inclined to dive in to
>> see what's going on if one existed already.
>>
>> Ray.
>>
>> >
>> > --
>> > Gintautas Miliauskas
>> >
>> >
>> > ------------------------------------------------------------------------------
>> >
>> > _______________________________________________
>> > Msys2-users mailing list
>> > Msys2-users@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/msys2-users
>> >
>
>
>
>
> --
> Gintautas Miliauskas



--
Gintautas Miliauskas