Re: [Msys2-users] Debugging undeterministic segfaults

+ghc-devs
Hi Ray,
thanks for the feedback. ghc-stage1 is a native application, it is built
using a host ghc and a mingw gcc bundled with the ghc distribution (in
order to keep Windows builds more predictable). The thing is, the same
builds (using make) that were stable on cygwin seem to be crashing on
msys2, even though the (bundled) gcc used for the build is the same. It
could be that msys2 is triggering a bug in ghc somehow, but it could be
something going on in msys2 itself.
The tricky part is that the crashes are rare, one in thousands of ghc
invocations within a make build, and I'm having trouble pinning one down to
investigate. I'll try some basic tracing first, but ideas for something
more sophisticated would be very helpful.
A PKGBUILD for ghc should be feasible, although the bootstrapping is a bit
tricky (some Haskell tools need to be preinstalled). I'm not sure how
useful it would be since for Windows there is already a nicely packaged
native Haskell Platform installer.
On Mon, Nov 3, 2014 at 3:30 AM, Ray Donnelly
On Sun, Nov 2, 2014 at 11:45 PM, Gintautas Miliauskas
wrote: Hello,
I have been working on building GHC, the Glasgow Haskell Compiler, on Windows using msys2 [1]. We have been having some strange trouble with ghc segfaulting during the bootstrapping process (i.e., during make). The crashes are not very hard to reproduce, but they are not predictable either. This is almost certainly a bug in ghc itself and not with msys2 (although the crashes do not seem to occur in other environments), but I've been having some trouble pinning it down.
Hi Gintuatas,
Great. I spotted that MSYS2 was the recommended env. for GHC on Windows, hopefully it will remain so!
What's a good way to debug such crashes? Is it possible to somehow stop the ghc process when it segfaults and attach gdb, or to dump core somehow for later inspection?
[1] https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows
Here's one example crash:
"inplace/bin/ghc-stage1.exe" -hisuf hi -osuf o -hcsuf hc -static -H32m -O -Werror -Wall -H64m -O0 -this-package-key ghc_4ugNArSu5ba0Z1uHXrbTlU -hide-all-packages -i -icompiler/basicTypes -icompiler/cmm -icompiler/codeGen -icompiler/coreSyn -icompiler/deSugar -icompiler/ghci -icompiler/hsSyn -icompiler/iface -icompiler/llvmGen -icompiler/main -icompiler/nativeGen -icompiler/parser -icompiler/prelude -icompiler/profiling -icompiler/rename -icompiler/simplCore -icompiler/simplStg -icompiler/specialise -icompiler/stgSyn -icompiler/stranal -icompiler/typecheck -icompiler/types -icompiler/utils -icompiler/vectorise -icompiler/stage2/build -icompiler/stage2/build/autogen -Icompiler/stage2/build -Icompiler/stage2/build/autogen -Icompiler/. -Icompiler/parser -Icompiler/utils -Icompiler/../rts/dist/build -Icompiler/stage2 -optP-DGHCI -optP-include -optPcompiler/stage2/build/autogen/cabal_macros.h -package-key Win32_43THQMouBnk2wpnouztX1X -package-key array_GX4NwjS8xZkC2ZPtjgwhnz -package-key base_ESD4aQEEWwsHtYJVc1BwtJ -package-key binpa_17GphrLqCXt1H1lm4Kse1p -package-key bytes_Kc0PyaputnzDnBdZW0y2Gv -package-key conta_ChF4XLXB9JmByIycPzerow -package-key direc_HU5aFxMIQNwGQFzisjuinu -package-key filep_34DFDFT9FVD9pRLLgh8IdQ -package-key hoopl_IZAd44CED5NCOlpg8p2Kaj -package-key hpc_1QTsfQSN40FHN9p3mydARY -package-key proce_7ZlAbRkwiRO8qgXx3NNP0G -package-key templ_F6UJgDtBcDIFWuHmMGEFzy -package-key time_HGs4JcQCd4wF6U8vJQ5fNH -package-key trans_5jw4w9yTgmZ89ByuixDAKP -Wall -fno-warn-name-shadowing -this-package-key ghc -XHaskell2010 -optc-DTHREADED_RTS -DGHCI_TABLES_NEXT_TO_CODE -DSTAGE=2 -Rghc-timing -O2 -fwarn-tabs -O -dcore-lint -no-user-package-db -rtsopts -odir compiler/stage2/build -hidir compiler/stage2/build -stubdir compiler/stage2/build -c compiler/basicTypes/UniqSupply.lhs -o compiler/stage2/build/UniqSupply.o
compiler/ghc.mk:657: recipe for target 'compiler/stage2/build/UniqSupply.o' failed make[1]: *** [compiler/stage2/build/UniqSupply.o] Segmentation fault
Well, it's pretty much the same story as with Linux. Build the executable and as many libraries as you can with (something like) "-ggdb -O0" then use gdb command line or some IDE of choice. Personally I use Qt Creator (we have packages for this) as it can debug external programs and, even though I don't dislike command lines, I find debugging from them a bit too masochistic.
Is ghc-stage1.exe an MSYS2 program or a native one? Which compiler is it built with exactly?
What would be involved in creating a PKGBUILD for ghc? We'd love to have one, and it'd certainly make me a lot more inclined to dive in to see what's going on if one existed already.
Ray.
-- Gintautas Miliauskas
------------------------------------------------------------------------------
_______________________________________________ Msys2-users mailing list Msys2-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/msys2-users
-- Gintautas Miliauskas

On 4. 11. 2014 0:07, Gintautas Miliauskas wrote:
A PKGBUILD for ghc should be feasible, although the bootstrapping is a bit tricky (some Haskell tools need to be preinstalled). I'm not sure how useful it would be since for Windows there is already a nicely packaged native Haskell Platform installer.
It definitely helps MSYS2 users by providing an easily installable ghc package. That in turn could help with setting up ghc development environment. Hopefully after Windows ghc migrates to gcc 4.8, ghc package will be able to use the MSYS2 packaged toolchain to reduce duplication. I have this so far, but I give no guarantees: https://github.com/elieux/MINGW-packages/tree/ghc To use: - download ghc and put it in PATH - optionally alter ghc/lib/settings to use MSYS2 toolchain instead of the bundled one (see below) - use cabal-install PKGBUILD to build Cabal and put it in PATH - use built Cabal to download and build Alex and Happy and put them in PATH (I think they're installed somewhere inside AppData by default) - run ghc PKGBUILD Related: https://www.haskell.org/pipermail/ghc-devs/2014-October/006759.html My ghc/lib/settings (from version 7.6.3): [("GCC extra via C opts", " -fwrapv"), ("C compiler command", "gcc.exe"), ("C compiler flags", " -fno-stack-protector -Wl,--hash-size=31 -Wl,--reduce-memory-overheads"), ("ar command", "ar.exe"), ("ar flags", "q"), ("ar supports at file", "YES"), ("touch command", "$topdir/touchy.exe"), ("dllwrap command", "dllwrap.exe"), ("windres command", "windres.exe"), ("perl command", "perl.exe"), ("target os", "OSMinGW32"), ("target arch", "ArchX86_64"), ("target word size", "8"), ("target has GNU nonexec stack", "False"), ("target has .ident directive", "True"), ("target has subsections via symbols", "False"), ("LLVM llc command", ""), ("LLVM opt command", "") ] -- David Macek

Nice!
This needs a host ghc (with alex and happy) preinstalled, correct?
On Tue, Nov 4, 2014 at 12:55 AM, David Macek
On 4. 11. 2014 0:07, Gintautas Miliauskas wrote:
A PKGBUILD for ghc should be feasible, although the bootstrapping is a bit tricky (some Haskell tools need to be preinstalled). I'm not sure how useful it would be since for Windows there is already a nicely packaged native Haskell Platform installer.
It definitely helps MSYS2 users by providing an easily installable ghc package. That in turn could help with setting up ghc development environment. Hopefully after Windows ghc migrates to gcc 4.8, ghc package will be able to use the MSYS2 packaged toolchain to reduce duplication.
I have this so far, but I give no guarantees: https://github.com/elieux/MINGW-packages/tree/ghc
To use: - download ghc and put it in PATH - optionally alter ghc/lib/settings to use MSYS2 toolchain instead of the bundled one (see below) - use cabal-install PKGBUILD to build Cabal and put it in PATH - use built Cabal to download and build Alex and Happy and put them in PATH (I think they're installed somewhere inside AppData by default) - run ghc PKGBUILD
Related: https://www.haskell.org/pipermail/ghc-devs/2014-October/006759.html
My ghc/lib/settings (from version 7.6.3):
[("GCC extra via C opts", " -fwrapv"), ("C compiler command", "gcc.exe"), ("C compiler flags", " -fno-stack-protector -Wl,--hash-size=31 -Wl,--reduce-memory-overheads"), ("ar command", "ar.exe"), ("ar flags", "q"), ("ar supports at file", "YES"), ("touch command", "$topdir/touchy.exe"), ("dllwrap command", "dllwrap.exe"), ("windres command", "windres.exe"), ("perl command", "perl.exe"), ("target os", "OSMinGW32"), ("target arch", "ArchX86_64"), ("target word size", "8"), ("target has GNU nonexec stack", "False"), ("target has .ident directive", "True"), ("target has subsections via symbols", "False"), ("LLVM llc command", ""), ("LLVM opt command", "") ]
-- David Macek _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
-- Gintautas Miliauskas

On 4. 11. 2014 1:05, Gintautas Miliauskas wrote:
Nice!
This needs a host ghc (with alex and happy) preinstalled, correct?
Maybe I misunderstood, but the answer should be clear from my message. Yes, the ghc PKGBUILD expects working ghc, alex and happy in PATH. ghc has to be downloaded (until an MSYS2 package becomes available), cabal can be built using cabal-install PKGBUILD and alex and happy can then be downloaded and built using cabal. -- David Macek

thanks for the feedback. ghc-stage1 is a native application, it is built using a host ghc and a mingw gcc bundled with the ghc distribution (in order to keep Windows builds more predictable). The thing is, the same builds (using make) that were stable on cygwin seem to be crashing on msys2, even though the (bundled) gcc used for the build is the same. It could be that msys2 is triggering a bug in ghc somehow, but it could be something going on in msys2 itself.
Interesting data point: in one of my experiments the following command segfaulted: "inplace/bin/ghc-stage1.exe" -optc-fno-stack-protector -optc-Werror -optc-Wall -optc-Wall -optc-Wextra -optc-Wstrict-prototypes -optc-Wmissing-prototypes -optc-Wmissing-declarations -optc-Winline -optc-Waggregate-return -optc-Wpointer-arith -optc-Wmissing-noreturn -optc-Wnested-externs -optc-Wredundant-decls -optc-Iincludes -optc-Iincludes/dist -optc-Iincludes/dist-derivedconstants/header -optc-Iincludes/dist-ghcconstants/header -optc-Irts -optc-Irts/dist/build -optc-DCOMPILING_RTS -optc-fno-strict-aliasing -optc-fno-common -optc-Wno-error=inline -optc-O2 -optc-fomit-frame-pointer -optc-fno-omit-frame-pointer -optc-g -optc-O0 -optc-DRtsWay=\"rts_thr_debug\" -static -optc-DTHREADED_RTS -optc-DDEBUG -H32m -O -Werror -Wall -H64m -O0 -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Irts -Irts/dist/build -DCOMPILING_RTS -this-package-key rts -dcmm-lint -i -irts -irts/dist/build -irts/dist/build/autogen -Irts/dist/build -Irts/dist/build/autogen -O2 -O0 -c rts/sm/GCUtils.c -o rts/dist/build/sm/GCUtils.thr_debug_o [...] rts/ghc.mk:236: recipe for target 'rts/dist/build/sm/GCUtils.thr_debug_o' failed make[1]: *** [rts/dist/build/sm/GCUtils.thr_debug_o] Segmentation fault make[1]: *** Deleting file 'rts/dist/build/sm/GCUtils.thr_debug_o' Note that while the command is invoking ghc, actually it is only building a C file, so it's just forwarding on the work to gcc The issue is thus unlikely to be somewhere deep in ghc, it's either on the surface or in msys2. There could be a problem with our gcc build, but other compiler versions crash too, so I don't think that's it. -- Gintautas Miliauskas
participants (2)
-
David Macek
-
Gintautas Miliauskas