#8834: 64-bit windows cabal.exe segfaults in GC ---------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: None | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: None/Unknown | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ---------------------------------+---------------------------------- Comment (by awson): Probably, it's worth to add that I observed this problem since I was able to build 64-bit windows GHC 7.7+ (a month or so). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ---------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: None | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: None/Unknown | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ---------------------------------+---------------------------------- Comment (by awson): Since things were changed since that commit, I've revised it and made manual reversal. This commit was mainly a set of comments, relevant changes were more or less local and I've created this reversal patch mechanically. I still don't understand underlying machinery, but it looks this patch makes things work again on win64. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Changes (by awson): * failure: None/Unknown => Runtime crash * component: None => Compiler -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Changes (by simonpj): * cc: simonmar, jstolarek (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by simonpj): Could this account for #8870? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): Well, separating to modules also causes the bug. I put Cmm dumps here. But is this enough? Could the bug manifest itself somewhere else (e. g. `base` or `bytestring` libraries)? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by jstolarek): I see something that looks suspicious. In the correct version we have: {{{ c1zB: _r1xg::P64 = R1; if ((Sp + -16) < SpLim) goto c1zC; else goto c1zD; c1zC: R1 = _r1xg::P64; call (stg_gc_enter_1)(R1) args: 8, res: 0, upd: 8; c1zD: (_c1zx::I64) = call "ccall" arg hints: [PtrHint, PtrHint] result hints: [PtrHint] newCAF(BaseReg, _r1xg::P64); if (_c1zx::I64 == 0) goto c1zz; else goto c1zy; c1zz: call (I64[_r1xg::P64])() args: 8, res: 0, upd: 8; c1zy: I64[Sp - 16] = stg_bh_upd_frame_info; I64[Sp - 8] = _c1zx::I64; R2 = c1zA_str; Sp = Sp - 16; call GHC.CString.unpackCString#_info(R2) args: 24, res: 0, upd: 24; }}} In the incorrect one we have: {{{ c1zy: if ((Sp + -16) < SpLim) goto c1zz; else goto c1zA; c1zz: R1 = R1; call (stg_gc_enter_1)(R1) args: 8, res: 0, upd: 8; c1zA: (_c1zu::I64) = call "ccall" arg hints: [PtrHint, PtrHint] result hints: [PtrHint] newCAF(BaseReg, R1); if (_c1zu::I64 == 0) goto c1zw; else goto c1zv; c1zw: call (I64[R1])() args: 8, res: 0, upd: 8; c1zv: I64[Sp - 16] = stg_bh_upd_frame_info; I64[Sp - 8] = _c1zu::I64; R2 = c1zx_str; Sp = Sp - 16; call GHC.CString.unpackCString#_info(R2) args: 24, res: 0, upd: 24; }}} Notice how correct version saves `R1` to local variable. I'm especially worried about this call: {{{ call (I64[_r1xg::P64])() args: 8, res: 0, upd: 8; CORRECT.CMM call (I64[R1])() args: 8, res: 0, upd: 8; WRONG.CMM }}} '''If''' `R1` gets clobbered be earlier ccall to `newCAF(BaseReg, R1)` then this is probably the reason why things go wrong. In that case the right solution would be to tell GHC that the call to `newCAF` defines register `R1` (see `DefinerOfRegs` instance declaration for `GlobalReg`). Then this case should be caught by first guard in `conflicts`. Also, Note [Sinking and calls] seems very relevant here. As a side note: it is really annoying to see these `R1 = R1` assignments. I recall they are eliminated by the code generator but it is frustrating to see them at the Cmm level. I believe the sinking pass should eliminate these assignments but I didn't have enough time to investigate into this further during my internship. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): Replying to [comment:14 jstolarek]:

So this works:

{{{ isTrivial :: CmmExpr -> Bool isTrivial (CmmReg _) = True isTrivial _ = False }}}

but this doesn't:

{{{ isTrivial :: CmmExpr -> Bool isTrivial (CmmReg (CmmLocal _)) = True isTrivial _ = False }}}

Conversely. The first does not work, the second works. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by simonmar): I think the bad code is much more likely to be in a library somewhere, not in the code that calls `readFile`/`lines`, so we're not going to see anything bad in that file. @jstolarek - R1 is not caller-saves (on either Linux or Win64), so it is safe to leave it live across a foreign call. Also, that code fragment is the standard `newCAF` call which appears everywhere, if this was broken then everything would be broken. I'm surprised that the change to `isTrivial` all by itself causes this failure. That must mean that there was something wrong with the code before the "bad patch", because the change to `isTrivial` should be safe. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): Replying to [comment:17 simonmar]:

I'm surprised that the change to `isTrivial` all by itself causes this failure. That must mean that there was something wrong with the code before the "bad patch", because the change to `isTrivial` should be safe.

It probably would worth to add that reverting `isTrivial` only does not help either. I. e. `isTrivial` all by itself broke things as they were before that bad commit, but changing *only* `isTrivial` to it's pre-commit state does not help either. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): Replying to [comment:20 simonpj]:

Could this have anything to do with #8870, a much simpler case than Cabal?

(As #8870 says, I can't even build a working stage-2 GHC on Windows now.)

Both this and #8870 look like release blockers to me.

Simon

I can't reproduce #8870 using 32-bit GHC on 64-bin Windows. Are you using 32-bit Windows? If no, then your problem can be different from #8870 (but still related to #8834). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): Yes, they are in RTS. More precise location is rts/sm/Evac.c line 390. Also I think this bug is triggered near foreign `memchr` call in function `elemIndex` from `Data.ByteString` module in `bytestring` package, which is inlined down to `lines` function from `Data.ByteString.Char8`. Also I checked this bug is not triggered if relevant modules from `bytestring` package are compiled with `-O` flag instead of `-O2`, hence this bug could be more ubiquitous if more applications code was compiled with `-O2`, perhaps. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:23 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by thoughtpolice): It's not true that those are the *only* things enabled by -O2 - you must also search for `optLevel`, which client code can depend on for specific instances if they wish (for example, maybe it's *not* an entire Core->Core pass, but an otherwise small micro-optimization). Actually, now that I'm searching and thinking about it - the only other case where we do this is when we short-cut PAPs - see 4d1ea482885481073d2fee0ea0355848b9d853a1 and `Note [avoid intermediate PAPs]` in `StgCmmLayout`. Simon committed this a while ago. I also have a Win32 build going, so I'll test this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:25 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

It would be great if we had a test case that does not depend on any

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): Replying to [comment:24 jstolarek]: library code. This way we could eyeball the problem by looking at Cmm. Do you think you would be able to create such a test case. I think, this code should be enough: {{{ module BugIso (lines1) where import Prelude hiding (null, take, drop) import Data.ByteString hiding (elemIndex) import Data.ByteString.Internal import Data.Word import Foreign elemIndex1 :: Word8 -> ByteString -> Maybe Int elemIndex1 c (PS x s l) = inlinePerformIO $ withForeignPtr x $ \p -> do let p' = p `plusPtr` s q <- memchr p' c (fromIntegral l) return $! if q == nullPtr then Nothing else Just $! q `minusPtr` p' {-# INLINE elemIndex1 #-} elemIndex :: Char -> ByteString -> Maybe Int elemIndex = elemIndex1 . c2w {-# INLINE elemIndex #-} lines1 :: ByteString -> [ByteString] lines1 ps | null ps = [] | otherwise = case search ps of Nothing -> [ps] Just n -> take n ps : lines1 (drop (n+1) ps) where search = elemIndex '\n' }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:27 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by thoughtpolice): I can confirm Kyril's example above works properly and produces the segfault as well. Here's my updated version: {{{#!haskell module Main (main) where import Prelude hiding (null, take, drop) import Data.ByteString hiding (elemIndex) import qualified Data.ByteString as B import Data.ByteString.Internal import Data.Word import Foreign elemIndex1 :: Word8 -> ByteString -> Maybe Int elemIndex1 c (PS x s l) = inlinePerformIO $ withForeignPtr x $ \p -> do let p' = p `plusPtr` s q <- memchr p' c (fromIntegral l) return $! if q == nullPtr then Nothing else Just $! q `minusPtr` p' {-# INLINE elemIndex1 #-} elemIndex :: Char -> ByteString -> Maybe Int elemIndex = elemIndex1 . c2w {-# INLINE elemIndex #-} lines1 :: ByteString -> [ByteString] lines1 ps | null ps = [] | otherwise = case search ps of Nothing -> [ps] Just n -> take n ps : lines1 (drop (n+1) ps) where search = elemIndex '\n' main = do f <- B.readFile "00-index.cache" print (Prelude.length $ lines1 f) }}} You can grab the `00-index.cache` file from `/c/Users/$USER/AppData/Roaming/cabal/packages/hackage.haskell.org/00-index.cache` I'm investigating reverting the PAP patch (still building). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:29 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): Replying to [comment:30 jstolarek]:

I applied awson's patch that reverts some of my changes in `CmmSink` and surprisingly I'm still getting a segfault with code posted in comment [[comment:#27]].

Very strange. I have no segfault here with patched HEAD. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:31 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): These are versions, compiled with `-O2 -fmax-simplifier-iterations=10 -fdicts-cheap -fspec-constr-count=6` (`bytestring` package flags) by 7.8rc2 (bad) and patched HEAD (good). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:33 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by simonmar): Here is the broken bit of code, from `lines1_bad`: {{{ c2gC: _s2cV::I64 = R5; _s2cY::I64 = R2 + R4; _c2f5::I64 = R5; (_s2d3::I64) = call "ccall" arg hints: [PtrHint, `signed',] result hints: [PtrHint] memchr(_s2cY::I64, 10, _c2f5::I64); if (_s2d3::I64 == 0) goto c2gK; else goto c2gL; c2gK: call MO_Touch(R3); I64[Hp - 128] = Data.ByteString.Internal.PS_con_info; P64[Hp - 120] = R3; I64[Hp - 112] = R2; I64[Hp - 104] = R4; I64[Hp - 96] = _s2cV::I64; I64[Hp - 88] = :_con_info; P64[Hp - 80] = Hp - 127; P64[Hp - 72] = GHC.Types.[]_closure+1; _c2gw::P64 = Hp - 86; Hp = Hp - 72; R1 = _c2gw::P64; call (P64[Sp])(R1) args: 8, res: 0, upd: 8; }}} Note how R2, R3 and R4 are live across the C call. This is wrong, because on Win64, R3 and R4 are caller-saves and therefore clobbered by the C call. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:35 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): All looks quite the contrary: {{{ ... #define REG_R2 r14 #define REG_R3 rsi #define REG_R4 rdi ... }}} [http://msdn.microsoft.com/en-us/library/6t169e9c.aspx The registers RBX, RBP, RDI, RSI, RSP, R12, R13, R14, and R15 are considered nonvolatile...] And we have *not defined* here: {{{ #if !defined(mingw32_HOST_OS) #define CALLER_SAVES_R3 #define CALLER_SAVES_R4 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:37 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by simonpj): So where are we on this? * Simon M says "here is the broken bit of code" (comment 35), and then says "I misread" (comment 38). Does that mean that the broken bit of code isn't broken? * If we use Karel's reversion patch (between comments 2 and 3 above) does that cure all known crashes? I confirm that it ''does'' fix my own "ghc- stage2 segfaults" problem, reported in #8870. But what about the hello- world problem in #8834, and Ganesh's new report in #8890? * If reverting the `CmmSink` change does in fact solve the problem, we should probably go ahead and revert it, and un-block the GHC 7.8 release. * But even if we do that we are still stuck with not knowing WHY it solves the problem. Perhaps the patch was fine, but it exposes a problem somewhere else? And we really want the `CmmSink` improvements. We really need someone to dig into it with GDB. Austin, can you get a Windows box on the network that exhibits the bug, so that Simon M can crank up gdb? Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:39 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): I've inspected "bad" Cmm a bit and found [https://ghc.haskell.org/trac/ghc/attachment/ticket/8834/lines1_bad#L2276 this]: {{{ call (stg_gc_fun)(R1) args: 40, res: 0, upd: 8; c2gF: if (%MO_S_Le_W64(R5, 0)) goto c2gB; else goto c2gC; }}} `R5` is `r8` and is volatile on Win64. I don't see it restored before `%MO_S_Le_W64(R5, 0)` (honestly, I don't know what `%MO_S_Le_W64(R5, 0)` is). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:41 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): But, AFAIUI, `R5` shall be restored anyway after calling `stg_gc_fun`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:43 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): [https://ghc.haskell.org/trac/ghc/attachment/ticket/8834/lines1_good#L2295 Corresponding code] from "good" cmm is: {{{ I64[Sp - 32] = _s2d4::I64; P64[Sp - 24] = _s2d5::P64; I64[Sp - 16] = _s2d6::I64; I64[Sp - 8] = _s2d7::I64; Sp = Sp - 32; call (stg_gc_fun)(R1) args: 40, res: 0, upd: 8; c2gR: if (%MO_S_Le_W64(_s2d7::I64, 0)) goto c2gN; else goto c2gO; }}} Not that it restores `R5` - it use no `R5` at all. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:45 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): Ah, thanks, assembler intuition was wrong for Cmm :) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:47 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by Simon Peyton Jones ): In [changeset:"a79613a75c7da0d3d225850382f0f578a07113b5/ghc"]: {{{ #!CommitTicketReference repository="ghc" revision="a79613a75c7da0d3d225850382f0f578a07113b5" Revert ad15c2, which causes Windows seg-faults (Trac #8834) We don't yet understand WHY commit ad15c2, which is to do with CmmSink, causes seg-faults on Windows, but it certainly seems to. So reverting it is a stop-gap, but we need to un-block the 7.8 release. Many thanks to awson for identifying the offending commit. }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:49 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by simonpj): I don't think Austin is planning to make the final release in the next day or two, so yes, please go ahead. My reversion was just to un-glue the Windows builds which otherwise are totally stuck. Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:51 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by simonpj): Fine. But reversion works fine too, right? I'm not suggesting abandoning the `CmmSink` improvements, once you figure out what is going on. If meanwhile you want to re-apply, and make the smaller change you propose, that's fine with me Simon -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:53 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by jstolarek): Here's a quick summary of what I've learned: * As we noticed earlier the bug does not happen when only `-O1` is used. Enabling `-O1 -fspec-constr` triggers the bug, but with `-O1 -fspec-constr -fno-cmm-sink` everything works perfectly fine. * I believe that Simon Marlow correctly identified offending piece of Cmm code, although my knowledge of calling conventions doesn't allow me to say why that piece of code is incorrect. Here's how that piece of code looks only with `-O1`: {{{ c2h9: _s2cz::I64 = _s2ct::I64 + _s2cv::I64; (_s2cE::I64) = call "ccall" arg hints: [PtrHint, `signed',] result hints: [PtrHint] memchr(_s2cz::I64, 10, _s2cw::I64); if (_s2cE::I64 == 0) goto c2h4; else goto c2h5; c2h4: Hp = Hp - 32; _s2cH::P64 = Data.Maybe.Nothing_closure+1; goto s2cF; c2h5: I64[Hp - 24] = GHC.Types.I#_con_info; I64[Hp - 16] = _s2cE::I64 - _s2cz::I64; I64[Hp - 8] = Data.Maybe.Just_con_info; P64[Hp] = Hp - 23; _s2cH::P64 = Hp - 6; goto s2cF; s2cF: call MO_Touch(_s2cu::P64); I64[Sp - 40] = c2f7; R1 = _s2cH::P64; I64[Sp - 32] = _s2ct::I64; P64[Sp - 24] = _s2cu::P64; I64[Sp - 16] = _s2cv::I64; I64[Sp - 8] = _s2cw::I64; Sp = Sp - 40; if (R1 & 7 != 0) goto c2f7; else goto c2f8; }}} It looks that sinking is not performed here for some reason that I was unable to figure out. Enabling `-O1 -fspec-constr` changes the above code to: {{{ c2hi: _s2dr::I64 = R5; _s2du::I64 = R2 + R4; _c2fB::I64 = R5; (_s2dz::I64) = call "ccall" arg hints: [PtrHint, `signed',] result hints: [PtrHint] memchr(_s2du::I64, 10, _c2fB::I64); if (_s2dz::I64 == 0) goto c2hd; else goto c2he; c2hd: call MO_Touch(R3); I64[Hp - 128] = Data.ByteString.Internal.PS_con_info; P64[Hp - 120] = R3; I64[Hp - 112] = R2; I64[Hp - 104] = R4; I64[Hp - 96] = _s2dr::I64; I64[Hp - 88] = :_con_info; P64[Hp - 80] = Hp - 127; P64[Hp - 72] = GHC.Types.[]_closure+1; _c2h8::P64 = Hp - 86; Hp = Hp - 72; R1 = _c2h8::P64; call (P64[Sp])(R1) args: 8, res: 0, upd: 8; }}} Enabling `-fspec-constr` sinks R2, R3 and R4 past the call to `memchr` (which BTW. is consistent with awson's earlier observation that segfault is happening somewhere around the call to `memchr`). * Changing definition of `isTrivial` from: {{{ isTrivial :: CmmExpr -> Bool isTrivial (CmmReg _) = True isTrivial (CmmLit _) = True isTrivial _ = False }}} to {{{ isTrivial :: CmmExpr -> Bool isTrivial (CmmReg (CmmLocal _)) = True isTrivial (CmmLit _) = True isTrivial _ = False }}} makes the bug disappear. I believe this is not a proper fix, because it only hides the bug - global registers should be safe to inline! The above Cmm code changes to (`-O1 -fspec-constr` enabled): {{{ 2hi: _s2du::I64 = _s2do::I64 + _s2dq::I64; (_s2dz::I64) = call "ccall" arg hints: [PtrHint, `signed',] result hints: [PtrHint] memchr(_s2du::I64, 10, _s2dr::I64); if (_s2dz::I64 == 0) goto c2hd; else goto c2he; 2hd: call MO_Touch(_s2dp::P64); I64[Hp - 128] = Data.ByteString.Internal.PS_con_info; P64[Hp - 120] = _s2dp::P64; I64[Hp - 112] = _s2do::I64; I64[Hp - 104] = _s2dq::I64; I64[Hp - 96] = _s2dr::I64; I64[Hp - 88] = :_con_info; P64[Hp - 80] = Hp - 127; P64[Hp - 72] = GHC.Types.[]_closure+1; _c2h8::P64 = Hp - 86; Hp = Hp - 72; R1 = _c2h8::P64; call (P64[Sp])(R1) args: 8, res: 0, upd: 8; }}} * Since it looks like the problem is caused by sinking registers past the unsafe foreign call to `memchr` I changed the definition of `foreignTargetRegs` in the instance definition of `DefinerOfRegs` in `CmmNode` (lines 345-346) from: {{{ foreignTargetRegs (ForeignTarget _ (ForeignConvention _ _ _ CmmNeverReturns)) = [] foreignTargetRegs _ = activeCallerSavesRegs }}} to: {{{ foreignTargetRegs (ForeignTarget _ (ForeignConvention _ _ _ CmmNeverReturns)) = [] foreignTargetRegs _ = activeRegs }}} This says that any global register can be clobbered by unsafe foreign call, in practice preventing any global register sinking past such calls. After this change the Cmm dump looks like this (again, `-O1 -fspec-constr` enabled): {{{ c2hi: _s2dr::I64 = R5; _s2dq::I64 = R4; _s2dp::P64 = R3; _s2do::I64 = R2; _s2du::I64 = R2 + R4; _c2fB::I64 = R5; (_s2dz::I64) = call "ccall" arg hints: [PtrHint, `signed',] result hints: [PtrHint] memchr(_s2du::I64, 10, _c2fB::I64); if (_s2dz::I64 == 0) goto c2hd; else goto c2he; c2hd: call MO_Touch(_s2dp::P64); I64[Hp - 128] = Data.ByteString.Internal.PS_con_info; P64[Hp - 120] = _s2dp::P64; I64[Hp - 112] = _s2do::I64; I64[Hp - 104] = _s2dq::I64; I64[Hp - 96] = _s2dr::I64; I64[Hp - 88] = :_con_info; P64[Hp - 80] = Hp - 127; P64[Hp - 72] = GHC.Types.[]_closure+1; _c2h8::P64 = Hp - 86; Hp = Hp - 72; R1 = _c2h8::P64; call (P64[Sp])(R1) args: 8, res: 0, upd: 8; }}} Based on above facts I believe that the problem lies in incorrect definition of caller saves registers. On the other hand I believe we are defining R2, R3 and R4 as callee-saves according to the specification. So there is still possibility that the bug lies somewhere else. At this point I am stuck. I will attach Cmm dumps in a moment so others can verify my findings. All dumps are for the `BugIso` module - they don't include boilerplate from `Main`. Replying to [comment:54 awson]:

Hmm, on my experiments changing `isTrivial` to {{{ isTrivial :: CmmExpr -> Bool isTrivial (CmmReg (CmmLocal _)) = True -- isTrivial (CmmLit _) = True isTrivial _ = False }}} alone was not enough to fix things - see my comment 19. So doing `CmmLit` branch `True` makes this work?

On my experiments the change of `isTrivial` that you just posted is enough to prevent the segmentation fault. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:55 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

* Can you (by experiment) find which of registers is causing the

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by jstolarek): Replying to [comment:56 simonpj]: problem? You fixed it by making them ''all'' caller saves, but that might be more than what's needed.

* You hypothesise that the C procedure `memchr` is destroying a callee-

saves register. Might it be possible to test this hypothesis directly? I looked at the implementation of `memchr` in `glibc` and it looks that on 64 bits it is touching `%rsi` and `%rdi` registers (our R3 and R4, respectively). I changed this definition in [[GhcFile(includes/stg/MachRegs.h)]]: {{{ #if !defined(mingw32_HOST_OS) #define CALLER_SAVES_R3 #define CALLER_SAVES_R4 #endif }}} to: {{{ #define CALLER_SAVES_R3 #define CALLER_SAVES_R4 }}} (ie. I removed the conditional) and the segfault has disappeared. Looking at the Cmm confirms that R3 are R4 are now not sunk past the call to `memchr`: {{{ c2hy: _s2dH::I64 = R5; _s2dG::I64 = R4; _s2dF::P64 = R3; _s2dK::I64 = R2 + R4; _c2fR::I64 = R5; (_s2dP::I64) = call "ccall" arg hints: [PtrHint, `signed',] result hints: [PtrHint] memchr(_s2dK::I64, 10, _c2fR::I64); if (_s2dP::I64 == 0) goto c2ht; else goto c2hu; c2ht: call MO_Touch(_s2dF::P64); I64[Hp - 128] = Data.ByteString.Internal.PS_con_info; P64[Hp - 120] = _s2dF::P64; I64[Hp - 112] = R2; I64[Hp - 104] = _s2dG::I64; I64[Hp - 96] = _s2dH::I64; I64[Hp - 88] = :_con_info; P64[Hp - 80] = Hp - 127; P64[Hp - 72] = GHC.Types.[]_closure+1; _c2ho::P64 = Hp - 86; Hp = Hp - 72; R1 = _c2ho::P64; }}} But one thing is not right here: {{{ #if !defined(mingw32_HOST_OS) #define CALLER_SAVES_R3 #define CALLER_SAVES_R4 #endif }}} Under 64 bit Windows `mingw32_HOST_OS` should not be declared and therefore R3 and R4 should be defined as caller saves! As a quick sanity check I wrote this small C program and compiled it with the in-tree mingw gcc: {{{ #include #if !defined(mingw32_HOST_OS) int x = 1; #else int x = 2; #endif int main() { printf("%d",x); } }}} This program prints `1` as expected. Does anyone have any idea what might be going on here? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:57 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by awson): `glibc` is completely irrelevant here. `memchr` comes from system dll `msvcrt.dll` and I doubt it is wrong. So, perhaps, our idea of what is callee-saves is wrong. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:59 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by refold): Replying to [comment:60 jstolarek]:

...

glibc is completely irrelevant here. memchr comes from system dll msvcrt.dll and I doubt it is wrong.

Hmm... I doubt we can take a look at source of that dll ;-)

I believe that Microsoft ships the CRT source with Visual Studio. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:61 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by jstolarek): Replying to [comment:61 refold]:

Replying to [comment:60 jstolarek]:

...

...

glibc is completely irrelevant here. memchr comes from system dll msvcrt.dll and I doubt it is wrong.

Hmm... I doubt we can take a look at source of that dll ;-)

I believe that Microsoft ships the CRT source with Visual Studio.

Great. Does anyone have access to Visual Studio and could check the implementation of `memchr`? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:63 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: patch Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by igloo): http://msdn.microsoft.com/en-us/library/6t169e9c.aspx says {{{ The registers RAX, RCX, RDX, R8, R9, R10, R11 are considered volatile and must be considered destroyed on function calls (unless otherwise safety- provable by analysis such as whole program optimization). The registers RBX, RBP, RDI, RSI, RSP, R12, R13, R14, and R15 are considered nonvolatile and must be saved and restored by a function that uses them. }}} so the saving info looks right to me. It wouldn't be too surprising if saving and restoring registers more worked around a register corruption bug. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:65 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Changes (by thoughtpolice): * owner: => jstolarek -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:67 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: new Priority: high | Milestone: 7.8.2 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by simonmar): Hold the release! I've found the bug. It's in the native codegen. Patch coming soon. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:69 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by simonmar): Oh, and you also have to revert a79613a75c7da0d3d225850382f0f578a07113b5 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:71 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: jstolarek Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by simonmar): Yeah, this is definitely the cause of the crash on 64-bit Windows, so if 32-bit is also crashing then there must be another bug somewhere. @thoughtpolice is going to test again and see if it's still crashing. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:73 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by Austin Seipp ): In [changeset:"c4eeacdfdf4578eb6e75bbf2e067bfe70ec94ab0/ghc"]: {{{ #!CommitTicketReference repository="ghc" revision="c4eeacdfdf4578eb6e75bbf2e067bfe70ec94ab0" Use the correct callClobberedRegs on Windows/x64 (#8834) Signed-off-by: Austin Seipp }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:75 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by thoughtpolice): I've definitely managed to reproduce this finally after a bunch of hunting - it doesn't really seem related to OS version, JUST related to MSYS2. #8870 is the same thing, I'm quite certain (failure at `CPSZ` output during segfault if you check `-v3`). It just doesn't make sense to me why the testsuite runs clean on my machine where I built the bindist, and everything works and compiles, but it fails for users and here. I'm running the testsuite right now and I can see some isolated failures that I'm quite sure are illegitimate (several segfaults), i'll report the results here, `./validate` is about half done. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:77 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by thoughtpolice): Okay, I spent some time boiling some things down, and I've at least determined the approximate location of the segfault in the code during compilation, which is `stmtToInstrs` in `compiler/nativeGen/X86/CodeGen.hs`. Here's just a quick dump (to not loose findings) and I'll keep looking around. The fault is when compiling `System.Time` in profiling. Run under gdb: {{{ $ gdb --args "inplace/bin/ghc-stage2.exe" -v3 -hisuf p_hi -osuf p_o -hcsuf p_hc -static -prof -H32m -O -package-name old-time-1.i -ilibraries/old-time/. -ilibraries/old-time/dist-install/build -ilibraries /old-time/dist-install/build/autogen -Ilibraries/old-timearies/old-time /dist-install/build/autogen -Ilibraries/old-time/include -optP-include -optPlibraries/old-time/dist-install/build/auage base-4.7.0.0 -package old-locale-1.0.0.6 -Wall -XHaskell2010 -O2 -no-user-package-db -rtsopts -odir libraries/old-time/distaries/old-time/dist-install/build -stubdir libraries/old-time/dist-install/build -c libraries/old-time/dist- install/build/System/Tie/dist-install/build/System/Time.p_o +RTS -DS GNU gdb (GDB) 7.6.1 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i686-pc-msys". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Traceback (most recent call last): File "<string>", line 3, in <module> ImportError: No module named libstdcxx.v6.printers /etc/gdbinit:6: Error in sourced command file: Error while executing Python code. Reading symbols from /home/Administrator/ghc/inplace/bin/ghc- stage2.exe...done. warning: File "/home/Administrator/ghc/.gdbinit" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto- load". To enable execution of this file add add-auto-load-safe-path /home/Administrator/ghc/.gdbinit line to your configuration file "/home/Administrator/.gdbinit". To completely disable this security protection add set auto-load safe-path / line to your configuration file "/home/Administrator/.gdbinit". For more information about this security protection see the "Auto-loading safe path" section in the GDB manual. E.g., run from the shell: info "(gdb)Auto-loading safe path" (gdb) load .gdbinit You can't do that when your target is `exec' (gdb) source .gdbinit (gdb) r Starting program: /home/Administrator/ghc/inplace/bin/ghc-stage2.exe -v3 -hisuf p_hi -osuf p_o -hcsuf p_hc -static -prof -H32m -O -package-name old-time-1.1.0.2 -hide-all-packages -i -ilibraries/old-time/. -ilibraries /old-time/dist-install/build -ilibraries/old-time/dist- install/build/autogen -Ilibraries/old-time/dist-install/build -Ilibraries /old-time/dist-install/build/autogen -Ilibraries/old-time/include -optP- include -optPlibraries/old-time/dist-install/build/autogen/cabal_macros.h -package base-4.7.0.0 -package old-locale-1.0.0.6 -Wall -XHaskell2010 -O2 -no-user-package-db -rtsopts -odir libraries/old-time/dist-install/build -hidir libraries/old-time/dist-install/build -stubdir libraries/old-time /dist-install/build -c libraries/old-time/dist- install/build/System/Time.hs -o libraries/old-time/dist- install/build/System/Time.p_o +RTS -DS [New Thread 1136.0xcc8] cc8: cap 0: initialised [New Thread 1136.0x15e8] [New Thread 1136.0x1658] [New Thread 1136.0x11b8] [New Thread 1136.0x11e8] [New Thread 1136.0x1718] Glasgow Haskell Compiler, Version 7.9.20140329, stage 2 booted by GHC version 7.6.3 Using binary package database: C:\Users\Administrator\Desktop\msys32\home\Administrator\ghc\inplace\lib\package.conf.d\package.cache wired-in package ghc-prim mapped to ghc-prim-0.3.1.0-inplace wired-in package integer-gmp mapped to integer-gmp-0.5.1.0-inplace wired-in package base mapped to base-4.7.0.0-inplace wired-in package rts mapped to builtin_rts wired-in package template-haskell mapped to template- haskell-2.10.0.0-inplace wired-in package dph-seq not found. wired-in package dph-par not found. Hsc static flags: *** Checking old interface for old-time-1.1.0.2:System.Time: *** Parser: *** Renamer/typechecker: *** Desugar: Result size of Desugar (after optimization) = {terms: 5,701, types: 3,843, coercions: 29} ... *** Tidy Core: Result size of Tidy Core = {terms: 15,413, types: 10,079, coercions: 582} Created temporary directory: C:\Users\Administrator\Desktop\msys32\tmp\ghc1136_0 *** CorePrep: Result size of CorePrep = {terms: 18,936, types: 12,028, coercions: 582} *** Stg2Stg: *** CodeOutput: *** New CodeGen: *** CPSZ: *** CPSZ: *** CPSZ: *** CPSZ: *** CPSZ: Program received signal SIGSEGV, Segmentation fault. 0x02137032 in c1hhA_info () (gdb) bt #0 0x02137032 in c1hhA_info () Cannot access memory at address 0x28a874 (gdb) disassemble Dump of assembler code for function c1hhA_info: 0x02137024 <+0>: sub $0x3510,%esp 0x0213702a <+6>: mov 0x8(%ebp),%eax 0x0213702d <+9>: mov 0x4(%ebp),%ecx 0x02137030 <+12>: mov %esi,%edx => 0x02137032 <+14>: mov %eax,0x184(%esp) 0x02137039 <+21>: mov -0x1(%edx),%eax 0x0213703c <+24>: movzwl -0x2(%eax),%eax 0x02137040 <+28>: cmp $0x1e,%eax 0x02137043 <+31>: ja 0x214916f 0x02137049 <+37>: mov %eax,0x190(%esp) 0x02137050 <+44>: mov 0x1c(%ebp),%eax 0x02137053 <+47>: mov %eax,0xa0(%esp) 0x0213705a <+54>: mov 0x190(%esp),%eax 0x02137061 <+61>: jmp *0x2b86708(,%eax,4) 0x02137068 <+68>: inc %edx 0x02137069 <+69>: add %al,(%eax) 0x0213706b <+71>: add %ah,(%eax) 0x0213706d <+73>: add %al,(%eax) 0x0213706f <+75>: add %al,0x3510ec(%ecx) End of assembler dump. (gdb) (gdb) info registers eax 0x6b3b05d 112439389 ecx 0x6b49524 112497956 edx 0x67a40a9 108675241 ebx 0x2bc3470 45888624 esp 0x28a874 0x28a874 ebp 0x4697cc8 0x4697cc8 esi 0x67a40a9 108675241 edi 0x6b4ac20 112503840 eip 0x2137032 0x2137032 eflags 0x10202 [ IF RF ] cs 0x23 35 ss 0x2b 43 ds 0x2b 43 es 0x2b 43 fs 0x53 83 gs 0x2b 43 (gdb) p8 $ebp 0x4697ce4: 0x6b488d9 0x4697ce0: 0x6b4ac18 0x4697cdc: 0x6300ef1e 0x4697cd8: 0x6b4954d 0x4697cd4: 0x67a40a9 0x4697cd0: 0x6b3b05d 0x4697ccc: 0x6b49524 0x4697cc8: 0x2137024 (gdb) pinfo &c1hhA_info $1 = {layout = {payload = {ptrs = 903, nptrs = 0}, bitmap = 903, large_bitmap_offset = 903, selector_offset = 903}, type = 32, srt_bitmap = 7, code = 0x2137024 "\201\354\020\065"} (gdb) prinfo &c1hhA_info $2 = {srt_offset = 8706840, i = {layout = {payload = {ptrs = 903, nptrs = 0}, bitmap = 903, large_bitmap_offset = 903, selector_offset = 903}, type = 32, srt_bitmap = 7, code = 0x2137024 "\201\354\020\065"}} }}} The fault is in `c1hhA_info`. Finding that symbol: {{{ $ find compiler/stage2 -type f | xargs grep c1hhA_info Binary file compiler/stage2/build/libHSghc-7.9.20140329.a matches Binary file compiler/stage2/build/X86/CodeGen.o matches }}} It's in `./nativeGen/X86/CodeGen.hs` - check the `-ddump-opt-cmm` out to get corresponding optimized code, finding the closure for the info table: {{{ ==================== Optimised Cmm ==================== a292_rFAj_entry() // [] { [(c18OH, block_c18OH_info: const u1hJf_srtd-block_c18OH_info; const 1; const 4294901792;), (c18OQ, block_c18OQ_info: const u1hJg_srtd-block_c18OQ_info; const 3; const 4294901792;), ... ... ... (c1hhA, block_c1hhA_info: const SUys_srt-block_c1hhA_info+2332; const 903; const 458784;), ... ... ... c1hyQ: I32[Hp - 8] = $w$j_sSgL_info; I32[Hp - 4] = I32[Sp + 24]; I32[Hp] = I32[Sp + 20]; I32[Sp] = block_c1hhA_info; R1 = P32[Sp + 12]; P32[Sp + 24] = Hp - 8; if (R1 & 3 != 0) goto c1hhA; else goto c1hhB; c1hhB: call (I32[R1])(R1) returns to c1hhA, args: 4, res: 4, upd: 4; c1hhA: _sSgE::P32 = P32[Sp + 8]; _sSgI::P32 = P32[Sp + 4]; _sSmu::P32 = R1; _c1hub::I32 = %MO_UU_Conv_W16_W32(I16[I32[_sSmu::P32 - 1] - 2]); if (_c1hub::I32 > 30) goto c1hug; else goto c1hue; ... ... }}} We can see this matches pretty closely with the assembly generated around the offending code (`-ddump-asm`): {{{ _c1hyQ: movl $$w$j_sSgL_info,-8(%edi) movl 24(%ebp),%eax movl %eax,-4(%edi) movl 20(%ebp),%eax movl %eax,(%edi) movl $block_c1hhA_info,(%ebp) movl 12(%ebp),%esi leal -8(%edi),%eax movl %eax,24(%ebp) testl $3,%esi jne _n1kED ... .text .align 4,0x90 .long SUys_srt-(block_c1hhA_info)+2332 .long 903 .long 458784 block_c1hhA_info: _c1hhA: subl $13584,%esp _n1kED: movl 8(%ebp),%eax movl 4(%ebp),%ecx movl %esi,%edx movl %eax,388(%esp) movl -1(%edx),%eax movzwl -2(%eax),%eax cmpl $30,%eax ja _c1hug }}} Next, looking at the STG output, `a292_rFAj` looks like: {{{ a292_rFAj :: forall e_a94q x_a94r. CmmNode.CmmNode e_a94q x_a94r -> NCGMonad.NatM_State -> (X86.CodeGen.InstrBlock, NCGMonad.NatM_State) ... }}} If you search for calls to this, there is one call to it from `a290_rFAh`, which looks like: {{{ a290_rFAh :: CmmMachOp.CallishMachOp -> Data.Maybe.Maybe CmmNode.CmmFormal -> [CmmNode.CmmActual] -> NCGMonad.NatM_State -> (X86.CodeGen.InstrBlock, NCGMonad.NatM_State) ... let { sat_sMWp [Occ=Once] :: CmmNode.CmmNode Compiler.Hoopl.Block.O Compiler.Hoopl.Block.O [LclId, Str=DmdType] = NO_CCS CmmNode.CmmUnsafeForeignCall! [GHC.Prim.coercionToken# GHC.Prim.coercionToken# sat_sMWb sat_sMWd sat_sMWo]; } in a292_rFAj sat_sMWp st'_sMWa; }}} The only thing that has a type of `CallishMachOp -> ...` is `outOfLineCmmOp`, which does indeed call `stmtToInstrs (CmmUnsafeForeignCall ...)` like in the above snippet. The type of `stmtToInstrs` also matches the (desugared) type of `a292_rFAj`. So this is certainly where the fault is occurring. Unfortunately `stmtToInstrs` becomes incredibly large it seems, so pinning it down further is proving challening. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:79 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: new Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by thoughtpolice): The only way I can get this to *not* segfault is if I completely disable optimization with `{-# OPTIONS_GHC -O0 #-}` in `CodeGen.hs`. I'm going to see if the build completes and run the testsuite again to see what it says... (BTW, this is a typical performance build, so everything will be compiled with -O, at least). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:81 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: closed Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: fixed | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Changes (by thoughtpolice): * status: new => closed * resolution: => fixed Comment: Whoops, yes, this is fixed. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:83 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#8834: 64-bit windows cabal.exe segfaults in GC ----------------------------------+---------------------------------- Reporter: awson | Owner: Type: bug | Status: closed Priority: highest | Milestone: 7.8.1 Component: Compiler | Version: 7.8.1-rc2 Resolution: fixed | Keywords: Operating System: Windows | Architecture: x86_64 (amd64) Type of failure: Runtime crash | Difficulty: Unknown Test Case: | Blocked By: Blocking: | Related Tickets: ----------------------------------+---------------------------------- Comment (by Austin Seipp ): In [changeset:"c6c86789c95462216a3167d7b98b202a5bf4c0b2/ghc"]: {{{ #!CommitTicketReference repository="ghc" revision="c6c86789c95462216a3167d7b98b202a5bf4c0b2" Revert "Revert ad15c2, which causes Windows seg-faults (Trac #8834)" This reverts commit a79613a75c7da0d3d225850382f0f578a07113b5. }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8834#comment:85 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler