#16057: GHC 8.6.3 hangs with Template Haskell on Windows 10 -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Template Haskell | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: x86 Type of failure: Compile-time | Test Case: crash or panic | Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Description changed by gizmo.mk0: Old description:

Have these two files in a directory:

Main.hs: {{{#!hs {-# LANGUAGE TemplateHaskell #-} module Main where import Helper

main = return ()

test }}}

Helper.hs: {{{#!hs {-# LANGUAGE TemplateHaskell #-} module Helper where import Language.Haskell.TH

test :: Q [Dec] test = return [] }}}

If compiled with `ghc Main.hs`, the compilation freezes, when it reaches `Main`.

This works well on Linux (Ubuntu 18) using GHC 8.6.3, and there is no problem in GHC 8.4.3 on either Windows or Linux.

GCC version: 8.2.0 Rev3 (from MSYS2) / 7.2.0 Rev1 (from cmd.exe) Adding -dcore-lint does not change the outcome.

New description: Have these two files in a directory: Main.hs: {{{#!hs {-# LANGUAGE TemplateHaskell #-} module Main where import Helper main = return () test }}} Helper.hs: {{{#!hs {-# LANGUAGE TemplateHaskell #-} module Helper where import Language.Haskell.TH test :: Q [Dec] test = return [] }}} If compiled with `ghc Main.hs`, the compilation freezes, when it reaches `Main`. This compiles without problems on Linux (Ubuntu 18) using GHC 8.6.3, and there is no problem in GHC 8.4.3 on either Windows or Linux. GCC version: 8.2.0 Rev3 (from MSYS2) / 7.2.0 Rev1 (from cmd.exe) Adding -dcore-lint does not change the outcome. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 hangs with Template Haskell on Windows 10 -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: normal | Milestone: Component: Template Haskell | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: x86 Type of failure: Compile-time | Test Case: crash or panic | Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): Yeah I've started taking a look at it, I can't reproduce it in HEAD nor can I with 8.6.2, only 8.6.3. Where it ends is an endless loop in {{{ #0 0x0000000003d53506 in interpretBCO (cap=0x4604380 <MainCapability>) at rts\Interpreter.c:395 #1 0x0000000003d588b7 in schedule (initialCapability=initialCapability@entry=0x4604380 <MainCapability>, task=<optimized out>, task@entry=0x4d6ef00) at rts\Schedule.c:458 #2 0x0000000003d5914b in scheduleWaitThread (tso=0x7505ba8, ret=ret@entry=0x0, pcap=pcap@entry=0x4cdfd18) at rts\Schedule.c:2533 #3 0x0000000003d50968 in rts_evalLazyIO (cap=cap@entry=0x4cdfd18, p=p@entry=0x3d9f0d8 , ret=ret@entry=0x0) at rts\RtsAPI.c:530 #4 0x0000000003d4fc95 in hs_main (argc=<optimized out>, argv=<optimized out>, main_closure=0x3d9f0d8 , rts_config=...) at rts\RtsMain.c:72 }}} But since it's a release build everything has been optimized out. I started a debug build of 8.6.3 to see. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 hangs with Template Haskell on Windows 10 -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: Template Haskell | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: x86 Type of failure: Compile-time | Test Case: crash or panic | Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by RyanGlScott): Replying to [comment:4 Phyx-]:

Another question remains.. do we not have any actual TH tests in the testsuite???

We certainly do—see the `testsuite/tests/th` directory. I have no idea if any of those tests are exercising this particular code path, however. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 hangs with Template Haskell on Windows 10 -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: Template Haskell | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: x86 Type of failure: Compile-time | Test Case: crash or panic | Blocked By: | Blocking: Related Tickets: #16071 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by RyanGlScott): * related: => #16071 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 hangs with Template Haskell on Windows 10 -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: Template Haskell | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: x86 Type of failure: Compile-time | Test Case: crash or panic | Blocked By: | Blocking: Related Tickets: #16071 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Phyx-): Hi Simon, I started looking last weekend and will continue coming one. Luckily the byte code interpreter seems to crash already at the first BCO it's trying to interpret. Just working my way through the scheduler to try to figure out what it was supposed to be. If I don't find the cause this weekend I'll be sure to ask for help. I only have time to look at GHC in the weekends so don't want to hold things up on me :) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 byte-code interpreter segfaults on any object. -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: Runtime System | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #16071 #13617 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by Phyx-): * architecture: x86 => Unknown/Multiple * related: #16071 => #16071 #13617 Comment: Like all good and bad things in life, this comes down to alignment. The change in ed86e3b531322f74d2c2d00d7ff8662b08fabde6 triggers a missing feature in the 8.6 branch. The `-Wa,-mbig-obj` changes the object code format, causing the COFF header to grow by 2 bytes. These two bytes push everything down, and causes the symbols which just happened to be aligned to the required section alignment to be misaligned. The `.text` section requires 16 byte alignment, and the ByteCode interpeter makes use of this to store tag bits in the lower bits. So a couple of things go wrong here. The new misaligned sections require the linker to actually do section alignment. Before: {{{ addSymbol 000000000c6900f0 `Main_main_info' addSymbol 000000000c6901e0 `Main_main_closure' addSymbol 000000000c690158 `ZCMain_main_info' addSymbol 000000000c690220 `ZCMain_main_closure' addSymbol 000000000c690260 `Main_zdtrModule_closure }}} Everything happened to be aligned in most cases. Now everything is unaligned: {{{ addSymbol 000000000c690114 `Main_main_info addSymbol 000000000c690204 `Main_main_closure' addSymbol 000000000c69017c `ZCMain_main_info' addSymbol 000000000c690244 `ZCMain_main_closure' addSymbol 000000000c690284 `Main_zdtrModule_closure' }}} The byte code interpreter then gets the unaligned address and thinks it's a tagged object and strips the bottom bits off {{{ bcoSize = 3 Sp = 00000000061f0be0 pc = 0 PUSH_G 000000000d8801b4 Sp = 00000000061f0bd8 pc = 2 ENTER --------------------------------------------------------------- Evaluating: Object 000000000d8801b0 = }}} And segfaults as the address is now nonsense. The change works on HEAD as I added alignment support to the linker in head for GHC 8.8. Now the question is how to proceed. There are two options 1) Revert this patch for the 8.6 branches and lose profiling support for a while till GHC 8.8 2) I can produce a much simpler alignment fix patch, but it'll consume more memory then the bigger linker overhaul in HEAD currently, but.. it won't have had as much testing as the version in HEAD has.. so will need to be able to reason about it's correctness instead. I suppose most people would favor no# 2 so I'll get started on that. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 byte-code interpreter segfaults on any object. -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: Runtime System | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #16071 #13617 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by awson): Hi, Tamar, which command-line options should I use to get such an output

{{{ bcoSize = 3 Sp = 00000000061f0be0 pc = 0 PUSH_G 000000000d8801b4 Sp = 00000000061f0bd8 pc = 2 ENTER

--------------------------------------------------------------- Evaluating: Object 000000000d8801b0 = }}} ?

-- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 byte-code interpreter segfaults on any object. -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: Runtime System | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #16071 #13617 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Replying to [comment:12 Phyx-]:

hmm having looked at the linker status in 8.6 it's non-trivial to add alignment support there, and changing the linker in the stable branch this much seems like not a good idea..

I think the best option would be to revert ed86e3b531322f74d2c2d00d7ff8662b08fabde6 in the 8.6 branch, we were able to build before without it and nothing changed in the interim between 8.6.2 and 8.6.3 should have increased the amount of symbols.

The fix is required in HEAD and works as intended there. Thoughts @bgamari?

Yes, I can't think of any reasonable alternatives to reverting. Let's do that and cut a yet another release. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 byte-code interpreter segfaults on any object. -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: Component: Runtime System | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #16071 #13617 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Thank you for tracking this down, Tamar! We would be truly lost without you. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 byte-code interpreter segfaults on any object. -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: new Priority: highest | Milestone: 8.6.4 Component: Runtime System | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #16071 #13617 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by juhpetersen): * milestone: => 8.6.4 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#16057: GHC 8.6.3 byte-code interpreter segfaults on any object. -------------------------------------+------------------------------------- Reporter: gizmo.mk0 | Owner: (none) Type: bug | Status: closed Priority: highest | Milestone: 8.6.4 Component: Runtime System | Version: 8.6.3 Resolution: fixed | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash or panic | Test Case: Blocked By: | Blocking: Related Tickets: #16071 #13617 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by Phyx-): * status: new => closed * resolution: => fixed Comment: Ty bgamari :) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16057#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler