#10672: checkProddableBlock crash during Template Haskell linking -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Revisions: #8237 #9907 | -------------------------------------+------------------------------------- Changes (by lukexi): * related: #9297 #10563 #8237 => #9297 #10563 #8237 #9907 Comment: I've tested on GHC 7.8.4 now and get this error instead: {{{ Loading package cxxylib-0.1.0.0 ... ghc.exe: Unknown PEi386 section name `.gcc_except_table' (while processing: C:\msys64\home\lukex_000\cxx-link- fail-repro\dist\build\HScxxylib-0.1.0.0.o) <no location info>: ghc.exe: panic! (the 'impossible' happened) (GHC version 7.8.4 for x86_64-unknown-mingw32): loadObj "C:\\msys64\\home\\lukex_000\\cxx-link-fail- repro\\dist\\build\\HScxxylib-0.1.0.0.o": failed Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug }}} so it looks like this is related to #9907 and friends. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: checkProddableBlock crash during Template Haskell linking -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Revisions: #8237 #9907 | -------------------------------------+------------------------------------- Comment (by lukexi): Results of 7.8.4 with {{{bullet-mini}}}, which is using {{{-fno- exceptions}}} to avoid linking {{{libgcc_s_sjlj-1.dll}}} {{{ Loading package bullet-mini-0.1.0.0 ... ghc.exe: Unknown PEi386 section name `.text$_ZN21btBroadphaseInterfaceD1Ev' (while processing: C:\msys64\home\lukex_000\Projects\bullet-mini\dist\build\HSbullet- mini-0.1.0.0.o) <no location info>: ghc.exe: panic! (the 'impossible' happened) (GHC version 7.8.4 for x86_64-unknown-mingw32): loadObj "C:\\msys64\\home\\lukex_000\\Projects\\bullet- mini\\dist\\build\\HSbullet-mini-0.1.0.0.o": failed Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: checkProddableBlock crash during Template Haskell linking -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Type: bug | Status: new Priority: high | Milestone: 7.10.3 Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Revisions: #8237 #9907 | -------------------------------------+------------------------------------- Changes (by lukexi): * milestone: => 7.10.3 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: checkProddableBlock crash during Template Haskell linking -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Type: bug | Status: new Priority: high | Milestone: 7.10.3 Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Revisions: #8237 #9907 | -------------------------------------+------------------------------------- Comment (by lukexi): To anyone else bitten by this: While this is figured out, I'm just stripping out all uses of Template Haskell in my software since I'm only using it for generating lenses. I put up a simple preprocessor here to do that during the build phase https://github.com/lukexi/strip-ths, designed to work with stack or cabal. (this workaround works because, as I mention in the README for the repro,

Everything also works [...] if I just call cabal exec -- ghc app/Main.hs. This is presumably because GHC doesn't try to link cxxylib when it is just compiling the Template Haskell splices, whereas Cabal asks it to. )

-- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: checkProddableBlock crash during Template Haskell linking -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Type: bug | Status: new Priority: high | Milestone: 7.10.3 Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Revisions: #8237 #9907 | -------------------------------------+------------------------------------- Changes (by Phyx-): * cc: Phyx- (added) Comment: Hmm, this seems to happen because in `checkProddableBlock` the `oc->proddables` is NULL. So it has no blocks to check and it errors out. Don't know why it errors out and doesn't skip that ObjectCode and continue when there are no `proddables`. I'm pretty new at the linker so hopefully someone who knows more about this part can explain it. Also it won't work on anything prior to 7.10.x since you'll get the Unknown PE section error fixed in #9907 I don't think #9297 is related either, which seems to have more to do with the name manglings. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: checkProddableBlock crash during Template Haskell linking -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Type: bug | Status: new Priority: high | Milestone: 7.10.3 Component: Compiler | Version: 7.10.1 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Revisions: #8237 #9907 | -------------------------------------+------------------------------------- Comment (by lukexi): Hi Edward, thanks for the reply! Just to be sure, do you mean compiling a GHC with GhcDebugged=YES and then using cabal build --ghc-options="+RTS -Dl"? Luke -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: GHCi linker does not understand C++ exception tables on Windows -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Type: bug | Status: new Priority: high | Milestone: 7.10.3 Component: Runtime System | Version: 7.10.1 (Linker) | Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Revisions: #8237 #9907 | -------------------------------------+------------------------------------- Changes (by ezyang): * component: Compiler => Runtime System (Linker) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: GHCi linker does not understand C++ exception tables on Windows -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Phyx- Type: bug | Status: new Priority: high | Milestone: 7.10.3 Component: Runtime System | Version: 7.10.1 (Linker) | Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Revisions: #8237 #9907 | -------------------------------------+------------------------------------- Comment (by lukexi): Hi Phyx-/all — let me know if there are any public repos with work-in- progress on this; I'd love to help out! Planning on giving it some time later this week. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: GHCi linker does not understand C++ exception tables on Windows -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Phyx- Type: bug | Status: new Priority: high | Milestone: 7.10.3 Component: Runtime System | Version: 7.10.1 (Linker) | Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Revisions: #8237 #9907 | -------------------------------------+------------------------------------- Comment (by lukexi): Awesome stuff Tamar!! I confirmed your branch works on my test case and also backported the patches to 7.10.2 at https://github.com/lukexi/ghc/tree/ghc-7.10.2 -release-plus-rework-windows-pe-linker (git did 99% of the work, they mostly applied cleanly) and was able to build all of `bullet-mini` flawlessly, hurray! Thanks so much for your hard work. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:17 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: GHCi linker does not understand C++ exception tables on Windows -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Phyx- Type: bug | Status: patch Priority: high | Milestone: 7.10.3 Component: Runtime System | Version: 7.10.1 (Linker) | Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Rev(s): Phab:D1244 #8237 #9907 | -------------------------------------+------------------------------------- Comment (by Thomas Miedema ): In [changeset:"620fc6f909cd6e51b5613454097ec1c9f323839a/ghc" 620fc6f9/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="620fc6f909cd6e51b5613454097ec1c9f323839a" Make Windows linker more robust to unknown sections The Windows Linker has 3 main parts that this patch changes. 1) Identification and classification of sections 2) Adding of symbols to the symbols tables 3) Reallocation of sections 1. Previously section identification used to be done on a whitelisted basis. It was also exclusively being done based on the names of the sections. This meant that there was a bit of a cat and mouse game between `GCC` and `GHC`. Every time `GCC` added new sections there was a good chance `GHC` would break. Luckily this hasn't happened much in the past because the `GCC` versions `GHC` used were largely unchanged. The new code instead treats all new section as `CODE` or `DATA` sections, and changes the classifications based on the `Characteristics` flag in the PE header. By doing so we no longer have the fragility of changing section names. The one exception to this is the `.ctors` section, which has no differentiating flag in the PE header, but we know we need to treat it as initialization data. The check to see if the sections are aligned by `4` has been removed. The reason is that debug sections often time are `1 aligned` but do have relocation symbols. In order to support relocations of `.debug` sections this check needs to be gone. Crucially this assumption doesn't seem to be in the rest of the code. We only check if there are at least 4 bytes to realign further down the road. 2. The second loop is iterating of all the symbols in the file and trying to add them to the symbols table. Because the classification of the sections we did previously are (currently) not available in this phase we still have to exclude the sections by hand. If they don't we will load in symbols from sections we've explicitly ignored the in # 1. This whole part should rewritten to avoid this. But didn't want to do it in this commit. 3. Finally the sections are relocated. But for some reason the PE files contain a Linux relocation constant in them `0x0011` This constant as far as I can tell does not come from GHC (or I couldn't find where it's being set). I believe this is probably a bug in GAS. But because the constant is in the output we have to handle it. I am thus mapping it to the constant I think it should be `0x0003`. Finally, static linking *should* work, but won't. At least not if you want to statically link `libgcc` with exceptions support. Doing so would require you to link `libgcc` and `libstd++` but also `libmingwex`. The problem is that `libmingwex` also defines a lot of symbols that the RTS automatically injects into the symbol table. Presumably because they're symbols that it needs. like `coshf`. The these symbols are not in a section that is declared with weak symbols support. So if we ever want to get this working, we should either a) Ask mingw to declare the section as such, or b) treat all a imported symbols as being weak. Though this doesn't seem like it's a good idea.. Test Plan: Running ./validate for both x86 and x86_64 Also running the specific test case for #10672 make TESTS="T10672_x86 T10672_x64" Reviewed By: ezyang, thomie, austin Differential Revision: https://phabricator.haskell.org/D1244 GHC Trac Issues: #9907, #10672, #10563 }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:19 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10672: GHCi linker does not understand C++ exception tables on Windows -------------------------------------+------------------------------------- Reporter: lukexi | Owner: Phyx- Type: bug | Status: merge Priority: high | Milestone: 7.10.3 Component: Runtime System | Version: 7.10.1 (Linker) | Resolution: | Keywords: Operating System: Windows | Architecture: | Unknown/Multiple Type of failure: Compile-time | Test Case: crash | rts/T10672/T10672_x86 T10672_x64 Blocked By: | Blocking: Related Tickets: #9297 #10563 | Differential Rev(s): Phab:D1244 #8237 #9907 | -------------------------------------+------------------------------------- Comment (by thomie): If #10726 is going to be merged, a path of without merge conflicts is: * 9f7cdfee3e9f9ca6fbfa27d3b2dc2d86ac4ee226 (#10705) * a442800fd27952bff9bf9773f514ee062f4b55d0 (#10705) * 54b7dc5537f8b871e655752bace1ad6f5e6fe89a * 7b211b4e5a38efca437d76ea442495370da7cc9a (#10726) * 620fc6f909cd6e51b5613454097ec1c9f323839a (this ticket) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10672#comment:21 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler