[GHC] #16084: Improve link times on Windows
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Keywords: | Operating System: Windows Architecture: | Type of failure: Compile-time Unknown/Multiple | performance bug Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- Currently the Windows CI builds are typically lasting over 3 hours. The primary cause of this appears to be poor performance in `ld.bfd`, especially when linking testsuite tests. One option to fix this is to try using LLD for linking. Unfortunately the msys2 `gcc` does not support `-fuse-ld=lld`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I have opened an `msys2` [[https://github.com/Alexpux/MSYS2-packages/issues/1531|ticket]] requesting that `gcc` ship with support for `-fuse-ld=lld`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: upstream Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => upstream -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: upstream Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * cc: Phyx (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: upstream Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Tamar has done a bit of working optimising `ld.bfd`: https://github.com/Mistuke/binutils-gdb -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
One option to fix this is to try using LLD for linking. Unfortunately
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: upstream Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by awson): Replying to [ticket:16084 bgamari]: the msys2 `gcc` does not support `-fuse-ld=lld`. LLD won't magically help here. Latest improvements by Martin Storsjö (mstorsjo) have made LLD able to link some mingw **gcc**-generated code, but, alas, when assembling **GHC**-generated assembly, mingw **gas** produces (I believe this is a sort of "optimisation") non-standard relocations, which LLD is unable to deal with (honestly, I haven't checked if this is still the case, but it was definitely so a year or two ago). For quite a bit of time I have a GHC port which works flawlessly against native Windows SDK and uses clang in MSVC mode and MS or LLD linker. After recent LLD improvements by mstorsjo I've decided to try a less intrusive approach — use clang in mingw mode and LLD linker. That required quite a bit of work and I finally managed to produce stage2 GHC executable, which appeared to be severely broken — it is unable to do anything. TL;DR the current state of affairs is that `lld` **can't** serve as a drop-in replacement for `ld` not only when GNU binutils toolchain is used, but even when `clang` is used as an assembler. Perhaps, the latter might work against mingw SDK built using LLVM tools (e.g. https://github.com/mstorsjo/llvm-mingw), but I never tried this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: upstream Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by adamse): * cc: adamse (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: upstream Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari):
LLD won't magically help here. Latest improvements by Martin Storsjö (mstorsjo) have made LLD able to link some mingw gcc-generated code, but, alas, when assembling GHC-generated assembly, mingw gas produces (I believe this is a sort of "optimisation") non-standard relocations.
Frankly I wonder how difficult it would be to add support for these relocations. Given that LLVM code is generally pretty approachable I suspect that would be the easiest path forward. In the meantime I have been pursuing testing Tamar's binutils patch. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: upstream Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by awson): I think it's easy, but (as I've already mentioned) the other problems exist. Mingw binutils introduce a lot of non-standard things, which `LLD` didn't support at all until recently, now some support have appeared, but AFAIUI, things go best when mingw sdk is built by LLVM toolsuite, not by mingw gcc/binutils. And a general problem is that wrong linkage bugs are *very* hard to debug, this is why I haven't ever tried to continue my mingw clang/lld experiment — it might consume an unpredictable amount of time to understand what is wrong with the generated ghc executable — all the symptoms are that the image file is invalid, since the OS can't even load it properly. Replying to [comment:7 bgamari]:
Frankly I wonder how difficult it would be to add support for these relocations. Given that LLVM code is generally pretty approachable I suspect that would be the easiest path forward.
In the meantime I have been pursuing testing Tamar's binutils patch.
-- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#16084: Improve link times on Windows
-------------------------------------+-------------------------------------
Reporter: bgamari | Owner: (none)
Type: bug | Status: upstream
Priority: high | Milestone:
Component: Compiler | Version: 8.6.3
Resolution: | Keywords:
Operating System: Windows | Architecture:
Type of failure: Compile-time | Unknown/Multiple
performance bug | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#16084: Improve link times on Windows
-------------------------------------+-------------------------------------
Reporter: bgamari | Owner: (none)
Type: bug | Status: upstream
Priority: high | Milestone:
Component: Compiler | Version: 8.6.3
Resolution: | Keywords:
Operating System: Windows | Architecture:
Type of failure: Compile-time | Unknown/Multiple
performance bug | Test Case:
Blocked By: | Blocking:
Related Tickets: | Differential Rev(s):
Wiki Page: |
-------------------------------------+-------------------------------------
Comment (by Ben Gamari
#16084: Improve link times on Windows -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: upstream Priority: high | Milestone: Component: Compiler | Version: 8.6.3 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple performance bug | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): I have pushed a new branch testing a new `binutils` build provided by Tamar to `wip/fix-windows-ci` (see !495). -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/16084#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC