[GHC] #10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Keywords: | Operating System: Windows Architecture: | Type of failure: Compile-time Unknown/Multiple | crash Test Case: | Blocked By: Blocking: | Related Tickets: Differential Revisions: | -------------------------------------+------------------------------------- I've seen three different issues on Github about this: two in the stack repo for stack users, and one in the yesod repo for a cabal sandbox user: https://github.com/yesodweb/yesod/issues/1020 https://github.com/commercialhaskell/stack/issues/466 https://github.com/commercialhaskell/stack/issues/795 The problem comes down to this: Windows has a limit of 32k on the size of a command string, which can be tripped by GHC's link phase when using a large number of libraries. The ideal situation would be for GHC to have a different way of passing large argument lists to the linker; I don't know how feasible this is, but I wanted to open up a ticket before I started investigating myself in case others have seen this. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by awson): I have a patch for `Cabal` which introduces response files usage for `ar` and `ld`. This speeds things up dramatically in some cases and eliminates some problems with prelinked object files creation when `split-objs` and gcc > 4.8 are used. This technique can clearly be applied to `gcc` invocations also because `gcc` on windows also supports response files (for ages). The problem with this patch is that: 1. I am lazy, and since I use this patch only internally I didn't bother to factor common code out and simply almost copypasted (added a very minor edit) the same code both for `ar` and `ld`. 2. I didn't test anything on the platforms other than `windows` -- perhaps something could be broken there. OTOH, the patch is very simple and could easily be polished to play nicely with a platform specificity. 2. It deals with `Cabal`, not with `ghc`. I didn't read linked content thoroughly, do we need to modify `ghc`, or modifying `Cabal` would suffice? Btw, feature request for adding it to `ghc` [https://ghc.haskell.org/trac/ghc/ticket/8596 exists]. Anyway, if anybody is interested to look at how this works I can attach the patch. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by snoyberg): I'd very much like to see the Cabal diff, and I'm sure other users would too. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by awson): * Attachment "cabal-rsp.patch" added. Quick and dirty Cabal patch to make it use response files in a couple of ar and ld invocations -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by awson): More or less relevant is the part behind `withTempFile`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #8596 #9685 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by Phyx-): * related: => #8596 #9685 Comment: The limits are a bit all over the place. For the record: - The maximum command line length for the CreateProcess function is 32767 characters. This limitation comes from the UNICODE_STRING structure. CreateProcess is the core function for creating processes, so if you are talking directly to Win32, then that's the only limit you have to worry about. But if you are reaching CreateProcess by some other means, then the path you travel through may have other limits. - If you are using the CMD.EXE command processor, then you are also subject to the 8192 character command line length limit imposed by CMD.EXE. - If you are using the ShellExecute/Ex function, then you become subject to the INTERNET_MAX_URL_LENGTH (around 2048) command line length limit imposed by the ShellExecute/Ex functions. (If you are running on Windows 95, then the limit is only MAX_PATH.) [1] http://blogs.msdn.com/b/oldnewthing/archive/2003/12/10/56028.aspx [2] https://support.microsoft.com/en-us/kb/830473 -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: snoyberg Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #8596 #9685 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by snoyberg): * owner: => snoyberg -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: snoyberg Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #8596 #9685 | Differential Revisions: -------------------------------------+------------------------------------- Comment (by snoyberg): I've sent a diff for this: https://phabricator.haskell.org/D1158 Unfortunately, I haven't been successful yet at building GHC on Windows, so I haven't tested there yet. If someone who's experienced this problem has success building with that change, please report back. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing
error messages
-------------------------------------+-------------------------------------
Reporter: snoyberg | Owner: snoyberg
Type: bug | Status: new
Priority: normal | Milestone:
Component: Compiler | Version: 7.10.2
Resolution: | Keywords:
Operating System: Windows | Architecture:
Type of failure: Compile-time | Unknown/Multiple
crash | Test Case:
Blocked By: | Blocking:
Related Tickets: #8596 #9685 | Differential Revisions:
-------------------------------------+-------------------------------------
Comment (by Ben Gamari

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: snoyberg Type: bug | Status: merge Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #8596 #9685 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by ezyang): * status: new => merge Comment: Merge maybe? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: snoyberg Type: bug | Status: merge Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 (Linking) | Resolution: | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #8596 #9685 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by JohnWiegley): * component: Compiler => Compiler (Linking) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: snoyberg Type: bug | Status: closed Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 (Linking) | Resolution: fixed | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #8596 #9685 | Differential Revisions: -------------------------------------+------------------------------------- Changes (by bgamari): * status: merge => closed * resolution: => fixed -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler

#10777: Overlong linker arguments on Windows leads to broken builds with confusing error messages -------------------------------------+------------------------------------- Reporter: snoyberg | Owner: snoyberg Type: bug | Status: closed Priority: normal | Milestone: Component: Compiler | Version: 7.10.2 (Linking) | Resolution: fixed | Keywords: Operating System: Windows | Architecture: Type of failure: Compile-time | Unknown/Multiple crash | Test Case: Blocked By: | Blocking: Related Tickets: #8596 #9685 | Differential Rev(s): -------------------------------------+------------------------------------- Comment (by bgamari): This was merged to `ghc-7.10` as 6b08e42ad99bb7857b631f28db869def046bff35. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10777#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC