I am helping Mark with the Haskell Platform, doing the Windows builds (32- and 64-bit). I want you to be aware of a problem I am encountering, and solicit suggestions and possible help. In building for HP 2015.2.0.0 on Windows 7, 64-bit (haven't gotten to 32-bit yet but likely the same problem will occur), I seem to be hitting the 32K limit for the length of arguments to a process, encountered while cabal is invoking haddock to build the docs for the OpenGLRaw package. For HP2014.2.0.0, the argument list was ~25K (from looking at my old build logs) but now is ~36K, which exceeds the maximum for CreateProcess (not a limit of the command-line, but of the OS call itself). Is there a way to build haddock docs for a single package but in multiple haddock invocations (maybe building a .haddock file for portions, then combining them, with the goal that the command line is kept short)? Seems this would also require a corresponding cabal change, as cabal is the invocator when this happens. Barring any existing mechanism, the typical solution to this problem on the Windows OS is (when possible, of course) to modify the program to accept a "response file" of command-line arguments. In this case, we could add an option to haddock to accept either a complete "response file" (i.e., allowing *all* options and arguments to come from a file) or just a file containing the files to process. Either of these changes to haddock are rather trivial to write (but adding another option implies more testing, documentation, other cases to handle, etc.). Since haddock ships with the ghc release, that's another wrinkle for this particular release. The other implication of such a solution is that cabal would need a change to utilize this change for it to be effective, checking haddock's version for support of this new haddock-flag, and either use it if the haddock version supports it, or do it optionally (which implies a new flag for cabal's haddock sub-command). This change to cabal is also rather trivial to implement (this is not to imply insensitivity to the incurred cost of each line of code, nor to the added burden of user-visible changes such as a command-line option). (Less desirable possibilities, mentioned only for completeness: skip the documentation for OpenGLRaw for this version of the Haskell Platform; split up the OpenGLRaw package itself in some way.) Other possible solutions and work-arounds? Thoughts on either using haddock in a different way (and the cabal change that would be required to break up the doc build into multiple steps for a single package)? Thoughts on the "response file" solution? Randy
To ship `gl`, we wound up having to rename all of the modules to make it
work on Windows.
Alas, the module name conventions used by `OpenGLRaw` are longer.
-Edward
On Fri, Mar 27, 2015 at 1:16 PM, Randy Polen
I am helping Mark with the Haskell Platform, doing the Windows builds (32- and 64-bit). I want you to be aware of a problem I am encountering, and solicit suggestions and possible help.
In building for HP 2015.2.0.0 on Windows 7, 64-bit (haven't gotten to 32-bit yet but likely the same problem will occur), I seem to be hitting the 32K limit for the length of arguments to a process, encountered while cabal is invoking haddock to build the docs for the OpenGLRaw package. For HP2014.2.0.0, the argument list was ~25K (from looking at my old build logs) but now is ~36K, which exceeds the maximum for CreateProcess (not a limit of the command-line, but of the OS call itself).
Is there a way to build haddock docs for a single package but in multiple haddock invocations (maybe building a .haddock file for portions, then combining them, with the goal that the command line is kept short)? Seems this would also require a corresponding cabal change, as cabal is the invocator when this happens.
Barring any existing mechanism, the typical solution to this problem on the Windows OS is (when possible, of course) to modify the program to accept a "response file" of command-line arguments. In this case, we could add an option to haddock to accept either a complete "response file" (i.e., allowing *all* options and arguments to come from a file) or just a file containing the files to process. Either of these changes to haddock are rather trivial to write (but adding another option implies more testing, documentation, other cases to handle, etc.). Since haddock ships with the ghc release, that's another wrinkle for this particular release. The other implication of such a solution is that cabal would need a change to utilize this change for it to be effective, checking haddock's version for support of this new haddock-flag, and either use it if the haddock version supports it, or do it optionally (which implies a new flag for cabal's haddock sub-command). This change to cabal is also rather trivial to implement (this is not to imply insensitivity to the incurred cost of each line of code, nor to the added burden of user-visible changes such as a command-line option).
(Less desirable possibilities, mentioned only for completeness: skip the documentation for OpenGLRaw for this version of the Haskell Platform; split up the OpenGLRaw package itself in some way.)
Other possible solutions and work-arounds? Thoughts on either using haddock in a different way (and the cabal change that would be required to break up the doc build into multiple steps for a single package)? Thoughts on the "response file" solution?
Randy _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
On 03/27/2015 05:16 PM, Randy Polen wrote:
I am helping Mark with the Haskell Platform, doing the Windows builds (32- and 64-bit). I want you to be aware of a problem I am encountering, and solicit suggestions and possible help.
In building for HP 2015.2.0.0 on Windows 7, 64-bit (haven't gotten to 32-bit yet but likely the same problem will occur), I seem to be hitting the 32K limit for the length of arguments to a process, encountered while cabal is invoking haddock to build the docs for the OpenGLRaw package. For HP2014.2.0.0, the argument list was ~25K (from looking at my old build logs) but now is ~36K, which exceeds the maximum for CreateProcess (not a limit of the command-line, but of the OS call itself).
Is there a way to build haddock docs for a single package but in multiple haddock invocations (maybe building a .haddock file for portions, then combining them, with the goal that the command line is kept short)? Seems this would also require a corresponding cabal change, as cabal is the invocator when this happens.
Barring any existing mechanism, the typical solution to this problem on the Windows OS is (when possible, of course) to modify the program to accept a "response file" of command-line arguments. In this case, we could add an option to haddock to accept either a complete "response file" (i.e., allowing *all* options and arguments to come from a file) or just a file containing the files to process. Either of these changes to haddock are rather trivial to write (but adding another option implies more testing, documentation, other cases to handle, etc.). Since haddock ships with the ghc release, that's another wrinkle for this particular release. The other implication of such a solution is that cabal would need a change to utilize this change for it to be effective, checking haddock's version for support of this new haddock-flag, and either use it if the haddock version supports it, or do it optionally (which implies a new flag for cabal's haddock sub-command). This change to cabal is also rather trivial to implement (this is not to imply insensitivity to the incurred cost of each line of code, nor to the added burden of user-visible changes such as a command-line option).
(Less desirable possibilities, mentioned only for completeness: skip the documentation for OpenGLRaw for this version of the Haskell Platform; split up the OpenGLRaw package itself in some way.)
Other possible solutions and work-arounds? Thoughts on either using haddock in a different way (and the cabal change that would be required to break up the doc build into multiple steps for a single package)? Thoughts on the "response file" solution?
Hi Randy, We actually have an issue about response files already[1], I just haven't gotten around to doing it. When I looked into it last, cabal folks were willing to support it too so there should be no problem on that end. All that remains is that we write code for it in Haddock and for cabal and GHC to adapt. I can implement this in Haddock even today and you can cherry-pick a patch onto whatever you're working with. Is that satisfactory? [1]: https://github.com/haskell/haddock/issues/285 -- Mateusz K.
Mateusz, Haddock issue #285 indeed sounds like a win (or Win-dows). A logistical wrinkle I worry about for the short-term is that the HP uses the GHC release, and haddock is part of that release. I can certainly incorporate a newer haddock, but I want to mention this GHC-related release issue in case others here have a better approach for this part of the plan (e.g., "yes, go ahead and augment GHC 7.10.1 release with a custom haddock" or "no, let's spin up a 7.10.1.1 (?)"). There is also the Cabal piece as well, but that is separate from GHC and thus a bit easier to incorporate into the HP build. Randy ----------------------------------------
Date: Sat, 28 Mar 2015 15:12:34 +0000 From: fuuzetsu@fuuzetsu.co.uk To: ghc-devs@haskell.org Subject: Re: HP 2015.2.0.0 and GHC 7.10
On 03/27/2015 05:16 PM, Randy Polen wrote:
I am helping Mark with the Haskell Platform, doing the Windows builds (32- and 64-bit). I want you to be aware of a problem I am encountering, and solicit suggestions and possible help.
In building for HP 2015.2.0.0 on Windows 7, 64-bit (haven't gotten to 32-bit yet but likely the same problem will occur), I seem to be hitting the 32K limit for the length of arguments to a process, encountered while cabal is invoking haddock to build the docs for the OpenGLRaw package. For HP2014.2.0.0, the argument list was ~25K (from looking at my old build logs) but now is ~36K, which exceeds the maximum for CreateProcess (not a limit of the command-line, but of the OS call itself).
Is there a way to build haddock docs for a single package but in multiple haddock invocations (maybe building a .haddock file for portions, then combining them, with the goal that the command line is kept short)? Seems this would also require a corresponding cabal change, as cabal is the invocator when this happens.
Barring any existing mechanism, the typical solution to this problem on the Windows OS is (when possible, of course) to modify the program to accept a "response file" of command-line arguments. In this case, we could add an option to haddock to accept either a complete "response file" (i.e., allowing *all* options and arguments to come from a file) or just a file containing the files to process. Either of these changes to haddock are rather trivial to write (but adding another option implies more testing, documentation, other cases to handle, etc.). Since haddock ships with the ghc release, that's another wrinkle for this particular release. The other implication of such a solution is that cabal would need a change to utilize this change for it to be effective, checking haddock's version for support of this new haddock-flag, and either use it if the haddock version supports it, or do it optionally (which implies a new flag for cabal's haddock sub-command). This change to cabal is also rather trivial to implement (this is not to imply insensitivity to the incurred cost of each line of code, nor to the added burden of user-visible changes such as a command-line option).
(Less desirable possibilities, mentioned only for completeness: skip the documentation for OpenGLRaw for this version of the Haskell Platform; split up the OpenGLRaw package itself in some way.)
Other possible solutions and work-arounds? Thoughts on either using haddock in a different way (and the cabal change that would be required to break up the doc build into multiple steps for a single package)? Thoughts on the "response file" solution?
Hi Randy,
We actually have an issue about response files already[1], I just haven't gotten around to doing it. When I looked into it last, cabal folks were willing to support it too so there should be no problem on that end. All that remains is that we write code for it in Haddock and for cabal and GHC to adapt. I can implement this in Haddock even today and you can cherry-pick a patch onto whatever you're working with.
Is that satisfactory?
[1]: https://github.com/haskell/haddock/issues/285
-- Mateusz K. _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
On 03/28/2015 04:27 PM, Randy Polen wrote:
Mateusz,
Haddock issue #285 indeed sounds like a win (or Win-dows).
A logistical wrinkle I worry about for the short-term is that
the HP uses the GHC release, and haddock is part of that release. I can certainly
incorporate a newer haddock, but I want to mention this GHC-related release
issue in case others here have a better approach for this part of the plan (e.g.,
"yes, go ahead and augment GHC 7.10.1 release with a custom haddock" or
"no, let's spin up a 7.10.1.1 (?)").
There is also the Cabal piece as well, but that is separate from GHC and thus
a bit easier to incorporate into the HP build.
Randy
We cut a Haddock release when GHC comes out so that it's easy to track versions but that's simply to try to keep some sanity when users report --version. But there is nothing stopping us from releasing a new Haddock version without forcing a GHC release. If I release 2.16.1 tomorrow with the argument list thing, you can just cabal install install it. As long as your system knows to look into binaries built through cabal first and binaries shipped with GHC second, you'll be golden. For HP purposes, you probably want to ship 2.16.1 (or whatever version) when that comes out because tools like cabal-install depend on the version number to determine what features (such as response files) are available. I don't have a date for next point release. Do you have a date for HP? -- Mateusz K.
participants (3)
-
Edward Kmett -
Mateusz Kowalczyk -
Randy Polen