On Tue, Mar 29, 2011 at 12:54 PM, Frank Murphy <anirishduck@gmail.com> wrote:
I've been looking around and trying to develop a plan for parallelizing builds
in cabal-install.
Something I'd love to see, for sure. But the devil's in the details; see below.
- Parallelize executeInstallPlan. When given a target load average as a flag it
will determine whether it should spawn a worker (if below the target load
average) or wait.
Load average is a very very bad method to use, because it's very slow to respond to changes in real load on Unix, and it doesn't exist at all on Windows. Do like "make" instead and just accept a "-j" parameter for the maximum number of jobs to run simultaneously.
- Rewrite install.*Package and their callees to use the CHP (Communicating
Haskell Process) monad where possible. Use channels to communicate build
status back to the main thread.
Why not just use MVar or Chan?
- It might be necessary to parse the output of external builds in some way so
that meaningful status can be communicated back to the user.
Yuck, no. Too fragile. Just check the exit status of a process.
- Add a default parallel build log path template. Allow the user to specify one
on the command line to override the default. All output of parallel package
builds will be logged in the background silently instead of displayed to the
user.
- On single-threaded (sequential) builds, revert to the old output style. On
multi-threaded builds, display the current status of all running builds, load
averages and nothing else.
No to both of these, too. It's desirable that build outputs shouldn't scribble on each other via excessive interleaving, but telling people to go read a log file because a build failed is going to give them a very bad experience. For instance, that would defeat the usual way that users of Emacs and other IDEs jump to the first error.