Hi, Am Freitag, den 02.09.2011, 09:07 +0100 schrieb Simon Marlow:
On 01/09/2011 18:02, Evan Laforge wrote:
It's an interesting idea that I hadn't thought of. There would have to be an atomic file system operation to "commit" a compiled module - getting that right could be a bit tricky (compilation isn't deterministic, so the commit has to be atomic).
I suppose you could just rename it into place when you're done. -Edward
I was imagining that it could create Module.o.compiling and then rename into place when it's done. Then each ghc would do a work stealing thing where it tries to find output to produce that doesn't have an accompanying .compiling, or sleeps for a bit if all work at this stage is already taken, which is likely to happen since sometimes the graph would go through a bottleneck. Then it's easy to clean up if work gets interrupted, just rm **/*.compiling
Right, using a Module.o.compiling file as a lock would work.
Another way to do this would be to have GHC --make invoke itself to compile each module separately. Actually I think I prefer this method, although it might be a bit slower since each individual compilation has to read lots of interface files. The main GHC --make process would do the final link only. A fun hack for somebody?
this would also help building large libraries on architectures with little memory, as it seems to me that when one ghc instance is compiling multiple modules in a row, some leaked memory/unevaluated thunks pile up and eventually cause the compilation to abort. I suspect that building each file on its own avoids this issue. (But this is only based on observation, not on hard facts.) Greetings, Joachim -- Joachim "nomeata" Breitner mail@joachim-breitner.de | nomeata@debian.org | GPG: 0x4743206C xmpp: nomeata@joachim-breitner.de | http://www.joachim-breitner.de/