On Fri, 2007-12-28 at 09:54 -0500, Gregory Wright wrote:

Hi Duncan,

(Cross-posting to ghc-users since some of the issues were brought up there.)

On Dec 2, 2007, at 5:04 PM, Duncan Coutts wrote:

...

Sun Dec 2 14:06:20 PST 2007 Duncan Coutts * make rawSystemStdout put its temp files in the temp dir rather than cwd Should fixe reported wierdness with finding program version numbers

M ./Distribution/Simple/Utils.hs -2 +4

I saw this problem when trying to build 6.8.1 on a mac (Leopard/Intel) using 6.4 to build compiler. I edited rawSystemStdout to put the tmp files directory in "/tmp" instead of using getTemporaryDirectory. On this combination of compilers, moving the temporary file did not fix the problem of finding the program version number. The underlying error was that the temp file was reported to still be locked.

I've no idea what the problem could be but I would not expect changing the directory where the temp files are put to be related. What can cause file locked errors? I thought the OS did not enforce any locking and it was only the RTS that did so, and only within a process, not between processes using OS voluntary locking apis. As far as I know, only Windows enforces these kinds of file locks (like preventing deleting of open files).

Building 6.8.2 with another 6.8.2 eliminated the program version problem, but I have seen two other intermittent errors which seem to be related. (Reported in trac as issue #1992.) Building 6.8.2 with 6.8.2 using the MacPorts infrastructure, I get this error about half the time:

../compiler/stage1/ghc-inplace -M -optdep-f -optdep.depend-BASE - osuf o -I../includes -H16m -O -I/opt/local/include -I/usr/include -L/ [snip] .depend-BASE: openFile: resource busy (file is locked)

Cabal never directly calls ghc to generate Makefile style dependencies, though ghc's build system does use Cabal to generate Makefiles which call ghc to generate .depend files.

When I build in an eshell running under Aquamacs, I've consistently had the build fail with:

../utils/mkdependC/mkdependC -f .depend-BASE -D__GLASGOW_HASKELL__=608 -I../includes -Iutils -IbasicTypes -Itypes -IhsSyn -Iprelude -Irename - Itypecheck -IdeSugar -IcoreSyn -Ivectorise -Ispecialise -IsimplCore - Istranal -IstgSyn -IsimplStg -IcodeGen -Imain -Iprofiling -Iparser - IcprAnalysis -IndpFlatten -Iiface -Icmm -InativeGen -Ighci -I../ includes -- -O -I/opt/local/include -Iparser -I. -O -- parser/ cutils.c parser/hschooks.c /opt/local/bin/perl -pe 'binmode(stdin); binmode(stdout); s@(\S*[._]o)@stage2/$1@g; s@(\S*[._]hi)@stage2/$1@g; s@^.*/compat.* $@@g;' <.depend-BASE >.depend-2 /usr/bin/make -C compiler stage=2 ../compiler/stage1/ghc-inplace -cpp stage2/ghc-inplace.c -o stage2/ ghc-inplace ld: in /Users/gwright/Desktop/ghc-6.8.2/libraries/haskell98/dist/build/ libHShaskell98-1.0.1.0.a, archive has no table of contents collect2: ld returned 1 exit status make[2]: *** [stage2/ghc-inplace] Error 1 make[1]: *** [stage2] Error 2 make: *** [bootstrap2] Error 2

Again, this is running from a Makefile, not from Cabal directly.

In this last case, it looks as if libHShaskell98-1.0.1.0.a is being used before ranlib has been run on it.

The interesting thing is why this happens in your environment and not more generally. Admittedly, looking at the Makefile that cabal generates it does not mention ranlib but then if that were the problem we would expect all builds on OSX to fail always, which is clearly not the case. So again, I don't know what's going on :-)

The common theme of these errors is that they all come out of Cabal,

Actually all of these are Makefiles, though possibly Makefiles generated by Cabal.

and seem to involve using the results of external processes before they are ready. In the original case of the temporary file used to record the version number, runProcess/waitForProcess are used. In the last case (running ranlib), runProcessPosix/waitForProcess are used.

runProcessPosix?

Since the bug seems to be a race condition, it's not surprising that changing the code can cause it to disappear for a while. I will continue to look at what runProcess and waitForProcess are doing. Do you know of anything in Cabal that might confuse the handling of external processes?

No. It uses rawSystem for most purposes which waits for process termination or for the rawSystemStdout it uses runProcess and waitForProcess. However, as I said, the above examples were from Makefiles not from code in Cabal.

Were the original reports of weirdness in detecting the version number on OS X, or another operating system?

They were OS independent, it was just due to using a dir that was not writable rather than the proper tmp directory. It was not related to locking. The only other similar bug I've seen was on Windows where cabal-install sometimes fails to delete a file in a build directory due to it being locked, even though the last process to use that file has terminated. So all in all, I'm confused. Duncan