#8591: Concurrent executions of ghc-pkg can cause inconstant package.cache files -------------------------------------+------------------------------------- Reporter: janm | Owner: pgj Type: bug | Status: new Priority: normal | Milestone: Component: Package | Version: 7.6.3 system | Keywords: ghc-pkg race Resolution: | Architecture: Unknown/Multiple Operating System: FreeBSD | Difficulty: Unknown Type of failure: Other | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- Comment (by janm): I am running a test build now but code inspection shows that the same race is present. I expect it to fail. We resolved the issue by single-threading all package installation in the build process. We are running on 9.3-p5 at the moment -- It was probably 9.2-RELEASE or 9-STABLE at the time of the original bug report. I also see that the FreeBSD port for lang/ghc limits concurrency to 4 jobs, added in SVN rev 348842. Assuming you're also pgj@freebsd you committed the change, so probably know! This was in response to FreeBSD bug ports/186829 (see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186829 ). The same problem seems to be the underlying cause. The error trace in the bug report shows cabal failing to install a package, just like I saw in my system build processes. Resolving the problem in ghc-pkg should resolve this problem and allow lang/ghc builds to run with full parallelism. In principle, the withFileAtomic function should (in order): 1. take a lock on the target file (creating it if it is not present) 2. create the temporary file 3. write to the temporary file 4. close the temporary file 5. rename the temporary file on top of the locked target file 6. close the original target file which is now no longer present in the file system. Concurrent execution will now be serialised around the lock on the target file. This doesn't resolve the "big hairy race condition" on Windows, but it should resolve the big hairy race condition I'm hitting that isn't mentioned code comments. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8591#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler