#8591: Concurrent executions of ghc-pkg can cause inconstant package.cache files ------------------------------------+---------------------------- Reporter: janm | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Package system | Version: 7.6.3 Keywords: ghc-pkg race | Operating System: FreeBSD Architecture: Unknown/Multiple | Type of failure: Other Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | ------------------------------------+---------------------------- I am doing 24 way parallel builds of system images, including all packages on a system. This includes ghc and multiple ghc packages. I am seeing intermittent dependency failure from the ghc packaging system. From examining Main.hs in ghc-pkg, I see the function withFileAtomic write to a temporary file in package.conf.d and then atomically rename on top of a target file, package.cache in the case. With parallel execution the last rename would win, leading to lost entries in package.cache. In my case, the following things happened: ("Building" indicates a start, "Built" indicates completion, "Installing" is setup in a separate chroot'd environment and is isolated) The FreeBSD ports system is used to drive the Haskell build system. The process works single threaded and fails intermittently when done in parallel. Building: devel/hs-data-default-instances-base Building: devel/hs-data-default-instances-containers Building: devel/hs-data-default-instances-old-locale Built: devel/hs-dlist Building: devel/hs-data-default-instances-dlist Built: devel/hs-temporary Built: jail-image-full Installing: system-image__jail-image-full Built: devel/hs-base64-bytestring Built: archivers/hs-zlib Building: security/hs-digest Built: devel/hs-syb Building: textproc/hs-hs-bibutils Building: textproc/hs-pandoc-types Built: devel/hs-utf8-string Built: devel/hs-data-default-instances-old-locale Built: devel/hs-data-default-instances-containers Built: devel/hs-data-default-instances-base Built: devel/hs-data-default-instances-dlist Building: devel/hs-data-default Built: devel/hs-random Installed: system-image__lang/ghc Installing: system-image__archivers/hs-zlib Installing: system-image__devel/hs-utf8-string Installing: system-image__devel/hs-syb Installing: system-image__devel/hs-base64-bytestring Installing: system-image__devel/hs-data-default-class Installing: system-image__devel/hs-dlist Installing: system-image__devel/hs-random Installing: system-image__devel/hs-temporary Installing: system-image__devel/hs-extensible-exceptions Built: devel/hs-data-default FAILED The error from the Haskell data-default build was: setup: At least the following dependencies are missing: data-default-instances-base -any Looking in the in the package.conf.d directory shows that the data- default-instances-base-0.0.1-7bdf8678f0d8637e096e397e7910f82a.conf file was present, but running "ghc-pkg list" did not show data-default- instances-base Running /usr/local/lib/cabal/ghc-7.6.3/data-default-instances- base-0.0.1/register.sh (which was also present) caused ghc-pkg to now show data-default-instances-base. To me this looks like a race condition between multiple instances of ghc- pkg causing the cache to become inconsistent. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/8591 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler