Re: patch applied (cabal): make rawSystemStdout put its temp files in the temp dir rather than cwd

Hi Duncan, (Cross-posting to ghc-users since some of the issues were brought up there.) On Dec 2, 2007, at 5:04 PM, Duncan Coutts wrote:

Sun Dec 2 14:06:20 PST 2007 Duncan Coutts * make rawSystemStdout put its temp files in the temp dir rather than cwd Should fixe reported wierdness with finding program version numbers

M ./Distribution/Simple/Utils.hs -2 +4

I saw this problem when trying to build 6.8.1 on a mac (Leopard/Intel) using 6.4 to build compiler. I edited rawSystemStdout to put the tmp files directory in "/tmp" instead of using getTemporaryDirectory. On this combination of compilers, moving the temporary file did not fix the problem of finding the program version number. The underlying error was that the temp file was reported to still be locked. Building 6.8.2 with another 6.8.2 eliminated the program version problem, but I have seen two other intermittent errors which seem to be related. (Reported in trac as issue #1992.) Building 6.8.2 with 6.8.2 using the MacPorts infrastructure, I get this error about half the time: ../compiler/stage1/ghc-inplace -M -optdep-f -optdep.depend-BASE - osuf o -I../includes -H16m -O -I/opt/local/include -I/usr/include -L/ opt/local/lib -L/usr/lib -iutils -ibasicTypes -itypes -ihsSyn - iprelude -irename -itypecheck -ideSugar -icoreSyn -ivectorise - ispecialise -isimplCore -istranal -istgSyn -isimplStg -icodeGen -imain -iprofiling -iparser -icprAnalysis -indpFlatten -iiface -icmm - inativeGen -ighci -Wall -fno-warn-name-shadowing -fno-warn-orphans - Istage2 -package hpc -package bytestring -DGHCI -package template- haskell -DGHCI_TABLES_NEXT_TO_CODE -package readline -DUSE_READLINE - cpp -fglasgow-exts -fno-generics -Rghc-timing -I. -Iparser -package unix -package Cabal -ignore-package lang -recomp -Rghc-timing -H16M '- #include "cutils.h"' -package-name ghc-6.8.2 -fgenerics basicTypes/ BasicTypes.lhs basicTypes/DataCon.lhs basicTypes/Demand.lhs basicTypes/ Id.lhs basicTypes/IdInfo.lhs basicTypes/Literal.lhs basicTypes/ MkId.lhs basicTypes/Module.lhs basicTypes/Name.lhs basicTypes/ NameEnv.lhs basicTypes/NameSet.lhs basicTypes/NewDemand.lhs basicTypes/ OccName.lhs basicTypes/RdrName.lhs basicTypes/SrcLoc.lhs basicTypes/ UniqSupply.lhs basicTypes/Unique.lhs basicTypes/Var.lhs basicTypes/ VarEnv.lhs basicTypes/VarSet.lhs cmm/CLabel.hs cmm/Cmm.hs cmm/ CmmBrokenBlock.hs cmm/CmmCPS.hs cmm/CmmCPSGen.hs cmm/CmmCallConv.hs cmm/CmmInfo.hs cmm/CmmLex.hs cmm/CmmLint.hs cmm/CmmLive.hs cmm/ CmmOpt.hs cmm/CmmParse.hs cmm/CmmProcPoint.hs cmm/CmmUtils.hs cmm/ Dataflow.hs cmm/MachOp.hs cmm/PprC.hs cmm/PprCmm.hs codeGen/Bitmap.hs codeGen/CgBindery.lhs codeGen/CgCallConv.hs codeGen/CgCase.lhs codeGen/ CgClosure.lhs codeGen/CgCon.lhs codeGen/CgExpr.lhs codeGen/ CgForeignCall.hs codeGen/CgHeapery.lhs codeGen/CgHpc.hs codeGen/ CgInfoTbls.hs codeGen/CgLetNoEscape.lhs codeGen/CgMonad.lhs codeGen/ CgParallel.hs codeGen/CgPrimOp.hs codeGen/CgProf.hs codeGen/ CgStackery.lhs codeGen/CgTailCall.lhs codeGen/CgTicky.hs codeGen/ CgUtils.hs codeGen/ClosureInfo.lhs codeGen/CodeGen.lhs codeGen/ SMRep.lhs coreSyn/CoreFVs.lhs coreSyn/CoreLint.lhs coreSyn/ CorePrep.lhs coreSyn/CoreSubst.lhs coreSyn/CoreSyn.lhs coreSyn/ CoreTidy.lhs coreSyn/CoreUnfold.lhs coreSyn/CoreUtils.lhs coreSyn/ ExternalCore.lhs coreSyn/MkExternalCore.lhs coreSyn/PprCore.lhs coreSyn/PprExternalCore.lhs cprAnalysis/CprAnalyse.lhs deSugar/ Check.lhs deSugar/Coverage.lhs deSugar/Desugar.lhs deSugar/ DsArrows.lhs deSugar/DsBinds.lhs deSugar/DsCCall.lhs deSugar/ DsExpr.lhs deSugar/DsForeign.lhs deSugar/DsGRHSs.lhs deSugar/ DsListComp.lhs deSugar/DsMeta.hs deSugar/DsMonad.lhs deSugar/ DsUtils.lhs deSugar/Match.lhs deSugar/MatchCon.lhs deSugar/ MatchLit.lhs ghci/ByteCodeAsm.lhs ghci/ByteCodeFFI.lhs ghci/ ByteCodeGen.lhs ghci/ByteCodeInstr.lhs ghci/ByteCodeItbls.lhs ghci/ ByteCodeLink.lhs ghci/Debugger.hs ghci/GhciMonad.hs ghci/GhciTags.hs ghci/InteractiveUI.hs ghci/Linker.lhs ghci/ObjLink.lhs ghci/ RtClosureInspect.hs hsSyn/Convert.lhs hsSyn/HsBinds.lhs hsSyn/ HsDecls.lhs hsSyn/HsDoc.hs hsSyn/HsExpr.lhs hsSyn/HsImpExp.lhs hsSyn/ HsLit.lhs hsSyn/HsPat.lhs hsSyn/HsSyn.lhs hsSyn/HsTypes.lhs hsSyn/ HsUtils.lhs iface/BinIface.hs iface/BuildTyCl.lhs iface/IfaceEnv.lhs iface/IfaceSyn.lhs iface/IfaceType.lhs iface/LoadIface.lhs iface/ MkIface.lhs iface/TcIface.lhs main/BreakArray.hs main/CmdLineParser.hs main/CodeOutput.lhs main/Config.hs main/Constants.lhs main/ DriverMkDepend.hs main/DriverPhases.hs main/DriverPipeline.hs main/ DynFlags.hs main/ErrUtils.lhs main/Finder.lhs main/GHC.hs main/ HeaderInfo.hs main/HscMain.lhs main/HscStats.lhs main/HscTypes.lhs main/InteractiveEval.hs main/Main.hs main/PackageConfig.hs main/ Packages.lhs main/ParsePkgConf.hs main/PprTyThing.hs main/ StaticFlags.hs main/SysTools.lhs main/TidyPgm.lhs nativeGen/ AsmCodeGen.lhs nativeGen/GraphBase.hs nativeGen/GraphColor.hs nativeGen/GraphOps.hs nativeGen/GraphPpr.hs nativeGen/MachCodeGen.hs nativeGen/MachInstrs.hs nativeGen/MachRegs.lhs nativeGen/NCGMonad.hs nativeGen/PositionIndependentCode.hs nativeGen/PprMach.hs nativeGen/ RegAllocColor.hs nativeGen/RegAllocInfo.hs nativeGen/RegAllocLinear.hs nativeGen/RegAllocStats.hs nativeGen/RegArchBase.hs nativeGen/ RegArchX86.hs nativeGen/RegCoalesce.hs nativeGen/RegLiveness.hs nativeGen/RegSpill.hs nativeGen/RegSpillClean.hs nativeGen/ RegSpillCost.hs ndpFlatten/FlattenInfo.hs ndpFlatten/FlattenMonad.hs ndpFlatten/Flattening.hs ndpFlatten/NDPCoreUtils.hs ndpFlatten/ PArrAnal.hs parser/Ctype.lhs parser/HaddockLex.hs parser/ HaddockParse.hs parser/HaddockUtils.hs parser/LexCore.hs parser/ Lexer.hs parser/Parser.hs parser/ParserCore.hs parser/ ParserCoreUtils.hs parser/RdrHsSyn.lhs prelude/ForeignCall.lhs prelude/ PrelInfo.lhs prelude/PrelNames.lhs prelude/PrelRules.lhs prelude/ PrimOp.lhs prelude/TysPrim.lhs prelude/TysWiredIn.lhs profiling/ CostCentre.lhs profiling/SCCfinal.lhs rename/RnBinds.lhs rename/ RnEnv.lhs rename/RnExpr.lhs rename/RnHsDoc.hs rename/RnHsSyn.lhs rename/RnNames.lhs rename/RnSource.lhs rename/RnTypes.lhs simplCore/ CSE.lhs simplCore/FloatIn.lhs simplCore/FloatOut.lhs simplCore/ LiberateCase.lhs simplCore/OccurAnal.lhs simplCore/SAT.lhs simplCore/ SATMonad.lhs simplCore/SetLevels.lhs simplCore/SimplCore.lhs simplCore/ SimplEnv.lhs simplCore/SimplMonad.lhs simplCore/SimplUtils.lhs simplCore/Simplify.lhs simplStg/SRT.lhs simplStg/SimplStg.lhs simplStg/ StgStats.lhs specialise/Rules.lhs specialise/SpecConstr.lhs specialise/ Specialise.lhs stgSyn/CoreToStg.lhs stgSyn/StgLint.lhs stgSyn/ StgSyn.lhs stranal/DmdAnal.lhs stranal/SaAbsInt.lhs stranal/SaLib.lhs stranal/StrictAnal.lhs stranal/WorkWrap.lhs stranal/WwLib.lhs typecheck/FamInst.lhs typecheck/Inst.lhs typecheck/TcArrows.lhs typecheck/TcBinds.lhs typecheck/TcClassDcl.lhs typecheck/ TcDefaults.lhs typecheck/TcDeriv.lhs typecheck/TcEnv.lhs typecheck/ TcExpr.lhs typecheck/TcForeign.lhs typecheck/TcGadt.lhs typecheck/ TcGenDeriv.lhs typecheck/TcHsSyn.lhs typecheck/TcHsType.lhs typecheck/ TcInstDcls.lhs typecheck/TcMType.lhs typecheck/TcMatches.lhs typecheck/ TcPat.lhs typecheck/TcRnDriver.lhs typecheck/TcRnMonad.lhs typecheck/ TcRnTypes.lhs typecheck/TcRules.lhs typecheck/TcSimplify.lhs typecheck/ TcSplice.lhs typecheck/TcTyClsDecls.lhs typecheck/TcTyDecls.lhs typecheck/TcTyFuns.lhs typecheck/TcType.lhs typecheck/TcUnify.lhs types/Class.lhs types/Coercion.lhs types/FamInstEnv.lhs types/ FunDeps.lhs types/Generics.lhs types/InstEnv.lhs types/TyCon.lhs types/ Type.lhs types/TypeRep.lhs types/Unify.lhs utils/Bag.lhs utils/ Binary.hs utils/BufWrite.hs utils/Digraph.lhs utils/Encoding.hs utils/ FastMutInt.lhs utils/FastString.lhs utils/FastTypes.lhs utils/ FiniteMap.lhs utils/IOEnv.hs utils/ListSetOps.lhs utils/Maybes.lhs utils/OrdList.lhs utils/Outputable.lhs utils/Panic.lhs utils/ Pretty.lhs utils/State.hs utils/StringBuffer.lhs utils/UniqFM.lhs utils/UniqSet.lhs utils/Util.lhs vectorise/VectBuiltIn.hs vectorise/ VectCore.hs vectorise/VectMonad.hs vectorise/VectType.hs vectorise/ VectUtils.hs vectorise/Vectorise.hs .depend-BASE: openFile: resource busy (file is locked) When I build in an eshell running under Aquamacs, I've consistently had the build fail with: ../utils/mkdependC/mkdependC -f .depend-BASE -D__GLASGOW_HASKELL__=608 -I../includes -Iutils -IbasicTypes -Itypes -IhsSyn -Iprelude -Irename - Itypecheck -IdeSugar -IcoreSyn -Ivectorise -Ispecialise -IsimplCore - Istranal -IstgSyn -IsimplStg -IcodeGen -Imain -Iprofiling -Iparser - IcprAnalysis -IndpFlatten -Iiface -Icmm -InativeGen -Ighci -I../ includes -- -O -I/opt/local/include -Iparser -I. -O -- parser/ cutils.c parser/hschooks.c /opt/local/bin/perl -pe 'binmode(stdin); binmode(stdout); s@(\S*[._]o)@stage2/$1@g; s@(\S*[._]hi)@stage2/$1@g; s@^.*/compat.* $@@g;' <.depend-BASE >.depend-2 /usr/bin/make -C compiler stage=2 ../compiler/stage1/ghc-inplace -cpp stage2/ghc-inplace.c -o stage2/ ghc-inplace ld: in /Users/gwright/Desktop/ghc-6.8.2/libraries/haskell98/dist/build/ libHShaskell98-1.0.1.0.a, archive has no table of contents collect2: ld returned 1 exit status make[2]: *** [stage2/ghc-inplace] Error 1 make[1]: *** [stage2] Error 2 make: *** [bootstrap2] Error 2 In this last case, it looks as if libHShaskell98-1.0.1.0.a is being used before ranlib has been run on it. The common theme of these errors is that they all come out of Cabal, and seem to involve using the results of external processes before they are ready. In the original case of the temporary file used to record the version number, runProcess/waitForProcess are used. In the last case (running ranlib), runProcessPosix/ waitForProcess are used. Since the bug seems to be a race condition, it's not surprising that changing the code can cause it to disappear for a while. I will continue to look at what runProcess and waitForProcess are doing. Do you know of anything in Cabal that might confuse the handling of external processes? Were the original reports of weirdness in detecting the version number on OS X, or another operating system? Best Regards, Greg