- Glasgow-haskell-users

RE: Native Threads in the RTS
by Simon Peyton-Jones 10 Jan '03

10 Jan '03

| I've postponed writing up a new proposal again... | | But I'm going to sum up some requirements that I would like to see | fulfilled - to make it clearer to others why I'm proposing such strange | things... I've been watching the discussion about native threads, and getting thoroughly confused. In an attempt to get a handle on what is going on, I have (with Simon M) written a sort of operational semantics for Concurrent Haskell with foreign threads, that attempts to abstract away from everything except the issues that gave rise to these thread proposals. You can find the document in haskell-report/ffi/threads.tex. The style files are in haskell-report/styles/*.sty. There's a Makefile in haskell-report/ffi. It's very much a first cut, and it would clearly be good to merge the threads.txt document with this new Latex one, and in the end have a final spec. But before investing effort in fiddling with it, I thought it'd be good to see whether anyone finds it helpful. It's almost certainly inconsistent in minor ways, and may need adjustment to accurately reflect the proposals on the table, but I hope it may help to make the debate more precise. Feel free to modify it. E.g. add inline comments, alternative rules, and whatever. Simon |

2 1

RE: openFile and threads
by Peter Thiemann 10 Jan '03

10 Jan '03

Hi Simon, > > If the openFile isn't on the critical path, then it can help to do it in > another thread, but only if the openFile is blocking (not usually the > case if you're opening a normal file). Actually, we tried that using mergeIO, but there was no real improvement. > > You might consider bypassing the Handle interface and going to the bare > metal using the Posix library, which will cut down on the overhead in > openFile. That's what I was fearing. Is the conversion from Haskell Strings to C strings a performance problem? > > The best solution is to avoid the open altogether if possible (eg. with > caching), but I don't know about your particular application so this > might not be a possibility. No, that's not possible. The program reads every file at most once and then terminates. Although one might contemplate writing a separate server that keeps all the information in memory and refreshes it regularly. Thanks -Peter

1 0

RE: readping fd's and flushing buffers
by Simon Marlow 10 Jan '03

10 Jan '03

> A related problem... connections are refused when using accept if the > hostname doesn't resolve. Maybe something like this would > help, unless there > is a better way? > > accept sock = do > ~(sock', (SockAddrInet port haddr)) <- Socket.accept sock > (HostEntry peer _ _ _) <- ((getHostByAddr AF_INET haddr) > `Control.Exception.catch` (\_ -> > return (HostEntry ((showHex ((haddr `shiftR` 24) .&. 0xff) > . showChar '.' > . showHex ((haddr `shiftR` 16) .&. 0xff) > . showChar '.' . showHex ((haddr `shiftR` 8) .&. 0xff) > . showChar '.' > . showHex (haddr .&. 0xff)) "") [] AF_INET []))) > handle <- socketToHandle sock' ReadWriteMode > return (handle, peer, port) Yes, except that inet_ntoa is the right way to get the string representation of an IP address. I'll make this change. Cheers, Simon

1 0

RE: building stage=2 on solaris
by Simon Marlow 10 Jan '03

10 Jan '03

> ghc-5.05: unknown package name: readline > gmake[1]: *** [depend] Error 1 > gmake[1]: Leaving directory > `/nfs/nlg/users/hdaume/ghc-cvs/ghc/compiler' > gmake: *** [stage2] Error 2 > 106.14u 97.45s 5:15.85 64.4% > > > (sorry for the long clig), but the basic problem is this > missing package > readline. if i run ghc/compiler/stage1/ghc-inplace -v, I get the > following packages: > > data, hssource, net, text, util, posix, concurrent, lang, > unix, haskell-src, network, haskell98, base, rts > > but no readline. > > now, if i *remove* the '-package readline' from that commandline, it > builds properly. > > is this needed? I've fixed it so that -package readline should be omitted if stage 1 doesn't have a readline package. GHCi falls back to using hGetLine if there's no readline available, which is why the build goes through after removing the option. Cheers, Simon

1 0

RE: Two Questions: 'memory consumption' and '-pgmL'
by Simon Peyton-Jones 10 Jan '03

10 Jan '03

It's also entirely possible that GHC is using more memory than it really needs. We haven't done a space audit recently. Sometimes these things are program dependent. Would you like to cut down your program a bit, till it needs a mere 128M or so, and forward it to us? Or you could just wait... we are planning to look at space in the next month or so. Simon | -----Original Message----- | From: Peter Simons [mailto:simons@cryp.to] | Sent: 09 January 2003 20:54 | To: glasgow-haskell-users(a)haskell.org | Subject: Re: Two Questions: 'memory consumption' and '-pgmL' | | Simon Marlow writes: | | > I would try -c first (turn on the compacting collector). Adding | > more generations (eg. -G3) might help, and setting a maximum heap | > size (eg. -M512m) will cause GHC to try to trim down its memory use | > when it gets close to this boundary. | | Unfortunately neither of that helped. It appears that ghc simply | _needs_ that amount of memory. No matter what option I gave, at one | point it hit the 800 MB limit and aborted. (I specified -M800 because | if it used more memory than that, the machine stood basically still | with thrashing.) | | Looks like I'll have to support the memor chip industry ... The | problem is that if a 512MB machine cannot compile it, I wonder how the | _users_ of my program will get along. I guess they'll tend to have | smaller machines than the average software developer does. | | -peter | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users(a)haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

1 0

building stage=2 on solaris
by Hal Daume III 09 Jan '03

09 Jan '03

I know this is somewhat related to something in the FAQ, but I'm having a bit of trouble building GHC from CVS source (gmake bootstrap). Stage 1 completes find and I get a ghc-inplace as I should. The last bit looks like: gmake[1]: Entering directory `/nfs/nlg/users/hdaume/ghc-cvs/ghc/compiler' ../../glafp-utils/mkdirhier/mkdirhier stage2 for i in utils basicTypes types hsSyn prelude rename typecheck deSugar coreSyn specialise simplCore stranal stgSyn simplStg codeGen absCSyn main profiling parser usageSP cprAnalysis compMan ndpFlatten nativeGen ghci; do \ ../../glafp-utils/mkdirhier/mkdirhier stage2/$i; \ done for i in */*hi-boot*; do \ gln -s -f ../../$i stage2/$i; \ done ../../ghc/compiler/stage1/ghc-inplace -M -optdep-f -optdep.depend-BASE -osuf o -I../includes -H64m -O0 -fno-warn-unused-matches -lbfd -liberty -iutils -ibasicTypes -itypes -ihsSyn -iprelude -irename -itypecheck -ideSugar -icoreSyn -ispecialise -isimplCore -istranal -istgSyn -isimplStg -icodeGen -iabsCSyn -imain -iprofiling -iparser -iusageSP -icprAnalysis -icompMan -indpFlatten -inativeGen -ighci -DGHCI -package haskell-src -package unix -package readline -cpp -fglasgow-exts -I. -IcodeGen -InativeGen -Iparser -recomp -DDEBUG -H16M '-#include "hschooks.h"' absCSyn/AbsCSyn.lhs absCSyn/AbsCUtils.lhs absCSyn/CLabel.lhs absCSyn/CStrings.lhs absCSyn/Costs.lhs absCSyn/MachOp.hs absCSyn/PprAbsC.lhs basicTypes/BasicTypes.lhs basicTypes/DataCon.lhs basicTypes/Demand.lhs basicTypes/FieldLabel.lhs basicTypes/Id.lhs basicTypes/IdInfo.lhs basicTypes/Literal.lhs basicTypes/MkId.lhs basicTypes/Module.lhs basicTypes/Name.lhs basicTypes/NameEnv.lhs basicTypes/NameSet.lhs basicTypes/NewDemand.lhs basicTypes/OccName.lhs basicTypes/RdrName.lhs basicTypes/SrcLoc.lhs basicTypes/UniqSupply.lhs basicTypes/Unique.lhs basicTypes/Var.lhs basicTypes/VarEnv.lhs basicTypes/VarSet.lhs codeGen/CgBindery.lhs codeGen/CgCase.lhs codeGen/CgClosure.lhs codeGen/CgCon.lhs codeGen/CgConTbls.lhs codeGen/CgExpr.lhs codeGen/CgHeapery.lhs codeGen/CgLetNoEscape.lhs codeGen/CgMonad.lhs codeGen/CgRetConv.lhs codeGen/CgStackery.lhs codeGen/CgTailCall.lhs codeGen/CgUpdate.lhs codeGen/CgUsages.lhs codeGen/ClosureInfo.lhs codeGen/CodeGen.lhs codeGen/SMRep.lhs compMan/CompManager.lhs coreSyn/CoreFVs.lhs coreSyn/CoreLint.lhs coreSyn/CorePrep.lhs coreSyn/CoreSyn.lhs coreSyn/CoreTidy.lhs coreSyn/CoreUnfold.lhs coreSyn/CoreUtils.lhs coreSyn/ExternalCore.lhs coreSyn/MkExternalCore.lhs coreSyn/PprCore.lhs coreSyn/PprExternalCore.lhs coreSyn/Subst.lhs cprAnalysis/CprAnalyse.lhs deSugar/Check.lhs deSugar/Desugar.lhs deSugar/DsBinds.lhs deSugar/DsCCall.lhs deSugar/DsExpr.lhs deSugar/DsForeign.lhs deSugar/DsGRHSs.lhs deSugar/DsListComp.lhs deSugar/DsMeta.hs deSugar/DsMonad.lhs deSugar/DsUtils.lhs deSugar/Match.lhs deSugar/MatchCon.lhs deSugar/MatchLit.lhs ghci/ByteCodeAsm.lhs ghci/ByteCodeFFI.lhs ghci/ByteCodeGen.lhs ghci/ByteCodeInstr.lhs ghci/ByteCodeItbls.lhs ghci/ByteCodeLink.lhs ghci/InteractiveUI.hs ghci/Linker.lhs ghci/ObjLink.lhs hsSyn/Convert.lhs hsSyn/HsBinds.lhs hsSyn/HsCore.lhs hsSyn/HsDecls.lhs hsSyn/HsExpr.lhs hsSyn/HsImpExp.lhs hsSyn/HsLit.lhs hsSyn/HsPat.lhs hsSyn/HsSyn.lhs hsSyn/HsTypes.lhs main/BinIface.hs main/CmdLineOpts.lhs main/CodeOutput.lhs main/Config.hs main/Constants.lhs main/DriverFlags.hs main/DriverMkDepend.hs main/DriverPhases.hs main/DriverPipeline.hs main/DriverState.hs main/DriverUtil.hs main/ErrUtils.lhs main/Finder.lhs main/GetImports.hs main/HscMain.lhs main/HscStats.lhs main/HscTypes.lhs main/Interpreter.hs main/Main.hs main/MkIface.lhs main/Packages.lhs main/ParsePkgConf.hs main/SysTools.lhs main/TidyPgm.lhs nativeGen/AbsCStixGen.lhs nativeGen/AsmCodeGen.lhs nativeGen/AsmRegAlloc.lhs nativeGen/MachCode.lhs nativeGen/MachMisc.lhs nativeGen/MachRegs.lhs nativeGen/PprMach.lhs nativeGen/RegAllocInfo.lhs nativeGen/Stix.lhs nativeGen/StixMacro.lhs nativeGen/StixPrim.lhs ndpFlatten/FlattenInfo.hs ndpFlatten/FlattenMonad.hs ndpFlatten/Flattening.hs ndpFlatten/NDPCoreUtils.hs ndpFlatten/PArrAnal.hs parser/Ctype.lhs parser/Lex.lhs parser/LexCore.hs parser/Parser.hs parser/ParserCore.hs parser/ParserCoreUtils.hs parser/RdrHsSyn.lhs prelude/ForeignCall.lhs prelude/PrelInfo.lhs prelude/PrelNames.lhs prelude/PrelRules.lhs prelude/PrimOp.lhs prelude/PrimRep.lhs prelude/TysPrim.lhs prelude/TysWiredIn.lhs profiling/CostCentre.lhs profiling/SCCfinal.lhs rename/RnBinds.lhs rename/RnEnv.lhs rename/RnExpr.lhs rename/RnHiFiles.lhs rename/RnHsSyn.lhs rename/RnIfaces.lhs rename/RnNames.lhs rename/RnSource.lhs rename/RnTypes.lhs simplCore/CSE.lhs simplCore/FloatIn.lhs simplCore/FloatOut.lhs simplCore/LiberateCase.lhs simplCore/OccurAnal.lhs simplCore/SAT.lhs simplCore/SATMonad.lhs simplCore/SetLevels.lhs simplCore/SimplCore.lhs simplCore/SimplMonad.lhs simplCore/SimplUtils.lhs simplCore/Simplify.lhs simplStg/SRT.lhs simplStg/SimplStg.lhs simplStg/StgStats.lhs specialise/Rules.lhs specialise/SpecConstr.lhs specialise/Specialise.lhs stgSyn/CoreToStg.lhs stgSyn/StgLint.lhs stgSyn/StgSyn.lhs stranal/DmdAnal.lhs stranal/SaAbsInt.lhs stranal/SaLib.lhs stranal/StrictAnal.lhs stranal/WorkWrap.lhs stranal/WwLib.lhs typecheck/Inst.lhs typecheck/TcBinds.lhs typecheck/TcClassDcl.lhs typecheck/TcDefaults.lhs typecheck/TcDeriv.lhs typecheck/TcEnv.lhs typecheck/TcExpr.lhs typecheck/TcForeign.lhs typecheck/TcGenDeriv.lhs typecheck/TcHsSyn.lhs typecheck/TcIfaceSig.lhs typecheck/TcInstDcls.lhs typecheck/TcMType.lhs typecheck/TcMatches.lhs typecheck/TcMonoType.lhs typecheck/TcPat.lhs typecheck/TcRnDriver.lhs typecheck/TcRnMonad.lhs typecheck/TcRnTypes.lhs typecheck/TcRules.lhs typecheck/TcSimplify.lhs typecheck/TcSplice.lhs typecheck/TcTyClsDecls.lhs typecheck/TcTyDecls.lhs typecheck/TcType.lhs typecheck/TcUnify.lhs types/Class.lhs types/FunDeps.lhs types/Generics.lhs types/InstEnv.lhs types/PprType.lhs types/TyCon.lhs types/Type.lhs types/TypeRep.lhs types/Variance.lhs usageSP/UConSet.lhs usageSP/UsageSPInf.lhs usageSP/UsageSPLint.lhs usageSP/UsageSPUtils.lhs utils/Bag.lhs utils/Binary.hs utils/BitSet.lhs utils/Digraph.lhs utils/FastMutInt.lhs utils/FastString.lhs utils/FastTypes.lhs utils/FiniteMap.lhs utils/ListSetOps.lhs utils/Maybes.lhs utils/OrdList.lhs utils/Outputable.lhs utils/Panic.lhs utils/Pretty.lhs utils/PrimPacked.lhs utils/StringBuffer.lhs utils/UnicodeUtil.lhs utils/UniqFM.lhs utils/UniqSet.lhs utils/Util.lhs ghc-5.05: unknown package name: readline gmake[1]: *** [depend] Error 1 gmake[1]: Leaving directory `/nfs/nlg/users/hdaume/ghc-cvs/ghc/compiler' gmake: *** [stage2] Error 2 106.14u 97.45s 5:15.85 64.4% (sorry for the long clig), but the basic problem is this missing package readline. if i run ghc/compiler/stage1/ghc-inplace -v, I get the following packages: data, hssource, net, text, util, posix, concurrent, lang, unix, haskell-src, network, haskell98, base, rts but no readline. now, if i *remove* the '-package readline' from that commandline, it builds properly. is this needed? -- Hal Daume III "Computer science is no more about computers | hdaume(a)isi.edu than astronomy is about telescopes." -Dijkstra | www.isi.edu/~hdaume

1 0

RE: readping fd's and flushing buffers
by Simon Marlow 09 Jan '03

09 Jan '03

> This may be related to the answer just given to do with > finalizing the > Handles, as I have a > problem with sockets hanging around after a host name resolution has > failed (using the simple socket > library). Having looked at the code I would like to suggest the > following change: > > connectTo hostname (PortNumber port) = do > proto <- getProtocolNumber "tcp" > sock <- socket AF_INET Stream proto > he <- getHostByName hostname > connect sock (SockAddrInet port (hostAddress he)) > socketToHandle sock ReadWriteMode > > Should become: > > connectTo hostname (PortNumber port) = do > proto <- getProtocolNumber "tcp" > sock <- socket AF_INET Stream proto > (do > he <- getHostByName hostname > connect sock (SockAddrInet port (hostAddress he)) > socketToHandle sock ReadWriteMode) `Exception.catch` > (\e -> do > sClose sock;throw e) > > Is this a sensible change to make? Yes, well spotted. I'll add the exception handler. Cheers, Simon

2 1

RE: Two Questions: 'memory consumption' and '-pgmL'
by Simon Marlow 09 Jan '03

09 Jan '03

> first of all, let me thank you for writing and maintaining this > excellent compiler! I am using it a lot recently and I couldn't be > more happy with it. Thanks! :-) > > I wouldn't be posting here, though, if hadn't had a questions ... So > here I go: > > (1) Using the DtdToHaskell tool, I converted the XML Docbook DTD to > Haskell code. The resulting parser is amazing: It is almost 4 > megabyte large, 72800 lines of code. Now I tried to compile this > beast and ran fresh out of memory. > > I fiddled with the RTS options to no avail. At some point GHC was > consuming more than 800 megabytes of RAM, what resulted in > serious thrashing (my machine has only 512 MB) and eventually the > process was terminated. > > Does anyone have by any ideas how I could tackle this problem? > Can I reduce GHC's memory requirements somehow? Can I split the > module up and compile it in parts? In different phases? Anything? > (If nothing comes up, I guess buying some more RAM is the > answer.) You can try fiddling with GHC's GC settings to reduce memory consumption. The section of the User's Guide on runtime flags has some hints; I would try -c first (turn on the compacting collector). Adding more generations (eg. -G3) might help, and setting a maximum heap size (eg. -M512m) will cause GHC to try to trim down its memory use when it gets close to this boundary. Remember to surround any RTS options with +RTS ... -RTS. > (2) According to the documentation, GHC allows for setting the > literate pre-processor to be used when compiling an .lhs file. > This is supposed to occur with the '-pgmL' option, but no matter > what I try, GHC always tells me: > > | ghc-5.04.2: unrecognised flags: -pgmL > | Usage: For basic information, try the `--help' option. > > Am I doing something wrong or is the documentation out of synch > with the implementation? Hmm, good point. This option is missing from the implementation, thanks for pointing it out. Cheers, Simon

2 1

ghc -M
by Peter Thiemann 09 Jan '03

09 Jan '03

Folks, after using ghc --make happily in a project for a while I am switching to using Makefiles since I have a number of Main programs in the same directory that share a lot of modules. Now, since ghc -M does such a great job at gobbling up the dependencies, I wonder if it was not possible to extend its functionality a little bit so that it also determines the .o files that are required to link a Main program. I could imagine something like an -optdep-Main and then the program name. -Peter

2 2

RE: trying to build from cvs - "cc1: bad value (athlonmp) for -march= switch"
by Simon Marlow 09 Jan '03

09 Jan '03

> -------------------------------------------------------------- > ---------- > ==fptools== make all -wr; > in /home/shae/build/haskell/fpbuild/ghc/rts > -------------------------------------------------------------- > ---------- > ../../ghc/compiler/ghc-inplace -optc-O -optc-Wall -optc-W > -optc-Wstrict-prototypes -optc-Wmissing-prototypes > -optc-Wmissing-declarations -optc-Winline > -optc-Waggregate-return -optc-Wbad-function-cast > -optc-I../includes -optc-I. -optc-Iparallel > -optc-DCOMPILING_RTS -optc-O2 -optc-fomit-frame-pointer > -optc-mpreferred-stack-boundary=2 -optc-march=athlonmp > -optc-DTHREADED_RTS -H16m -O -O2 -static -c Adjustor.c -o > Adjustor.o > cc1: bad value (athlonmp) for -march= switch > cc1: bad value (athlonmp) for -mcpu= switch > make[2]: *** [Adjustor.o] Error 1 > make[1]: *** [all] Error 1 > > > running "gcc -v" gives: > Reading specs from /usr/lib/gcc-lib/i386-linux/3.2.2/specs > Configured with: ../src/configure -v > --enable-languages=c,c++,java,f77,proto,pascal,objc,ada > --prefix=/usr --mandir=/usr/share/man > --infodir=/usr/share/info > --with-gxx-include-dir=/usr/include/c++/3.2 --enable-shared > --with-system-zlib --enable-nls --without-included-gettext > --enable-__cxa_atexit --enable-clocale=gnu > --enable-java-gc=boehm --enable-objc-gc i386-linux > Thread model: posix > gcc version 3.2.2 20021231 (Debian prerelease) > > Athlon MP is the correct arch/cpu, and gcc 3.2.2 has > -march=athlonmp available. > Any ideas? Are you sure that GHC is running the same gcc? Try 'ghc -v -fvia-C' on some small Haskell source file. Cheers, Simon

2 1