September 2010 - Glasgow-haskell-users

clearing GHCi (and, by extension, hint) loaded module dependencies
by Alex Suraci 21 Sep '10

21 Sep '10

Context: My language uses hint to interpret Haskell code at runtime, via `load: "path/to/file.hs"`. Hint works similar to ":load foo.hs" in GHCi (it uses the GHC API). After the source is interpreted the module's `load` function is executed in the language's VM. There is no valuable result; it is executed for its side-effects, usually definitions. Problem: Around 80MB[1] is used up by the module's dependencies and never freed. Subsequent "load:"s are faster, but that 80MB overhead can be expensive on things like VPSes. This can be simulated in GHCi by doing ":load foo.hs" followed by a ":load" to clear the loaded modules; the memory usage doesn't go down (understandably, in this case), and :loading it again is much faster. Is there any way to "hard reset" or free the memory being used for the loaded module's dependencies? Thanks, Alex [1]: number determined by loading an "empty" script, i.e. `load = return ()`

2 1

Collecting all external names in a module
by Johan Tibell 20 Sep '10

20 Sep '10

Hi, I have a question regarding the GHC API. Given a module, I'm trying to collect * the Name and SrcSpan of all top-level definitions, * the Name and SrcSpan of all (local) uses of these top-level definition * the Name and SrcSpan of all uses of imported definitions. For example, given the file A.hs module B where data Foo = Bar | Baz String main = print $ "Hello, World!" ++ show test test = let x = 2 in x I would like to output: B.Foo - A.hs:3:5-7 B.Bar - A.hs:3:10-12 B.Baz - A.hs:3:10-12 GHC.Base.String - A.hs:3:13-18 B.main - A.hs:5:1-4 System.IO.print - A.hs:5:8-11 GHC.Base.++ - A.hs:5:18-19 etc. (The line/column numbers are made up.) * I do not want to output e.g. 'x' as it's not a top-level identifier (the code I've included below gets this wrong). * I want to output whether the SrcSpan corresponds to a use site or definition site of the Name. For example: 'Foo' is a definition site while 'print' is a use site. I started writing a manual traversal of the RenamedSource AST (as I want qualified names) but I thought I check if I'm going about this right before I spend all the time required to write the traversal for the whole AST. Here's the code I have so far, am I on the right track? ---------- -- | Collects all qualified names that are referred to in a module, -- that are either defineds at the top-level in that module or that -- are imported from some other module. module Main where import Bag import DynFlags ( defaultDynFlags ) import GHC import GHC.Paths ( libdir ) import Outputable import System.Environment -- | Is the 'Name' defined here? data Origin = Local | External type Use = (Name, Origin, SrcSpan) local :: Name -> Use local name = (name, Local, nameSrcSpan name) external :: Name -> SrcSpan -> Use external name loc = (name, External, loc) showName :: (Name, Origin, SrcSpan) -> String showName (name, org, loc) = showSDoc (ppr name) ++ "," ++ showSDoc (ppr loc) ++ "," ++ showOrg org where showOrg Local = "1" showOrg External = "0" main :: IO () main = do [targetFile] <- getArgs res <- defaultErrorHandler defaultDynFlags $ runGhc (Just libdir) $ do dflags <- getSessionDynFlags _ <- setSessionDynFlags dflags target <- guessTarget targetFile Nothing setTargets [target] _ <- load LoadAllTargets modSum <- getModSummary $ mkModuleName "B" p <- parseModule modSum t <- typecheckModule p let Just (r, _, _, _) = tm_renamed_source t return r putStrLn $ showSDoc $ ppr res putStrLn "" putStr $ unlines $ map showName (collectHsGroup res) ------------------------------------------------------------------------ -- AST traversal -- | Collect all external qualified names in the module. collectHsGroup :: HsGroup Name -> [Use] collectHsGroup = collectHsValBindsLR . hs_valds collectHsValBindsLR :: HsValBindsLR Name Name -> [Use] collectHsValBindsLR (ValBindsOut xs _) = concatMap collectHsBindNames . map unLoc . concatMap bagToList . map snd $ xs collectHsValBindsLR (ValBindsIn binds _) = concatMap (collectHsBindNames . unLoc) (bagToList binds) collectHsBindNames :: HsBindLR Name Name -> [Use] collectHsBindNames fb@(FunBind { fun_id = L _ f }) = [local f] ++ collectMatchGroupNames (fun_matches fb) collectHsBindNames _ = [] collectMatchGroupNames :: MatchGroup Name -> [Use] collectMatchGroupNames (MatchGroup matches _) = concat [collectGRHSsNames x | Match _ _ x <- map unLoc matches] collectGRHSsNames :: GRHSs Name -> [Use] collectGRHSsNames (GRHSs xs _) = concatMap (collectGRHSNames . unLoc) xs collectGRHSNames :: GRHS Name -> [Use] collectGRHSNames (GRHS _stmts exprs) = collectHsExprNames exprs -- For less typing collectHsExprNames :: LHsExpr Name -> [Use] collectHsExprNames = collect collect :: LHsExpr Name -> [Use] collect (L loc expr) = go expr where go (HsVar name) | isExternalName name = [external name loc] | otherwise = [] go (HsIPVar _) = [] go (HsOverLit _) = [] go (HsLit _) = [] go (HsLam mg) = collectMatchGroupNames mg go (HsApp e1 e2) = collect e1 ++ collect e2 go (OpApp e1 e2 _ e3) = collect e1 ++ collect e2 ++ collect e3 go (NegApp e1 _) = collect e1 -- ++ collect e2 -- ??? go (HsPar e) = collect e go (SectionL e1 e2) = collect e1 ++ collect e2 go (SectionR e1 e2) = collect e1 ++ collect e2 go (ExplicitTuple xs _) = concat [ collect x | Present x <- xs] go (HsCase e mg) = collect e ++ collectMatchGroupNames mg go (HsIf e1 e2 e3) = collect e1 ++ collect e2 ++ collect e3 go (HsLet binds e) = collectHsLocalBindsLR binds ++ collect e -- go (HsDo (HsStmtContext Name) [LStmt id] (LHsExpr id) PostTcType) = -- go (ExplicitList PostTcType [LHsExpr id]) = -- go (ExplicitPArr PostTcType [LHsExpr id]) = -- go (RecordCon (Located id) PostTcExpr (HsRecordBinds id)) = -- go (RecordUpd (LHsExpr id) (HsRecordBinds id) [DataCon] [PostTcType] [PostTcType]) = -- go (ExprWithTySig (LHsExpr id) (LHsType id)) = -- go (ExprWithTySigOut (LHsExpr id) (LHsType Name)) = -- go (ArithSeq PostTcExpr (ArithSeqInfo id)) = -- go (PArrSeq PostTcExpr (ArithSeqInfo id)) = -- go (HsSCC FastString (LHsExpr id)) = -- go (HsCoreAnn FastString (LHsExpr id)) = -- go (HsBracket (HsBracket id)) = -- go (HsBracketOut (HsBracket Name) [PendingSplice]) = -- go (HsSpliceE (HsSplice id)) = -- go (HsQuasiQuoteE (HsQuasiQuote id)) = -- go (HsProc (LPat id) (LHsCmdTop id)) = -- go (HsArrApp (LHsExpr id) (LHsExpr id) PostTcType HsArrAppType Bool) = -- go (HsArrForm (LHsExpr id) (Maybe Fixity) [LHsCmdTop id]) = -- go (HsTick Int [id] (LHsExpr id)) = -- go (HsBinTick Int Int (LHsExpr id)) = -- go (HsTickPragma (FastString, (Int, Int), (Int, Int)) (LHsExpr id)) = -- go (EWildPat) = -- go (EAsPat (Located id) (LHsExpr id)) = -- go (EViewPat (LHsExpr id) (LHsExpr id)) = -- go (ELazyPat (LHsExpr id)) = -- go (HsType (LHsType id)) = -- go (HsWrap HsWrapper (HsExpr id)) = go _ = [] collectHsLocalBindsLR :: HsLocalBindsLR Name Name -> [Use] collectHsLocalBindsLR (HsValBinds x) = collectHsValBindsLR x collectHsLocalBindsLR _ = []

4 8

Template Haskell and linking
by Jonas Almström Duregård 20 Sep '10

20 Sep '10

Hi GHC users! Does anyone know if an import that is only used by Template Haskell (i.e. not in "actual" code) is reflected in the produced executable? Example: import LargeModule(thFunction) $thfunction ... Is LargeModule linked in the executable file? (Assume thFunction is not referenced by the code it generates or elsewhere in the example) Best regards Jonas

2 1

Re: Cost Centre MAIN
by Christian Höner zu Siederdissen 19 Sep '10

19 Sep '10

Yeah that was a bit lazy by me (or bull, if you want...;-); looks like you have runStmLogger on the top-level of your Main module ("MAIN") as well. What functions are defined there? And what vars? Gruss, Christian * Stefan Wehr <stefan.wehr(a)gmail.com> [19.09.2010 20:28]: > 2010/9/19 Christian Höner zu Siederdissen <choener(a)tbi.univie.ac.at>: > > You should either try "-auto-all" while compiling to get cost centres > > for every top-level function or insert cost centres by hand with "SCC": > > > > http://haskell.cs.yale.edu/ghc/docs/6.12.2/html/users_guide/profiling.html > > Well, I did compile with -auto-all and -caf-all > > > Otherwise, MAIN is "main = ..." > > Really? Here are the first couple of lines of the detailed profiling > report I get: > > COST CENTRE MODULE > no. entries %time %alloc %time %alloc > > MAIN MAIN > 1 0 47.5 0.2 100.0 100.0 > runStmLogger MobileGateway.Util.Logging > 5223 0 0.0 0.0 0.0 0.0 > main Main > 5166 1 0.0 0.0 0.6 0.2 > main MobileGateway.LubSyncClient > 5167 5 0.0 0.0 0.6 0.2 > getUrl MobileGateway.Util.Config > 5274 1 0.0 0.0 0.0 0.0 > > Doesn't look like MAIN is the same as "main = ...". > > -- Stefan

1 0

Re: Cost Centre MAIN
by Christian Höner zu Siederdissen 19 Sep '10

19 Sep '10

You should either try "-auto-all" while compiling to get cost centres for every top-level function or insert cost centres by hand with "SCC": http://haskell.cs.yale.edu/ghc/docs/6.12.2/html/users_guide/profiling.html Otherwise, MAIN is "main = ..." Gruss, Christian * Stefan Wehr <stefan.wehr(a)gmail.com> [19.09.2010 14:11]: > Hi all, > > I'm profiling a Haskell program and now getting that the program > spents 56% of its individual time in the cost centre MAIN (note the > capital letters). > > I searched the documentation for this cost centre, with no success. In > all profiling results so far, the individual time spent in MAIN was > always 0%. > > Any hints? > > -- Stefan > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users(a)haskell.org > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

1 0

Allocation break-down by type
by Bruno Martínez Aguerre 18 Sep '10

18 Sep '10

Hi list, To understand better whether ghc is taking advantage of my UNPACK pragmas, I want to profile my program and sort the allocations by type. I couldn't find an option to do this. -p is only by cost-center, and -hy only works for live data, and I want all allocations. Am I missing some option? Any tips to better debug UNPACK, apart from inspecting core? Thanks, Bruno

3 2

Interruptible GHC
by Edward Z. Yang 17 Sep '10

17 Sep '10

So I did a writeup of what I thought might be the next direction to go with the interruptible patch: http://blog.ezyang.com/2010/09/towards-platform-agnostic-interruptibility/ The really interesting bit (which I didn't cover) is what information to give to the user functioSo I did a writeup of what I thought might be the next direction to go with the interruptible patch: http://blog.ezyang.com/2010/09/towards-platform-agnostic-interruptibility/ The really interesting bit (which I didn't cover) is what information to give to the user function. It needs to be somewhat under the hood: if someone wants to pthread_cancel they need to know what the pthreads ID is; if they want to transmit a Windows event they need some way to get a hold of the event handle associated with the foreign call. Edward

2 7

Directory-backed Handle?
by Dimitry Golubovsky 15 Sep '10

15 Sep '10

Hi, In the GHC I/O system, a Handle may be backed by a "device" (IODevice instance) with IODeviceType = Directory There doesn't seem to be any (at least Google search did not give anything) code that uses this; however, what is supposed to be read from such a Handle: newline-separated list of filenames (like in getDirectoryContents), or some sort of serialized FileStatus structures, or anything else? Thanks. -- Dimitry Golubovsky Anywhere on the Web

3 3

Demand for 64-bit Windows GHC?
by Satnam Singh 15 Sep '10

15 Sep '10

I am trying to get a sense for how much demand there is for a 64-bit version of GHC for Windows. Personally I have two reasons why I would like to have a 64-bit build: * To allow me to write services in Haskell to be deployed on Microsoft's Azure cloud computing system which requires 64-bit binaries (and life is too short for SysWOW64). * To allow me to interoperate with 64-bit native libraries e.g. for dynamic code generation for SSE3 vector instructions as supported by the Accelerator system (I've done this in F# http://blogs.msdn.com/b/satnam_singh/archive/2009/12/15/gpgpu-and-x64-multi… and it would be good to have a Haskell version too for SSE3 vector code - the GPU version works fine with 32-bit DLLs). Is anyone else interested in a 64-bit version of GHC for Windows? Simon Marlow reckons it is about 2 weeks work for him so it should only take a mere mortal about a month or two to do. Anyone interested in making a 64-bit Windows port of GHC? Cheers, Satnam ________________________________ Satnam Singh Microsoft 7 JJ Thomson Avenue Cambridge CB3 0FB United Kingdom Email: satnams(a)microsoft.com<mailto:satnams@microsoft.com> UK tel: +44 1223 479905 Fax: +44 1223 479 999 UK mobile: +44 7979 648412 USA cell: 206 330 1580 URL: http://research.microsoft.com/~satnams Live Messenger: satnam(a)raintown.org<mailto:satnam@raintown.org>

1 0

ghc_rts_options (defalting RTS flags)
by Thomas DuBuisson 15 Sep '10

15 Sep '10

All, A recent SO question [1] led me to do a quick test on hard-coding RTS options as suggested in the manual rts-hooks section [2]. Unfortunately the timing tests indicate "ghc_rts_opts" isn't being used; tests with flags besides -N (ex: "-H1024m") also show the ghc_rts_opts being ignored. Can someone shed light on what's happening? A grep of ghc source shows no users of ghc_rts_opts, though that isn't definitive. Cheers, Thomas -------- $ cat rts-opts.h char *ghc_rts_opts = "-N"; $ ghc --make -O2 -threaded hashPar.hs rts-opts.h -fforce-recomp [1 of 1] Compiling Main ( hashPar.hs, hashPar.o ) Linking hashPar ... $ time ./hashPar 10MB 10MB 10MB 10MB ["10MB","10MB","10MB"] real 0m2.411s $ time ./hashPar 10MB 10MB 10MB 10MB +RTS -N ["10MB","10MB","10MB"] real 0m1.690s ---------- [1] http://stackoverflow.com/questions/3704430/why-do-you-have-to-use-both-a-co… [2] http://haskell.org/ghc/docs/6.12.2/html/users_guide/runtime-control.html#rt…

2 1