- Glasgow-haskell-users

Cost Centre MAIN
by Stefan Wehr 23 Sep '10

23 Sep '10

Hi all, I'm profiling a Haskell program and now getting that the program spents 56% of its individual time in the cost centre MAIN (note the capital letters). I searched the documentation for this cost centre, with no success. In all profiling results so far, the individual time spent in MAIN was always 0%. Any hints? -- Stefan

2 2

GHC 6.13 + Data.Vector experiences
by Christian Höner zu Siederdissen 23 Sep '10

23 Sep '10

Hi, I am about to upload a number of packages to hackage. They are all needed by RNAFoldProgs. The main program therein, RNAFold, replicates RNAfold of the ViennaRNA package. The difference is that the algorithm already uses a newer set of parameters; the C functionality will follow soon from our lab. But that is not why this message: RNAFold requires a total of 15 lookup tables (multidimensional arrays), from 1-d arrays up to a maximum of 6 dimensions. We write into 5 different 2-dimensional tables. 15 different (and rather complex) functions are involved in filling the tables. Everything is built upon the Data.Vector package to facilitate fusion to happen (though it boils down to map+enumFromN most of the time). Some measurements on an input sequence show the following: ViennaRNA / C: 1.75s 6.13.20100826 / llvm / O2: 40s 6.13 / llvm O3 / Odph: 17.5s 6.12.3 / c O3 / Odph: 535s (really, nine minutes [1]) Unfortunately, between last and this week, the time went from ~13s to 18s but instead of fiddling around, I thought to release the packages for anyone interested. Please take it only as my personal testbed for Data.Vector and head -- though should you want to use the lib for actual rna secondary structure prediction, that should work, too. Anyways, there are probably a number of bugs in there, so be warned. The target is, of course, to have a runtime of 1.74s on my machine :-) Thanks to don stewart (uvector), roman leshchinskiy (vector) and simon peyton jones (default method inlining [1]) [1] "6.12.3" shows just how much the working default method inliner brings. With 6.12, everything goes through dictionaries which is not cool if you need to do many millions of min/+ operations (We work on a ring and instanciate very late in the game). Viele Gruesse, Christian PS: Why are my executables with HEAD like 60mbyte in size?!

1 0

GHCi 6.12.3 stdout redirection works for :type, but not for :info
by Claus Reinke 22 Sep '10

22 Sep '10

Dear GHCi / IO experts, I'm encountering some odd GHCi behaviour with some old ghci.conf code of mine (used to work in older GHCis). The problem is that my code for redirecting the output of GHCi commands still works for things like :type, but no longer for things like :info (this is my smallest example demonstrating the effect). With the attached GHCi script defining a :redir <var> <command> command that should redirect the ouput of <command> into variable <var>, we get the output appended below (note the Exception for WriteConsole "handle is invalid", which is written into the temporary file when :info is called while stdout is redirected to that temporary file; in other words, the error message appears where the output of the command fails to appear). Is that an expected change with a known workaround, so that I just need to update my code, or is that unexpected behaviour that can only be fixed in GHCi? Claus PS. This is on Windows 7, if that matters. ------------- example session output $ ghcii.sh -ignore-dot-ghci GHCi, version 6.12.3: http://www.haskell.org/ghc/ :? for help .. Prelude> :cmd readFile "redir.script" Prelude> :redir x :t map .. Prelude> x "-- logging GHCi stdout\nmap :: (a -> b) -> [a] -> [b]\n" Prelude> :redir y :i map Prelude> y "-- logging GHCi stdout\n*** Exception: WriteConsole: invalid argument (Das Handle ist ung\252ltig.)\n"

1 1

clearing GHCi (and, by extension, hint) loaded module dependencies
by Alex Suraci 21 Sep '10

21 Sep '10

Context: My language uses hint to interpret Haskell code at runtime, via `load: "path/to/file.hs"`. Hint works similar to ":load foo.hs" in GHCi (it uses the GHC API). After the source is interpreted the module's `load` function is executed in the language's VM. There is no valuable result; it is executed for its side-effects, usually definitions. Problem: Around 80MB[1] is used up by the module's dependencies and never freed. Subsequent "load:"s are faster, but that 80MB overhead can be expensive on things like VPSes. This can be simulated in GHCi by doing ":load foo.hs" followed by a ":load" to clear the loaded modules; the memory usage doesn't go down (understandably, in this case), and :loading it again is much faster. Is there any way to "hard reset" or free the memory being used for the loaded module's dependencies? Thanks, Alex [1]: number determined by loading an "empty" script, i.e. `load = return ()`

2 1

Collecting all external names in a module
by Johan Tibell 21 Sep '10

21 Sep '10

Hi, I have a question regarding the GHC API. Given a module, I'm trying to collect * the Name and SrcSpan of all top-level definitions, * the Name and SrcSpan of all (local) uses of these top-level definition * the Name and SrcSpan of all uses of imported definitions. For example, given the file A.hs module B where data Foo = Bar | Baz String main = print $ "Hello, World!" ++ show test test = let x = 2 in x I would like to output: B.Foo - A.hs:3:5-7 B.Bar - A.hs:3:10-12 B.Baz - A.hs:3:10-12 GHC.Base.String - A.hs:3:13-18 B.main - A.hs:5:1-4 System.IO.print - A.hs:5:8-11 GHC.Base.++ - A.hs:5:18-19 etc. (The line/column numbers are made up.) * I do not want to output e.g. 'x' as it's not a top-level identifier (the code I've included below gets this wrong). * I want to output whether the SrcSpan corresponds to a use site or definition site of the Name. For example: 'Foo' is a definition site while 'print' is a use site. I started writing a manual traversal of the RenamedSource AST (as I want qualified names) but I thought I check if I'm going about this right before I spend all the time required to write the traversal for the whole AST. Here's the code I have so far, am I on the right track? ---------- -- | Collects all qualified names that are referred to in a module, -- that are either defineds at the top-level in that module or that -- are imported from some other module. module Main where import Bag import DynFlags ( defaultDynFlags ) import GHC import GHC.Paths ( libdir ) import Outputable import System.Environment -- | Is the 'Name' defined here? data Origin = Local | External type Use = (Name, Origin, SrcSpan) local :: Name -> Use local name = (name, Local, nameSrcSpan name) external :: Name -> SrcSpan -> Use external name loc = (name, External, loc) showName :: (Name, Origin, SrcSpan) -> String showName (name, org, loc) = showSDoc (ppr name) ++ "," ++ showSDoc (ppr loc) ++ "," ++ showOrg org where showOrg Local = "1" showOrg External = "0" main :: IO () main = do [targetFile] <- getArgs res <- defaultErrorHandler defaultDynFlags $ runGhc (Just libdir) $ do dflags <- getSessionDynFlags _ <- setSessionDynFlags dflags target <- guessTarget targetFile Nothing setTargets [target] _ <- load LoadAllTargets modSum <- getModSummary $ mkModuleName "B" p <- parseModule modSum t <- typecheckModule p let Just (r, _, _, _) = tm_renamed_source t return r putStrLn $ showSDoc $ ppr res putStrLn "" putStr $ unlines $ map showName (collectHsGroup res) ------------------------------------------------------------------------ -- AST traversal -- | Collect all external qualified names in the module. collectHsGroup :: HsGroup Name -> [Use] collectHsGroup = collectHsValBindsLR . hs_valds collectHsValBindsLR :: HsValBindsLR Name Name -> [Use] collectHsValBindsLR (ValBindsOut xs _) = concatMap collectHsBindNames . map unLoc . concatMap bagToList . map snd $ xs collectHsValBindsLR (ValBindsIn binds _) = concatMap (collectHsBindNames . unLoc) (bagToList binds) collectHsBindNames :: HsBindLR Name Name -> [Use] collectHsBindNames fb@(FunBind { fun_id = L _ f }) = [local f] ++ collectMatchGroupNames (fun_matches fb) collectHsBindNames _ = [] collectMatchGroupNames :: MatchGroup Name -> [Use] collectMatchGroupNames (MatchGroup matches _) = concat [collectGRHSsNames x | Match _ _ x <- map unLoc matches] collectGRHSsNames :: GRHSs Name -> [Use] collectGRHSsNames (GRHSs xs _) = concatMap (collectGRHSNames . unLoc) xs collectGRHSNames :: GRHS Name -> [Use] collectGRHSNames (GRHS _stmts exprs) = collectHsExprNames exprs -- For less typing collectHsExprNames :: LHsExpr Name -> [Use] collectHsExprNames = collect collect :: LHsExpr Name -> [Use] collect (L loc expr) = go expr where go (HsVar name) | isExternalName name = [external name loc] | otherwise = [] go (HsIPVar _) = [] go (HsOverLit _) = [] go (HsLit _) = [] go (HsLam mg) = collectMatchGroupNames mg go (HsApp e1 e2) = collect e1 ++ collect e2 go (OpApp e1 e2 _ e3) = collect e1 ++ collect e2 ++ collect e3 go (NegApp e1 _) = collect e1 -- ++ collect e2 -- ??? go (HsPar e) = collect e go (SectionL e1 e2) = collect e1 ++ collect e2 go (SectionR e1 e2) = collect e1 ++ collect e2 go (ExplicitTuple xs _) = concat [ collect x | Present x <- xs] go (HsCase e mg) = collect e ++ collectMatchGroupNames mg go (HsIf e1 e2 e3) = collect e1 ++ collect e2 ++ collect e3 go (HsLet binds e) = collectHsLocalBindsLR binds ++ collect e -- go (HsDo (HsStmtContext Name) [LStmt id] (LHsExpr id) PostTcType) = -- go (ExplicitList PostTcType [LHsExpr id]) = -- go (ExplicitPArr PostTcType [LHsExpr id]) = -- go (RecordCon (Located id) PostTcExpr (HsRecordBinds id)) = -- go (RecordUpd (LHsExpr id) (HsRecordBinds id) [DataCon] [PostTcType] [PostTcType]) = -- go (ExprWithTySig (LHsExpr id) (LHsType id)) = -- go (ExprWithTySigOut (LHsExpr id) (LHsType Name)) = -- go (ArithSeq PostTcExpr (ArithSeqInfo id)) = -- go (PArrSeq PostTcExpr (ArithSeqInfo id)) = -- go (HsSCC FastString (LHsExpr id)) = -- go (HsCoreAnn FastString (LHsExpr id)) = -- go (HsBracket (HsBracket id)) = -- go (HsBracketOut (HsBracket Name) [PendingSplice]) = -- go (HsSpliceE (HsSplice id)) = -- go (HsQuasiQuoteE (HsQuasiQuote id)) = -- go (HsProc (LPat id) (LHsCmdTop id)) = -- go (HsArrApp (LHsExpr id) (LHsExpr id) PostTcType HsArrAppType Bool) = -- go (HsArrForm (LHsExpr id) (Maybe Fixity) [LHsCmdTop id]) = -- go (HsTick Int [id] (LHsExpr id)) = -- go (HsBinTick Int Int (LHsExpr id)) = -- go (HsTickPragma (FastString, (Int, Int), (Int, Int)) (LHsExpr id)) = -- go (EWildPat) = -- go (EAsPat (Located id) (LHsExpr id)) = -- go (EViewPat (LHsExpr id) (LHsExpr id)) = -- go (ELazyPat (LHsExpr id)) = -- go (HsType (LHsType id)) = -- go (HsWrap HsWrapper (HsExpr id)) = go _ = [] collectHsLocalBindsLR :: HsLocalBindsLR Name Name -> [Use] collectHsLocalBindsLR (HsValBinds x) = collectHsValBindsLR x collectHsLocalBindsLR _ = []

4 8

Template Haskell and linking
by Jonas Almström Duregård 20 Sep '10

20 Sep '10

Hi GHC users! Does anyone know if an import that is only used by Template Haskell (i.e. not in "actual" code) is reflected in the produced executable? Example: import LargeModule(thFunction) $thfunction ... Is LargeModule linked in the executable file? (Assume thFunction is not referenced by the code it generates or elsewhere in the example) Best regards Jonas

2 1

Re: Cost Centre MAIN
by Christian Höner zu Siederdissen 20 Sep '10

20 Sep '10

Yeah that was a bit lazy by me (or bull, if you want...;-); looks like you have runStmLogger on the top-level of your Main module ("MAIN") as well. What functions are defined there? And what vars? Gruss, Christian * Stefan Wehr <stefan.wehr(a)gmail.com> [19.09.2010 20:28]: > 2010/9/19 Christian Höner zu Siederdissen <choener(a)tbi.univie.ac.at>: > > You should either try "-auto-all" while compiling to get cost centres > > for every top-level function or insert cost centres by hand with "SCC": > > > > http://haskell.cs.yale.edu/ghc/docs/6.12.2/html/users_guide/profiling.html > > Well, I did compile with -auto-all and -caf-all > > > Otherwise, MAIN is "main = ..." > > Really? Here are the first couple of lines of the detailed profiling > report I get: > > COST CENTRE MODULE > no. entries %time %alloc %time %alloc > > MAIN MAIN > 1 0 47.5 0.2 100.0 100.0 > runStmLogger MobileGateway.Util.Logging > 5223 0 0.0 0.0 0.0 0.0 > main Main > 5166 1 0.0 0.0 0.6 0.2 > main MobileGateway.LubSyncClient > 5167 5 0.0 0.0 0.6 0.2 > getUrl MobileGateway.Util.Config > 5274 1 0.0 0.0 0.0 0.0 > > Doesn't look like MAIN is the same as "main = ...". > > -- Stefan

1 0

Re: Cost Centre MAIN
by Christian Höner zu Siederdissen 19 Sep '10

19 Sep '10

You should either try "-auto-all" while compiling to get cost centres for every top-level function or insert cost centres by hand with "SCC": http://haskell.cs.yale.edu/ghc/docs/6.12.2/html/users_guide/profiling.html Otherwise, MAIN is "main = ..." Gruss, Christian * Stefan Wehr <stefan.wehr(a)gmail.com> [19.09.2010 14:11]: > Hi all, > > I'm profiling a Haskell program and now getting that the program > spents 56% of its individual time in the cost centre MAIN (note the > capital letters). > > I searched the documentation for this cost centre, with no success. In > all profiling results so far, the individual time spent in MAIN was > always 0%. > > Any hints? > > -- Stefan > _______________________________________________ > Glasgow-haskell-users mailing list > Glasgow-haskell-users(a)haskell.org > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

1 0

Allocation break-down by type
by Bruno Martínez Aguerre 19 Sep '10

19 Sep '10

Hi list, To understand better whether ghc is taking advantage of my UNPACK pragmas, I want to profile my program and sort the allocations by type. I couldn't find an option to do this. -p is only by cost-center, and -hy only works for live data, and I want all allocations. Am I missing some option? Any tips to better debug UNPACK, apart from inspecting core? Thanks, Bruno

3 2

Interruptible GHC
by Edward Z. Yang 18 Sep '10

18 Sep '10

So I did a writeup of what I thought might be the next direction to go with the interruptible patch: http://blog.ezyang.com/2010/09/towards-platform-agnostic-interruptibility/ The really interesting bit (which I didn't cover) is what information to give to the user functioSo I did a writeup of what I thought might be the next direction to go with the interruptible patch: http://blog.ezyang.com/2010/09/towards-platform-agnostic-interruptibility/ The really interesting bit (which I didn't cover) is what information to give to the user function. It needs to be somewhat under the hood: if someone wants to pthread_cancel they need to know what the pthreads ID is; if they want to transmit a Windows event they need some way to get a hold of the event handle associated with the foreign call. Edward

2 7