- ghc-devs - Haskell.org

New space leak?
by Simon Marlow 01 Oct '13

01 Oct '13

My validate has a few perf failures (below). Now I know that sampling heap size is inaccurate, but the evidence here is pretty strong. Also validate fell over completely in my 2GB VM when Haddock ran out of memory, I had to give it more memory to complete. Something has got worse recently - this wasn't happening a couple of weeks ago. Does anyone have any idea what might have caused this? Going back in time is difficult. Are these failures happening for other people? Let me suggest something that might help with these perf tests: running the program with +RTS -h -i0.01 will give a more accurate indication of peak residency, by running major GCs much more often. Also you get a heap profile. Cheers, Simon =====> haddock.base(normal) 2474 of 3806 [0, 1, 0] max_bytes_used value is too high: Expected max_bytes_used: 96022312 +/-10% Lower bound max_bytes_used: 86420080 Upper bound max_bytes_used: 105624544 Actual max_bytes_used: 113537216 peak_megabytes_allocated value is too high: Expected peak_megabytes_allocated: 274 +/-10% Lower bound peak_megabytes_allocated: 246 Upper bound peak_megabytes_allocated: 302 Actual peak_megabytes_allocated: 323 *** unexpected failure for haddock.base(normal) =====> haddock.Cabal(normal) 2475 of 3806 [0, 2, 0] =====> haddock.compiler(normal) 2476 of 3806 [0, 2, 0] peak_megabytes_allocated value is too high: Expected peak_megabytes_allocated: 1250 +/-10% Lower bound peak_megabytes_allocated: 1125 Upper bound peak_megabytes_allocated: 1375 Actual peak_megabytes_allocated: 1397 bytes allocated value is too high: Expected bytes allocated: 25990254632 +/-10% Lower bound bytes allocated: 23391229168 Upper bound bytes allocated: 28589280096 Actual bytes allocated: 28680445952 *** unexpected failure for haddock.compiler(normal) =====> T3064(normal) 2480 of 3806 [0, 3, 0] cd ./perf/compiler && '/home/smarlow/ghc/bindisttest/install dir/bin/ghc' -fforce-recomp -dno-debug-output -no-user-package-db -rtsopts -fno-ghci-history -c T3064.hs +RTS -V0 -tT3064.comp.stats --machine-readable -RTS >T3064.comp.stderr 2>&1 max_bytes_used value is too high: Expected max_bytes_used: 12000480 +/-20% Lower bound max_bytes_used: 9600384 Upper bound max_bytes_used: 14400576 Actual max_bytes_used: 15258696 peak_megabytes_allocated value is too high: Expected peak_megabytes_allocated: 30 +/-20% Lower bound peak_megabytes_allocated: 24 Upper bound peak_megabytes_allocated: 36 Actual peak_megabytes_allocated: 37 *** unexpected failure for T3064(normal)

4 4

Is evacuate for StgMutArrPtrs and StgArrPtrs expensive to GC?
by Johan Tibell 01 Oct '13

01 Oct '13

Hi, When I benchmark Data.HashMap.insert from unordered-containers (inserting the keys [0..10000]) the runtime is dominated by GC: $ cat Test.hs module Main where import Control.DeepSeq import Control.Exception import Control.Monad import qualified Data.HashMap.Strict as HM import Data.List (foldl') main = do let ks = [0..10000] :: [Int] evaluate (rnf ks) forM_ ([0..1000] :: [Int]) $ \ x -> do evaluate $ HM.null $ foldl' (\ m k -> HM.insert k x m) HM.empty ks $ perf record -g ./Test +RTS -s 6,187,678,112 bytes allocated in the heap 3,309,887,128 bytes copied during GC 1,299,200 bytes maximum residency (1002 sample(s)) 118,816 bytes maximum slop 5 MB total memory in use (0 MB lost due to fragmentation) Tot time (elapsed) Avg pause Max pause Gen 0 11089 colls, 0 par 1.31s 1.30s 0.0001s 0.0005s Gen 1 1002 colls, 0 par 0.49s 0.51s 0.0005s 0.0022s INIT time 0.00s ( 0.00s elapsed) MUT time 1.02s ( 1.03s elapsed) GC time 1.80s ( 1.80s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 2.82s ( 2.84s elapsed) %GC time 63.7% (63.5% elapsed) Alloc rate 6,042,264,963 bytes per MUT second Productivity 36.3% of total user, 36.1% of total elapsed $ perf report 41.46% Test Test [.] evacuate 15.47% Test Test [.] scavenge_block 11.04% Test Test [.] s3cN_info 8.74% Test Test [.] s3aZ_info 3.59% Test Test [.] 0x7ff5 2.83% Test Test [.] scavenge_mut_arr_ptrs 2.69% Test libc-2.15.so [.] 0x147fd9 2.51% Test Test [.] allocate 2.00% Test Test [.] s3oo_info 0.91% Test Test [.] todo_block_full 0.87% Test Test [.] hs_popcnt64 0.80% Test Test [.] s3en_info 0.62% Test Test [.] s3el_info Is GC:ing StgMutArrPtrs and StgArrPtrs, which I create a lot of, more expensive than GC:ing normal heap objects (i.e. for standard data types)? -- Johan

4 6

Testsuite failures
by Simon Peyton-Jones 01 Oct '13

01 Oct '13

I've pushed some error-message-wibbling changes but I have to go before I've complete the testsuite wibbles. So don't worry about any testsuite error mis-matches for the next 12 hrs. Sorry... I got distracted Simon Microsoft Research Limited (company number 03369488) is registered in England and Wales Registered office is at 21 Station Road, Cambridge, CB1 2FB

1 0

Re: [commit: ghc] master: Improve pretty-printing of types (66c5ddb)
by Johan Tibell 01 Oct '13

01 Oct '13

This commit broke the build: ghc/InteractiveUI.hs:1986:18: Not in scope: data constructor 'Opt_PrintKindArgs' The commit is missing the author name and email so I don't know who to contact directly. On Tue, Oct 1, 2013 at 8:55 AM, <git(a)git.haskell.org> wrote: > Repository : ssh://git@git.haskell.org/ghc > > On branch : master > Link : http://ghc.haskell.org/trac/ghc/changeset/66c5ddba449e78a174e989ea35783eb6c… > >>--------------------------------------------------------------- > > commit 66c5ddba449e78a174e989ea35783eb6c793e483 > Author: unknown <simonpj(a)MSRC-4971295.europe.corp.microsoft.com> > Date: Tue Oct 1 16:34:00 2013 +0100 > > Improve pretty-printing of types > > * The main change is to suppress printing (in types) of > kind for-alls > kind applications > The new flag -fprint-explicit-kinds prints them as before > (by analogy with the existing -fprint-explicit-foralls) > > * I also took advantage of the fact that SDoc now has access > to DynFlags, to tidy up the way in which explicit for-alls > are printed. Instead of passing a boolean flag around, we > now simply consult the DynFlags. Much neater. > > I still need to add documentation for the flag > > >>--------------------------------------------------------------- > > 66c5ddba449e78a174e989ea35783eb6c793e483 > compiler/ghci/Debugger.hs | 5 +- > compiler/main/DynFlags.hs | 2 + > compiler/main/PprTyThing.hs | 156 ++++++++++++++++++++++--------------------- > compiler/types/TypeRep.lhs | 126 ++++++++++++++++++++++------------ > ghc/InteractiveUI.hs | 52 ++++++--------- > 5 files changed, 189 insertions(+), 152 deletions(-) > > Diff suppressed because of size. To see it, use: > > git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc 66c5ddba449e78a174e989ea35783eb6c793e483 > _______________________________________________ > ghc-commits mailing list > ghc-commits(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-commits

1 1

Initial state of gc_thread.mut_spin
by Patrick Palka 01 Oct '13

01 Oct '13

Hi everyone, In new_gc_thread(), why is gc_thread.mut_spin initialized to the 'released' state instead of the 'acquired' state (as with gc_spin)? I think this is wrong for two reasons: 1. During the first GC (but not during subsequent GCs), gcWorkerThread() releases mut_spin but mut_spin is not 'acquired' at this point. 2. The initial state of mut_spin is inconsistent with the final state of mut_spin at the end of a GC (it is in the 'acquired' state then). So it seems to me that mut_spin should be initialized to the 'acquired' state, like gc_spin is. Doing so would address these two points. Does this change make sense?

1 0

FW: [Haskell-cafe] The RTSOPTS "-qm" flag's impact on runtime
by Simon Peyton-Jones 01 Oct '13

01 Oct '13

Simon: did you see this? A factor of 50 in runtime seems pretty significant! Simon -----Original Message----- From: Haskell-Cafe [mailto:haskell-cafe-bounces@haskell.org] On Behalf Of Iustin Pop Sent: 30 September 2013 23:14 To: Haskell Cafe Subject: [Haskell-cafe] The RTSOPTS "-qm" flag's impact on runtime Hi all, I found an interesting case where the rtsopts -qm flag makes a significant difference in runtime (~50x). This is using GHC 7.6.3, llvm 3.4, program compiled with "-threaded -O2 -fllvm" and a couple of language extension. Source is at http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=chameneosre…, on the language shootout benchmarks. Running the code without -N results (on my computer) in around 4 seconds of runtime: $ time ./orig 6000000 … real 0m3.919s user 0m3.903s sys 0m0.010s This is reasonably consistent. Running -N4 (this is an 8-core machine) results in the surprising: $ time ./orig 6000000 +RTS -N4 … real 1m15.154s user 1m38.790s sys 2m7.947s The cores are all used very erratically (continuously changing 5%-20%-40%) and the overall cpu usage is ~27-28%. Note the surprising 2m7s of sys usage, which means the kernel is involved a lot… Note that removing the explicit forkOn and running with -N4 results in somewhat worse performance: real 2m6.548s user 2m13.470s sys 2m3.043s So in that sense the forkOn itself is not at fault. What I have found is that -qm is here a life saver: $ time ./orig 6000000 +RTS -N4 -qm real 0m2.773s user 0m5.610s sys 0m0.123s Adding -qa doesn't make a big difference. To summarise more runs (in terms of cpu used, user+sys): with forkOn: - -N4: 228s - -N4 -qa: 110s - -N4 -qm: 6s - -N4 -qm -qa: 6s without forkOn: - -N4: 253s - -N4 -qa: 252s - -N4 -qm: 5s - -N4 -qm -qa: 5s (Note that "without forkOn" is a bit slower in term of wall-clock, as the "with forkOn" version distributes the work a bit better, even if it uses overall a tiny bit more CPU.) So the question is, what does -qm actually do that it affects this benchmark so much (~50x)? (The docs are not very clear on it) And furthermore, could there be an heuristic inside the runtime such that automatic thread migration is suspended if threads are "over-migrated" (which is what I suppose happens here)? thanks for any explanations, iustin _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe(a)haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

1 0

I've moved the primitive package to github
by Bryan O'Sullivan 30 Sep '13

30 Sep '13

Roman seems to have gone silent, and primitive being broken with GHC HEAD means that there's a ton of packages we can't build right now. Accordingly, I've imported the old darcs repo into git and published it on github. https://github.com/haskell/primitive Austin, please plan to fix the GHC build to get its copy of primitive from here, once we have it working with HEAD.

5 14

Re: [commit: ghc] master: Fix AMP warnings for explicit Prelude imports (#8004) (b9127f4)
by Patrick Palka 30 Sep '13

30 Sep '13

On Sun, Sep 29, 2013 at 9:10 PM, <git(a)git.haskell.org> wrote: > + isPrelude :: ImportDecl Name -> Bool > + isPrelude = (== "Prelude") . moduleNameString . unLoc . ideclName Can't you just do unLoc (ideclName imp) == pRELUDE_NAME here?

2 1

Re: [commit: packages/base] master: Expose performMinorGC (#8257) (c1491e6)
by Christiaan Baaij 30 Sep '13

30 Sep '13

Shouldn't the last line of the patch be?: > +foreign import ccall "performGC" performMinorGC :: IO () the rts doesn't export a function called "performMinorGC". -- Christiaan On Sep 30, 2013, at 12:24 AM, git(a)git.haskell.org wrote: > Repository : ssh://git@git.haskell.org/base > > On branch : master > Link : http://ghc.haskell.org/trac/ghc/changeset/c1491e683d3edc31fd20bd3b3021d3a62… > >> --------------------------------------------------------------- > > commit c1491e683d3edc31fd20bd3b3021d3a620aa0184 > Author: Austin Seipp <austin(a)well-typed.com> > Date: Sun Sep 29 09:42:06 2013 -0500 > > Expose performMinorGC (#8257) > > Signed-off-by: Austin Seipp <austin(a)well-typed.com> > > >> --------------------------------------------------------------- > > c1491e683d3edc31fd20bd3b3021d3a620aa0184 > System/Mem.hs | 20 ++++++++++++++------ > 1 file changed, 14 insertions(+), 6 deletions(-) > > diff --git a/System/Mem.hs b/System/Mem.hs > index f162a75..2ad75ce 100644 > --- a/System/Mem.hs > +++ b/System/Mem.hs > @@ -14,11 +14,19 @@ > -- > ----------------------------------------------------------------------------- > > -module System.Mem ( > - performGC > - ) where > - > +module System.Mem > + ( performGC > + , performMajorGC > + , performMinorGC > + ) where > import Prelude > > --- | Triggers an immediate garbage collection > -foreign import ccall {-safe-} "performMajorGC" performGC :: IO () > +-- | Triggers an immediate garbage collection. > +performGC :: IO () > +performGC = performMajorGC > + > +-- | Triggers an immediate garbage collection. > +foreign import ccall "performMajorGC" performMajorGC :: IO () > + > +-- | Triggers an immediate minor garbage collection. > +foreign import ccall "performMinorGC" performMinorGC :: IO () > > _______________________________________________ > ghc-commits mailing list > ghc-commits(a)haskell.org > http://www.haskell.org/mailman/listinfo/ghc-commits

1 0

Re: [GHC] #8173: GHC uses nub
by Kim-Ee Yeoh 29 Sep '13

29 Sep '13

On Sun, Sep 29, 2013 at 2:56 PM, GHC <ghc-devs(a)haskell.org> wrote: > According to the results, sadly the 5% performance gain in compile time > was complete bogus. Sorry. =( > Does anyone have any insight into why? How can a future benchmark avoid such bogusity? The only -- I might add, implicit -- rec is to run things twice. -- Kim-Ee

1 0