you may want to try out using the Strict variant of IntMap
http://hackage.haskell.org/package/containers-0.5.3.1/docs/Data-IntMap-Stric...
I've definitely had some scary space usage explosions with the lazy one on
occasions! (one embarrassing instance nearly two years ago was a 40gb space
blowup that turned into 100mb usage once i switched to a strict map)
On Thu, Dec 5, 2013 at 9:00 PM, Carter Schonwald wrote: Hey Uri,
could you explain what those numbers represent? many of use the profiling
tools, but we may not remember what columns / numbers mean what. What
number seems wrong and what does the number mean? Also its very hard to
help debug if you don't link to the code If you're hitting space leak problems, one culprit could be your data
structure or the operations acting on it is too lazy! Have you tried making
it stricter? cheers
-Carter On Thu, Dec 5, 2013 at 8:13 PM, Uri Braun Hi Café, I'm attempting to profile my rather large (~18K LOC) Haskell program that
primarily manipulates graph structures based on Data.IntMap. The program
is chewing up _lots_ of memory (approx. 4G for 2.5K nodes & 2.5K edges)and I have profiled it in an attempt to figure out why. It
appears that the .prof and .hp (which I visualize with hp2ps -c) files
do not agree on the source of the problem. The hp results are located at:
*http://www.eecs.harvard.edu/~uribraun/sybil.ps*http://www.eecs.harvard.edu/~uribraun/sybil.psand the top of the .prof file looks like this: COST CENTRE MODULE %time %alloc compare Policy.Edge 21.7 50.0 eqPartition.edges Policy.EdgeQuad 6.5 3.7 toConstructedSet Types.TaggedIntMultimap 6.1 3.6 elems Types.TaggedIntMap 6.0 6.8 elems Types.TaggedIntSet 5.7 3.8 toPairs Types.TaggedIntMultimap 3.9 5.2 assocs Types.TaggedIntMap 3.2 4.1 eqIsValid Policy.EdgeQuad 2.9 0.5 rIsValid Policy.Relations 2.2 0.2 assocs.\ Types.TaggedIntMap 1.8 0.0 keysSet Types.TaggedIntMap 1.5 1.7 mergeIdenticalNodesT.exactMap Minimize.MergeIdentical 1.4 1.2 fromList Types.TaggedIntSet 1.3 1.6 tePartitionSrc Policy.TaggedEdge 1.2 0.9 null Types.TaggedIntSet 1.2 0.0 scc.comps Policy.Scc 1.2 0.9 tePartitionTgt Policy.TaggedEdge 1.1 0.8 member Types.TaggedIntSet 1.1 0.8 alter Types.TaggedIntMap 1.0 1.1 Also, the line in the .prof that corresponds to the anormUnion.name
(what the hprof image claims is taking up space) looks like (some spaces
removed): anormUnion.name Anormalize.Anormalize 2914 60
0.0 0.0 0.0 0.0 My main questions at this point are: 1. What are the chances the hprof image is just wrong? 2. What would you suggest I try next in debugging my space issue? This is my first big Haskell program, please do inform me of any big war
tools I don’t know about. Many thanks in advance! +Uri _______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe