RE: [Haskell-cafe] using Map rather than FiniteMap

On 25 January 2005 23:27, S. Alexander Jacobson wrote:
Oops. It pays to check your checking code before making posts like this.
After actually running the correct test, I am still getting semi-ridiculous space behavior (6k/pair)!
import qualified Map zipped =zip [1..] [1..100000]::[(Int,Int)] untup f (x,y) = f x y produce = foldr (untup Map.insert) Map.empty zipped fm = length $ Map.keys produce main = print $ fm
Has this profile:
example +RTS -p -K5M -RTS
total time = 5.10 secs (255 ticks @ 20 ms) total alloc = 612,534,236 bytes (excludes profiling overheads)
COST CENTRE MODULE %time %alloc balance Map 71.8 72.8 insert Map 12.2 10.8 size Map 9.0 9.7 bin Map 2.4 2.2 rotateR Map 1.6 0.3 zipped Main 0.8 1.9
Notice the 612Mb for storing a mapping from Int to Int!!
Notice that's 612Mb *allocation* to create the finite map and deconstruct it into a list. Not 612Mb of live heap to store the finite map. If you make sure everything is evaluated, and examine the heap profile I'm sure you'll find that the finite map is taking a reasonable amount of space (perhaps 20-30 bytes per node for storing integers, I guess). To get a rough idea of the total live heap without profiling, you can run the program with +RTS -sstderr and look at the memory "in use" figure. Cheers, Simon

Ah, ok. So I ran the code with 100000 items, 50000 items, and 25000 items and got total memory in use of 28Mb, 15Mb, and 8Mb respectively. That comes to 260-280 bytes per record which is still an order of magnitude higher than the 20-30 bytes per record we would expect. On the other hand, I found 10mb, 5mb, and 2.5mb maximum residency, but that is still ~100 bytes per record. Lastly, I tried "example +RTS -p -K5M -hc" and then looked at the resulting graph (attachment #2) and it shows a total of 1.6Mb heap allocated and if that data is correct, it does correspond roughly to our 20-30 bytes per record estimate. Regarding stack, I tried "example +RTS -p -K5M -xt -hc" and then ran hp2ps and looked at the resulting graph (attachment #1) SYSTEM appears to use 4mb of stack at the very beginning (presumably to create "zipped"), but I don't know why it would. Then this stack is consumed by the rest of the program. Can you provide some guidance on interpreting this data? -Alex- ______________________________________________________________ S. Alexander Jacobson tel:917-770-6565 http://alexjacobson.com On Wed, 26 Jan 2005, Simon Marlow wrote:
On 25 January 2005 23:27, S. Alexander Jacobson wrote:
Oops. It pays to check your checking code before making posts like this.
After actually running the correct test, I am still getting semi-ridiculous space behavior (6k/pair)!
import qualified Map zipped =zip [1..] [1..100000]::[(Int,Int)] untup f (x,y) = f x y produce = foldr (untup Map.insert) Map.empty zipped fm = length $ Map.keys produce main = print $ fm
Has this profile:
example +RTS -p -K5M -RTS
total time = 5.10 secs (255 ticks @ 20 ms) total alloc = 612,534,236 bytes (excludes profiling overheads)
COST CENTRE MODULE %time %alloc balance Map 71.8 72.8 insert Map 12.2 10.8 size Map 9.0 9.7 bin Map 2.4 2.2 rotateR Map 1.6 0.3 zipped Main 0.8 1.9
Notice the 612Mb for storing a mapping from Int to Int!!
Notice that's 612Mb *allocation* to create the finite map and deconstruct it into a list. Not 612Mb of live heap to store the finite map. If you make sure everything is evaluated, and examine the heap profile I'm sure you'll find that the finite map is taking a reasonable amount of space (perhaps 20-30 bytes per node for storing integers, I guess).
To get a rough idea of the total live heap without profiling, you can run the program with +RTS -sstderr and look at the memory "in use" figure.
Cheers, Simon
participants (2)
-
S. Alexander Jacobson
-
Simon Marlow