Just a few updates on this one:
I've upgraded to bytestring-0.9.0.5 from Darcs, no improvement.
Also this morning I tried using Data.HashMap with Bytestring's readInt
and HashMap's hashInt.. The result was a Stack space overflow :(
God damn it I don't want to have to go back to C++ but soon will have
no choice with time constraints :(
Dave,
On 10/03/2008, Dave Tapley
Hi all,
I've been plugging away at this all day and some discussion in #haskell has been fruitless. Perhaps you have the inspiration to see what's happening!
Concerning this minimal example: http://hpaste.org/6268
It works as required, loading K/V pairs into a Data.Map, the concern is the amount of memory used. Pausing (using a getLine) after the readFile one can see (through 'ps v') that the whole file 'f' is loaded in to memory. However once the Map is created the memory footprint swells to around five times the size. Surely this can't be just overhead from Map? My reasoning was that by using ByteString and getting the whole file into memory a small and linear increase would be seen for Map overhead..
I have tried using both Data.ByteString.Char8 and Data.ByteString.Lazy.Char8 with negligible difference. For a hoot I tried it with String and yes, it's ridiculous :)
Cheers,
Dave