On Wed, Mar 24, 2010 at 11:23:05AM +0100, Heinrich Apfelmus wrote:
A quick fix would be to intersperse a function between sort and readRecords that imposes proper evaluation order: [...] While I expect the above to work, I hesitate to claim that it actually does. The reason is that I don't understand your readRecords function at a glance, unlike the processFile pipeline, it is not apparent how it works. Since finding and fixing space leaks is easier if your code is more obvious, I recommend to reformulate it as a function composition as well, for instance something like this: [...]
Thanks, that was enlightening! With your version of readRecords the program takes as much memory as with mine (it's still more than 10x what the C version takes, but perhaps that's the cost of using Haskell). But your version taught me a lot. Exactly the kind of feedback I needed :) Sami