Daniel Fischer
Maybe I've misused the word segfault.
I think so. A segfault is the operating-system complaining about an illegal memory access. If you get them from Haskell, it is likely a bug in the compiler or run-time system (or you were using unsafeAt, or FFI).
The programme consumed more and more memory (according to top), kswapd started to have a higher CPU-percentage than my programme, programme died, system yelling 'Speicherzugriffsfehler', top displays 'kswapd<defunct>'.
I find that swapping the GHC heap is not productive, so I tend to limit memory to available RAM. You can do that either by limiting available memory to process in the shell (ulimit or limit), or by specifying RTS options to the haskell program (typically +RTS -Mxxx, where xxx is 80-90% of physical RAM).
From the GHC docs (6.4.2):
-Msize [Default: unlimited] Set the maximum heap size to size bytes. The I thought the default was set according to limits? heap normally grows and shrinks according to the memory requirements of the program. The only reason for having this option is to stop the heap growing without bound and filling up all the available swap space, which at the least will result in the program being summarily killed by the operating system. In my experience, a program which thrashes severely without heap limits can run fine if you just limit the heap. So it's not as much an issue of 'filling up swap' vs. 'killed by the OS', but 'takes forever, making the system unresponsive in the process' vs. 'tries hard to complete in a timely fashion, or halts with an error'. I much prefer the latter.
However the problem might be too little lazyness, because if I explicitly read the file line by line, memory usage remains low enough -- but ByteString is *much* faster anyway.
Right. You should still try to consume the file lazily, and make sure the data can get discarded and GC'ed when you have no further use for it. But a String is something like 8 or 12 bytes per character, a ByteString gets you down to 1. -k -- If I haven't seen further, it is by standing in the footprints of giants