Re: [Haskell-cafe] How can I improve the pipes's performance with a huge file?

On Fri, Nov 14, 2014 at 09:14:17PM +0800, zhangjun.julian wrote:

Dear Tom

I change Map to Strict，it be little fast when test with 18M rows, but it hanged again with 40M rows.

Do you have any other advice?

Dear Zhangjun Julian, Perhaps too much of the output string is being kept around when it is printed. I would try mapM_ (\((x,y), i) -> hPutStrLn writeHadle (show x ++ "," ++ show y ++ "," ++ show i)) (DM.toList rt) instead of hPutStr writeHadle $ mapToString rt Apart from that, I don't have any other ideas. Can you determine whether the large memory usage comes from the Pipe or from the printing of the result? Tom