19 Sep
2008
19 Sep
'08
3:17 p.m.
Hello Don, Friday, September 19, 2008, 9:12:43 PM, you wrote:
It is possible to implement a map reduce version that can handle gzipped log files?
Using the zlib binding on hackage.haskell.org, you can stream multiple zlib decompression threads with lazy bytestrings, and combine the results.
for gzip decompression you need to decompress all the previous data, so decompression of single file cannot be multithreaded -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com