Exactly. Someone on the list gave me this example awhile back for
reading CSV files. I can process a gigabyte (simple unpack and print
to dev/null for IO testing purposes) in about two and a half seconds
using this code.
import Data.ByteString.Lazy.Char8 as C
-- | Read a datafile and turn it into lists of columns
readDatafileAndTranspose name = do
sheet <- (transpose . map (C.split '\t') . C.lines) `fmap` C.readFile name
return $ foldl' go M.empty sheet
where go m (x:xs) = M.insert (C.unpack x) xs m
2008/5/29 Adrian Neumann
Isn't fast IO what ByteStrings where invented for?
Adrian
Tom Harper schrieb:
I'm trying to implement some file I/O where I can read in a file to an array, but do so without having to know how much space I will need. (Before you suggest it, lists are too slow/space consuming.) I was thinking that one possible solution is to use a strategy used in dynamic arrays, where everytime you run out of space you double the capacity of the array. Trouble is, most arrays in Haskell would require you to traverse the entire array and copy each individual cell, which would make it worthless.
Are there any low-level array types (MutableByteArray#, for example) that support a memcpy-like function where I could copy the entire array into the new one instead of copying it one value at a time? Is there another solution that I'm missing?
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- I try to take things like a crow; war and chaos don't always ruin a picnic, they just mean you have to be careful what you swallow. -- Jessica Edwards