On 01 August 2005 16:05, Cale Gibbard wrote:
Your problem is, as you pointed out, that readFile does lazy IO. Although the semantics of it can be a bit confusing at times, it is useful for applications where you have a large file which is being consumed, and you don't want to allocate all of the memory for it before doing any processing. Laziness lets you read the file as needed -- you may not even need it all, depending on what is being done. This is quite helpful when you have something like a couple gigabytes of data on disk which needs processing. It can however be confusing at first that it may not finish reading the file before the file is altered, or, in situations involving handles, before the handle is closed.
You can write a strict IO version of readFile in Haskell as follows:
import IO
hGetContents' hdl = do e <- hIsEOF hdl if e then return [] else do c <- hGetChar hdl cs <- hGetContents' hdl return (c:cs)
readFile' fn = do hdl <- openFile fn ReadMode xs <- hGetContents' hdl hClose hdl return xs
If you use readFile', it will ensure that the entire file is read and memory for the string is allocated before continuing. This ought to solve your problem.
Note that hGetContents' is likely to be slow, because it does repeated hGetChar. Also it is not tail-recursive, so it will run out of stack for a large file. It would be better to read the whole file into memory but lazilly convert it to a String, like so: import System.IO import System.IO.Unsafe import Foreign import Data.Char readFile' f = do h <- openFile f ReadMode s <- hFileSize h fp <- mallocForeignPtrBytes (fromIntegral s) len <- withForeignPtr fp $ \buf -> hGetBuf h buf (fromIntegral s) lazySlurp fp 0 len lazySlurp :: ForeignPtr Word8 -> Int -> Int -> IO String lazySlurp fp ix len | ix == len = return [] | otherwise = do c <- withForeignPtr fp $ \p -> peekElemOff p ix cs <- unsafeInterleaveIO (lazySlurp fp (ix+1) len) return (chr (fromIntegral c) : cs) Actually, I recommend always using this version unless you need to deal with really large files. Performance won't be quite as good as readFile, but I've attached a tweaked version that performs much better. Cheers, Simon