[Haskell-beginners] problem with lazy IO

9 Nov 2011


      I have the following code, which reads several thousand text files and
creates some Map data. On Windows XP, it crapped out with the error
"resource exhausted: too many open files." So I'm sure I need to make it
stricter. I've never tried to do that before, so I'm not sure where the
strictness is necessary. Maybe all I need is to switch to strict
ByteStrings?

-- Read a bunch of text files. The files are organized in pairs, where the
file names in each pair are
-- "a<number>.txt" and "f<fnumber>.txt". Example: "a00001.txt" and
"f00001.txt".
-- Each file is a list of float values specified in a certain text format
(which happens to be
--  an output format of the audio-related program Csound)
--
-- Input:
--   [(String,String)]  : a list of pairs of file names
--   String                : directory in which the files exist
readPvsFiles :: [(String,String)] -> String -> IO (Map Int (Map Float
Float))
readPvsFiles filenames dir = do
  -- contents :: [(Int,Map Float Float)]
  --      where Int is the number in the file name.
  --      and Map Float Float is a map of numbers in the
  --      in the first file mapped to numbers in the second file
  contents <- mapM (oneFilePair dir) filenames
  return $ M.fromList contents

-- Read one file pair.
--
-- Input:
--   String  : directory of the files
--   (String,String) : the two file names in the pair
--   (Int,Map Float Float) :
  --      Int is the number in the file name.
  --      and Map Float Float is a map of numbers in the
  --      in the first file mapped to numbers in the second file
oneFilePair :: String -> (String,String) -> IO (Int,Map Float Float)
oneFilePair dir (ffile,afile) = do
  -- Read the float values from each file (which is a list of floats in
  -- a text format.
  fvalues <- readTableValues (dir ++ "/" ++ ffile)
  avalues <- readTableValues (dir ++ "/" ++ afile)
  -- t is the number in the filename.
  let t = read . take 6 . drop 1 $ ffile
  return (t, M.fromList $ zip fvalues avalues)

-- Open the file via readFile, and parse all the text values in it.
readTableValues :: String -> IO [Float]
readTableValues s = do
  b <- readFile s
  let bs = lines b
      lenLine = head . drop 1 $ bs
      n = read (drop 6 lenLine) :: Int
      -- valueLines :: [String]. This is a list of the lines
      --   in the file that have the float values. One float
      --   value in text format per line.
      valueLines = take n . drop 25 $ bs
  return $ map read valueLines

[Haskell-beginners] problem with lazy IO

Dennis Raddle