Thanks everyone,

Dan, MapMI worked for me ... 

Regards,
Kashyap


On Mon, Mar 18, 2013 at 12:42 AM, Petr Pudlák <petr.mvd@gmail.com> wrote:
Hi Kashyap,

you could also use iteratees or conduits for a task like that. The beauty of such libraries is that they can ensure that a resource is always properly disposed of. See this simple example: https://gist.github.com/anonymous/5183107
It prints the first line of each file given as an argument. After each line is printed, the `fileConduit` pipe ensures that the handle is closed. It also makes the program nicely composable.

Best regards,
Petr



import Control.Monad
import Control.Monad.Trans.Class
import Control.Monad.IO.Class
import Data.Conduit
import Data.Conduit.List
import System.Environment
import System.IO
 
{- | Accept file paths on input, output opened file handle, and ensure that the
- handle is always closed after its downstream pipe finishes whatever work on it. -}
fileConduit :: MonadResource m => IOMode -> Conduit FilePath m Handle
fileConduit mode = awaitForever process
where
process file = bracketP (openFile file mode) closeWithMsg yield
closeWithMsg h = do
putStrLn "Closing file"
hClose h
 
{- | Print the first line from each handle on input. Don't care about the handle. -}
firstLine :: MonadIO m => Sink Handle m ()
firstLine = awaitForever (liftIO . (hGetLine >=> putStrLn))
 
main = do
args <- getArgs
runResourceT $ sourceList args =$= fileConduit ReadMode $$ firstLine



2013/3/17 C K Kashyap <ckkashyap@gmail.com>
Hi,

I am working on an automation that periodically fetches bug data from our bug tracking system and creates static HTML reports. Things worked fine when the bugs were in the order of 200 or so. Now I am trying to run it against 3000 bugs and suddenly I see things like - too  many open handles, out of memory etc ...

Here's the code snippet - http://hpaste.org/84197

It's a small snippet and I've put in the comments stating how I run into "out of file handles" or simply file not getting read due to lazy IO.

I realize that putting ($!) using a trial/error approach is going to be futile. I'd appreciate some pointers into the tools I could use to get some idea of which expressions are building up huge thunks.


Regards,
Kashyap

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe