Can someone help? The program below works fine with small files but when I try to use it on the one I need to (about 3 million lines of data) it produces no output. The hard disk is hammered - I assume this is the run time system paging. My suspicion is that the program is trying to read in the whole file before processing it. Is this correct? If so, how do I make the program lazy so that it processes a line at time?

I was about to apply GHood to your program to see whether such a tool could help to find the problem, so I started to cut down your code. However, in the simplified version of the program, one can see what is going on without any graphical tool.. (nevertheless, observation of the simplified code with GHood confirms your suspicion immediately, and it points out the spine of the list as the problem, too, so the tool is useful!-) (*) In effect, your anonymize comes down to a "mapM" over a list of actions applied to input lines, and all of the resulting IO-actions are placed before the single "hPutStr". So, even if the results of the individual actions in the list may not be needed until later, the whole spine of the list of lines has to be traversed before "hPutStr" can be executed, meaning that all input is read before any output is produced (and thus before any computation results are requested, blowing up memory usage). For the problem at hand, you could simply output each line as it is processed instead of just returning it into a list for later use (see variant _1 below). If you would want to keep both the modular program structure and the explicit line-by-line IO-style, you would need to interleave the input and output commands somehow (perhaps similar to variant _2 below?). Hth, Claus (*) Please note that our web-server is being upgraded today.. (web-pages and GHood download will not be available until tomorrow, hence no URL here :-( PS The simplified code (+ variations) with observations: module Main(main) where import Observe import IO(openFile, hPutStr, IOMode(ReadMode,WriteMode,AppendMode)) filename = "ldif1.txt" fileout = "ldif.out" readAndWriteAttrVals = do h <- openFile fileout WriteMode s <- readFile filename let action l = return (':':l) a <- mapM action (observe "input" (lines s)) hPutStr h (unlines (observe "output" a)) main = runO readAndWriteAttrVals readAndWriteAttrVals_1 = do h <- openFile fileout WriteMode s <- readFile filename let action_and_output l = hPutStr h (':':l) mapM_ (observe "output" action_and_output) (observe "input" (lines s)) main_1 = runO readAndWriteAttrVals_1 readAndWriteAttrVals_2 = do h <- openFile fileout WriteMode s <- readFile filename let { action l = return (':':l) ; as = map action (observe "input" (lines s)) ; os = repeat (hPutStr h) } mapM id (observe "output" (zipWith (>>=) as os)) main_2 = runO readAndWriteAttrVals_2