On Mon, Mar 22, 2010 at 03:06:08AM +0200, Sami Liedes wrote:
On Sun, Mar 21, 2010 at 08:28:23PM -0400, Patrick LeBoutillier wrote:
- Also, does memory usage improve if you remove the "sort"?
Yes. Then it only takes a few megabytes, regardless of how large the file is.
This led me to investigate further. The following hack (and nothing less crude I could invent) seems to significantly lessen the amount of memory needed by the program. Now it takes maybe 180 megabytes instead of 1.5 gigabytes. I guess the 180 megabytes consists of the output stream (string), which is now explicitly computed entirely before sorting and outputting it. It's still far from perfect. I think there definitely _should_ be a better way of doing this (forcing some computation to be strict so it takes less memory). Any ideas? And I especially despise the (sum (map (const 0) b)) thing, but it seems nothing less ugly does the trick. It seems seq doesn't force the entire computation to finish, only the next level. Is there some way to force the entire computation subtree to finish? That way I could say just seq' b (if seq' was the function that does that). (see my original posting for the entire code) ------------------------------------------------------------ -- Return the unsorted package list and the sorted package list processFile :: String -> (String,String) processFile str = let l = lines str pkgs = readRecords l in (unlines (map sprintPackage pkgs), unlines (map sprintPackage (sort pkgs))) -- Force the unsorted pkg list to be computed first (outputting it also -- does the trick, but mere (map id b) doesn't). This is really ugly. -- A mere "do putStrLn c" however eats 1.5 GB of memory instead of 180 MB. main = do a <- getContents let (b,c) = processFile a in seq (sum (map (const 0) b)) ( do putStrLn c) ------------------------------------------------------------ Sami