Re: [Haskell-cafe] "Out of memory" if compiled with -O2, why?

I've tried it out with and without -O2 and with and without manual inlining. I also used len=1024*1024. Just in case, the system is Linux x86-64, GHC 6.12.3. You can find output of memory profiler attached. It seems that list of pairs is shared only when lst is not manually inlined and -O2 is enabled. And the GC certainly cannot collect any of map nodes until M.size has returned. So, at the same time list [(Integer, Integer)] of length len and Map Integer Integer of size len are in memory, which means that an (Integer, Integer) and s single node of this map are taking together 100 bytes of memory, which I find plausible. Bottom line --- I see nothing strange about it, except for unnecessary sharing of lst and M.Map being quite memory-costy. -- Regards, Petr On Thu, Dec 2, 2010 at 1:27 PM, Dmitry Kulagin wrote:

Thank you, it is indeed very similar problem. Nevertheless it seems that the lst function is not the direct reason, because: 1) if I inline lst (by hands), the problem is still there 2) size of the list is actially not so large - just 15 millions elements

I am almost sure that the reason is Map.fromList - result of the function perhaps somehow memoized and not released by GC.

Dmitry.

On Wed, Dec 1, 2010 at 8:15 PM, Petr Prokhorenkov wrote:

...

Hi, Dmitry

I recently had the same problem: http://www.haskell.org/pipermail/haskell-cafe/2010-November/086450.html

Memory is taken by the list returned by your lst function wich is being shared across g,h,i,j,k,l,m,n. Apparently there is no safe and easy way to overcome this yet :(

-- Regards, Petr

On Wed, Dec 1, 2010 at 5:23 PM, Dmitry Kulagin wrote:

...

Hi,

I have problems with memory leaks and can't find out how to avoid them. I tried to reduce sample to demonstrate the following problems: 1) when compiled without -O2 option, it iconsumes 1582MB (!) total memory 2) when compiled with -O2 option it terminates with "out of memory"

Actually I don't understand the reasons, particulary why  GC can't collect already processed objects g,...,n (see code below)?

I would appreciate very much any help with this situation. Thanks!

module Main where

import qualified Data.Map as M

len = 15*1024*1024 lst from = take len $ zip [from..] [0..]

g = M.size $ M.fromList $ lst 0 h = M.size $ M.fromList $ lst 0 i = M.size $ M.fromList $ lst 0 j = M.size $ M.fromList $ lst 0 k = M.size $ M.fromList $ lst 0 l = M.size $ M.fromList $ lst 0 m = M.size $ M.fromList $ lst 0 n = M.size $ M.fromList $ lst 0

main = do    mapM_ print [g,h,i,j,k,l,m,n]

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe