On Thu, Jun 18, 2009 at 9:55 PM, Ross Mellgren wrote:

It looks offhand like you're not being strict enough when you put things back in the IORef, and so it's building up thunks of (+1)...

With two slight mods:

go 0 = return () go n = do modifyIORef ior (+1) go (n-1)

-->

go 0 = return () go n = do modifyIORef ior (\ x -> let x' = x+1 in x `seq` x') go (n-1)

Just a slight prettification of that line: modifyIORef ior ((1+) $!) Or applied prefix if you prefer. Prefix ($!) has the nice interpretation as the HOF that makes its argument into a strict function. Luke

D'oh, yeah that is better. You know, I actually had that and had expanded it because I was going to seq both the input and the result of the (+1), but punted on it and didn't switch back to the more compact format. -Ross On Jun 19, 2009, at 12:45 AM, Luke Palmer wrote:

On Thu, Jun 18, 2009 at 9:55 PM, Ross Mellgren wrote: It looks offhand like you're not being strict enough when you put things back in the IORef, and so it's building up thunks of (+1)...

With two slight mods:

go 0 = return () go n = do modifyIORef ior (+1) go (n-1)

-->

go 0 = return () go n = do modifyIORef ior (\ x -> let x' = x+1 in x `seq` x') go (n-1)

Just a slight prettification of that line:

modifyIORef ior ((1+) $!)

Or applied prefix if you prefer. Prefix ($!) has the nice interpretation as the HOF that makes its argument into a strict function.

Luke

dvde:

Don Stewart schrieb:

...

It is not possible to write a modifyIORef that *doesn't* leak memory!

Why? Or can one read about it somewhere?

Try writing a version of this program, using modifyIORef only, such that it doesn't exhaust the heap: import Data.IORef import Control.Monad import System.IO.Unsafe ref :: IORef Int ref = unsafePerformIO $ newIORef 0 {-# NOINLINE ref #-} main = do modifyIORef ref (\a -> a + 1) main Run it in a constrained environment, so you don't thrash: $ ./A +RTS -M100M Heap exhausted; Current maximum heap size is 99999744 bytes (95 MB); use `+RTS -M<size>' to increase it. The goal is to run in constant space. -- Don

dvde:

Don Stewart schrieb:

...

dvde:

...

Don Stewart schrieb:

...

It is not possible to write a modifyIORef that *doesn't* leak memory!

Why? Or can one read about it somewhere?

Try writing a version of this program, using modifyIORef only, such that it doesn't exhaust the heap:

import Data.IORef import Control.Monad import System.IO.Unsafe

ref :: IORef Int ref = unsafePerformIO $ newIORef 0 {-# NOINLINE ref #-}

main = do modifyIORef ref (\a -> a + 1) main

Run it in a constrained environment, so you don't thrash:

$ ./A +RTS -M100M Heap exhausted; Current maximum heap size is 99999744 bytes (95 MB); use `+RTS -M<size>' to increase it.

The goal is to run in constant space.

-- Don

Hm, do you say it is not possible to write a modifyIORef function that does not leak memory, or do you say it is not possible to use the (existing) modifyIORef without having memory leaks?

The latter. atomicModifyIORef is harder though still, since it is a primop with the same properties as modifyIORef :/

So would it make sense to create a strict modifyIORef' function?

Very much so. In fact, I'd argue the vast majority of uses are for the WHNF-strict version. -- Don

Evan Laforge writes:

The only workaround I could find is to immediately read the value back out and 'seq' on it, but it's ugly.

Yep! C'est la vie unfortunately. The way atomicModifyIORef works is that the new value isn't actually evaluated at all; GHC just swaps the old value with a thunk which will do the modification when the value is demanded. It's done like that so that the atomic modification can be done with a compare-and-swap CPU instruction; a fully-fledged lock would have to be taken otherwise, because your function could do an unbounded amount of work. While that's happening, other mutator threads could be writing into your memory cell, having read the same old value you did, and then *splat*, the souffle is ruined. Once you're taking a lock, you've got yourself an MVar. This is why IORefs are generally (always?) faster than MVars under contention; the lighter-weight lock mechanism means mutator threads don't block, if the CAS fails atomicModifyIORef just tries again in a busy loop. (I think!) Of course, the mutator threads themselves then tend to bump into each other or do redundant work when it's time to evaluate the thunks (GHC tries to avoid this using "thunk blackholing"). Contention issues here have gotten radically better in recent versions of GHC I think. Forgive me if I've gotten anything wrong here, I think Simon Marlow might be the only person who *really* understands how all this stuff works. :)

So two questions:

writeIORef doesn't have this problem. If I am just writing a simple value, is writeIORef atomic? In other words, can I replace 'atomicModifyIORef r (const (x, ())' with 'writeIORef r x'?

Any reason to not do solution 1 above?

Well if you're not inspecting or using the old value then it's safe to just blow it away, yes. Cheers, G -- Gregory Collins

Don Stewart wrote:

dvde:

...

Don Stewart schrieb:

...

It is not possible to write a modifyIORef that *doesn't* leak memory!

Why? Or can one read about it somewhere?

Try writing a version of this program, using modifyIORef only, such that it doesn't exhaust the heap:

import Data.IORef import Control.Monad import System.IO.Unsafe

ref :: IORef Int ref = unsafePerformIO $ newIORef 0 {-# NOINLINE ref #-}

main = do modifyIORef ref (\a -> a + 1) main

Run it in a constrained environment, so you don't thrash:

$ ./A +RTS -M100M Heap exhausted; Current maximum heap size is 99999744 bytes (95 MB); use `+RTS -M<size>' to increase it.

The goal is to run in constant space.

-- Don _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

Thanks, that's good to know. do x <- readIORef ior writeIORef ior $! (x+1) Works for me. The laziness of modifyIORef and workarounds would be a good thing to have documented in the modifyIORef docs [1], since it's probably a common source of memory leaks. I'd also be in favor of a strict version of modifyIORef. [1] http://www.haskell.org/ghc/dist/current/docs/libraries/base/Data-IORef.html#... -jim

...

It is not possible to write a modifyIORef that *doesn't* leak memory!

Why? Or can one read about it somewhere?

Possibly, Don meant that 'modifyIORef' is defined in a way that does not allow to enforce evaluation of the result of the modification function (a typical problem with fmap-style library functions): modifyIORef ref f = readIORef ref >>= writeIORef ref . f No matter whether 'f' is strict or not, the 'writeIORef r' doesn't evaluate its result, just stores the unevaluated application: > r<-newIORef 0 > modifyIORef r (\x->trace "done" $ x+1) > modifyIORef r (\x->trace "done" $ x+1) > readIORef r done done 2 If it had been defined like this instead mRef r ($) f = readIORef r >>= (writeIORef r $) . f it would be possible to transform the strictness of 'writeIORef r' to match that of 'f': > r<-newIORef 0 > mRef r ($) (\x->trace "done" $ x+1) > mRef r ($) (\x->trace "done" $ x+1) > readIORef r done done 2 > r<-newIORef 0 > mRef r ($!) (\x->trace "done" $ x+1) done > mRef r ($!) (\x->trace "done" $ x+1) done > readIORef r 2 Claus