Discussion: Add unboxed mutable references somewhere sensible

The problem they solve is perhaps not as well known as it should be: Code that frequently modifies an `STRef Int`, for example, will typically surprise the programmer by allocating a ton of memory. This happens because the reference holds a *boxed* Int. Code like modifySTRef ref (+1) will allocate a new Int box every time. To the best of my knowledge, GHC makes no attempt to figure out if this is actually necessary and do something about it. The Data.Ref.Unboxed module in the ArrayRef package attempts to address this, but it doesn't seem to get much visibility, and its code hasn't been touched since 2009. What can we do about this?

We could add something like this to primitive. It has a generalization of
things like STRef, under the name MutVar.
On Wed, Feb 11, 2015 at 7:03 PM, David Feuer
The problem they solve is perhaps not as well known as it should be:
Code that frequently modifies an `STRef Int`, for example, will typically surprise the programmer by allocating a ton of memory. This happens because the reference holds a *boxed* Int. Code like
modifySTRef ref (+1)
will allocate a new Int box every time. To the best of my knowledge, GHC makes no attempt to figure out if this is actually necessary and do something about it. The Data.Ref.Unboxed module in the ArrayRef package attempts to address this, but it doesn't seem to get much visibility, and its code hasn't been touched since 2009. What can we do about this? _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

I added this to mutable-containers. It's easiest to see this by looking at
three readme on
http://www.stackage.org/package/mutable-containers
I'd support something like this in primitive as well.
On Thu, Feb 12, 2015, 4:36 AM Dan Doel
We could add something like this to primitive. It has a generalization of things like STRef, under the name MutVar.
On Wed, Feb 11, 2015 at 7:03 PM, David Feuer
wrote: The problem they solve is perhaps not as well known as it should be:
Code that frequently modifies an `STRef Int`, for example, will typically surprise the programmer by allocating a ton of memory. This happens because the reference holds a *boxed* Int. Code like
modifySTRef ref (+1)
will allocate a new Int box every time. To the best of my knowledge, GHC makes no attempt to figure out if this is actually necessary and do something about it. The Data.Ref.Unboxed module in the ArrayRef package attempts to address this, but it doesn't seem to get much visibility, and its code hasn't been touched since 2009. What can we do about this? _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Yes, this is the sort of thing I was looking for. One question: if I
understand it correctly, it looks like ArrayRef duplicates the logic
of vector's Unbox to allow more general types without piggybacking on
Vector with it (small) overhead. Am I reading this right? If so, what
are the arguments against this approach?
Finally, I can't seem to stop wondering if there might be some
fundamentally more pleasant approach to this whole thing. The
underlying basis for this suspicion comes from the fact that, as
strange as it looks, `StateT v (ST s) a` actually works very well. For
example, the source below produces very nice core with -O2.
module STST (countUp) where
import Control.Monad.ST
import Control.Monad.Trans.State.Strict
import Control.Monad.Trans (lift)
import Control.Monad (when)
import Data.Functor ((<$))
import qualified Data.Vector.Unboxed.Mutable as V
import Data.Vector.Unboxed.Mutable (MVector, unsafeWrite)
step :: V.MVector s Int -> StateT Int (ST s) Bool
step v = do
i <- get
if i == V.length v
then return True
else do
lift (unsafeWrite v i i)
put (i+1)
return False
countUp :: V.MVector s Int -> ST s ()
countUp v = () <$ execStateT go 0
where
go = do
done <- step v
when (not done) go
On Wed, Feb 11, 2015 at 10:35 PM, Michael Snoyman
I added this to mutable-containers. It's easiest to see this by looking at three readme on
http://www.stackage.org/package/mutable-containers
I'd support something like this in primitive as well.
On Thu, Feb 12, 2015, 4:36 AM Dan Doel
wrote: We could add something like this to primitive. It has a generalization of things like STRef, under the name MutVar.
On Wed, Feb 11, 2015 at 7:03 PM, David Feuer
wrote: The problem they solve is perhaps not as well known as it should be:
Code that frequently modifies an `STRef Int`, for example, will typically surprise the programmer by allocating a ton of memory. This happens because the reference holds a *boxed* Int. Code like
modifySTRef ref (+1)
will allocate a new Int box every time. To the best of my knowledge, GHC makes no attempt to figure out if this is actually necessary and do something about it. The Data.Ref.Unboxed module in the ArrayRef package attempts to address this, but it doesn't seem to get much visibility, and its code hasn't been touched since 2009. What can we do about this? _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

If you're talking about Data.Ref.Unboxed, it only seems to do what vector
and primitive classify as Prim types. Not tuples and other
structure-of-arrays type stuff. And all that classification is contained in
primitive (Data.Primitive.Types). It covers some pointer types that Prim
doesn't, but doesn't have Addrs, which Prim does.
On Thu, Feb 12, 2015 at 12:12 PM, David Feuer
Yes, this is the sort of thing I was looking for. One question: if I understand it correctly, it looks like ArrayRef duplicates the logic of vector's Unbox to allow more general types without piggybacking on Vector with it (small) overhead. Am I reading this right? If so, what are the arguments against this approach?
Finally, I can't seem to stop wondering if there might be some fundamentally more pleasant approach to this whole thing. The underlying basis for this suspicion comes from the fact that, as strange as it looks, `StateT v (ST s) a` actually works very well. For example, the source below produces very nice core with -O2.
module STST (countUp) where import Control.Monad.ST import Control.Monad.Trans.State.Strict import Control.Monad.Trans (lift) import Control.Monad (when) import Data.Functor ((<$)) import qualified Data.Vector.Unboxed.Mutable as V import Data.Vector.Unboxed.Mutable (MVector, unsafeWrite)
step :: V.MVector s Int -> StateT Int (ST s) Bool step v = do i <- get if i == V.length v then return True else do lift (unsafeWrite v i i) put (i+1) return False
countUp :: V.MVector s Int -> ST s () countUp v = () <$ execStateT go 0 where go = do done <- step v when (not done) go
On Wed, Feb 11, 2015 at 10:35 PM, Michael Snoyman
wrote: I added this to mutable-containers. It's easiest to see this by looking at three readme on
http://www.stackage.org/package/mutable-containers
I'd support something like this in primitive as well.
On Thu, Feb 12, 2015, 4:36 AM Dan Doel
wrote: We could add something like this to primitive. It has a generalization
of
things like STRef, under the name MutVar.
On Wed, Feb 11, 2015 at 7:03 PM, David Feuer
wrote: The problem they solve is perhaps not as well known as it should be:
Code that frequently modifies an `STRef Int`, for example, will typically surprise the programmer by allocating a ton of memory. This happens because the reference holds a *boxed* Int. Code like
modifySTRef ref (+1)
will allocate a new Int box every time. To the best of my knowledge, GHC makes no attempt to figure out if this is actually necessary and do something about it. The Data.Ref.Unboxed module in the ArrayRef package attempts to address this, but it doesn't seem to get much visibility, and its code hasn't been touched since 2009. What can we do about this? _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Oh, I see now why my thinking is flawed. Yes, one of these primitive
type things would be nice to have somewhere nice.
On Thu, Feb 12, 2015 at 12:12 PM, David Feuer
Yes, this is the sort of thing I was looking for. One question: if I understand it correctly, it looks like ArrayRef duplicates the logic of vector's Unbox to allow more general types without piggybacking on Vector with it (small) overhead. Am I reading this right? If so, what are the arguments against this approach?
Finally, I can't seem to stop wondering if there might be some fundamentally more pleasant approach to this whole thing. The underlying basis for this suspicion comes from the fact that, as strange as it looks, `StateT v (ST s) a` actually works very well. For example, the source below produces very nice core with -O2.
module STST (countUp) where import Control.Monad.ST import Control.Monad.Trans.State.Strict import Control.Monad.Trans (lift) import Control.Monad (when) import Data.Functor ((<$)) import qualified Data.Vector.Unboxed.Mutable as V import Data.Vector.Unboxed.Mutable (MVector, unsafeWrite)
step :: V.MVector s Int -> StateT Int (ST s) Bool step v = do i <- get if i == V.length v then return True else do lift (unsafeWrite v i i) put (i+1) return False
countUp :: V.MVector s Int -> ST s () countUp v = () <$ execStateT go 0 where go = do done <- step v when (not done) go
On Wed, Feb 11, 2015 at 10:35 PM, Michael Snoyman
wrote: I added this to mutable-containers. It's easiest to see this by looking at three readme on
http://www.stackage.org/package/mutable-containers
I'd support something like this in primitive as well.
On Thu, Feb 12, 2015, 4:36 AM Dan Doel
wrote: We could add something like this to primitive. It has a generalization of things like STRef, under the name MutVar.
On Wed, Feb 11, 2015 at 7:03 PM, David Feuer
wrote: The problem they solve is perhaps not as well known as it should be:
Code that frequently modifies an `STRef Int`, for example, will typically surprise the programmer by allocating a ton of memory. This happens because the reference holds a *boxed* Int. Code like
modifySTRef ref (+1)
will allocate a new Int box every time. To the best of my knowledge, GHC makes no attempt to figure out if this is actually necessary and do something about it. The Data.Ref.Unboxed module in the ArrayRef package attempts to address this, but it doesn't seem to get much visibility, and its code hasn't been touched since 2009. What can we do about this? _______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries
_______________________________________________ Libraries mailing list Libraries@haskell.org http://www.haskell.org/mailman/listinfo/libraries

Henning, the idea is to make something friendly to use, like an
unboxed Vector or Array. How that works under the hood is something I
don't want to have to think about too much, as long as it's efficient.
On Thu, Feb 12, 2015 at 5:06 AM, Henning Thielemann
On Wed, 11 Feb 2015, David Feuer wrote:
The Data.Ref.Unboxed module in the ArrayRef package attempts to address this, but it doesn't seem to get much visibility, and its code hasn't been touched since 2009. What can we do about this?
Use Ptr, Storable, poke and peek.
participants (4)
-
Dan Doel
-
David Feuer
-
Henning Thielemann
-
Michael Snoyman