Re: [Haskell-cafe] ST not strict enough?

On Wed, Nov 16, 2011 at 07:58:51PM +0000, Jason Dusek wrote:

2011/11/15 Johan Tibell :

...

On Tue, Nov 15, 2011 at 12:08 PM, Jason Dusek wrote:

...

Should I be annotating my functions with strictness, for the vector reference, for example? Should I be using STUArrays, instead?

From http://www.haskell.org/ghc/docs/latest/html/libraries/base-4.4.1.0/Control-M...

    "The >>= and >> operations are strict in the state (though not in values stored in the state)."

which implies that

     modifySTRef counter (+1)

is too lazy.

As a first cut at strictifying the ST operations, I introduced a strict plus and strict vector write operation, strictifying every parameter that admitted it.

(+!) a b = ((+) $!! a) $!! b w v n b = (Vector.unsafeWrite v $!! n) $!! b

This did not alter memory usage in any noticeable way. (Tried it with strict and lazy ByteStrings and both had the same memory usage as they did without the extra strictness.)

It does seem off odd that building a vector byte by byte is so hard to do performantly. Maybe the memory usage ends up being okay when working with larger structures, though.

I think your use of (+1) in the STRef was safe here because unsafeWrite was forcing it relatively quickly. The laziness in ST values just bit me a few days ago because I wasn't forcing the values soon enough and was getting stack overflows when evaluation was finally triggered. Have you tried building the vector using things besides write/ST? It might be a bit faster to use something like Vector.unfoldr or Vector.generateM and ByteString.index to build up a pure Vector. After that you could use Vector.unsafeThaw to convert that pure Vector into an MVector.