
I split the string in lines, then map some functions on each line to parse the data, and finally calling toU, for converting to an UArr.
Just to make sure (code fragments or, better, reduced examples would make it easier to see what the discussion is about): are you forcing the UArr to be constructed before putting it into the Map?
parse handle = contents <- S.hGetContents handle let v = map singleton' $ ratings contents let m = foldl1' (unionWith appendU) v v `seq` return $! m
where -- Build a Map with a single movie rating singleton' :: (Word32, Word8) -> MovieRatings singleton' (id, rate) = singleton (fromIntegral $ id) (singletonU $ pairS (id, rate))
That helps to make things clearer, I think. One issue is the nature of Maps (strict in keys, non-strict in values). - neither singleton nor unionWith are strict in the Map values, so nothing here forces the evaluation of rate or construction of UArr Prelude Data.IntMap> (unionWith (++) (singleton 1 undefined) (singleton 2 undefined)) `seq` () () - singletonU is strict, but that only means that it will evaluate its parameter if it is evaluated itself (which it isn't, because singleton isn't strict) - seq on a list only forces the first node of the list ((:),[],_|_), so (v `seq`) isn't likely to help much. Also, you probably do not want to force the whole list of singletons before builing the Map, you want the singletons to be constructed and consumed incrementally. - forcing a Map doesn't force any of the values, nor does it force more than the top-level node of whatever the internal Map representation is, so (return $! m) isn't much help, either (by nature of unionWith and foldl1', it'll force all keys before it can say anything much about the Map, but the values remain untouched, burried further under unevaluated (++)s)
type Rating = Word32 :*: Word8 type MovieRatings = IntMap (UArr Rating) -- UArr from uvector
A standard trick to keep Map values evaluated by construction is to make the availability of keys dependent on their values, eg (singleton key) $! value. That won't help with unionWith and the appendUs, but it should allow the source string references to be dropped early, as the singletons are constructed. Hth, Claus