Thanks for the reply. I did have a version along those lines at some point, but I felt it was cheating rather to print the dots not at the correct point in the stream. Perhaps I've over-complicated for the sake of the learning experience, but I do like to have a version that passes on the correct number of bytes, then prints the ".", and then continues. David "Ertugrul Soeylemez" wrote in message news:20110406163105.13cbc9a3@angst.streitmacht.eu...

"David Hotham" wrote:

...

I've spent some time over the last couple of days trying to write an enumeratee that prints a "." every n bytes (with obvious intended use as a progress tracker). Seems like it oughtn't be hard, but it has been a steep learning curve...

I have come up with something that seems to do the job but I don't know that I'm completely happy with it (or even that I completely understand it, to be honest).

If anyone more expert would be kind enough either to reassure me that I'm doing it right or - more likely - to offer improvements / suggestions on what obvious simplifications I have overlooked, I'd be grateful.

I think that using lazy bytestrings does not have any advantage here, since the enumerator creates the strict bytestrings at some point, then your enumeratee converts them into lazy ones just for counting. Just use the straightforward approach: Take the chunks and count the bytes like here:

{-# LANGUAGE ScopedTypeVariables #-}

dotsAt :: forall b m. MonadPeelIO m => Int -> Enumeratee ByteString ByteString m b dotsAt n = loop 0

where loop :: Int -> Enumeratee ByteString ByteString m b loop i' step@(Continue k) = continue go

where go :: Stream ByteString -> Iteratee ByteString m (Step ByteString m b) go EOF = return step go ch@(Chunks strs) = do let (numDots, i) = divMod (i' + sum (L.map BC.length strs)) n tryIO $ BC.putStr (BC.replicate numDots '.') k ch >>== loop i loop i' step = return step

I think, this is about the most straightforward and also the fastest approach.

Greets, Ertugrul

-- nightmare = unsafePerformIO (getWrongWife >>= sex) http://ertes.de/

The desired behaviour (certainly my desired behaviour, but I think also the most useful behaviour generally) is that the enumeratee passes n bytes to its iteratee, prints a dot, and repeats. Given that, printing the dots all in one bunch after passing bytes to the iteratee isn't any improvement over printing the dots all in one bunch before passing them to the iteratee. I think that mostly I want it the way that I want it because that's the bit that I struggled most over and I'm now reluctant to give it up! However this might actually make a useful difference to behaviour in the case of an iteratee that did very expensive processing, or that itself performed IO. In such cases, my behaviour could be expected to give a more accurate indication of how far through processing we'd actually got. David PS Yes, I already tried out the code ;-). You'll see that both of my versions did indeed hFlush stdout. "Ertugrul Soeylemez" wrote in message news:20110406182931.0238561f@angst.streitmacht.eu...

"David Hotham" wrote:

...

I did have a version along those lines at some point, but I felt it was cheating rather to print the dots not at the correct point in the stream.

Perhaps I've over-complicated for the sake of the learning experience, but I do like to have a version that passes on the correct number of bytes, then prints the ".", and then continues.

Well, then just do the printing after calling the continuation:

dotsAt :: forall b m. MonadPeelIO m => Int -> Enumeratee ByteString ByteString m b dotsAt n = loop 0

where loop :: Int -> Enumeratee ByteString ByteString m b loop i' step@(Continue k) = continue go

where go :: Stream ByteString -> Iteratee ByteString m (Step ByteString m b) go EOF = return step go ch@(Chunks strs) = do let (numDots, i) = divMod (i' + sum (L.map BC.length strs)) n printDots = tryIO $ BC.putStr (BC.replicate numDots '.') >> hFlush stdout k ch >>== (\step -> printDots >> loop i step) loop i' step = return step

By the way, after trying out the code, I found that you should use hFlush after printing. Otherwise you may see the dots delayed.

Greets, Ertugrul

-- nightmare = unsafePerformIO (getWrongWife >>= sex) http://ertes.de/

As a side note, even though GHC seems to handle this properly, I would force the value of 'i' before passing data to the continuation. Otherwise a less smart compiler may eat memory. I believe, it can only eat memory proportional to 'n', but nevertheless real constant space is better: seq i $ k ch >>== (\step -> printDots >> loop i step) Greets, Ertugrul -- nightmare = unsafePerformIO (getWrongWife >>= sex) http://ertes.de/

Very interesting - thanks! "Gregory Collins" wrote in message news:BANLkTikEh7aQfe2-JDq6DoCZ+SYw5MJ5Lg@mail.gmail.com...

My enumerator style may not be the best (I'm long-winded), but personally when the stream types are the same on input and output I often skip the Enumeratee stuff and just write an Enumerator wrapper. To address your complaint here:

...

PS Implementations which involve "EB.take count" seem to me unsatisfactory; one surely oughtn't need to have a large buffer to solve this problem

I'd write a helping combinator:

...

module Main where

import Control.Monad (when) import Control.Monad.Trans import Data.ByteString.Char8 (ByteString) import qualified Data.ByteString.Lazy.Char8 as L import Data.Enumerator import qualified Data.Enumerator.List as EL import System.IO

takeUpTo :: Monad m => Int -> Iteratee ByteString m (Stream ByteString, Int) takeUpTo n' = continue k where n = toEnum n'

k EOF = yield (EOF,0) EOF k (Chunks xs) = if taken == 0 then takeUpTo n' else yield (stream, taken) rest where s = L.fromChunks xs (a,b) = L.splitAt n s taken = fromEnum $ L.length a stream = Chunks $ L.toChunks a rest = Chunks $ L.toChunks b

The code to run a side effect every N bytes is then pretty short (and should be efficient):

...

sideEffectEveryNBytes :: Monad m => Int -- ^ run the side effect every N bytes -> m () -- ^ side effect -> Step ByteString m a -> Iteratee ByteString m a sideEffectEveryNBytes n act = flip checkContinue1 n $ \loop i k -> do (str, taken) <- takeUpTo i when (taken == i) $ lift act (lift $ runIteratee $ k str) >>= loop (nextI $ i - taken) where nextI 0 = n nextI i = i

Here's your particular example:

...

example :: IO [ByteString] example = run_ $ enumList 1 [ "the quick brown " , "fox " , "jumped " , "over " , "the lazy dog" ] $$ it where it = do xs <- sideEffectEveryNBytes 10 (putStr "." >> hFlush stdout) $$ EL.consume lift $ putStrLn "" return xs

Running it:

...

*Main> example .... ["the quick ","brown ","fox ","jumped ","ove","r ","the lazy"," dog"]

G -- Gregory Collins