
I've been looking for a way to compose enumeratees in the enumerator package, but I've come up with nothing so far. I want this function (=$=) :: Monad m => Enumeratee a0 a1 m b -> Enumeratee a1 a2 m b -> Enumeratee a0 a2 m b I'm building a modular library on top of enumerator that facilitates reading time series data from a DB, applying any number of transformations to it, and then writing it back / doing something else with it. I'd like to be able to write simple transformations (enumeratees) and compose them without binding them to either a db reader (enumerator) or db writer (iteratee). I've been looking at the iterIO package as a possible alternative, because it seems to allow easy composition of Inums (enumeratees). I'm a little skittish of it because it seems unpopular next to enumerator. Thoughts on these issues? Cheers, Mike S Craig

Michael Craig wrote:
I've been looking for a way to compose enumeratees in the enumerator package, but I've come up with nothing so far. I want this function
(=$=) :: Monad m => Enumeratee a0 a1 m b -> Enumeratee a1 a2 m b -> Enumeratee a0 a2 m b
I think part of the problem here is that Enumeratee is defined as: type Enumeratee ao ai m b = Step ai m b -> Iteratee ao m (Step ai m b) If you expand out your type signature you get: (=$=) :: Monad m => (Step a1 m b -> Iteratee a0 m (Step a1 m b)) -> (Step a2 m b -> Iteratee a1 m (Step a2 m b)) -> (Step a2 m b -> Iteratee a0 m (Step a2 m b)) which to me looks rather painful to implement.
I've been looking at the iterIO package as a possible alternative, because it seems to allow easy composition of Inums (enumeratees). I'm a little skittish of it because it seems unpopular next to enumerator.
Thoughts on these issues?
I think these issues are actually common to all implementations of the Iteratee concept. Basically they do not compose as nicely and as cleanly as they would be expected to. I recently ran into this difficulty in composition in my project which was solved by nesting an iteratee inside an enumerator. https://github.com/erikd/http-proxy/commit/73775555c1cc695b21b7c13b823abc6c3... There is work being done to address these issues. See Michael Snoyman's work on Conduits: https://github.com/snoyberg/conduit Cheers. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/

On Sat, Dec 24, 2011 at 2:00 PM, Erik de Castro Lopo
Michael Craig wrote:
I've been looking for a way to compose enumeratees in the enumerator package, but I've come up with nothing so far. I want this function
(=$=) :: Monad m => Enumeratee a0 a1 m b -> Enumeratee a1 a2 m b -> Enumeratee a0 a2 m b
I think part of the problem here is that Enumeratee is defined as:
type Enumeratee ao ai m b = Step ai m b -> Iteratee ao m (Step ai m b)
If you expand out your type signature you get:
(=$=) :: Monad m => (Step a1 m b -> Iteratee a0 m (Step a1 m b)) -> (Step a2 m b -> Iteratee a1 m (Step a2 m b)) -> (Step a2 m b -> Iteratee a0 m (Step a2 m b))
which to me looks rather painful to implement.
I've been looking at the iterIO package as a possible alternative, because it seems to allow easy composition of Inums (enumeratees). I'm a little skittish of it because it seems unpopular next to enumerator.
Thoughts on these issues?
I think these issues are actually common to all implementations of the Iteratee concept. Basically they do not compose as nicely and as cleanly as they would be expected to. I recently ran into this difficulty in composition in my project which was solved by nesting an iteratee inside an enumerator.
https://github.com/erikd/http-proxy/commit/73775555c1cc695b21b7c13b823abc6c3...
There is work being done to address these issues. See Michael Snoyman's work on Conduits:
https://github.com/snoyberg/conduit
Cheers. Erik -- ---------------------------------------------------------------------- Erik de Castro Lopo http://www.mega-nerd.com/
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
By the way, conduit defines *precisely* that operator, for "fusing" (iterIO's term) two conduits together. Michael

On 24 December 2011 05:47, Michael Craig
I've been looking for a way to compose enumeratees in the enumerator package, but I've come up with nothing so far. I want this function
(=$=) :: Monad m => Enumeratee a0 a1 m b -> Enumeratee a1 a2 m b -> Enumeratee a0 a2 m b
I'm building a modular library on top of enumerator that facilitates reading time series data from a DB, applying any number of transformations to it, and then writing it back / doing something else with it. I'd like to be able to write simple transformations (enumeratees) and compose them without binding them to either a db reader (enumerator) or db writer (iteratee).
I've been looking at the iterIO package as a possible alternative, because it seems to allow easy composition of Inums (enumeratees). I'm a little skittish of it because it seems unpopular next to enumerator.
Hi Michael, You could also look at the iteratee package. This is the signature of the (><>) operator: (><>) :: (Nullable s1, Monad m) => (forall x. Enumeratee s1 s2 m x) -> Enumeratee s2 s3 m a -> Enumeratee s1 s3 m a it's quite useful for composing enumeratees, likewise its friend (<><) swims the other way. http://hackage.haskell.org/packages/archive/iteratee/0.8.7.5/doc/html/Data-I... cheers, Conrad.

Thanks for the replies, all. It's good to see that the other iteratee
packages out there are addressing this issue.
I still don't get why it's an issue in the first place. It seems to me like
a pretty simple thing to implement:
(=$=) :: (Monad m)
=> Enumeratee a0 a1 m (Step a2 m b) -> Enumeratee a1 a2 m b
-> Enumeratee a0 a2 m b
(=$=) e01 e12 step = Iteratee $ do
step' <- runIteratee $ e12 step
runIteratee . joinI $ e01 step'
This puts a type restriction on the LHS enumeratee, but enumeratees are
generally polymorphic in the last type param anyway. (And joinE has a
similar restriction when composing an enumerator with an enumeratee.)
Is there a good reason why enumerator doesn't export this or something
analogous?
Mike Craig
On Sun, Dec 25, 2011 at 10:20 PM, Conrad Parker
On 24 December 2011 05:47, Michael Craig
wrote: I've been looking for a way to compose enumeratees in the enumerator package, but I've come up with nothing so far. I want this function
(=$=) :: Monad m => Enumeratee a0 a1 m b -> Enumeratee a1 a2 m b -> Enumeratee a0 a2 m b
I'm building a modular library on top of enumerator that facilitates reading time series data from a DB, applying any number of transformations to it, and then writing it back / doing something else with it. I'd like to be able to write simple transformations (enumeratees) and compose them without binding them to either a db reader (enumerator) or db writer (iteratee).
I've been looking at the iterIO package as a possible alternative, because it seems to allow easy composition of Inums (enumeratees). I'm a little skittish of it because it seems unpopular next to enumerator.
Hi Michael,
You could also look at the iteratee package. This is the signature of the (><>) operator:
(><>) :: (Nullable s1, Monad m) => (forall x. Enumeratee s1 s2 m x) -> Enumeratee s2 s3 m a -> Enumeratee s1 s3 m a
it's quite useful for composing enumeratees, likewise its friend (<><) swims the other way.
http://hackage.haskell.org/packages/archive/iteratee/0.8.7.5/doc/html/Data-I...
cheers,
Conrad.

I wound up emailing John Millikin about this and he made a good case
against these kinds of simplified operators, which is basically the problem
of handling left-over input. The joinI and joinE combinators discard the
left-over Stream that is yielded by the inner iteratee. (As John explains
it, this is a trade-off between ease of use and programming complexity.)
Simplified operators (like $=, =$, and now =$=) use repeated joinI's, so
left-over input may be lost in various places. When using simple iteratees
that never yield left-over input, this isn't a problem and the operators
make sense.
For more complex pipelines, John advocates a style like this:
joinI (foo $$ (bar $$ baz))
so that left over data is only discarded once after the computation is
otherwise complete.
In any case, there's now a new release of enumerator (0.4.17) which
includes an enumeratee composition operator: (=$=) :: Monad m => Enumeratee
a1 a2 m (Step a3 m b) -> Enumeratee a2 a3 m b -> Enumeratee a1 a3 m b.
Cheers,
Mike Craig
On Tue, Dec 27, 2011 at 10:12 AM, Michael Craig
Thanks for the replies, all. It's good to see that the other iteratee packages out there are addressing this issue.
I still don't get why it's an issue in the first place. It seems to me like a pretty simple thing to implement:
(=$=) :: (Monad m) => Enumeratee a0 a1 m (Step a2 m b) -> Enumeratee a1 a2 m b -> Enumeratee a0 a2 m b (=$=) e01 e12 step = Iteratee $ do step' <- runIteratee $ e12 step runIteratee . joinI $ e01 step'
This puts a type restriction on the LHS enumeratee, but enumeratees are generally polymorphic in the last type param anyway. (And joinE has a similar restriction when composing an enumerator with an enumeratee.)
Is there a good reason why enumerator doesn't export this or something analogous?
Mike Craig
On Sun, Dec 25, 2011 at 10:20 PM, Conrad Parker
wrote: On 24 December 2011 05:47, Michael Craig
wrote: I've been looking for a way to compose enumeratees in the enumerator package, but I've come up with nothing so far. I want this function
(=$=) :: Monad m => Enumeratee a0 a1 m b -> Enumeratee a1 a2 m b -> Enumeratee a0 a2 m b
I'm building a modular library on top of enumerator that facilitates reading time series data from a DB, applying any number of transformations to it, and then writing it back / doing something else with it. I'd like to be able to write simple transformations (enumeratees) and compose them without binding them to either a db reader (enumerator) or db writer (iteratee).
I've been looking at the iterIO package as a possible alternative, because it seems to allow easy composition of Inums (enumeratees). I'm a little skittish of it because it seems unpopular next to enumerator.
Hi Michael,
You could also look at the iteratee package. This is the signature of the (><>) operator:
(><>) :: (Nullable s1, Monad m) => (forall x. Enumeratee s1 s2 m x) -> Enumeratee s2 s3 m a -> Enumeratee s1 s3 m a
it's quite useful for composing enumeratees, likewise its friend (<><) swims the other way.
http://hackage.haskell.org/packages/archive/iteratee/0.8.7.5/doc/html/Data-I...
cheers,
Conrad.
participants (4)
-
Conrad Parker
-
Erik de Castro Lopo
-
Michael Craig
-
Michael Snoyman