On Mon, Feb 4, 2013 at 3:47 PM, Simon Marechal
On 03/02/2013 16:06, Felipe Almeida Lessa wrote:
I guess you could use the Flush datatype [1] depending on how your data is generated.
Thank you for this suggestion. I tried to do exactly this by modifying my bulk Redis source so that it can timeout and send empty lists [1]. Then I wrote a few helpers conduits[2], such as :
concatFlush :: (Monad m) => Integer -> Conduit [a] m (Flush a)
which will convert a stream of [a] into a stream of (Flush a), sending Flush whenever it encounters and empty list or it send a tunable amount of data downstream.
I finally modified my examples [3]. I realized then it would be nice to have fmap for conduits (but I am not sure how to write such a type signature). Suggestions are welcome !
Actually `fmap` already exists on the Pipe datatype, it just probably doesn't do what you want. It modifies the return value, which is only relevant for Sinks. What you probably are looking for is mapOutput[1]. Michael [1] https://haskell.fpcomplete.com/hoogle?q=mapOutput
[1]
https://github.com/bartavelle/hslogstash/commit/663bf8f5e6058b476c9ed9b5c9cf... [2] https://github.com/bartavelle/hslogstash/blob/master/Data/Conduit/Misc.hs [3]
https://github.com/bartavelle/hslogstash/blob/master/examples/RedisToElastic...