Hello Fabien,
your expectations are correct, but in order to make this really
happen your consumer function should be aware of the
conduit pipeline in order to consume only the required amount of data.
That may happen automatically in two cases:
1. your function consumes an entire chunk
2. your function can work in an iterative way and can return unprocessed
data or continuation that may consume more data (the case of iterative API
in binary)
A nice example of the function that is related to your use case and
aware of the conduit pipeline:
https://hackage.haskell.org/package/conduit-extra-1.3.0/docs/Data-Conduit-At...
If you pass a parser in sinkParser, it will consume only the required
amount of data.
--
Alexander
On Sun, 4 Nov 2018 at 11:48, Fabien R
Thanks Alexander, The package fixed the problem. I thought that, since a conduit is driven by downstream, if bsToData requested a record, sourceFile would only send the corresponding Bytestrings.
-- Fabien _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
-- Alexander