I'd like to use conduits to (1) read a binary file in chunks (2) take an md5 of that chunk (3) write the chunk to a file (4) write md5 to file. I'm looking at Data.Conduit.Binary. Do I want sourceFileRange for (1)?
Hi Michael,
The sourceFile function reads the file in chunks and passes each chunk down
the conduit, although the sizes of the chunks are not specified. If that's
ok with you, you may be able to do something as simple as this:
sourceFile inFile `fuse` accumulateMD5 `fuseUpstream` sinkFile outFile
where accumulateMD5 does the MD5 calculation.
If you need control over the sizes of the chunks, I'd add something else to
the pipeline between sourceFile and accumulateMD5 that sorts that out,
rather than using sourceFileRange which will open the file and seek to the
right place each time you call it.
Hope that helps,
On 3 September 2016 at 18:37, Michael Litchard
I'd like to use conduits to (1) read a binary file in chunks (2) take an md5 of that chunk (3) write the chunk to a file (4) write md5 to file. I'm looking at Data.Conduit.Binary. Do I want sourceFileRange for (1)?
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.
That does help, thanks!
On Sat, Sep 3, 2016 at 10:49 AM, David Turner wrote: Hi Michael, The sourceFile function reads the file in chunks and passes each chunk
down the conduit, although the sizes of the chunks are not specified. If
that's ok with you, you may be able to do something as simple as this: sourceFile inFile `fuse` accumulateMD5 `fuseUpstream` sinkFile outFile where accumulateMD5 does the MD5 calculation. If you need control over the sizes of the chunks, I'd add something else
to the pipeline between sourceFile and accumulateMD5 that sorts that out,
rather than using sourceFileRange which will open the file and seek to the
right place each time you call it. Hope that helps, On 3 September 2016 at 18:37, Michael Litchard I'd like to use conduits to (1) read a binary file in chunks
(2) take an md5 of that chunk (3) write the chunk to a
file
(4) write md5 to file. I'm looking at
Data.Conduit.Binary.
Do I want sourceFileRange for (1)? _______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post. _______________________________________________
Haskell-Cafe mailing list
To (un)subscribe, modify options or view archives go to:
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.
participants (2)
-
David Turner -
Michael Litchard