Re: [Haskell-cafe] Streaming to JuicyPixels

Hi, I didn't thought it would be a full rewrite, when I finish Jpeg writing functionality, I will look at it in depth, which will not be immediately. Feel free to use any code snippet from JuicyPixels if it help your for your own library. Regards Vincent Berthoux Le 24 février 2012 06:42, Myles C. Maxfield a écrit : So I started working on moving JuicyPixels to a streaming interface, and

have some observations. This is going to be a pretty major change, touching pretty much every function, and the end result will end up looking very close to the code that I have already written. I'm not nearly as close to the code as the author is, and I've already made some mistakes due to not understanding how the original code is structured and layed out.

Because this is essentially turning out to be a rewrite, I think it makes more sense for me to just work on my own library, and release it as a streaming alternative to JuicyPixels. How do you feel about this, Vincent?

Thanks, Myles

On Wed, Feb 22, 2012 at 12:30 PM, Vincent Berthoux < vincent.berthoux@gmail.com> wrote:

...

Hi, Please go ahead, and github is the perfect medium for code sharing :)

Regards

Vincent Berthoux

Le 22 février 2012 20:20, Myles C. Maxfield a écrit :

Let's put aside the issue of getting access to the pixels before the

...

stream is complete.

How would you feel if I implemented use of the STT monad transformer on top of Get in JuicyPixels, in order to get rid of the (remaining >> getBytes) call, and then expose the underlying Get interface to callers? This would allow use for streaming.

Is this something that you feel that I should pursue? I can send you a GitHub Pull Request when I'm done.

Thanks, Myles

On Wed, Feb 22, 2012 at 5:01 AM, Vincent Berthoux < vincent.berthoux@gmail.com> wrote:

...

Hi,

I can understand your performance problems, I bumped into them before the first release of Juicy Pixels and took a long time to get 'correct' performance out of the box, and the IDCT is not the only 'hot point', I got problems with intermediate data structure as well. Any list has proven a bit problematic performance wise, so I rewrote some things which would have been easily implemented with forM_ or mapM_ with manual recursion.

I didn't knew STT existed, so it open a new area of reflection for the streaming Jpeg decoder, instead of using the (remaining >> getBytes) combo, staying in the Get monad might help. The reason to expose the ST monad is that I use it internally to mutate the final image directly, and I'd prefer to avoid freezing/unfreezing the underlying array. So in order to give access to the intermediate representation, a type could be (STT s (StateT JpegDecodingState Get) (MutableImage s PixelYCbCr8)) (the Jpeg decoder only produce (Image PixelYCbCr8 internally)). This should allow a freeze then a color conversion.

As it would induce performance loss, this version should exist alongside current implementation. This is not trivial, but it's far from impossible. For the IDCT implementation, I don't think a package make much sense, if you want to use it, just grab it and customize the interface to your needs :).

Regards

Vincent Berthoux

Le 21 février 2012 06:16, Myles C. Maxfield a écrit :

Hello, and thanks for the quick reply.

...

You're right that using (remaining >> getBytes) won't work for streaming, as it would pull the rest of the stream into a buffer, thereby delaying further processing until the image is completely downloaded. :-(

I'm a little unclear about what you mean about the use of the ST monad. There is an STThttp://hackage.haskell.org/packages/archive/STMonadTrans/0.2/doc/html/Contro... monad transformer, so you could wrap that around Get. Is that what you're referring to?

As an aside: I didn't realize that JuicyPixels existed, so I wrote a JPEG decoder specifically designed for streaming - it doesn't request a byte of input unless it has to, uses a StateT (wrapped around Attoparsec) to keep track of which bit in the current byte is "next", and does the Huffman decoding in-line. However, I didn't use ST for the IDCT, so my own decoder has serious performance problems. This prompted me to start searching around for a solution, and I came across JuicyPixels, which already exists and is much faster than my own implementation. I'm hoping to get rid of my own decoder and just use JuicyPixels. If you're curious, my own code is here: https://github.com/litherum/jpeg.

Is it reasonable to extend JuicyPixels to fit my use case? It sounds like JuicyPixels wouldn't work so well as it stands. I'd be happy to do whatever work is necessary to help out and get JuicyPixels usable for me. However, if that would require a full (or near-full) rewrite, it might make more sense for me to use my own implementation with your IDCT. Is there a way we can share just the IDCT between our two repositories? Perhaps making a new IDCT8 library that we can both depend on?

As for what API I'd like to be able to use, just a "Get DynamicImage" should suffice (assuming it has streaming semantics as described above). It would be really nice if it was possible to get at the incomplete image before the stream is completed (so the image could slowly update as more data arrives from the network), but I'm not quite sure how to elegantly express that. Do you have any ideas?

I think that having 2 native jpeg decoders (Actually 3, because of this package http://hackage.haskell.org/package/jpeg) is detrimental to the Haskell community, and I would really like to use JuicyPixels :D

Thanks, Myles C. Maxfield

On Mon, Feb 20, 2012 at 3:01 PM, Vincent Berthoux < vincent.berthoux@gmail.com> wrote:

...

Hi,

I can expose the low level parsing, but you would only get the chunks/frames/sections of the image, Cereal is mainly used to parse the structure of the image, not to do the raw processing. For the raw processing, I rely on `remaining >> getBytes` to be able to manipulate data at bit level or to feed it to zlib, and the documentation clearly state that remaining doesn't work well with runGetPartial, so no read ahead, but even worse for streaming :).

To be fair, I never thought of this use case, and exposing a partially decoded image would impose the use of the ST Monad somehow, and Serialize is not a monad transformer, making it a bit hard to implement.

By curiosity what kind of API would you hope for this kind of functionality?

Regards

Vincent Berthoux

Le 20 février 2012 22:08, Myles C. Maxfield a écrit :

Hello, > I am interested in the possibility of using JuicyPixels for > streaming images from the web. It doesn't appear to expose any of its > internally-used Serialize.Get functionality, which is problematic for > streaming - I would not like to have to stream the whole image into a > buffer before the decoder can start decoding. Are there any plans on > exposing this API, so I can use the runGetPartial function to facilitate > streaming? > > In addition, does the library do much readahead? There's no point in > exposing a Get interface if it's just going to wait until the stream is > done to start decoding anyway. > > Thanks, > Myles C. Maxfield >

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe