On Fri, 2010-03-26 at 17:06 +0000, Duncan Coutts wrote:

On Fri, 2010-03-26 at 16:50 +0000, Maciej Piechotka wrote:

...

On Fri, 2010-03-26 at 13:29 +0000, Duncan Coutts wrote:

...

On Fri, 2010-03-26 at 02:11 +0000, Maciej Piechotka wrote:

...

Hello,

I have written patch (attached) which introduce new API for zlib - personally I wrote it to have easy implementation of compression/decompression in iteratee (example file - no yet working -attached).

Earlier version of patch I tried to sent to maintainer of zlib. However I received no response (possibly I get into spam folder by helpful spamassassin or something like that). I definitely don't want to fork this library - especially that it is in haskell platform.

Ivan is right :-)

I understand. However given the last update of zlib it could be that the address is no longer valid.

...

I've discussed iterator approaches with zlib before. Tibbe and I discussed it in the context of web servers that want precise control over outputting blocks.

I'm not happy however that something that is essentially pure (compression/decompression) should have to have be in IO just to fit the iterator pattern. That seems wrong. I could tolerate ST however.

Well. Iteration can be made pure again iff: 1. It will be possible to copy the z_stream. Currently it is not possible with zlib.

It is in principle possible, however it is rather expensive. You would not want to copy it in practise.

I know. It is a list why I put it in IO monad. (The same for point 3)

...

2. It is deal with great care in iteration. It is possible to use safely unsafePerformIO and seq. Iteratee can be pure as it have no copy of the context. Existsing compress/decompress functions as well are 'safe' for the same reason.

I'm not quite sure I follow what you're referring to.

While the iterating API is in IO monad you can have something like.

...

...

I think that it is possible for you to construct the impure interface you want on top of the existing pure interface, using threads and MVars. It is a general pattern, to transform a lazy [a] -> [a] function into an imperative "push block / pull block" interface. You connect up the lazy function to an input channel and an output MVar (running in a new thread). Then externally you push blocks into the input channel and wait for blocks on the output channel. If benchmarks prove that to have too high an overhead then we might consider doing something lower level.

1. That was my first approach. It was so horribly complicated that I gave up. Especially if you wanted any level of error handling.

On the plus side you would only have to do it once and it would let you wrap any lazy [a] -> [a] function into the iterator stuff.

If you consider the error handling and/or trailing data it is not so nice and for sure not general.

...

3. If anyone is going to write such function - I'm fine. I just chosen what I believe it is a simpler and more correct approach.

I understand it was easier to write it that way, however it is not a nice general API to expose from the zlib package

I tried to be as close as possible to Data.Digest.Pure.MD5[1]. However due to mentioned above problems with z_stream it cannot.

and it messes with the layering of the library.

Could you explain?

...

...

Another alternative that I would be satisfied with would be a lower level interface using the ST monad. That should allow the existing pure interface to be implemented safely on top, and hopefully would allow the kind of impure interface you're after.

As the function have side effects I have chosen IO.

Right but they only have local side effects. As such IO is overkill and it forces users to use unsafePerformIO/unsafeIOToST which means they need to know about the library internals to know if that is safe.

If they would be prefixed with unsafePerformIO/unsafeIOToST user still would have to know something about internals of library but he would have no visual clue.

...

The patch was intended as a low-level interface for specific applications.

Sure, but I'm convinced that it is possible to make a low level ST interface that is not ugly and that the rest of the library can be based upon too.

The rest of library can be based onto this (if you want I can prove it by providing patch). Also I'm not quite sure what you mean by ugly.

Duncan

Regards [1] http://hackage.haskell.org/packages/archive/pureMD5/1.1.0.0/doc/html/Data-Di...