
Hi all, I've just pushed a relatively minor change to the WAI spec to Github, which ties off the last hole in the specification I'm aware of. One annoyance in creating streaming responses is the issue of flushing. By using Builders, it's more difficult to make a truly streaming, realtime app, as the server will wait until a buffer is filled before sending any data to the client. This is actually a very easy problem to solve. blaze-builder provides a value called `flush` that forces the buffer to be sent, even if it's not full. The problem arises when we add one more piece: the gzip middleware. With gzip, we have an extra layer of buffering in place.[1] We need some way to force gzip to flush the buffer as well. I've put together patches to zlib-bindings, conduit, blaze-builder-conduit, zlib-conduit, and wai, that do the following: * zlib-bindings can now explicitly flush a buffer. * conduit exposes a new datatype: `data Flush a = Chunk a | Flush` * blaze-builder-conduit exposes a new function: builderToByteStringFlush, that keeps flush boundaries intact * zlib-conduit similarly has compressFlush/decompressFlush * The type of ResponseSource now has a stream of `Flush Builder` instead of `Builder` I've updated the affected WAI packages. This may seem like a major overhaul, but it actually has very limited scope. The plus side is that we can now easily do something like an eventsource app that has gzip compression applied. Please let me know your thoughts on this, or if people know of any other holes in WAI that should be plugged right now as well. Michael [1] Regarding previous discussions of using a ByteString instead of a Builder for the stream: this issue would affect either a Builder or ByteString approach, and is thus an orthogonal issue.