On Tue, Nov 18, 2014 at 4:59 PM, Thomas Koster
-- | This version has a space leak. zerosAppA :: Application zerosAppA _req respond = withZeros 100000000 $ \ largeLBS -> respond $ responseStream status200 [] $ \ write _flush -> write $ fromLazyByteString largeLBS
-- | This version does not have a space leak. zerosAppB :: Application zerosAppB _req respond = respond $ responseStream status200 [] $ \ write _flush -> withZeros 100000000 $ \ largeLBS -> write $ fromLazyByteString largeLBS
On 20 November 2014 16:59, Bardur Arantsson
I would recommend avoiding lazy I/O altogether and using "responseStream" instead. This will let you decide exactly what to write and when.
I am already using responseStream. It's just that in my examples and my real application, the 'octets' to be used for the response entity ("largeLBS" from my examples) are provided to the callback as a lazy bytestring built from lazy I/O, until I get my head around pipes and conduit.
Or will replacing the lazy bytestring with a streaming abstraction like pipes or conduit make the problem go away in both versions?
Not magically, but both of those can certainly be used to avoid the problem. You'll still have to arrange it so that you don't close over a huge (lazy) byte string.
OK, but could I use pipes or conduit (no lazy I/O) to remove the space leak in version A without transforming it into version B (which doesn't have the space leak anyway)? That is, can I switch out the lazy bytestring in version A with a pipe or conduit so that I can start streaming the zeros *outside* of responseStream in order to determine the HTTP status code and headers (which is only possible in version A) and continue to stream them *inside* of responseStream for the rest of the response entity, without incurring a space leak? If the answer is "yes" then that's that and I will go and learn pipes and/or conduit right away.
If what you're trying to do is simple, then I'd just recommend writing your responses using "responseStream" and writing your "genereate output" function in terms of the "write" and "flush" callbacks you get access to write "responseStream". It's pretty easy when you get the hang of it.
...except that the HTTP status code and headers are arguments to responseStream and therefore I cannot determine them from within the callback passed as responseStream's third argument. This is why some of my 'long stream' needs to be accessed before invoking responseStream and why my real application looks like the leaky version A. Thanks, -- Thomas Koster