Re: [Haskell-cafe] Space leak in WAI 3.0 application

On 2014-11-20 08:29, Thomas Koster wrote:

On Tue, Nov 18, 2014 at 4:59 PM, Thomas Koster wrote:

...

-- | This version has a space leak. zerosAppA :: Application zerosAppA _req respond = withZeros 100000000 $ \ largeLBS -> respond $ responseStream status200 [] $ \ write _flush -> write $ fromLazyByteString largeLBS

-- | This version does not have a space leak. zerosAppB :: Application zerosAppB _req respond = respond $ responseStream status200 [] $ \ write _flush -> withZeros 100000000 $ \ largeLBS -> write $ fromLazyByteString largeLBS

On 20 November 2014 16:59, Bardur Arantsson wrote:

...

I would recommend avoiding lazy I/O altogether and using "responseStream" instead. This will let you decide exactly what to write and when.

I am already using responseStream. It's just that in my examples and my real application, the 'octets' to be used for the response entity ("largeLBS" from my examples) are provided to the callback as a lazy bytestring built from lazy I/O, until I get my head around pipes and conduit.

Lol, sorry, I guess I shouldn't be posting when I'm sick. :) I guess I got confused by the fact what you're using a huge LBS when you could just write it in pieces by providing the "write" and "flush" callbacks directly to the code which is actually generating the response (and let it be resposible for avoiding generating huge blobs).

built from lazy I/O,

I would suggest *never* *ever* using lazy I/O. Even though it looks like the easy route, it's much harder to reason about than any of the alternatives. Btw, there are a couple of alternatives which you haven't mentioned (and thus may not be familiar with:) - Strict/direct I/O: Just use write/flush directly, do your own looping "until done" instead of hGetContents, etc. (I realize I've already mentioned it, I just repeat it here for completeness.) - io-streams: A very thin layer of stream abstractions directly on top of I/O. I found this much easier to get a handle on than either conduit or pipes, but then it's also less powerful and gives you very few abstractions independent of IO. Works very well in practice, though, if you just want to "get the job done". See https://hackage.haskell.org/package/io-streams Anyway, sorry about the dud reply before -- hopefully this'll be more helpful! Regards,