On Tue, Nov 18, 2014 at 4:59 PM, Thomas Koster
-- | This version has a space leak. zerosAppA :: Application zerosAppA _req respond = withZeros 100000000 $ \ largeLBS -> respond $ responseStream status200 [] $ \ write _flush -> write $ fromLazyByteString largeLBS
-- | This version does not have a space leak. zerosAppB :: Application zerosAppB _req respond = respond $ responseStream status200 [] $ \ write _flush -> withZeros 100000000 $ \ largeLBS -> write $ fromLazyByteString largeLBS
On 20 November 2014 05:30, Gregory Collins
Why does version A not process the LBS in constant space?
The lazy bytestring is let-lifted out of the function so that subsequent calls reuse the same heap value.
What in version A is preventing the GC from collecting the LBS chunks after they have been fed to Warp?
The value is re-used (and the closure holds a reference) so the GC can't collect it.
What is it about version B that permits the LBS chunks to be collected?
The allocation is performed underneath the lambda in version B and so you get a fresh copy every time.
Thanks for your reply. My coding style makes heavy use of let to keep lines succinct so am I going to have to be more careful about the order of lets in case I accidentally introduce unwanted sharing? I was able to re-arrange the leaky version A into version B in my example, but I may not be able to do this in my real application. What if I need the first few bytes of "largeLBS" to determine the response status code or headers? This is only possible in version A. Or will replacing the lazy bytestring with a streaming abstraction like pipes or conduit make the problem go away in both versions? Thanks, -- Thomas Koster