Quoth Manfred Lotz
The problem seems to lie in the HaskellNet package. If for example I only fetch a specific message m <- fetch con 2092 having a size of some 1.2m then I get the same stack overflow.
If at runtime I specify +RTS -K40M -RTS it works but takes over 40 seconds to fetch the message.
That's not so good, but I wouldn't be surprised if it's a natural parsing problem, I mean it's just a lot of data to run through a Haskell parser. IMAP does give you the means to mitigate the problem - the big data transfer in a FETCH response is preceded by a byte count - but to really take advantage of that, how far do you go? I don't have much experience with general purpose parsers, do they often support an efficient counted string read? Is it OK to return String, or do we need a more efficient type (e.g., ByteString?) Is it OK to return any kind of string value - given that a message part could be arbitrarily long (and needs to be decoded), do you go to a lot of trouble to support large message parts but not extremely large ones? For me, the answer is for the parser to bail out, reporting the counted input as a count but leaving it to the application to actually effect the data transfer and return to finish the parse. That's only moderately complicated, but it's part of a general philosophy about application driven I/O vs. protocol parsing that seems to be mine alone. I have no idea how much could be done to tighten up HaskellNet.IMAP. Someone who understands it well enough might be able to get a miraculous improvement with a strictness annotation or something. Maybe you could track that down with profiling. Donn