Hello,
I guess I should stick my hand up as the supposed maintainer of HaskellNet. Unfortunately I can't say that I know the code that well. Two years ago I rescued it
from bitrot cabalized it and when I couldn't get any response from the original author put myself down as the maintainer.
It is a package which is starting to show its age. Michael Snoyman and I had a conversation in February agreeing that I should try to revamp it by applying some techniques such as those used in blaze html. Unfortunately, I haven't had the time.
I agree with the post above that for mime mail HaskellNet shouldn't be retreiving all of the messages with their message bodies. I might see if I can get a chance to work on it a little this weekend but if someone is using the library and has some time to make some changes that person woould be very welcome (and I'd be more than happy if someone wishes to take over as the maintenance).
-Rob
On Wed, Jul 27, 2011 at 5:47 PM, Donn Cave
<donn@avvanta.com> wrote:
> The problem seems to lie in the HaskellNet package. If for example I
> only fetch a specific message
> m <- fetch con 2092
> having a size of some 1.2m then I get the same stack overflow.
>
> If at runtime I specify +RTS -K40M -RTS it works but takes over 40
> seconds to fetch the message.
That's not so good, but I wouldn't be surprised if it's a natural parsing
problem, I mean it's just a lot of data to run through a Haskell parser.
IMAP does give you the means to mitigate the problem - the big data
transfer in a FETCH response is preceded by a byte count - but to really
take advantage of that, how far do you go? I don't have much experience
with general purpose parsers, do they often support an efficient counted
string read? Is it OK to return String, or do we need a more efficient
type (e.g., ByteString?) Is it OK to return any kind of string value -
given that a message part could be arbitrarily long (and needs to be
decoded), do you go to a lot of trouble to support large message parts
but not extremely large ones?
For me, the answer is for the parser to bail out, reporting the counted
input as a count but leaving it to the application to actually effect
the data transfer and return to finish the parse. That's only moderately
complicated, but it's part of a general philosophy about application
driven I/O vs. protocol parsing that seems to be mine alone.
I have no idea how much could be done to tighten up HaskellNet.IMAP.
Someone who understands it well enough might be able to get a miraculous
improvement with a strictness annotation or something. Maybe you
could track that down with profiling.
Donn