Re: Is it an error to call hGetLine on a non-blocking Handle?

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue 06 May 2014 05:35:15 PM UTC, Alain O'Dea wrote:

On Tue 06 May 2014 04:39:19 PM UTC, Bryan O'Sullivan wrote:

...

On Mon, May 5, 2014 at 10:04 PM, Alain O'Dea mailto:alain.odea@gmail.com> wrote:

I'm trying to get to the bottom of a very tenacious bug in the GHC Builder.

The details of how this happens are documented separately for reference: https://gist.github.com/AlainODea/dc841046524cd7833da8

The Handles involved are returned by System.Process.runInteractiveProcess. They are non-blocking at the Illumos kernel level.

It appears that maybeFillReadBuffer (called by hGetLine under the hood) treats EAGAIN as an error:

http://hackage.haskell.org/package/base-4.7.0.0/docs/src/GHC-IO-Handle-Text....

Is it expected that runInteractiveProcess would return non-blocking Handles for stdout and stderr?

The non-blockingness should be hidden.

Your attempt to follow EAGAIN down the programmatic rabbit hole hasn't gotten you nearly far enough, I'm afraid. You stopped at a layer where the underlying non-blocking nature of the IO shouldn't be visible, so it would be inappropriate to check for EAGAIN there. As you'll notice if you follow the breadcrumbs further, it becomes quite difficult to spelunk through the various vtables to see what's really going on.

I /think/ that you should be looking at GHC.IO.FD.readRawBufferPtr, as that's where the IO is ultimately happening. The trouble is, it calls throwErrnoIfMinus1RetryMayBlock, which should already be doing the right thing with EAGAIN. Sadly, I don't know what's going wrong here, but I'm sure you will need to go down a level or two of abstraction from where you are now before you'll find the source of the problem.

Thank you Bryan. I'm probably too low-level with my current DTrace of syscalls and definitely too high-level in the GHC sources as you have identified. I'll find my way to the middle here.

I'll take a look at GHC.IO.FD.readRawBufferPtr and throwErrorIfMinus1RetryMayBlock. It appears to be working normally for most of the operations, but hits a gap somewhere that allows the ResourceExhausted IOError to leak out.

I looked through he readRawBufferPtr and friends and could not see any obvious gaps for EAGAIN to slip through. I didn't look as hard as I'd like, but I was getting a bit lost after a few hours of constructing possible code paths. On a whim I tried setting LC_ALL to en_US.UTF-8 and the problem disappeared. Does hGetLine have undefined behavior if locale environment variables (LANG, LC_*) are unset? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTelh6AAoJEP0rIXJNjNSAxAwH/1iKbSyDK8RChSTSrVUFE7UU KLU9e1g7bU31CGcvwdMG4MfEKT1LgGf9YNoOJCLdimT3RxdnExx/zTuQlfaa/PSo Bb8bcagJvau62RjGgDebIOxarUUzZ+vhNditopsmV5lNnvn600ryORmyBQOI6+Pw SCU83BYMNGdjzSm4/C0KIz5lfaX+OA4UiE1JuIS4/I83/mmnniY59dAiQ8SNzkRD XbrOtdbsrbdoYG48OWy1auQTim+zzzpXVvuzn4vxx/oQ7A1k8kkrfl+UJJNClkAo MLXUZY6CC8pl5nACWlnPsM+WswEpRBDEr9BQ1ZX5psNB+4j4NL76xR28+glek5Y= =0Kkg -----END PGP SIGNATURE-----