First, thanks for the very detailed investigation. I've CC:ed Andreas and
Kazu who has the most knowledge of the I/O manager nowadays.
-- Johan
On Fri, Mar 15, 2013 at 12:54 PM, PHO
I found the HEAD stopped working on MacOS X 10.5.8 since the parallel I/O manager got merged to HEAD. Stage-2 compiler successfully builds (including Language.Haskell.TH.Syntax contrary to the report by Kazu Yamamoto) but the resulting binary is very unstable especially for ghci:
% inplace/bin/ghc-stage2 --interactive GHCi, version 7.7.20130313: http://www.haskell.org/ghc/ :? for help Loading package ghc-prim ... linking ... done. Loading package integer-gmp ... linking ... done. Loading package base ... linking ... done. Prelude> <stdin>: hGetChar: failed (Operation not supported)
So I took a dtruss log and found it was kevent(2) that returned ENOTSUP. GHC.Event.KQueue was just registering the stdin for EVFILT_READ, whose type was of course tty, and then kevent(2) said "tty is not supported". Didn't the old I/O manager do the same thing? Why was it working then?
After a hard investigation, I concluded that the old I/O manager was not really working. It just looked fine but in fact wasn't. Here's an explanation: If a fd to be registered is unsupported by kqueue, kevent(2) returns -1 iff no incoming event buffer is passed together. Otherwise it successfully returns with an incoming kevent whose "flags" is EV_ERROR and "data" contains an errno. The I/O manager has always been passing a non-empty event buffer until the commit e5f5cfcd, while it wasn't (and still isn't) checking if a received event in fact represents an error. That is, the KQueue backend asks the kernel to monitor the stdin's readability. The kernel then immediately delivers an event saying ENOTSUP. The KQueue backend thinks "Hey, the stdin is now readable!" so it invokes a callback associated with the fd. The thread which called "threadWaitRead" is now awakened and performs a supposedly non-blocking read on the fd, which in fact blocks but works anyway.
However the situation has changed since the commit e5f5cfcd. The I/O manager now registers fds without passing an incoming event buffer, so kevent(2) no longer successfully delivers an error event instead it directly returns -1 with errno set to ENOTSUP, hence the "Operation not supported" exception.
The Darwin's kqueue has started supporting tty since MacOS X 10.7 (Lion), but I heard it still doesn't support some important devices like /dev/random. FreeBSD's kqueue has some difficulties too. It's no doubt kqueue is a great mechanism for sockets, but IMHO it's not something to use for all kinds of file I/O. Should we then try kqueue(2) first and fallback to poll(2) if failed? Sadly no. Darwin's poll(2) is broken too, and select(2) is the only method reliable: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#OS_X_AND_DARWIN_BUGS
I wrote a small program to test if a given stdin is supported by kqueue, poll and select: https://gist.github.com/phonohawk/5169980#file-kqueue-poll-select-cpp
MacOS X 10.5.8 is hopelessly broken. We can't use anything other than select(2) for tty and other devices:
https://gist.github.com/phonohawk/5169980#file-powerpc-apple-darwin9-8-0-txt
FreeBSD 8.0 does support tty but not /dev/random. I don't know what about the latest FreeBSD 9.1: https://gist.github.com/phonohawk/5169980#file-i386-unknown-freebsd8-0-txt
NetBSD 6.99.17 works perfectly here:
https://gist.github.com/phonohawk/5169980#file-x86_64-unknown-netbsd6-99-17-...
Just for reference, Linux 2.6.16 surely doesn't have kqueue(2) but it supports poll(2)ing on devices:
https://gist.github.com/phonohawk/5169980#file-x86_64-unknown-linux2-6-16-tx...
I accordingly suggest that we should have some means to run two independent I/O managers on BSD-like systems: KQueue for sockets, pipes and regular file reads, and Select for any other devices and regular file writes. Note also that no implementations of kqueue support monitoring writability of regular file writes, which also endangers the current use of kqueue-based I/O manager namely "threadWaitWrite".
Any ideas?
Thanks, PHO _______________________________________________________ - PHO - http://cielonegro.org/ OpenPGP public key: 1024D/1A86EF72 Fpr: 5F3E 5B5F 535C CE27 8254 4D1A 14E7 9CA7 1A86 EF72
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs