Hello John, Thursday, February 09, 2006, 3:19:30 AM, you wrote:
JM> If we had a good standard poll/select interface in System.IO then we JM> actually could implement a lot of concurrency as a library with no JM> (required) run-time overhead. I'd really like to see such a thing get JM> into the standard. Well, mainly it would just be a really useful thing JM> to have in general. If others think it is a good idea I can try to come JM> up with a suitable API and submit it to the repo.
i have delayed answering to this letter until i announced my Streams library. now i can say that such API already exists - in terms of my library you need just to write an transformer that intercepts vGetBuf/vPutBuf calls and pass them to the select/poll machinery. so you can write such transformer just now and every program that uses Streams will benefit from its usage. Converting programs that use Handles to using Streams should be also an easy task.
JM> I was actually asking for something much more modest, which was the JM> routine needed to pass them to the select/poll machinery. but yeah, what JM> you say is one of my expected uses of such a routine. Once a standard IO JM> library settles down, then I can start working on the exact API such a JM> routine would have. but if all will wait while the library settles down, it will never occur :) your work can change design of library, like the my library itself can change the shape of haskell' :) at this moment, i just developed the library which satisfy demands in extending current I/O library by new features, such as Unicode support, high speed, portability to other compilers, binary i/o, i/o for packed strings, and asynchronous i/o using methods other than select(). but i don't implement all these features actually, i just developed infrastructure, in which all these features can be easily added. unlike the System.IO library, you don't need to ask someone to implement new features or make corrections in foreign sources. you just need to develop module what implements this standard Stream interface and then it can be used as easy as transformers from the library itself as i understand this idea, transformer implementing async i/o should intercept vGetBuf/vPutBuf calls for the FDs, start the appropriate async operation, and then switch to another Haskell threads. the I/O manager thread should run select() in cycle and when the request is finished, wake up the appropriate thread. what's all. if you will ever need, this implementation can then be used to extend GHC's System.IO internals with the support for new async i/o managers (as i understand, select() is now supported by GHC, but poll(), kqueue() is not supported?). the only difference that my lib gives an opportunity to test this implementation without modifying GHC I/O internals, what is somewhat simpler. so, interface for async vGetBuf/vPutBuf routines should be the same as for read/write: type FD = Int vGetBuf_async :: FD -> Ptr a -> Int -> IO Int vPutBuf_async :: FD -> Ptr a -> Int -> IO Int i think that implementations for ghc and jhc should be slightly different, though, because of different ways to implement multi-threading. but the I/O manager should be the same - it just receives info about I/O operations to run and returns information about completed ones. ... well, this I/O manager should implement just one operation: performIO :: Request -> IO () type Request = (IOType, FD, Ptr a, Int, Notifier) data IOType = Read | Write | ... type Notifier = Result -> IO () data Result = OK Int | Fail ErrorInfo "performIO" starts new I/O operation. On the completion of this operation, Notifier is called with information about results of execution. so, for the GHC the following should work: vGetBuf_async fd ptr size = do done <- newMVar let notifier = putMVar done () performIO (Read, fd, ptr, size, notifier) takeMVar done for JHC, the body of "vGetBuf_async" may be different if you will find this interface reasonable, at least for the first iteration, i will develop appropriate transformer, so for you remains "only" the implementation of "performIO"
of course, Streams library is not some standard just now, and moreover - it is not compatible with JHC. the greatest problem is what i using type classes extensions available in GHC/Hugs what is not in H98 standard. so, i'm interested in pushing Haskell' to accept most advanced possible extensions in this area and, of course, in actual implementing these extensions in the Haskell compilers. alternative way to make Streams available to wider range of Haskell compilers is to strip support of streams working in monads other that IO.
JM> Don't take the absence of a feature in jhc to mean I don't like or want JM> that feature. There are a lot of things I don't have but that I'd JM> definitly want to see in the language simply because I was only shooting JM> for H98 to begin with and was more interested in a lot of the back end JM> stuff. You should figure out the nicest design that uses just the JM> extensions needed for the design you want. it could help us decide what JM> goes into haskell-prime to know what is absolutely needed for good JM> design and what is just nice to have. this simply means that the Streams library cannot be used with JHC, what is bad news, because it is even more rich than GHC's System.IO. jhc had chance to get modern I/O library. but it lost that chance :)
if you can make select/poll transformer, at least for testing purposes, that will be really great.
JM> Yeah, I will look into this. the basic select/poll call will have to be JM> pretty low level, but hopefully it will allow interesting higher level JM> constructs based on your streams or an evolution of them. please look. at this moment Sreams library lacks only a few important features, already implemented in GHC's System.IO: sockets, line buffering and async i/o. moreover, i don't have an experience in implementing the async i/o, so foreign help is really necessary addressing these three issues will allow to propose the Streams library as possible System.IO replacement. and as you can see, implementing the "performIO" will allow us to use async i/o for all possible i/o operations, including "get/put_" or vGetContents, for example -- Best regards, Bulat mailto:bulatz@HotPOP.com