On 27.01 13:10, Bulat Ziganshin wrote:
i'm now write some sort of new i/o library. one area where i currently lacks in comparision to the existing Handles implementation in GHC, is the asynchronous i/o operations. can you please briefly describe how this is done in GHC and partially - why the multiple buffers are used?
One simple optimization is that you can omit all buffering with unbuffered operation. Then simply add the buffer (which is ok because Handles are mutable) if the user ever calls hLookAhead.
moreover, i have an idea how to implement async i/o without complex burecreacy: use mmapped files, may be together with miltiple buffers. for example, we can allocate four 16kb buffers. when one buffer is filled with written data, the program unmaps it and switches to use the next buffer. i don't tested it, but OS can guess that unmapped buffer now should be asynchronously written to disk. the same for reading - when we completely read one buffer, we can unmap it, switch to the second buffer and map the third so that the OS can asynchronously fill the third buffer while we are reading second. should this work, at least on the main desktop OSes?
Please no. There are multiple reasons to avoid mmapped files. 1) They make very few performance guarantees for reading (i.e. a Haskell thread touches memory which has not yet been read from the file causing IO and all the other Haskell threads are blocked too) 2) The time of writes is unpredictable making implementing a hFlush harder? (not sure about this) 3) Not all file descriptors will support it - i.e. we will need the read/write path in any case. 4) Mmap cannot be used for random access for arbitrary files since they may be larger than the address space. This means some kind of window needs to be implemented - and this is easily done with read/write. - Einar Karttunen