- Haskell-Cafe - Haskell.org

RE: [Haskell-cafe] Re: Hugs vs GHC (again)was: Re: Somerandomnewbiequestions
by Simon Marlow 20 Jan '05

20 Jan '05

On 11 January 2005 14:15, Gracjan Polak wrote: > Simon Marlow wrote: > > There's a big lock on File. If you want to do truly concurrent > reading, > you can make multiple FileInputStreams, each of which has > its own file > descriptor (the Unix implementation uses dup(2)). > > > > Original and descriptor returned by dup or dup2 share file pointer. *blink* You're right! Serves me right for assuming that POSIX would have sensible semantics. Perhaps this API isn't implementable, in its current state. Cheers, Simon

10 26

RE: [Haskell-cafe]Re: Hugsvs GHC (again)was: Re: Somerandomnewbiequestions
by Simon Marlow 20 Jan '05

20 Jan '05

On 20 January 2005 10:01, Keean Schupke wrote: > Simon Marlow wrote: > >> We're getting a bit confused here. Keean: the original question was >> about whether a disk read will stop all other *Haskell* threads. >> Not OS threads. The two are quite different beasts in GHC. >> >> Cheers, >> Simon >> >> > But if GHC is running with the -threaded flag, then other > Haskell-threads can keep running using the second OS-thread, even > though one Haskell-thread (and its associated OS thread) is blocking > - right? If the call to read() was "safe" (i.e. the FFI "safe" attribute), then yes. But GHC currently does and "unsafe" call to read(), even with -threaded, which doesn't give other threads a chance to run. Perhaps it should be a safe call. That's going to hurt performance unless you really need to parallelise disk I/O, though. Maybe it wouldn't hurt too much, someone should measure it. > In other words even with disk IO (as I said the kernel would not > busy-wait - so there are > only two options, this OS-thread is put to sleep, or the data is > already in a buffer and is > returned immediately), all the other Haskell-threads should not block > (provided it is > running with -threaded) > > Have I got that right? Yes, except that you forgot that not all foreign calls can run concurrently with Haskell code. Only the "safe" ones can. Cheers, Simon

2 1

RE: [Haskell-cafe] Re: Hugsvs GHC (again)was: Re: Somerandomnewbiequestions
by Simon Marlow 20 Jan '05

20 Jan '05

On 20 January 2005 09:56, Keean Schupke wrote: > Why is disk a special case? I have never heard that all processes > under linux wait for a disk read... You were talking about Haskell threads, not processes! These are quite different things. > The kernel most certainly does not busy wait for disks > to respond, so the only alternative is that the process that needs to > wait (and only that process) is put to sleep. In which case a second > thread would be unaffected. > > Linux does not busy wait in the Kernel! (don't forget the kernel does > read-ahead, > so it could be that read really does return 'immediately' and without > any delay > apart from at the end of file - In which case asynchronous IO just > slows you down with extra context switches). The question is whether Haskell threads block when one of them is reading from the disk, and this has nothing to do with what happens in the kernel. I never said that Linux busy waits in the kernel. The reason that disk I/O is different is because select() treats it differently, and select() is what GHC's runtime currently uses to multiplex I/O. Cheers, Simon

2 1

RE: [Haskell-cafe]Re: Hugsvs GHC (again)was: Re: Somerandomnewbiequestions
by Simon Marlow 20 Jan '05

20 Jan '05

On 19 January 2005 20:31, Glynn Clements wrote: > Keean Schupke wrote: > >>>> Okay, my ignorance of Posix is showing again. Is it currently the >>>> case, then, that every GHC thread will stop running while a disk >>>> read is in progress in any thread? Is this true on all platforms? >>> >>> It's true on Unix-like systems, I believe. Even with -threaded. It >>> might not be true on Win32. >> >> I think this is not true on linux, where a thread is just a process >> created with special flags to keep the same fds and memory. >> >> As threads on linux are scheduled like processes, one thread >> blocking should not affect the others? > > That should be true of all POSIX-like thread implementations > (including Linux, whose threads aren't quite POSIX-compliant, e.g. in > regard to signal handling, but aren't that far off). > > Essentially, blocking system calls only block the calling kernel > thread. > > OTOH, if you are implementing multiple user-space threads within a > single kernel thread, if that kernel thread blocks, all of the > user-space threads within it will be blocked. We're getting a bit confused here. Keean: the original question was about whether a disk read will stop all other *Haskell* threads. Not OS threads. The two are quite different beasts in GHC. Cheers, Simon

2 1

RE: [Haskell-cafe]Re: Hugsvs GHC (again)was: Re: Somerandomnewbiequestions
by Simon Marlow 20 Jan '05

20 Jan '05

On 19 January 2005 20:33, Glynn Clements wrote: > Simon Marlow wrote: > >> We do use a thread pool. But you still need as many OS threads as >> there are blocked read() calls, unless you have a single thread >> doing select() as I described. > > How does the select() help? AFAIK, select() on a regular file or block > device will always indicate that it is readable, even if a subsequent > read() would have to read the data from disk. Again we've lost track of the discussion a bit. Yes you're right, but I was answering a different question. Cheers, Simon

1 0

Re: Why is getArgs in the IO monad?
by Ashley Yakeley 20 Jan '05

20 Jan '05

In article <41ECD49F.9020601(a)imperial.ac.uk>, Keean Schupke <k.schupke(a)imperial.ac.uk> wrote: > Surely both requirements can be satisfied if the programs arguments are made > parameters of main: > > main :: [String] -> IO () > > Keean. Better yet, it should be an implicit parameter so as not to break existing programs. main :: (?args :: [String]) => IO () You could do the same with standard input and output. -- Ashley Yakeley, Seattle WA

6 13

RE: [Haskell-cafe] Re: Hugsvs GHC (again)was: Re: Somerandomnewbiequestions
by Simon Marlow 19 Jan '05

19 Jan '05

We do use a thread pool. But you still need as many OS threads as there are blocked read() calls, unless you have a single thread doing select() as I described. BTW our Haskell Workshop paper from last year describes this stuff: http://www.haskell.org/~simonmar/papers/conc-ffi.ps.gz Cheers, Simon On 19 January 2005 15:07, Keean Schupke wrote: > Why not use a thread-pool, and a "safe" call to read, provided there > is an OS thread available, > defaulting to "unsafe" if no thread is available... You could make the > thread pool size an argument... > > Keean. > > Simon Marlow wrote: > >> On 19 January 2005 13:50, William Lee Irwin III wrote: >> >> >> >>> On 19 January 2005 09:45, Ben Rudiak-Gould wrote: >>> >>> >>>>> Okay, my ignorance of Posix is showing again. Is it currently the >>>>> case, then, that every GHC thread will stop running while a disk >>>>> read is in progress in any thread? Is this true on all platforms? >>>>> >>>>> >>> On Wed, Jan 19, 2005 at 01:39:05PM -0000, Simon Marlow wrote: >>> >>> >>>> It's true on Unix-like systems, I believe. Even with -threaded. >>>> It might not be true on Win32. >>>> >>>> >>> How does forkOS fit into this picture? It's described in the >>> documentation as allowing concurrent execution of system calls >>> and other activity by other threads. >>> >>> >> >> forkOS doesn't fix this. It forks another OS thread which can be >> used to make concurrent foreign calls, if they are not marked >> "unsafe". However, the standard I/O library, in -threaded mode, does >> read like this: >> >> - non-blocking, "unsafe", read() to see what's there >> - if read() would block, then hand off to another >> Haskell thread which does select() on all the outstanding >> IO requests. >> >> This scheme is just for efficiency. We could (and used to) just call >> "safe" read() for every read - that would give you the right >> concurrency with -threaded, but unfortunately you'd really notice >> the difference if you had 1000s of threads all doing IO, because >> each one would need its own OS thread. The current scheme is rather >> snappy (even snappier than non-threaded, as it happens). >> >> You can always do System.Posix.fileRead to get around it. >> >> Cheers, >> Simon >> _______________________________________________ >> Haskell-Cafe mailing list >> Haskell-Cafe(a)haskell.org >> http://www.haskell.org/mailman/listinfo/haskell-cafe

3 2

RE: [Haskell-cafe] Re: Hugsvs GHC (again)was: Re: Somerandomnewbiequestions
by Simon Marlow 19 Jan '05

19 Jan '05

On 19 January 2005 09:45, Ben Rudiak-Gould wrote: > Glynn Clements wrote: > > >Ben Rudiak-Gould wrote: > > > >>GHC really needs non-blocking > >>I/O to support its thread model, and memory-mapped I/O always > blocks. > > >If, by "blocks", you mean that execution will be suspended until the > >data has been read from the device into the buffer cache, then Unix > >non-blocking I/O (i.e. O_NONBLOCK) also blocks. > > Okay, my ignorance of Posix is showing again. Is it currently the > case, then, that every GHC thread will stop running while a disk read > is in progress in any thread? Is this true on all platforms? It's true on Unix-like systems, I believe. Even with -threaded. It might not be true on Win32. > (By the way, are the GHC folks aware that the description of Win32 I/O > at [2] is wrong? It seems to assume that ReadFile doesn't return until > the buffer is full.) I don't think that description corresponds with the implementation, but then I'm not terribly familiar with that part of the code. It doesn't get used when -threaded is on. Cheers, Simon

4 3

tree with labeled edges as a monad
by Andrew Pimlott 19 Jan '05

19 Jan '05

This is a "have you seen this monad?" post. I was trying to construct a search tree, and decided I wanted to do it in a monad (so I could apply StateT and keep state as I explored the space). I discovered that a tree with labeled leaves is a monad, but I wanted to label internal nodes, and such a tree (eg, Data.Tree.Tree) is not a monad (because it can only have one result type). Finally, I realized I could get a similar effect by labeling the edges and the leaves with different types: data Tree l a = Leaf a | Branch [(l, Tree l a)] instance Monad (Tree l) where return = Leaf Leaf a >>= f = f a Branch c >>= f = Branch [(l, t >>= f) | (l, t) <- c] You might use it as turn = do board <- getBoard move <- lift (Branch [(move, return move) | move <- findMoves board]) applyMove move turn I found this quite pleasing, though I hadn't run across trees as monads before. Has anyone else found this useful? Is it in a library somewhere? Andrew

3 2

RE: [Haskell-cafe] Re: Hugsvs GHC (again)was: Re: Somerandomnewbiequestions
by Simon Marlow 19 Jan '05

19 Jan '05

On 19 January 2005 16:49, Keean Schupke wrote: > But what about the continuing computation... we do not want > the fastest IO system, but we want the program to comlete the > fastest... So ideally we want 2 threads! > > One runs the Haskell code that is not waiting for IO. (IE other > Haskell threads)... The other runs a select loop as you suggest! > > This way the number of threads is fixed (2) and execution never > 'blocks' for IO. (Simon, what about this scheme?) This is what GHC does, if I understand you correctly. The thread running select() does so in its own OS thread, while another OS thread runs the Haskell code. As long as you use -threaded, that is. Oh, and before GHC 6.4 it was done a different way - the scheduler used to do the select() between running Haskell threads. Cheers, Simon

2 1