(I keep forgetting to correctly fill out the To: and Cc: fields before sending my reply... so here's the copy that should have been sent to the list 10 minutes ago...) Daan Leijen wrote:
Hi Wolfgang,
I feel like you are beating my proposal to death here, and I find it hard to react individually to all your remarks.
Sorry, maybe I shouldn't shoot all my arguments at you at once :-)
I'll try to focus on the main issue:
You are worried that the forkOS and forkIO distinction is too primitive and that it rule out sophisticated scheduling on SMP processors for example.
That is closely related to the point that I don't want implementation details (that several Haskell threads can run in one OS thread) to be part of a defined interface that has to be supported by all implementations. My main point is probably my strong phobia of "safe" calls: they are not safe to use. FFI newbies are repeatedly getting bitten by the fact that "safe" calls block everything else. There is no "natural" reason for a call to block other threads, it's a consequence of an implementation detail (lightweight threads). The intention behind threadsafe is to make this implementation detail invisible, and therefore harmless. The forkOS/forkIO proposal seems to rely on having foreign calls that block other threads (that run in the same OS thread)... or doesn't it?
Maybe, the forkOS/forkIO approach is flawed, but I think we should only rule it out when we can provide a convincing example where only the keyword approach would work, and where we can't use combinators to achieve the same effect.
That's unfair ;-) --- I could also claim the reverse and say that we stick with threadsafe/bound until we have a convincing example... Using combinators sounds good, but there are things where combinators are not automatically the best choice (we're usually not using combinators to implement lazyness, either), and we don't know yet whether this is the case here or not. So let's just get on with the discussion....
About the example: [snip] That is amazing :-) [...]
Well, of course, it's "none of your business" to know. Different Haskell runtimes could use completely different schemes, and you wouldn't even notice. In GHC, the second OS thread is spawned earlier, the first time a threadsafe call is made. When the threadsafe call is made, GHC a) makes sure that there is a second OS thread available (some overhead the first time) and b) makes sure that all non-bound [in the current implementation: all] Haskell threads are executed by the second OS thread from now on. Now if a bound callback is invoked [not yet implemented], the bound callback is executed in the thread that the wrapper was called in (the first OS thread). All other Haskell threads (if any) continue to run in the second OS thread. If forkIO is called, the new Haskell thread is just added to the list of threads to be run by the second OS thread, no new OS thread has to be spawned, and forkIO doesn't have to know about threadsafe or about bound. If a non-bound callback is invoked [available now in the CVS HEAD], the callback is executed along with the other background Haskell threads in the second OS thread.
Since the RTS can't guess whether to use new OS threads or not at forkIO, I assumed that you could mark "wrapper" functions (callbacks) with a "threadsafe" attribute. If this is not the case, I don't understand how the implementation could work in this particular example.
It doesn't need to guess, "threadsafe" is just for imports, and it works anyway. See above, and if I haven't explained it clearly, try http://www.cse.unsw.edu.au/~chak/haskell/ghc/comm/rts-libs/multi- thread.html (which I really should update, it's outdated), or ask again. Maybe we need a more exact specification of how forkOS/forkIO should behave, especially with respect to foreign calls blocking other threads. Could you elaborate on how you would expect "normal" (safe) foreign calls to behave in different situations? Cheers, Wolfgang