I don't like runAtomically either, because once again it assumes a global lock. This is fine for GHC or Hugs or NHC on single processors, but it would be a pain if you had multiple processors. You can't avoid locking between processors altogether of course; atomicModifyPVars and the unsafePerformIO hack I just posted require data to be updated consistently (you don't want the same unsafePerformIO being evaluated twice). However that is not so bad because the locking is only on particular bits of data; it's reasonable that access to particular thunks or PVars will be "owned" by one processor or another, so that if two processors are working on independent bits of state they can chug along perfectly happily without locking each other, and only need to talk when one processor wants to get at a bit of state owned by another. Although Alastair suggests compilers might try to spot when runAtomically can be used safely without stopping all but one processor, I think analysis of IO actions to check for potential conflicts might be even harder than getting Hugs to run Haskell finalizers 8-) It would require region analysis and probably worse. However we don't really need to discuss this anyway, since I don't think either runAtomically or atomicallyModifyIORef need to be in the FFI standard. I'm quite happy to leave this open.
However we don't really need to discuss this anyway, since I don't think either runAtomically or atomicallyModifyIORef need to be in the FFI standard. I'm quite happy to leave this open.
As usual, I disagree. I think the FFI spec would be incomplete if it provided Haskell finalizers but no mechanism to write them safely. -- Alastair
Alastair Reid wrote:
However we don't really need to discuss this anyway, since I don't think either runAtomically or atomicallyModifyIORef need to be in the FFI standard. I'm quite happy to leave this open.
As usual, I disagree. I think the FFI spec would be incomplete if it provided Haskell finalizers but no mechanism to write them safely.
Since we've talked about mutable state quite a lot, my suggestion would be that we write, in addition to the FFI specification, a Mutable State specification which documented newIORef, readIORef, writeIORef, atomicModifyIORef (and possibly, for reasons of efficiency, atomicModifyIORef_). I don't think it need be very long. It wouldn't have to be frozen right away; it would be good if some other working group could be formed to carry it further, so that for example it could also include mutable arrays.
Since we've talked about mutable state quite a lot, my suggestion would be that we write, in addition to the FFI specification, a Mutable State specification which documented newIORef, readIORef, writeIORef, atomicModifyIORef (and possibly, for reasons of efficiency, atomicModifyIORef_). I don't think it need be very long. It wouldn't have to be frozen right away; it would be good if some other working group could be formed to carry it further, so that for example it could also include mutable arrays.
It's already stable, well documented, etc. (and has been for something like 6 years now). We can reformat the Hugs-GHC library docs into Haskell Report style and give it the official Haskell committee seal of approval if it makes you feel any better but the spec won't change at all in the process so it doesn't alter any of the questions currently on the table. -- Alastair
Alastair Reid wrote:
Since we've talked about mutable state quite a lot, my suggestion would be that we write, in addition to the FFI specification, a Mutable State specification which documented newIORef, readIORef, writeIORef, atomicModifyIORef (and possibly, for reasons of efficiency, atomicModifyIORef_). I don't think it need be very long. It wouldn't have to be frozen right away; it would be good if some other working group could be formed to carry it further, so that for example it could also include mutable arrays.
It's already stable, well documented, etc. (and has been for something like 6 years now). We can reformat the Hugs-GHC library docs into Haskell Report style and give it the official Haskell committee seal of approval if it makes you feel any better
Good, then I suggest that's what we do (with the addition of atomicModifyIORef).
but the spec won't change at all in the process so it doesn't alter any of the questions currently on the table. Well, we can take it as settling the question of whether it is possible to do useful communication between finalizers and the rest of the Haskell world in a portable and useful way. It seems to me that the only remaining question we need to answer is whether Haskell finalizers can and should be implemented at all.
I don't like runAtomically either, because once again it assumes a global lock.
It has to be a global lock if we are to implement it on NHC and Hugs. Since we can't block finalizers once they start running, the only way to get atomicity is to stop them running. This requires use of a global lock unless we had (in advance, before the finalizer starts running) a list of the set of objects that the finalizer is going to access. It'd be hard to do this while treating libraries as black boxes.
This is fine for GHC or Hugs or NHC on single processors, but it would be a pain if you had multiple processors.
By 'pain' you mean slow? Since > 90% of uses of runAtomically will be with modifyIORef, we can avoid this overhead by providing atomicModifyIORef in the IORef library as well. Multiprocessor GHC is free to implement it more efficiently if necessary/ convenient. -- Alastair
Alastair Reid wrote: [snip]
This is fine for GHC or Hugs or NHC on single processors, but it would be a pain if you had multiple processors.
By 'pain' you mean slow? Yes.
Since > 90% of uses of runAtomically will be with modifyIORef, we can avoid this overhead by providing atomicModifyIORef in the IORef library as well. Multiprocessor GHC is free to implement it more efficiently if necessary/ convenient. [snip] I'm not convinced the overhead is going to be all in modifyIORef. For one thing runAtomically once again assumes the existence of some linear ordering on all state operations performed by the program. I don't have any experience of programming for multiple processors so I don't know what repercussions it would have for the run-time system, but it seems possible that there would be some, even in stateful operations which have nothing to do with runAtomically. It seems to me that the majority of locking in a multi-processor system is going to be one-to-one; one processor wants permission to get at a bit of state owned by another processor. However you are asking for something different, where one processor wants to broadcast a stop-running message to every single other processor, at the same time as perhaps other processors are also trying to do the same thing (so you need some mechanism for arbitration). Of course it will be possible, but since Alastair seems to be happy for the IORef to contain atomicModifyIORef anyway, I don't see a pressing need for the additional primitive.
Alastair:
Since > 90% of uses of runAtomically will be with modifyIORef, we can avoid this overhead by providing atomicModifyIORef in the IORef library as well. Multiprocessor GHC is free to implement it more efficiently if necessary/ convenient.
[snip] I'm not convinced the overhead is going to be all in modifyIORef.
If you don't believe that runAtomically is necessary, you can't believe that we will ever call it. That being so, the performance overhead shouldn't concern you. (Implementation effort would be a different kettle of fish.)
For one thing runAtomically once again assumes the existence of some linear ordering on all state operations performed by the program.
It assumes that side effects can be linearized. Every theory of concurrency I know of makes the same assumption.
I don't have any experience of programming for multiple processors so I don't know what repercussions it would have for the run-time system, but it seems possible that there would be some, even in stateful operations which have nothing to do with runAtomically.
The only repercussion in a multiprocessor system is that calls to runAtomically must take a global lock. Global locks have been implemented enough times that the issues have been figured out many times.
Of course it will be possible, but since Alastair seems to be happy for the IORef to contain atomicModifyIORef anyway, I don't see a pressing need for the additional primitive.
The need is for the small number of cases where the state is spread over a number of mutable variables (IORefs, maybe some mutable arrays, maybe some C state) and we want to update it atomically. Even if George's unsafePerformIO trick is correct (I remain doubtful), its unlikely that Joe Haskell User will apply the trick correctly so it's better to provide them with a simple alternative. -- Alastair
Alastair Reid wrote: [snip]
It assumes that side effects can be linearized. Every theory of concurrency I know of makes the same assumption. [snip] I don't know if the Java Language Specification counts as a "theory of concurrency" but you will find if you check that side effects are *not* linearised in that language. This is also true of the Java Virtual Machine. Thus it would be a problem if, say, someone were to compile Haskell to JVM.
George Russell wrote:
Alastair Reid wrote: [snip]
It assumes that side effects can be linearized. Every theory of concurrency I know of makes the same assumption. [snip] I don't know if the Java Language Specification counts as a "theory of concurrency" but you will find if you check that side effects are *not* linearised in that language. This is also true of the Java Virtual Machine. Thus it would be a problem if, say, someone were to compile Haskell to JVM.
Having said that, of course JVM's do permit global locking, so you could implement runAtomically. The effect would be that while you wouldn't have a single global ordering of side effects, runAtomically would appear to work in any of the linear orderings as observed by the separate threads (or something like that, I'm not digging up the fiendishly complicated chapter 7 or 8 or whatever it is of the JVM specification at this time of night). I suppose I have to concede that runAtomically is at least implementable. I still don't think it belongs in the FFI specification though; as Simon M says we seem to be able to do everything anyone can think of using atomicModifyIORef. It's not that I don't think locking primitives are a good idea, but runAtomically seems to be an awfully crude one. It also incidentally would trash any functions which implicitly rely on, say, worker threads. For example, some optimisation algorithm could repeatedly try to improve on the best possible solution until interrupted by an Exception from a worker thread indicating that time was up. Of course you couldn't write such a function in NHC, but in GHC such a technique is potentially useful, and you wouldn't necessarily want to have to tell the user that the function could not be called from inside runAtomically. Could we not at least replace the global lock by a local one? For example newLock :: IO Lock synchronize :: Lock -> IO a -> IO a would specify a Java-like mechanism, by which synchronize would delay the action if necessary so that no two actions were simultaneously performed on the same lock. The big question would be, can this be implemented with Haskell finalizers in NHC? I think for NHC you would simply implement Alastair's runAtomically, and wrap synchronize in it. Then you would know that if in the middle of "synchronize lock action", there was another attempt to synchronize on "lock", the lock could not be due to a finalizer running inside action (since new finalizers are blocked and finalizers are properly nested), so must be due to action itself, so we must deadlock. Thus for NHC this is mildly more complicated (and inefficient) than having a global runAtomically. For Hugs/GHC we can easily implement a Lock as an MVar (). The only disadvantage I can see is that for NHC you are delaying new finalizers indefinitely during the duration of the synchronize'd action, but of course this is equally true of runAtomically in NHC. Where this solution wins is that you don't have to delay anything in Hugs or GHC except where the user explicitly requests it.
participants (2)
-
Alastair Reid -
George Russell