
Hello, We wrote a small library (1) which offers a few extra synchronization primitives. These primitives are found in the standard libraries of languages like Java and Python, but not in Haskell. Before releasing concurrent-extra on hackage, we would like some comments on its name, design, implementation, documentation (2) and general usefulness. All primitives are implemented with MVars and IORefs, except for Control.Concurrent.STM.Event. There is a small test suite which tests some properties: cabal configure -ftest cabal build cabal test The test suite still needs to be expanded a great deal. We found that testing concurrent algorithms is really hard. Now we test things like "this shouldn't deadlock" by checking if it doesn't take too long. But that doesn't feel very formal. Is there a more principled approach to testing concurrent algorithms in Haskell? After writing our version of a read-write lock we noticed the package rwlock (3) on hackage. But there is a difference: rwlock is implemented with STM while our version is implemented entirely with MVars. Regards, Roel & Bas van Dijk 1 - http://code.haskell.org/~roelvandijk/code/concurrent-extra/ 2 - http://code.haskell.org/~roelvandijk/code/concurrent-extra/doc/html/concurre... 3 - http://hackage.haskell.org/package/rwlock

Hi, I had a look at the code for Event (both versions) and Lock (but not the others just yet) and it seemed fine. If you do lots of calls to waitTimeout before a set you will accumulate old locks in the list, but that won't cause any error that I can see, so it would only be a problem in pathological cases. I'm not sure there is a good way to test MVar algorithms. One way to do it (which the Concurrent Haskell Debugger did online, IIRC: http://www.informatik.uni-kiel.de/~fhu/chd/) is to reimplement IO to explore different interleavings of the various MVar calls to look for deadlocks. If you're testing STM, generally you can look to capture invariants of your transactions and check that they hold after each transaction (e.g. that if the state is Set, there shouldn't be any locks waiting with an event). Thanks, Neil. Roel van Dijk wrote:
Hello,
We wrote a small library (1) which offers a few extra synchronization primitives. These primitives are found in the standard libraries of languages like Java and Python, but not in Haskell.
Before releasing concurrent-extra on hackage, we would like some comments on its name, design, implementation, documentation (2) and general usefulness.
All primitives are implemented with MVars and IORefs, except for Control.Concurrent.STM.Event.
There is a small test suite which tests some properties:
cabal configure -ftest cabal build cabal test
The test suite still needs to be expanded a great deal. We found that testing concurrent algorithms is really hard. Now we test things like "this shouldn't deadlock" by checking if it doesn't take too long. But that doesn't feel very formal. Is there a more principled approach to testing concurrent algorithms in Haskell?
After writing our version of a read-write lock we noticed the package rwlock (3) on hackage. But there is a difference: rwlock is implemented with STM while our version is implemented entirely with MVars.
Regards, Roel & Bas van Dijk
1 - http://code.haskell.org/~roelvandijk/code/concurrent-extra/ 2 - http://code.haskell.org/~roelvandijk/code/concurrent-extra/doc/html/concurre... 3 - http://hackage.haskell.org/package/rwlock _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

2010/2/16 Neil Brown
I had a look at the code for Event (both versions) and Lock (but not the others just yet) and it seemed fine. If you do lots of calls to waitTimeout before a set you will accumulate old locks in the list, but that won't cause any error that I can see, so it would only be a problem in pathological cases.
I think I can fix the garbage locks on waitTimeout by tupling each lock with the ThreadId of the thread that created it. When a timeout occurs I can then simply remove the unnecessary lock from the list. The extra overhead would be the construction of a tuple and acquiring a ThreadId each time you wait for an event.
I'm not sure there is a good way to test MVar algorithms. One way to do it (which the Concurrent Haskell Debugger did online, IIRC: http://www.informatik.uni-kiel.de/~fhu/chd/) is to reimplement IO to explore different interleavings of the various MVar calls to look for deadlocks. If you're testing STM, generally you can look to capture invariants of your transactions and check that they hold after each transaction (e.g. that if the state is Set, there shouldn't be any locks waiting with an event).
Interesting. It reminds me of Wouter Swierstra's recent paper "Beauty in the Beast": http://www.cs.nott.ac.uk/~txa/publ/beast.pdf Thanks, Roel

Roel van Dijk wrote:
2010/2/16 Neil Brown
: I had a look at the code for Event (both versions) and Lock (but not the others just yet) and it seemed fine. If you do lots of calls to waitTimeout before a set you will accumulate old locks in the list, but that won't cause any error that I can see, so it would only be a problem in pathological cases.
I think I can fix the garbage locks on waitTimeout by tupling each lock with the ThreadId of the thread that created it. When a timeout occurs I can then simply remove the unnecessary lock from the list. The extra overhead would be the construction of a tuple and acquiring a ThreadId each time you wait for an event.
You don't need to do use ThreadId: MVar has an Eq instance, so you could make your Lock type derive an Eq instance, and then you can just compare the Locks to remove it after the timeout occurs (e.g. using delete to take it out of the list; it should be quite near the head of the list anyway). In fact, you may as well make most of your types derive an Eq instance where possible, as this can be useful sometimes. Thanks, Neil.

2010/2/17 Neil Brown
You don't need to do use ThreadId: MVar has an Eq instance, so you could make your Lock type derive an Eq instance, and then you can just compare the Locks to remove it after the timeout occurs (e.g. using delete to take it out of the list; it should be quite near the head of the list anyway). In fact, you may as well make most of your types derive an Eq instance where possible, as this can be useful sometimes.
Now I am wondering why I didn't think of that before. It's an elegant solution. Thanks!

On 16/02/2010 15:51, Roel van Dijk wrote:
Hello,
We wrote a small library (1) which offers a few extra synchronization primitives. These primitives are found in the standard libraries of languages like Java and Python, but not in Haskell.
Before releasing concurrent-extra on hackage, we would like some comments on its name, design, implementation, documentation (2) and general usefulness.
All primitives are implemented with MVars and IORefs, except for Control.Concurrent.STM.Event.
There is a small test suite which tests some properties:
cabal configure -ftest cabal build cabal test
The test suite still needs to be expanded a great deal. We found that testing concurrent algorithms is really hard. Now we test things like "this shouldn't deadlock" by checking if it doesn't take too long. But that doesn't feel very formal. Is there a more principled approach to testing concurrent algorithms in Haskell?
You might want to take a look at the concurrency part of the GHC test suite: http://darcs.haskell.org/testsuite/tests/ghc-regress/concurrent/should_run/ Not that we've really solved the problem you're talking about, but you might get some ideas. Often deadlocks are caught by the RTS and result in an exception which you can catch, though with the threaded RTS this relies on detecting idle time and running the GC, which by default happens after 0.3s of idle time. Some people find this annoying and turn off the idle-time GC (+RTS -I0) but then you lose deadlock detection. Swings and roundabouts. Cheers, Simon

2010/2/16 Simon Marlow
You might want to take a look at the concurrency part of the GHC test suite:
http://darcs.haskell.org/testsuite/tests/ghc-regress/concurrent/should_run/
Not that we've really solved the problem you're talking about, but you might get some ideas.
The method of testing appears to be similar to what I do now using unit tests. But the contents of the tests are interesting. I'll see if they are applicable to our primitives. Thanks, Roel
participants (3)
-
Neil Brown
-
Roel van Dijk
-
Simon Marlow