Hi cafe, I am happy to announce a new release of Déjà Fu, my library for testing concurrent Haskell programs! Things have advanced significantly since the paper I presented at the Haskell Symposium this year, both in terms of performance and functionality. Git: https://github.com/barrucadu/dejafu Hackage: https://hackage.haskell.org/package/dejafu-0.2.0.0 ## What is it? Déjà Fu is a library for developing and testing concurrent Haskell programs, it provides a typeclass-abstraction over GHC's regular concurrency API, allowing the concrete implementation to be swapped out. Why do we need this? Well, concurrency is really hard to get right. Empirical studies have found that many real-world concurrency bugs can be exhibited with small test cases using as few as two threads: so it’s not just big concurrent programs that are hard, small ones are too. We as programmers just don’t seem to have a very good intuition for traditional threads-and-shared-memory-style concurrency. The typical approach to testing concurrent programs is to just run them lots of times, but that doesn’t provide any hard coverage guarantees, and then we need to wonder: how many runs do we need? Fortunately, there has been a lot of research into testing concurrency in the past several years. Systematic concurrency testing is an approach where the source of nondeterminism, the actual scheduler, is swapped out for one under the control of the testing framework. This allows possible schedules to be systematically explored, giving real coverage guarantees for our tests. This is a library implementing systematic concurrency testing. It provides two typeclasses, MonadConc to abstract over much of Control.Concurrent and related modules, and MonadSTM, to similarly abstract over Control.Monad.STM. For examples, see the test suite and the async-dejafu source: - https://github.com/barrucadu/dejafu/tree/master/dejafu-tests - https://github.com/barrucadu/dejafu/tree/master/async-dejafu If you want to read more: - Recent technical report: http://misc.barrucadu.co.uk/pub/dejafu-techreport.pdf - Haskell Symposium paper about version 0.1.0.0: http://www.barrucadu.co.uk/publications/dejafu-hs15.pdf - A slew of things on my blog: http://www.barrucadu.co.uk/posts.html ## Changelog New stuff: - Relaxed memory: CRefs now behave just like IORefs, and testing can use one of three different memory models: sequential consistency, total store order (the default), and partial store order. - Support for 'yield'. - Integration with HUnit and tasty: https://hackage.haskell.org/package/hunit-dejafu https://hackage.haskell.org/package/tasty-dejafu - Version of the async library reimplemented to use MonadConc: https://hackage.haskell.org/package/async-dejafu - Fair bounding: which allows yield-based constructs like spinlocks to be tested without causing infinitely-long executions. - Length bounding: which allows for potentially non-terminating computations to be tested. Improvements: - Now uses partial-order reduction to cut down on the number of schedules to try, which is a *huge* improvement over just pre-emption bounding, which is what 0.1.0.0 did. - The 'autocheck' function has prettier output. ## How to use it: If you’re not making use of any IO in your code other than for concurrency, the transition to using MonadConc and MonadSTM will probably just be a textual substitution: - IO is replaced with MonadConc m => m - STM with MonadSTM m => m - *IORef with *CRef - *MVar with *CVar - *TVar with *CTVar - Most other things have the same name, and so can be replaced by just swapping imports around. If you are using other IO, you will need a gentle sprinkling of MonadIO and liftIO in your code as well. ## Is this really just a drop-in replacement for IO/STM? That’s the idea, yes. More specifically, the IO instance of MonadConc and the STM instance of MonadSTM just use the regular IO and STM functions, and so should have no noticeable change in behaviour. There are some differences which can lead to incorrect results when testing, but which should not affect code when used in an IO or STM context. Specifically: Control.Monad.Conc.Class.getNumCapabilities can lie to encourage more concurrency when testing; and Control.Exception.catch can catch exceptions from pure code, but Control.Monad.Conc.Class.catch can’t (except for the IO instance). -- Michael Walker (http://www.barrucadu.co.uk)