[ANN](and feedback request) unagi-chan: Fast and scalable concurrent queues for x86, with a Chan-like API

I'm happy to finally release unagi-chan, an implementation of high-performance concurrent FIFO queues that have an API very similar to Control.Concurrent.Chan. You can see benchmarks and documentation here: http://hackage.haskell.org/package/unagi-chan If you have a moment (especially if you're on a non-x86 architecture) please take 10 minutes and run the tests with: $ cabal configure --enable-tests $ cabal build $ time ./dist/build/test/test Thanks to Ryan Newton for helping answer some low-level questions about his atomic-primops package which provides the CAS and fetch-and-add implementations that are the core of unagi-chan. REQUEST FOR FEEDBACK: would anyone be interested in any functionality like the following: - concurrent builder for Text, something like: new :: IO (InTextChan , Lazy.Text) write :: String -> InTextChan -> IO () - something similar for ByteString (is there a class for types convertable to ByteString?) - concurrent Text and/or ByteString readers, something like: reader :: Text -> IO TextReader takeText :: Int -> TextReader -> IO Text That's very sketchy, but I ask because I'm very close to that already with Unagi.Unboxed, and should be able to implement something like any of the above without too much trouble if it would be useful. Brandon http://brandon.si

Hi, I get lots of import errors trying to build:
tests/Main.hs:19:8: Could not find module `IndexedMVar'
Changing that import line to 'import Utilities':
tests/Main.hs:18:8: Could not find module `Atomics'
Changing to 'import Data.Atomics':
tests/Main.hs:15:8: Could not find module `UnagiUnboxed'
and so on.
På Thu, 10 Jul 2014 20:39:00 +0200, skrev Brandon Simmons
I'm happy to finally release unagi-chan, an implementation of high-performance concurrent FIFO queues that have an API very similar to Control.Concurrent.Chan. You can see benchmarks and documentation here:
http://hackage.haskell.org/package/unagi-chan
If you have a moment (especially if you're on a non-x86 architecture) please take 10 minutes and run the tests with:
$ cabal configure --enable-tests $ cabal build $ time ./dist/build/test/test
Thanks to Ryan Newton for helping answer some low-level questions about his atomic-primops package which provides the CAS and fetch-and-add implementations that are the core of unagi-chan.
REQUEST FOR FEEDBACK: would anyone be interested in any functionality like the following: - concurrent builder for Text, something like: new :: IO (InTextChan , Lazy.Text) write :: String -> InTextChan -> IO ()
- something similar for ByteString (is there a class for types convertable to ByteString?)
- concurrent Text and/or ByteString readers, something like: reader :: Text -> IO TextReader takeText :: Int -> TextReader -> IO Text
That's very sketchy, but I ask because I'm very close to that already with Unagi.Unboxed, and should be able to implement something like any of the above without too much trouble if it would be useful.
Brandon http://brandon.si _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- Audun Skaugen

On Fri, Jul 11, 2014 at 3:32 AM, Audun Skaugen
Hi, I get lots of import errors trying to build:
Oops, I forgot to add some things to other-modules for the tests. Should be fixed in 0.1.0.2 now: cabal sandbox init cabal install --only-dependencies cabal configure --enable-tests cabal build test Thanks for the catch, Brandon
-- Audun Skaugen

On Thu, Jul 10, 2014 at 8:39 PM, Brandon Simmons < brandon.m.simmons@gmail.com> wrote:
I'm happy to finally release unagi-chan, an implementation of high-performance concurrent FIFO queues that have an API very similar to Control.Concurrent.Chan. You can see benchmarks and documentation here:
It's difficult to overstate how much faster unagi-chan is than
Control.Concurrent under contention, I'm seeing >10x improvements in the
100 readers/100 writers benchmark on this quad-core machine. I'll have to
try it on the 8-core at work. Performance seems to be predictably fast as
you scale the number of workers, unlike the others which asymptotically
explode during contention. Really impressive work, Brandon.
How close is unagi-chan to being a drop-in replacement for MVar? Since this
implementation seems to clearly be better, personally I would wonder if it
shouldn't just replace the existing one in the standard library.
Also: can you use the same technique to get a fast implementation of
bounded queues?
G
--
Gregory Collins

On Tue, Jul 15, 2014 at 2:44 AM, Gregory Collins
On Thu, Jul 10, 2014 at 8:39 PM, Brandon Simmons
wrote: I'm happy to finally release unagi-chan, an implementation of high-performance concurrent FIFO queues that have an API very similar to Control.Concurrent.Chan. You can see benchmarks and documentation here:
It's difficult to overstate how much faster unagi-chan is than Control.Concurrent under contention, I'm seeing >10x improvements in the 100 readers/100 writers benchmark on this quad-core machine. I'll have to try it on the 8-core at work. Performance seems to be predictably fast as you scale the number of workers, unlike the others which asymptotically explode during contention. Really impressive work, Brandon.
Thanks a lot. And thank you for trying it out.
How close is unagi-chan to being a drop-in replacement for MVar? Since this implementation seems to clearly be better, personally I would wonder if it shouldn't just replace the existing one in the standard library.
It's a drop-in replacement for all non-deprecated functions, except for the splitting of input and output ends of the chan and the handling of async exceptions in the reader. I wouldn't be comfortable advocating for it as a replacement, personally. Mostly because of a few questions and misgivings I have about some implementation details (especially related to memory model stuff). They're all documented in the source and I'm very comfortable with people using it in production, but only after running the test suite.
Also: can you use the same technique to get a fast implementation of bounded queues?
There's a pretty straightforward path to at least some sort of "fuzzy" bounded queue that just fails (rather than the writer block), which might end up as simply a `chanSize :: InChan a -> IO Int`. Please feel free to add to the discussion here: https://github.com/jberryman/unagi-chan/issues/1 Thanks again, Brandon
G -- Gregory Collins
participants (3)
-
Audun Skaugen
-
Brandon Simmons
-
Gregory Collins