Patrick Caldon writes:

...

it takes about a second to run on a PC. It's functional except it whacks the rng, which needs IO. I run 5-10 of these jobs, and then use:

Which RNG are you using that it needs so much IO? Mersenne Twister, System.Random.Mersenne. The ordinary rng kills

Ivan Lazar Miljenovic wrote: performance. Patrick.

Neil Brown wrote:

Patrick Caldon wrote:

...

I'm looking for the "right" concurrency library/semantics for what should be a reasonably simple problem.

I have a little simulator:

runWorldSim :: MTGen -> SimState -> IO SimState

it takes about a second to run on a PC. It's functional except it whacks the rng, which needs IO. I run 5-10 of these jobs, and then use:

mergeWorld :: [SimState] -> SimState

to pick the best features of the runs and build another possible world (state). Then I use this new world to run another 5-10 jobs and so on. I run this through ~20000 iterations.

It's an obvious place for parallelism.

I'm looking for a concurrency library with something like:

forkSequence :: Int -> [IO a] -> IO [a]

which I could call with something like this:

forkSequence 4 (take 10 (repeat (runWorldSim g ss)))

this would construct 4 threads, then dispatch the 10 jobs onto the threads, and pack up the results into a list I could run through my merger.

Why particularly do you want to run the 10 jobs on 4 threads? Haskell's run-time is quite good at spreading out the lightweight threads onto all your cores, so the easiest thing to do is run the 10 jobs on 10 (light-weight) threads and let the run-time sort out the rest.

Thanks so much for that! I'll give it a go. Different threads is just because some of the jobs are memory hogs, and I want to minimize the number running simultaneously. I'll see what happens with a runPar-like approach, and use a queue-based approach if it becomes a problem.

So if what you want is a function:

runPar :: [IO a] -> IO [a]

you can easily construct this. Shameless plug: my CHP library effectively has this function already, runParallel :: [CHP a] -> CHP [a] (CHP being a slight layer on top of IO). But you can do it just as easily with, say, STM. Here is a version where order doesn't matter (apologies for the point-free style):

import Control.Concurrent import Control.Concurrent.STM import Control.Monad

modifyTVar :: TVar a -> (a -> a) -> STM () modifyTVar tv f = readTVar tv >>= writeTVar tv . f

runPar :: [IO a] -> IO [a] runPar ps = do resVar <- newTVarIO [] mapM_ (forkIO . (>>= atomically . modifyTVar resVar . (:))) ps atomically $ do res <- readTVar resVar when (length res < length ps) retry return res

If order does matter, you can zip the results with an index, and sort by the index afterwards. If efficiency matters, you can perform other tweaks. But the principle is quite straightforward. Or you can refactor your code to take the IO dependency out of your random number generation, and run the sets of pure code in parallel using the parallel library. If all you are using IO for is random numbers, that's probably the nicest approach.

Good, fast random numbers are unfortunately necessary - I had a nice implementation using System.Random, but had to rewrite it because performance was poor :( .

P.S. take 10 . repeat is the same as replicate 10

Thanks again! Patrick.

On Wed, Dec 9, 2009 at 2:17 PM, Mario Blazevic wrote:

       It appears there are several implementations existing on Hackage of the following function, in various disguises:

  runPar :: [IO a] -> IO [a]

the idea being that the IO computations are run in parallel, rather than sequentially. My own Streaming Component Combinators package contains a similar function, but somewhat generalized:

  class Monad m => ParallelizableMonad m where      parallelize :: m a -> m b -> m (a, b)

  instance ParallelizableMonad IO  -- implemented using forkIO   instance ParallelizableMonad Identity  -- implemented using par   instance ParallelizableMonad Maybe  -- implemented using par

       Would there be any interest in having this class packaged in a separate library? If so, can you sugest a better name or some additional functionality?

A similar function that I'm fond of: forkExec :: IO a -> IO (IO a) forkExec k = do result <- newEmptyMVar _ <- forkIO $ k >>= putMVar result return (takeMVar result) Although I don't think it can be generalized to non-IO monads. Antoine

It's a good thing then that forkExec and return are denotationally equal (though not operationally). Otherwise, I'd be worried. Matthew Brecknell wrote:

Antoine Latter wrote:

...

A similar function that I'm fond of:

forkExec :: IO a -> IO (IO a)

It's cute that forkExec already has a dual operation with just the right name (specialised to IO):

join :: IO (IO a) -> IO a

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Wed, Dec 9, 2009 at 3:44 PM, Mario Blazevic wrote:

       I can't test it right now, but wouldn't the following do the job in the Identity monad?

forkExec :: Identity a -> Identity (Identity a) forkExec k = let result = runIdentity k             in result `par` return (Identity result)

Since Identity is a newtype, would that be equivalent to "result `par` result"? The forkExec in the IO monad let's other computations keep going until I need the result from the forked computation. In a pure computation, I can already get the same result with `par` and laziness, right? Antoine