On Fri, Sep 28, 2012 at 1:01 PM, Greg Fitzgerald <garious@gmail.com> wrote:

I'm new to concurrent programming in Haskell. I'm looking for a
drop-in replacement for 'mapM' to parallelize a set of independent IO
operations. I hoped 'mapConcurrently' might be it, but I need
something that will only spawn as many threads as I have CPUs
available [1].

I also tried Control.Parallel.Strategies [2]. While that route works,
I had to use unsafePerformIO. Considering that IO is for sequencing
effects and my IO operation doesn't cause any side-effects (besides
hogging a file handle), is this a proper use of unsafePerformIO?

Attempt 1
--------------

import System.Process(readProcess)
import Control.Concurrent.Async(mapConcurrently)

main :: IO [String]
main = mapConcurrently (\n -> readProcess "echo" ["test: " ++ show n]
"") [0..1000]

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.6.1

$ runghc test.hs
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: runInteractiveProcess: pipe: Too many open files
test.hs: echo: createProcess: resource exhausted (Too many open files)

Attempt 2
--------------

import System.Process(readProcess)
import Control.Parallel.Strategies(parMap, rpar)
import System.IO.Unsafe(unsafePerformIO)

main :: IO [String]
main = myMapConcurrently (\n -> readProcess "echo" ["test: " ++ show
n] "") [0..1000]
where
myMapConcurrently f = return . parMap rpar (unsafePerformIO . f)

$ runghc test.hs > /dev/null && echo Success
Success

Thanks,
Greg

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe