On 13/06/2011 15:23, Henning Thielemann wrote:
On Thu, 19 May 2011, Simon Marlow wrote:
I've put together a tutorial on Parallel and Concurrent programming in Haskell, here:
http://community.haskell.org/~simonmar/par-tutorial.pdf
The main reason for writing this was that I needed some lecture notes for a course at the CEFP summer school next month, but I hope the material is useful to the Haskell community in general.
Thank you a lot for this overview!
Concerning the parallelisation of K-means: It seems to be difficult to choose a size for the pieces of points.
In practice it's not too sensitive - as long as the value is much larger than the number of cores, but less than the spark pool size (4096), you should be ok.
Since the 'combine' function is associative something map-reduce-like seems to be applicable. That is, if there is a library function for a parallelised 'reduce' and I promise, that I call it with an associative function as argument, then the library could choose an appropriate size for the pieces. Is there already library support for parallelism of this style?
Not really, but there ought to be. Various people have looked at this problem, but not much in Haskell (look at "Lazy Tree Splitting" for example). In practice you usually want to add a cut-off point to the tree below which you stop trying to parallelise. Cheers, Simon