Thanks for the link; my ultimate interest, though, is in an architecture that could scale to multiple machines rather than multiple cores with shared memory on a single machine. Has there been any interest and/or progress in making DPH run on multiple machines and other NUMA architectures? Cheers, Greg On Apr 19, 2010, at 3:33 PM, Sebastian Sylvan wrote:

On Mon, Apr 19, 2010 at 11:03 PM, Gregory Crosswhite wrote: Hey everyone,

Has anyone done any work with bulk synchronous parallel computing in Haskell? The idea behind the model is that you divide your computation into a series of computation and communication phases, and it has recently occurred to me that this might be an ideal setup for parallelizing a pure language like Haskell because you can think of it as alternating between a stage that independently applies a bunch of functions to a bunch of independent chunks of data and a stage that applies a big function to all of the chunks that recombines them into new chunks for the next parallel phase, so that all stages are conceptually pure even if eventually the second stage is turned into something involving communication and hence side-effectful under the hood.

Experiences? Thoughts?

You may want to check out NDP, e.g. here: http://www.haskell.org/haskellwiki/GHC/Data_Parallel_Haskell

It's at a higher level of abstraction, in a way. You don't need to worry about the dicing up and recombining, the compiler takes care of it for you. You just write things in terms of parallel arrays (which can be small, e.g. 2 element wide) and the compiler will fuse/flatten these together into big bulk parallel computations with communication between them.

-- Sebastian Sylvan

On Apr 20, 2010, at 11:05 AM, Jason Dusek wrote:

...

Thanks for the link; my ultimate interest, though, is in an architecture that could scale to multiple machines rather than multiple cores with shared memory on a single machine. Has there been any interest and/or progress in making DPH run on multiple machines and other NUMA architectures?

I wonder what it would take to do this.

Erlang. ;0)

On Tue, Apr 20, 2010 at 14:05, Jason Dusek wrote:

One approach is some compiler "magic" that provides you with an RTS that can communicate with other RTSen over TCP and chunks the computation "appropriately".

The approaches to Haskell multi-host parallelism I've seen all seem to be (a) dead and (b) overly complicated, which I can't help but suspect are related. I don't need a tool that automatically figures out how to distribute any workload in an intelligent way and handles all the communication for me. If I have the basic building block, which is the ability to serialize a Haskell expression with its dependencies and read them into another Haskell instance where I can evaluate them, I can handle the other pieces, which are - passing strings back and forth in whatever way is convenient - deciding how to divide up my workload. In the Ruby universe, DRb combines the serialization and "passing strings around" job and lets me figure out how to divide up the work, and it would be delightful if there were something similarly simple in the Haskell world.

On Wed, Apr 21, 2010 at 14:14, Don Stewart wrote:

Eden is active, afaik,

   http://www.mathematik.uni-marburg.de/~eden/

Unfortunately, Eden is one of the examples I had in mind when referring to distributed Haskell projects as overly complicated and [for practical purposes] dead. Their last release available for download was in 2006. Their beta is "available upon request", which doesn't raise my confidence in the level of active development or openness of the project. From the notes here:

They just had a hackathon in St Andrews,

   http://hackage.haskell.org/trac/ghc/wiki/HackPar

they don't seem to have even a source code repository yet, and they appear to be bogged down in that complexity I mentioned. It looks like they want to engage large, academically interesting problems and produce papers, as opposed to producing small tools that solve simple problems in simple ways. The practical solution, as a result, continues to be "use another language".

On Wed, Apr 21, 2010 at 8:07 PM, Aaron D. Ball wrote:

If I have the basic building block, which is the ability to serialize a Haskell expression with its dependencies and read them into another Haskell instance where I can evaluate them, I can handle the other pieces

How I wish we had Clean's dynamics[1]! regards, Bas [1] See section 8 of the Clean report: http://clean.cs.ru.nl/download/Clean20/doc/CleanLangRep.2.1.pdf

On Thu, Apr 22, 2010 at 4:07 AM, Aaron D. Ball wrote:

I don't need a tool that automatically figures out how to distribute any workload in an intelligent way and handles all the communication for me.  If I have the basic building block, which is the ability to serialize a Haskell expression with its dependencies and read them into another Haskell instance where I can evaluate them, I can handle the other pieces, which are

- passing strings back and forth in whatever way is convenient - deciding how to divide up my workload.

Alice/ML is the place to look for this technology. http://www.ps.uni-saarland.de/alice/ The project may be dead (I don't know), but they did have the most sophisticated take on pickling that I've seen. It's an ML variant, with futures, running on top of the same platform used by Mozart/Oz of CTM fame. Why do you want to pass strings around? cheers peter