parallel and distributed haskell?

It looks like there was a recent hackathon focusing on implementing distributed haskell. http://hackage.haskell.org/trac/ghc/wiki/HackPar I feel there is quite a bit of latent interest in the subject here, but relatively little active development (compared to erlang, clojure, etc.) Can anyone involved give a quick overview (or pointers to one)? It would be good to hear what directions people are taking, and why, and where it's going. Personally, I'd love to know the current thinking on a variety of subjects: * erlang-like port/actor implementations v. more implicit constructs * serialization * node failure/error handling * "grid" compatibility v. various ad-hoc cluster approaches. Anyone actively looking at this kind of question who would want to talk about it? Thanks, --ts

tswaterman:
It looks like there was a recent hackathon focusing on implementing distributed haskell. http://hackage.haskell.org/trac/ghc/wiki/HackPar
I feel there is quite a bit of latent interest in the subject here, but relatively little active development (compared to erlang, clojure, etc.)
Distributed Clojure?
Can anyone involved give a quick overview (or pointers to one)? It would be good to hear what directions people are taking, and why, and where it's going.
Personally, I'd love to know the current thinking on a variety of subjects:
* erlang-like port/actor implementations v. more implicit constructs * serialization * node failure/error handling * "grid" compatibility v. various ad-hoc cluster approaches.
Anyone actively looking at this kind of question who would want to talk about it?
The people at the HackPar are behind the main groups doing all the distributed Haskell work. Parallel/multicore Haskell is a separate thing, of course, and GHC is long established there. -- Don

Hello Scott, Wednesday, December 16, 2009, 10:21:51 PM, you wrote:
Can anyone involved give a quick overview (or pointers to one)?
GHC has great support for SMP systems. there is further work on parallel libraries and i believe that it was main part of HackPar. there are good serialization libraries, "binary" at very least. Distributed Haskell was implemented once about 10 years ago, but now there are no work in this area NB: i'm not expert in this area, just regular haskell user -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com

bulat.ziganshin:
Hello Scott,
Wednesday, December 16, 2009, 10:21:51 PM, you wrote:
Can anyone involved give a quick overview (or pointers to one)?
GHC has great support for SMP systems. there is further work on parallel libraries and i believe that it was main part of HackPar. there are good serialization libraries, "binary" at very least. Distributed Haskell was implemented once about 10 years ago, but now there are no work in this area
Distributed Haskell is active again, on a number of projects: http://hackage.haskell.org/trac/ghc/wiki/HackPar

On Dec 16, 8:21 pm, "Scott A. Waterman"
It looks like there was a recent hackathon focusing on implementing distributed haskell. http://hackage.haskell.org/trac/ghc/wiki/HackPar
I feel there is quite a bit of latent interest in the subject here, but relatively little active development (compared to erlang, clojure, etc.)
With cloud services on the rise, profound distribution support will be vital in the future. Fully ack.
Can anyone involved give a quick overview (or pointers to one)? It would be good to hear what directions people are taking, and why, and where it's going.
Personally, I'd love to know the current thinking on a variety of subjects:
* erlang-like port/actor implementations v. more implicit constructs * serialization * node failure/error handling
There has been some discussion lately on that topic.There are two limitations: - Exceptions can only be caught in impure code - The lack of an agreed model for "stack traces", i.e. where and why an error / exception occured (I blur the distinction here) This is only about error/exception handling. I would assume that a node failure is sthg. which should be handled by a supervising process.
* "grid" compatibility v. various ad-hoc cluster approaches.
I assume you know http://www.haskell.org/haskellwiki/Applications_and_libraries/Concurrency_an... There are libraries for actors and the the concurrency-library should support messages comparable to that of Erlang. Clearly, supporting applications (e.g. to monitor running "processes") are missing. While e.g. Erlang supports only one interaction pattern between processes, it was natural to develop a solid framework and supporting tools around that concept. AFAIK Haskell is exploring numerous ways to handle that challenge (see the link above),that's wy there is no "standard way" to handle concurrency and only very limited tools support for monitoring. Actually, are there any?
Anyone actively looking at this kind of question who would want to talk about it?
Well, Christmas is approaching. I wrote some simple Erlang style distributed toy applications and found the message passing / actor approach dead simple and effective. Would be nice if this model would be fully supported by eg. GHC. So add me to the wish-list ;)
Thanks, --ts
_______________________________________________ Haskell-Cafe mailing list Haskell-C...@haskell.orghttp://www.haskell.org/mailman/listinfo/haskell-cafe

On 16/12/2009 19:21, Scott A. Waterman wrote:
It looks like there was a recent hackathon focusing on implementing distributed haskell. http://hackage.haskell.org/trac/ghc/wiki/HackPar
I feel there is quite a bit of latent interest in the subject here, but relatively little active development (compared to erlang, clojure, etc.) Can anyone involved give a quick overview (or pointers to one)? It would be good to hear what directions people are taking, and why, and where it's going.
The main directions are: GUM, which was one of the first parallel implementations of Haskell many years ago [1]. The programming API is the same as GHC has: par/pseq and strategies (indeed this API was invented in the context of GUM, we just re-used it in GHC). GUM uses PVM message passing to implement a distributed heap, and can run on clusters of machines or a multicore, or a combination of the two. GUM has in the past been integrated with GHC, but has sufferred from a lack of development effort so has rotted in recent years. Efforts are now underway to get it working with GHC HEAD again. Eden [2] also uses PVM, but does not have a distributed heap. It's implementation is much simpler, and the API is rather more explicit than par/seq and strategies. Eden has been tracking GHC more closely than GUM, but it's still a research project and there's little effort available to make releases. Neither of these are really what you'd call "Distributed Haskell", they are implementations of parallel variants of Haskell running on distributed hardware. There was a Distributed Haskell project, but it is not active at the moment [3]. Cheers, Simon [1] GUM: a portable parallel implementation of Haskell http://www.macs.hw.ac.uk/~dsg/gph/papers/abstracts/gum.html [2] Parallel functional programming in Eden Journal of Functional Programming (2005), 15 [3] http://www.macs.hw.ac.uk/~dsg/gdh/

Please note: We have a version of GUM in which PVM has been replaced by direct network communication. However, this project has been idle since the appearance of GHC-6, and I have not been able to get a binary of the parallel version (which is based on GHC-5) to compile itself. Murray Gross Brooklyn College, City University of New York On Thu, 17 Dec 2009, Simon Marlow wrote:
On 16/12/2009 19:21, Scott A. Waterman wrote:
It looks like there was a recent hackathon focusing on implementing distributed haskell. http://hackage.haskell.org/trac/ghc/wiki/HackPar
I feel there is quite a bit of latent interest in the subject here, but relatively little active development (compared to erlang, clojure, etc.) Can anyone involved give a quick overview (or pointers to one)? It would be good to hear what directions people are taking, and why, and where it's going.
The main directions are:
GUM, which was one of the first parallel implementations of Haskell many years ago [1]. The programming API is the same as GHC has: par/pseq and strategies (indeed this API was invented in the context of GUM, we just re-used it in GHC).
GUM uses PVM message passing to implement a distributed heap, and can run on clusters of machines or a multicore, or a combination of the two. GUM has in the past been integrated with GHC, but has sufferred from a lack of development effort so has rotted in recent years. Efforts are now underway to get it working with GHC HEAD again.
Eden [2] also uses PVM, but does not have a distributed heap. It's implementation is much simpler, and the API is rather more explicit than par/seq and strategies. Eden has been tracking GHC more closely than GUM, but it's still a research project and there's little effort available to make releases.
Neither of these are really what you'd call "Distributed Haskell", they are implementations of parallel variants of Haskell running on distributed hardware. There was a Distributed Haskell project, but it is not active at the moment [3].
Cheers, Simon
[1] GUM: a portable parallel implementation of Haskell http://www.macs.hw.ac.uk/~dsg/gph/papers/abstracts/gum.html
[2] Parallel functional programming in Eden Journal of Functional Programming (2005), 15
[3] http://www.macs.hw.ac.uk/~dsg/gdh/ _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Dec 16, 8:21 pm, "Scott A. Waterman"
Can anyone involved give a quick overview (or pointers to one)? It would be good to hear what directions people are taking, and why, and where it's going.
Personally, I'd love to know the current thinking on a variety of subjects:
* erlang-like port/actor implementations v. more implicit constructs
You may also take a look at http://hackage.haskell.org/package/Holumbus-Distribution or http://holumbus.fh-wedel.de/trac/wiki/Distribution The researchers involved are especially refering to Erlang (and Oz) as an inspirational source for the distribution technology and mechanism used. Johann

On 17 December 2009 06:21, Scott A. Waterman
I feel there is quite a bit of latent interest in the subject here, but relatively little active development (compared to erlang, clojure, etc.) Can anyone involved give a quick overview (or pointers to one)? It would be good to hear what directions people are taking, and why, and where it's going.
I've recently moved into HPC and am now quite interested in using Haskell on large clusters. My first goal was to get hMPI working (HaskellMPI). The original version appears to be from Michael Weber: http://www.foldr.org/~michaelw/hmpi/ which was followed up by Hal Daume III: http://hal3.name/newhmpi.tar.gz It doesn't appear to be cabalised. I wonder if anyone else has been using it? Cheers, Bernie.
participants (7)
-
Bernie Pope
-
Bulat Ziganshin
-
Don Stewart
-
Johann Höchtl
-
Murray Gross
-
Scott A. Waterman
-
Simon Marlow