Re: [Haskell-cafe] MapReduce reverse engineered

2009/2/25 Alberto G. Corona

And it would solve a lot of problem: scalability, system re-configuraition and installation: just by adding or removing nodes at runtime.. heavy numerical computations are also good candidates.

2009/2/25 Rick R

I agree. A distributed database could be made as usable as a standard

...

RDBMS by offering an interface by which you supply a map/reduce pair of functions and a list (range?) of keys.

This could be easily implemented with a database such as Scalaris, in which the Chord algorithm is responsible for placing and finding the data among nodes.

The user would interface with any node in the distributed database, supplying a map and reduce function. It would distribute the map function to nodes of its choosing (weighted by some metrics such as idle cpu), retrieve the intermediate sets and run reduce if supplied.

2009/2/25 Alberto G. Corona :

...

Galchin,

Maybe you are asking not only about remote execution, but also mobility of code. This is a problem that is previous to mapReduce, since mapReduce assumes that all the code (and the data) is in place in the respective nodes. In fact, the distribution of resources in order to efficiently use mapReduce is a design problem that the google people has done by hand. But my intuition says that there are a general algorithm for distribution of code, data, bandwidth and resources in general that moves around at execution time to achieve better and better performance for a given grid of nodes and for any task, for example, a mapReduce task. I would be very interesting to read something about this. I know that some efforts have been carried out the past , for example mobile haskell

...

which is a first step for this goal but I this has been discontinued and

http://homepages.inf.ed.ac.uk/stg/workshops/TFP/book/DuBois/duboismhaskell/c... the

...

source code is not available.

2009/2/25 Galchin, Vasili

...

Hello,

Here is an interesting paper of Google's MapReduce reverse

engineered

...

into Haskell. I apologize if already posted ..... http://www.cs.vu.nl/~ralf/MapReduce/

Kind regards, Vasili

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

-- We can't solve problems by using the same kind of thinking we used when we created them. - A. Einstein