Thanks; I saw you mention the paper before, but now I finally started reading it :) By the way, the paper *does* arrange an extra infrastructure for mappending only adjacent results. Looks like with a commutative monoid, a fold could be done in a fully distributed fashion, however it would no more be a scan. 2009/1/22 Dan Piponi <dpiponi@gmail.com>:
On Wed, Jan 21, 2009 at 11:12 PM, Eugene Kirpichov <ekirpichov@gmail.com> wrote:
To my mind, in the map-reduce case you generally need a commutative monoid. Or, you need an extra infrastructure that mappend's only results from adjacent machines, or something like that.
This is a good paper on the stuff I'm talking about: http://citeseer.ist.psu.edu/blelloch90prefix.html It doesn't explicitly mention monoids but it's all about associative operations with identity. -- Dan