On 3/11/07, Donald Bruce Stewart
bulat.ziganshin:
Hello Claus,
Saturday, March 10, 2007, 4:36:22 AM, you wrote:
ah, ok, i'm not used to thinking in such scales;-) (perhaps you should get in touch with those SAC people, after all - i don't know what their state of play is, but many years ago, they started in an office near mine, and they were definitely thinking about large arrays, even about how to distribute them, and computations on them;
last days i learned details of google's MapReduce system. seems that this approach is very interesting for dealing with large arrays. files (arrays) are splitted into chunks, operations are splitted into chunks, too. afaik, some C compilers are already able to automatically split vector operations into several threads? at least, it will be interesting to implement same technique for GHC, may be just in form of library, like google does
See the data parallel arrays library:
http://haskell.org/haskellwiki/GHC/Data_Parallel_Haskell
-- Don
(which, btw, is much more interesting than google's mapreduce stuff, since it does *nested* data parallelism) -- Sebastian Sylvan +46(0)736-818655 UIN: 44640862