On 2/11/16 10:28 AM, Joachim Durchholz wrote: We're still not really communicating, so it may be time to draw this thread to a close soon.
Am 01.11.2016 um 01:37 schrieb Richard A. O'Keefe:
On 1/11/16 9:54 AM, Joachim Durchholz wrote:
And you need to control memory coherence, i.e. you need to define what data goes together with what processes.
At this point I'm puzzled. Languages like Occam, ZPL, and Co-Array Fortran basically say NO! to memory coherence.
Sure, but unrelated.
How is it unrelated? If you don't need, don't want, and don't have "memory coherence", then you don't have to control it.
The hope with FPLs was that you do not need to explicitly specify it anymore, because the compiler can manage that.
The Alan Perlis quote applies: When someone says: "I want a programming language in which I need only say what I wish done", give him a lollipop. In declarative languages, you give up explicit control over some things in order to make other things easier to express. A compiler has three ways to decide which are the "important" things: - by analysis - by being told (there's the famous anecdote about the original Fortran FREQUENCY statement being implemented backwards and nobody noticing) - by measuring (standard technology in C compilers for years now) Nearly 20 years ago I was using a C compiler that would shuffle functions around in your executable in order to reduce page faults. That's the "measurement" approach.
If the data need to be available in some other process, there is some sort of fairly explicit communication.
Which means that you do not have a simple function call anymore, but an extra API layer.
Here you have left me behind. WHAT was "a simple function call"? WHAT is "an extra API layer", as opposed to annotations like the distribution annotations in ZPL and HPFortran?
For what it's worth, the "Clean" programming language used to be called "Concurrent Clean" because it was set up to run on a cluster of Macintoshes.
Clean is strict ;-)
That one line provoked this reply. Clean isn't any stricter than Haskell. It does strictness inference, just like GHC. It allows strictness annotations, just like GHC (though not in Haskell 2010). Plus, the more I read about various
forms of partitioning computations (not just NUMA but also IPC and networking), the more it seems that hardware moves towards making the barriers higher, not lower (the reason being that this helps making computations within the barrier more efficient).
If that's a general trend, that's bad news for network, IPC, or NUMA transparency. Which is going to make programming for these harder, not easier :-(
On the one hand, agreed. On the other hand, while programming in the 1960s still *is* getting harder, it's not clear that we cannot find an approach that will be easier. To my astonishment, quite a lot of the people using NZ's supercomputer facility are programming in Python (basically glue code hooking up existing applications), or Matlab (serious number-crunching, which they say they've benchmarked against C), or even R (yes, R; NOT one of the world's faster languages, BUT again we're mostly talking about glue code hooking together existing building-blocks). And this facility has people who will help researchers with their supercomputer programming for free. The Science Data Processor for the Square Kilometre Array will have a multi-level structure - compute elements contain multiple cores, probably GPUs, and probably FPGAs. These are roughly the equivalent of laptops. - compute nodes are tightly integrated clusters containing multiple compute elements + interconnects + extra storage - compute islands are looser clusters containing multiple compute nodes and management stuff - the SDP will contain multiple compute islands + massive interconnects (3TB/sec data coming in) - there will be two SDPs, each sharing a multimegawatt power station with a signal processing system that talks directly to the telescopes. Managing massive data flows and scheduling the computational tasks is going to be seriously hard work. One of the main tools for managing stuff like this is ISOLATION, and LIMITING communication as much as possible, which is the very opposite of the large flat shared memory model.