If I may ask, I'm not quite sure what O(2^n) and O(1) are? "Besides Moore's Law, digital computing also benefits from mature tools and expertise for optimizing performance at all levels of the system: process technology, fundamental circuits, layout and algorithms. Many engineers are simultaneously working to improve every aspect of digital technology, while alternative technologies like analog computing do not have the same kind of industry juggernaut pushing them forward." I'm curious, were not all these built on the foundation of Moore's Law? Everything Vigoda lists has Moore's Law in mind. If Moore's Law were to suddenly disappear, could these survive on their own merit? Let me rephrase that, of course they will survive politically. People built these tools and if built, they will be use but will they survive efficiently? In the future, if a particular specialized architecture is somewhat better than the rest on it's own merit for a particular need while the stock architecture is reaching a point of low returns for all the energy put into it - could the specialized architecture reach a point where it becomes useful? Could there be a competitive advantage to specialized architecture if Moore's Law were to go away? On 3/17/13, Gwern Branwen wrote:

On Sun, Mar 17, 2013 at 5:56 PM, OWP wrote:

...

These "stock architectures", were they really so good that they out performed the specialized ones on it's own merits or was this mainly due to Moore's Law on transistors? In other words, suppose we separate Moore's Law from the stock architecture, would it still outperform the specialized ones?

It's not really meaningful to separate them. Any time you use a custom architecture, you are forfeiting all sorts of network effects - and sooner or later, the custom architecture falls behind. If you want to make an analogy, when you go with a custom architecture, you are trading a process where your computing power increases O(2^n) for one with a big constant factor but where computing power increases O(1)...

...

"In practice replacing digital computers with an alternative computing paradigm is a risky proposition. Alternative computing architectures, such as parallel digital computers have not tended to be commercially viable, because Moore's Law has consistently enabled conventional von Neumann architectures to render alternatives unnecessary. Besides Moore's Law, digital computing also benefits from mature tools and expertise for optimizing performance at all levels of the system: process technology, fundamental circuits, layout and algorithms. Many engineers are simultaneously working to improve every aspect of digital technology, while alternative technologies like analog computing do not have the same kind of industry juggernaut pushing them forward."

from Benjamin Vigoda, "Analog Logic: Continuous-Time Analog Circuits for Statistical Signal Processing" (2003 PhD thesis)

-- gwern http://www.gwern.net

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

On Mon, Mar 18, 2013 at 4:31 PM, OWP wrote:

Let me rephrase that, of course they will survive politically. People built these tools and if built, they will be use but will they survive efficiently? In the future, if a particular specialized architecture is somewhat better than the rest on it's own merit for a particular need while the stock architecture is reaching a point of low returns for all the energy put into it - could the specialized architecture reach a point where it becomes useful? Could there be a competitive advantage to specialized architecture if Moore's Law were to go away?

There is now, in some narrow specializations. GPUs and DSP come to mind --- while both are also done on commodity CPUs to some extent, the specialized architectures are used where speed is of the essence. (DSP started out on specialized architectures, but many commodity uses are on commodity architectures these days, reserving the specialized ones to those niches that require them.) -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

Ironically, you made an interesting point on how Moore's Law created the on chip "real estate" that made specialized machines possible. As transistor sizing shrinks and die sizes increase, more and more real estate should now be available for usage. Oddly, what destroyed specialized machines in the past seemed to be the same cause in reviving it from the dead. The ARM Jazelle interface - I'm not familiar with it's but it's got me curious. Has there been any though (even in the most lighthearted discussions) on what a physical "Haskell Machine" could look like? Mainly, what could be left to compile to the stock architecture and what could be sent out to more specialized areas? On 3/18/13, Richard A. O'Keefe wrote:

On 19/03/2013, at 9:31 AM, OWP wrote:

...

If I may ask, I'm not quite sure what O(2^n) and O(1) are?

Check any data structures and algorithms textbook.

Reverting to the original topic, THIS is the age of specialised machines. A lot of the chips out there are not just a CPU but a SoC (System on a Chip). Start with the ARM1176JZF-S chip whose manual I currently have open. - sorta kinda RISCish ARM instruction set processor + including SIMD DSP/graphics support - native hardware execution of (most) Java bytecodes (This is ARM's "Jazelle" extension.) - vector floating point co-processor You can get other chips with ARM cores and a mix of - analogue<->digital converters, comparators, Flash controllers, Ethernet controllers, USB controllers, other interface controllers, hardware encryption (especially in ARM v8), more kinds of timers than you knew existed, hardware random number generation, You can even get ARM chips with on-board FPGAs.

Of course SoC systems are not limited to the ARM architecture. SPARC T4 chips have "Crypto Instruction Accelerators … [that] … enable high speed encryption for over a dozen industry standard ciphers" "plus random number generation" and "high speed 10 GbE networking directly on … the silicon" and two PCI Express controllers. SPARC systems offered, and still do offer, special hardware support for dynamic programming languages in which immediate integers have tag 00 in the bottom 2 bits. However, when they went 64-bit, they didn't bother to extend that to 64-minus-2-bit integers.

And of course there are Intel and AMD chips with all sorts of extra hardware support for all sorts of things. Notably, people are integrating GPUs onto the same chip as the CPU. (Where are the APL compilers that can take advantage of this? It's the perfect APLlication for the language!)

The key point is that cpu designers/vendors take existing workloads of commercial significance and figure out how to optimise that. If a heavy-duty web server, or a high end gaming system, or a mobile phone, &c could clearly benefit from some kind of acceleration, someone will get it. If Javascript had a common abstract instruction set, there'd be hardware acceleration for that.

Haskell programs are not yet a workload of commercial significance.

Sadly.

And if you are, you may be interested in https://github.com/tommythorn/Reduceron The underlying topic is a fascinating one. The fact that people ignore is that silicon cycle time improvements have been fairly modest - perhaps 2-3 orders of magnitude and we have long been at the point where wire delays are what matters. Without significant innovation, silicon scaling would just have given you a moderately faster cpu, but ridiculously tiny. Those innovations are countless, pipelining, caches, superscalar, out-of-order, speculation, ... Now that conventional single-threaded performance is peaking out, the time is ripe to revisit functional machines and apply these innovations. Reduceron is an amazing accomplishment (do and there are other projects in this space too. I hope the research on this continues. My only contribution here is to try to expand the usefulness of Reduceron and get it running on cheaper hardware. I would love more contributors, especially on compiler side (using a "real" Haskell front-end would be just lovely). Anyway, check it out and play around. I'll be happy to help. Tommy Thorn On Mar 19, 2013, at 05:07 , Simon Farnsworth wrote:

OWP wrote:

...

Ironically, you made an interesting point on how Moore's Law created the on chip "real estate" that made specialized machines possible. As transistor sizing shrinks and die sizes increase, more and more real estate should now be available for usage. Oddly, what destroyed specialized machines in the past seemed to be the same cause in reviving it from the dead.

The ARM Jazelle interface - I'm not familiar with it's but it's got me curious. Has there been any though (even in the most lighthearted discussions) on what a physical "Haskell Machine" could look like? Mainly, what could be left to compile to the stock architecture and what could be sent out to more specialized areas?

You might be interested in looking at the Reduceron - http://www.cs.york.ac.uk/fp/reduceron/ - it was an FPGA-based effort to design a CPU explicitly for a Haskell-like language.

-- Simon Farnsworth

_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe