| I think that we should have, as David Roundy pointed out, a | restriction to code that is actually used frequently. However, I | think we should make a distinction between micro-benchmarks, that | test some specific item, and real-life benchmarks. As many of you will know, the nofib benchmark suite made just such a distinction from the beginning. In fact, it has 3 sub-suites: * Imaginary: very tiny programs whose merit is that they sometimes hit missing optimisations very hard indeed. Useful for compiler writers. * Spectral: these are often called "kernels" and are meant to be typical of the inner loops of some real programs. * Real: these are unadulterated real applications, of various sizes. We found these categories to be useful and robust, and I think they'd be useful for the new suite. In particular, the imaginary suite is useless for (say) choosing a compiler, but fantastic for exposing particular weak spots. But if the imaginary programs were mixed with the real ones, the whole thing would lose credibility. You can find more info here: http://research.microsoft.com/~simonpj/Papers/nofib.ps Simon