To my recent message
(2) I wrote in the BAL paper that meaningful standard algebraic categories (classes) may, in principle, help the compiler to optimize programs using the properties related to the category names (associativity, commutativity ...). [..] I kept in mind implicitly that as soon as these categories enter a library standard their names can also be added to the _key words of a language_. In this way the compiler is enabled to use the relevant properties. There remains a question of expressions like _|_ + 1 === 1 + _|_, but this is another matter, maybe, a matter of flags for the compiler and language version.
Dylan Thurston <dpt@math.harvard.edu> responds on 23 Aug 2001
One question I have here is, how much are compilers prepared to take advantage of such knowledge? (Also, does it really belong in the compiler? In, e.g., FFTW, all this is optimized by a user program rather than the compiler.) I'm slightly skeptical: it's not entirely obvious how to take proper advantage of commutativity/associativity, and this seems like something I'd want to program directly rather than trust the compiler to do. But I haven't thought about this much; other opinions?
I am skeptical too. Anyway, if a compiler `knows' the word CommutativeRing, knows that by definition, CommutativeRing requires associativity, commutativity ..., observes the instance CommutativeRing T for some type T in some program, sees further expr = (2*a + b) - (a+a) :: T and sees the language version (compiler) flag -fignore-bottom, (something of this sort) then, has it right to convert expr --> b :: T ? To my mind, there is something more in this than intension. In practice, the users do not write explicitly such expressions as expr. But they write them implicitly, by applying functions in special cases, for example, one has f(x,y) and applies f(x,2*x) ... I do not ask compilers to exploit now this possibility. Only noted that a meaningful category organization provides, in principle, room for such optimization. Am I missing something? For I can mistake. Further, to my
(1): As I wrote earlier, Haskell instances cannot model such domains as the residue ring Integer/(n), when n changes dynamically. This is why the BAL library applies the sample argument approach, and this somewhat complicates the program meaning.
Now, we observe in the Aldor manual (http:/www.aldor.org), Section 7.8: `` Zmod(n: Integer) : Ring with { if prime? n then Field; } == Integer add { if prime? n then {inv (x: %) : % == ...} }
Z_n, the domain of integers modulo n, is always a Ring. However, if n is prime, then Z_n is also a Field, meaning that it should provide a multiplicative inverse for nonzero values. In an `add' expression, a definition which appears in the consequence of an `if' expression is said to be a conditional definition ... ''
Dylan Thurston <dpt@math.harvard.edu> writes
Z_n indeed sometimes supports additional operations. But how is the user expected to use these operations in a type-safe, statically checked way? Any use of 'inv' for Z_n will necessarily involve a run-time check on n to see whether it is prime. [..]
I believe, Aldor checks some types at run-time. And it has types as values. In fact, the above Zmod is a _function_ that takes values in a type called Ring. Each value Zmod(n) is a domain (abstract data type) of type Ring. Regards, ----------------- Serge Mechveliani mechvel@botik.ru
Sergey:
Anyway, if a compiler `knows' the word CommutativeRing, knows that by definition, CommutativeRing requires ...
I think this is too much to ask from a compiler. You quickly run into all sorts of undecidable problems when you want to do term rewriting, and even more so if you rewrite modulo associativity and so on. Of course program transformations would be nice, for the reasons you mentioned. But this would definitely require a dedicated toolset, separate from the compiler itself. Remember, the compiler needs to prove, for each transformationen a) that it preserves meaning (semantics) b) it increases efficiency b) the transformation itself terminates quickly Of course this impossible to do in general. On the other hand there *is* quite a bit of research (for instance, by the term rewriting community, http://rewriting.org) into special methods that work in special cases. What a (Haskell) compiler could provide is a nice interface (to its internal code representation) where external program analysis and transformation tools could be plugged in. This would also add a more "pratical" drive to the research mentioned above. I'd definitely love to see this. Best regards, -- -- Johannes Waldmann ---- http://www.informatik.uni-leipzig.de/~joe/ -- -- joe@informatik.uni-leipzig.de -- phone/fax (+49) 341 9732 204/252 --
On Fri, 24 Aug 2001, S.D.Mechveliani wrote:
I am skeptical too. Anyway, if a compiler `knows' the word CommutativeRing, knows that by definition, CommutativeRing requires associativity, commutativity ..., observes the instance CommutativeRing T for some type T in some program, sees further
expr = (2*a + b) - (a+a) :: T
and sees the language version (compiler) flag -fignore-bottom, (something of this sort) then, has it right to convert
expr --> b :: T ? To my mind, there is something more in this than intension. In practice, the users do not write explicitly such expressions as expr. But they write them implicitly, by applying functions in special cases, for example, one has f(x,y) and applies f(x,2*x) ...
On a purely pragmatic note ( :-) ) these optimisations can't necessarily be safely used in three of the four most common cases, namely Int, Float and Double because of the restricted range of intermediates. In particular, somewhere in my copy of the source code for MetaFont Donald Knuth has a routine containing (from memory): t <- (p-q) + p; {compute t=2p-q without overflow; let us hope an optimising compiler does not `optimise' this to 2*p-q} There's a thread on the gcc mailing list (`Unsafe FP optimisations' I think) that covers some of the even less obvious things that can happen in floating point. Clearly people who write code which is both likely to use the entire range of a numeric type and also do enough reasoning to know that they're particular sequence of operations does not overflow even though other `equivalent' reorderings will, is a set of measure zero. However I wouldn't think it was reasonable to ignore the possibility the program writer knows what he's doing better than the programmer. (In the fourth case, Integer, there're no problems that I can see with these optimisations.) So the potential benefits of optimisations based on commutativity and associativity seem to be restricted to Integers and non-elementary numeric types and algebraic types. ___cheers,_dave________________________________________________________ www.cs.bris.ac.uk/~tweed/pi.htm |tweed's law: however many computers email: tweed@cs.bris.ac.uk | you have, half your time is spent work tel: (0117) 954-5250 | waiting for compilations to finish.
D. Tweed: On Fri, 24 Aug 2001, S.D.Mechveliani wrote:
I am skeptical too. Anyway, if a compiler `knows' the word CommutativeRing, knows that by definition, CommutativeRing requires associativity, commutativity ..., observes the instance CommutativeRing T for some type T in some program, sees further
expr = (2*a + b) - (a+a) :: T
and sees the language version (compiler) flag -fignore-bottom, (something of this sort) then, has it right to convert
expr --> b :: T ? (...) On a purely pragmatic note ( :-) ) these optimisations can't necessarily be safely used in three of the four most common cases, namely Int, Float and Double because of the restricted range of intermediates. In particular, somewhere in my copy of the source code for MetaFont Donald Knuth has a routine containing (from memory):
t <- (p-q) + p; {compute t=2p-q without overflow; let us hope an optimising compiler does not `optimise' this to 2*p-q}
There's a thread on the gcc mailing list (`Unsafe FP optimisations' I think) that covers some of the even less obvious things that can happen in floating point. Clearly people who write code which is both likely to use the entire range of a numeric type and also do enough reasoning to know that they're particular sequence of operations does not overflow even though other `equivalent' reorderings will, is a set of measure zero. However I wouldn't think it was reasonable to ignore the possibility the program writer knows what he's doing better than the programmer. (In the fourth case, Integer, there're no problems that I can see with these optimisations.)
So the potential benefits of optimisations based on commutativity and associativity seem to be restricted to Integers and non-elementary numeric types and algebraic types.
I don't fully agree. There are plenty of cases where transformations based on these properties are useful and safe in practice, although it may be hard or impossible to prove the safety formally. Or, it might be the case that the damage caused by an occasional numeric failure that manifests itself (lack of convergence, overflow) is more than compensated by the average gain in speed. Restructuring compilers, in particular parallelising compilers, sometimes rely heavily on these algebraic laws for their optimizations. On the other hand, it must of course be possible to turn off such optimizations when they are deemed unsafe enough to be harmful. So the solution seems to be compiler flags or code annotations. Björn Lisper
participants (4)
-
Bjorn Lisper -
D. Tweed -
Johannes Waldmann -
S.D.Mechveliani