The GHC API ought to make this relatively easy, but sadly doesn't at the moment. What you'd want: - read in each module in turn, typecheck, desugar, perform modest simplification, and keep the resulting Core bindings - now glom all those bindings together and optimize - code generate, write out a .o file There is absolutely no difficulty in principle, but it'd require care. For example, do we write out one .hi file or many? If one, what should its name be? If many, which bindings should end up in which file? Similarly for the .o file. How would you control it, so that your multi-module project got compiled in the groups you wanted? Of course one group would be best, but you might run out of memory... It'd be an interesting project. Some transformations (e.g. defunctionalisation) only work if you can guarantee to see the whole program. But many others should just work better when you can see more of the program at once. Simon | -----Original Message----- | From: glasgow-haskell-users-bounces@haskell.org [mailto:glasgow-haskell- | users-bounces@haskell.org] On Behalf Of Neil Mitchell | Sent: 23 December 2007 12:48 | To: Isaac Dupree | Cc: GHC Users | Subject: Re: whole program optimization | | Hi Isaac, | | I've had similar thoughts before. Once GHC can read and write external | Core this should become a half hour hack. I did a similar thing in | Yhc, and it really is trivial | (http://darcs.haskell.org/yhc/src/compiler98/Core/Linker.hs). I'm sure | there would be a few corner cases, but those kind of things can easily | be patched over. | | I think there would be speed improvements, but not as dramatic as for | something that was designed to do whole-program from the outset. | | Thanks | | Neil - who really wants external Core | | | On 12/23/07, Isaac Dupree wrote: | > GHC optimizes less effectively when parts of a program are in different | > modules, or are exported from a module. Therefore, merging all the code | > into one "module Main (main)" would help. Unfortunately, Haskell source | > syntax is ill-suited to this; for example: | > | > module-boundary monomorphism restriction | > different defaults in different modules | > deriving (Show, Read) --for renaming everything that's | > -- in different modules, so it doesn't conflict | > new GHC "{..}" etc. record stuff | > imports as, unknown import lists.... | > | > Also, program correctness can be wrongly increased; for example, using | > instances that were defined in some module imported by Main, but not by | > the module that uses the instance. I suppose this might affect | > overlapping instances somehow? | > | > But it seems plausible to merge modules on the GHC Core level, after | > desugaring and type-checking. Then the one infelicity would be that | > orphan RULES could apply in more places (RULES are converted and applied | > to Core!). Is this a problem? Does it already happen when modules | > import small inlinings from .hi files of modules that don't have access | > to the RULES? | > | > It would not actually be quite "whole-program" as long as all the | > libraries are compiled separately and only available through their .o | > and incomplete .hi files. Which is okay; but it would be nice if they | > could all also be compiled to the "complete-.hi" that we thought might | > serve for External Core -- and then one would just need to tell GHC to | > take all those complete-.hi's and compile them together into one | > executable. (except for complications like C _stub files, etc, etc. | > Also some parts of `base` might be wired-in enough that they need some | > thought before messing with them.) Then _any_ function that was only | > used once would be inlined, and GHC would generally be free to make | > decisions like that (at the GHC-speed price of having to look at all | > program and library code in detail) (however the GHC interface to this | > works, Cabal could probably help automate it somehow?) | > | > (Unfortunately, ghc-haskell source syntax doesn't provide an easy way to | > encode Core, even if we wanted to convert it back into GHC-compilable | > Haskell, without unsafeCoerce everywhere, I think?) | > | > So it's more complicated than it seems on the surface, but it could be | > done; I only wonder how effective it would be on various programs. I | > assume GHC doesn't currently implement any optimizations that can _only_ | > be whole-program, so it would still not be quite like Jhc, though this | > basic support might allow some such optimizations to be implemented. | > | > Isaac | > _______________________________________________ | > Glasgow-haskell-users mailing list | > Glasgow-haskell-users@haskell.org | > http://www.haskell.org/mailman/listinfo/glasgow-haskell-users | > | _______________________________________________ | Glasgow-haskell-users mailing list | Glasgow-haskell-users@haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

On 12/24/07, Simon Peyton-Jones wrote:

The GHC API ought to make this relatively easy, but sadly doesn't at the moment. What you'd want:

- read in each module in turn, typecheck, desugar, perform modest simplification, and keep the resulting Core bindings

The API can do this already :-)

- now glom all those bindings together and optimize - code generate, write out a .o file

I'm working on this part. In fact, I'm almost ready to check it in -- at least for the single-module case. Cheers, Tim -- Tim Chevalier * http://cs.pdx.edu/~tjc * Often in error, never in doubt "The blues isn't about feeling better, it's about making other people feel worse, and making a few bucks while you're at it." -- Bleeding Gums Murphy

Stefan O'Rear wrote:

It's been done. http://www.cs.utah.edu/~hal/HAllInOne/index.html

Stefan

which alas seems to have a ridiculous, probably unenforcable license :-) Interesting though - I wonder how correct it is. " This code is licensed under the Big-Kazooba-Ritual license: You may copy, modify or distribute this software provided that you first perform the ritual described in section a. If you ever meet one of the authors of this work you can be asked, and must comply, to perform the ritual as described in section a. a. Lift your left leg, close your eyes and jump in the air. Now tap the point of your nose exactly two times with your right index finger. To complete: say the words "thank you Big Kazooba" You may open your eyes now. The disabled may opt not to perform this ritual but have a volunteer perform the ritual on their behalf. Anyone, of suited position, may opt to command any number of subordinates to perform the ritual instead. Moreover, this code makes use of the GMap library from Strafunski. Any licenses on that of course carry over to this program, if they need to. " Isaac

g9ks157k:

Am Sonntag, 23. Dezember 2007 13:35 schrieb Isaac Dupree:

...

GHC optimizes less effectively when parts of a program are in different modules, or are exported from a module.

By the way, does GHC do cross-package optimization (not just cross-module optimization within packages)?

Yep, esp. with high levels of -O, code is unfolded into .hi files, distributed with the package. You can, with enough INLINE, get full inlining of your library into the unsuspecting user's program :) (Data.Binary attempts this) -- Don