Hi, As some of you may know, I'm working on adding support for dynamically loaded plugins to GHC this summer. As part of this we need a way to specify ordering on the compiler phases installed, so e.g. you can say that a phase you install should run after strictness analysis does. This problem begs for a more modular and principled solution than just reusing the current system where we use the natural numbers to impose an ordering on INLINE/RULE activations. I have proposed a system at http://hackage.haskell.org/trac/ghc/wiki/Plugins/Phases that replaces this with one of named compiler phases not only for plugins but for all activation control in the compiler. The proposal mostly remains backwards compatible with the most common uses of the numeric format, with an exception noted on the page and below. I would be interested in feedback on the design before the implementation is complete and in the wild. I'm especially interested in hearing if you believe that loss of support for numeric phase numbers > 2 is a problem, as this is the only breaking change that I'm proposing. Cheers, Max
Hi Max, sorry for replying so late, I've completely forgotten about this.
I would be interested in feedback on the design before the implementation is complete and in the wild. I'm especially interested in hearing if you believe that loss of support for numeric phase numbers > 2 is a problem, as this is the only breaking change that I'm proposing.
Three things. Firstly, what would lenient ordering be useful for? You probably had a specific use case in mind? Secondly, I think it is quite important to be able to specify dependencies for already declared phases. That is, I (probably) want to be able to say in module C that phase X from module A should come before phase Y from module B. I suspect that's not quite possible with your current design. Lastly, in addition to < and > I'd also like to have = (i.e., phase X = phase Y which would make X an alias for Y). As to the numeric phase ids, I'm all in favour of dropping them altogether. For legacy code, you could provide wired-in names 0, 1 and 2 for now but I would deprecate them. Also, I'm one of the probably very few people who have ever used phase 3 and beyond and dropping support for that is perfectly fine with me. Thanks for working on this! Roman
Hi Roman,
Three things. Firstly, what would lenient ordering be useful for? You probably had a specific use case in mind?
I suspect that when you have multiple plugins all specifying constraints on the phase ordering independently it is possible to end up in a situation where using each plugin individually results in a consistent phase ordering but a combination of multiple plugins causes a cycle to appear in the order. Hence it is useful to let plugin authors flag up dependencies they don't >need< to have so we can intellegently break the cycle using those constraints rather than just giving up. Admittedly I only have a superstition that this will be a practical problem.
Secondly, I think it is quite important to be able to specify dependencies for already declared phases. That is, I (probably) want to be able to say in module C that phase X from module A should come before phase Y from module B.
This in interesting. You're right that it's not possible with this system: to support this you would need a more class + instance rather than single declaration flavour for phases. Do you have a practical example in mind of how you would use it?
I suspect that's not quite possible with your current design. Lastly, in addition to < and > I'd also like to have = (i.e., phase X = phase Y which would make X an alias for Y).
Ah! This would also let you do what you wanted with your second point: module C where {-# PHASE XAlias = A.X, < B.Y #-} This could indeed be useful, and since it subsumes your other proposal without complicating the phase declaration experience this looks like a good extension to have.
As to the numeric phase ids, I'm all in favour of dropping them altogether. For legacy code, you could provide wired-in names 0, 1 and 2 for now but I would deprecate them. Also, I'm one of the probably very few people who have ever used phase 3 and beyond and dropping support for that is perfectly fine with me.
Agreed: we should deprecate numeric ids if we have this alternative. Thanks for your interesting comments! Max
Max Bolingbroke wrote:
Hi Roman,
Three things. Firstly, what would lenient ordering be useful for? You probably had a specific use case in mind?
I suspect that when you have multiple plugins all specifying constraints on the phase ordering independently it is possible to end up in a situation where using each plugin individually results in a consistent phase ordering but a combination of multiple plugins causes a cycle to appear in the order. Hence it is useful to let plugin authors flag up dependencies they don't >need< to have so we can intellegently break the cycle using those constraints rather than just giving up. Admittedly I only have a superstition that this will be a practical problem.
If you don't need a dependency and it can be ignored anyway, why would you want to specify it in the first place? I just can't quite imagine a situation in which I would use this. As to cycles, I think just aborting if they occur would be ok.
Secondly, I think it is quite important to be able to specify dependencies for already declared phases. That is, I (probably) want to be able to say in module C that phase X from module A should come before phase Y from module B.
This in interesting. You're right that it's not possible with this system: to support this you would need a more class + instance rather than single declaration flavour for phases. Do you have a practical example in mind of how you would use it?
Yeah, I'd prefer the class/instance model. An example: suppose you have two independent libraries which implement two optimisations and you want one of them to run before another. I think we really need this if we want to compose optimisations.
I suspect that's not quite possible with your current design. Lastly, in addition to < and > I'd also like to have = (i.e., phase X = phase Y which would make X an alias for Y).
Ah! This would also let you do what you wanted with your second point:
module C where {-# PHASE XAlias = A.X, < B.Y #-}
True. It's a bit of a hack, though :-) Roman
If you don't need a dependency and it can be ignored anyway, why would you want to specify it in the first place? I just can't quite imagine a situation in which I would use this.
I think it makes sense because many of the inter-pass dependencies we have in the GHC pipeline today are actually somewhat lenient. For example, something quite sensible will happen if we do CSE before full laziness, it will simply lead to less efficient code. In contrast, it's imperative that we perform strictness analysis before applying worker-wrapper based on the annotations it adds.
Yeah, I'd prefer the class/instance model. An example: suppose you have two independent libraries which implement two optimisations and you want one of them to run before another. I think we really need this if we want to compose optimisations.
This is an interesting observation. So, you imagine that the >user program itself< might import the compiler plugins and add a constraint between the phases they export. I'm not sure how many users would really want to do this, but I can see it being useful. Thanks for your input: I've got a lot to think about now. Max
Max Bolingbroke wrote:
If you don't need a dependency and it can be ignored anyway, why would you want to specify it in the first place? I just can't quite imagine a situation in which I would use this.
I think it makes sense because many of the inter-pass dependencies we have in the GHC pipeline today are actually somewhat lenient. For example, something quite sensible will happen if we do CSE before full laziness, it will simply lead to less efficient code. In contrast, it's imperative that we perform strictness analysis before applying worker-wrapper based on the annotations it adds.
I see. I suspect I'd prefer if GHC asked me to manually resolve such conflicts rather than silently generating suboptimal code but then again, perhaps not. In any case, my comments mostly apply to phase control for inlining and rule matching; other passes probably have different requirements which I haven't thought about at all.
Yeah, I'd prefer the class/instance model. An example: suppose you have two independent libraries which implement two optimisations and you want one of them to run before another. I think we really need this if we want to compose optimisations.
This is an interesting observation. So, you imagine that the >user program itself< might import the compiler plugins and add a constraint between the phases they export. I'm not sure how many users would really want to do this, but I can see it being useful.
What I had in mind is a library/plugin which uses other libraries/plugins. For instance, let's say we have two libraries which both implement rule-based fusion, one for some high-level stuff and one for something low level. Now, a third library which uses the first two might want to specify that high-level fusion should be done before low-level fusion. Or consider something like loop optimisations, where you might have separate plugins which implement unrolling, PRE and so on and one plugin which aggregates all these to provide a complete loop optimiser. It's all about compositionality, really. Roman
| giving up. Admittedly I only have a superstition that this will be a | practical problem. I agree with Roman -- let's not bother with lenience until we need it | > Secondly, I think it is quite | > important to be able to specify dependencies for already declared phases. | > That is, I (probably) want to be able to say in module C that phase X from | > module A should come before phase Y from module B. | | This in interesting. You're right that it's not possible with this | system: to support this you would need a more class + instance rather | than single declaration flavour for phases. Do you have a practical | example in mind of how you would use it? Ah -- Roman you mean you want to add a phase-ordering constraint at some time *other* than when you declare one or other of the phases. Are you sure this is important? It's an awkward addition because, like orphan instances, it means there's an interface file with perhaps-vital info which you might have no other reason to visit. Also, why do you want phase aliases? Simon
Ah -- Roman you mean you want to add a phase-ordering constraint at some time *other* than when you declare one or other of the phases. Are you sure this is important? It's an awkward addition because, like orphan instances, it means there's an interface file with perhaps-vital info which you might have no other reason to visit.
I've begun to think that we need some generalized framework for gathering "orphaned" information since, as we discussed, the same problem occurs with my new annotations system (http://blog.omega-prime.co.uk/2008/07/05/compiler-plugins-for-ghc-weeks-thre...) if you wish to implement the ability to annotate identifiers from other modules. Max
Simon Peyton-Jones wrote:
Ah -- Roman you mean you want to add a phase-ordering constraint at some time *other* than when you declare one or other of the phases. Are you sure this is important?
Fairly. I've explained why in a follow-up to Max's message.
Also, why do you want phase aliases?
Modularity. Let's take NDP fusion as an example. We have 2 logically distinct fusion phases, one for fusing distributed stuff and one for stream fusion (let's call the DIST and STREAM). At the moment, the two are performed simultaneously, i.e., we have DIST = STREAM. However, I suspect that having DIST < STREAM might produce better results. This means that I'd like to use INLINE DIST when implementing distributed fusion, INLINE STREAM for stream fusion and only have one place where I declare DIST = STREAM or DIST < STREAM. I don't quite see how to achieve this without aliases. This will be even more of a problem once I add additional fusion layers. Incidentially, this is also an example of why adding ordering constraints on already declared phases is useful. Neither of the two fusion systems really depends on the other so it would be nice to be able to put the DIST < STREAM or DIST = STREAM declaration into a module which *integrates* the two. Roman
Also, why do you want phase aliases?
I don't quite see how to achieve this without aliases. This will be even more of a problem once I add additional fusion layers.
I've added phase equality to the implementation. It seems like a nice clean extension. Since this lets you add constraints to existing phases (given that the module defining the phase alias that accomplishes this is directly or indirectly imported into every affected module) I'm going to hold off on the "class/instance" style. Although it might be nice, it's rather complicated, and we can retrofit it at a later date if it turns out to be essential. Cheers, Max
participants (3)
-
Max Bolingbroke -
Roman Leshchinskiy -
Simon Peyton-Jones