Joachim Breitner writes:

Hi Ben,

thanks for kicking this off.

Am Montag, den 19.02.2018, 22:04 -0500 schrieb Ben Gamari:

...

The proposal addresses a long-standing limitation (#7414) of the GHC plugin interface which has become increasingly visible to users in recent years. While the particular approach proposed breaks existing plugin users, it is expressive enough to allow GHC to perform more aggressive recompilation avoidance in the future [1]. Moreover, the smart constructors proposed should make for a reasonably smooth migration path.

Given that the plugins appears to have buy-in from a few notable plugin authors, I recommend that we accept this proposal.

How realistic is it to do the partial module recompilation thing that warrants multiple PluginRecompile values per module? For this to be useful the compiler would have to

* calculate the typechecker plugin’s hash for module M * notice that it has changed, and start compiling M * but then, when we have core, magically notice that the output of the desugarer has not changed, and consider to abort calculation * but then notice that a core2core plugin was involved, and ask that for the new hash * and then this hash must be unchanged * and then really the compilation has ended.

Is that realistic? Can someone fill this example with concrete life?

Maybe I am just not seeing what people have in mind – but I feel unless this really is more than a very vague idea, we should consider the simpler variant where each plugin gets to calculate a single hash per module – and not possibly many in various stages.

Another way of putting it: The compiler pipeline is (simplified)

1. decide what to compiler 2. parse 3. typecheck 4. desguar 5. optimize 6. code gen

Plugins so far hook into step 3 and 5. Every one of these steps probably eventually deserves a plugin hook. This proposal is about adding one to step 1. With this view, I find the “add a single function to Plugin”, as described in https://github.com/ghc-proposals/ghc-proposals/pull/108#issuecomment-3626714... much more convincing.

I understand the concern; your description indeed matches how GHC currently handles recompilation, which makes it quite difficult to bail out after deciding to recompile. Simon PJ has mentioned in the past that one of the goals that would be worth pursuing is more aggressive recompilation avoidance (e.g. perhaps even some day on a sub-module granularity), especially given the recent complaints surrounding compilation times. I don't believe there is any motion on this and it would certainly be a significant undertaking. Given that we have very little idea of what such a design would look like, I agree that we shouldn't try to design the plugin interface too carefully around this (likely quite far-off) goal. However, I personally feel like the proposed design is a reasonable compromise. Lumping together all of the recompilation checks into a single hash would expose a relatively ugly facet of GHC's implementation to plugin authors and force authors to tie together logic from conceptually-distinct compilation phases. Furthermore, we would tie ourselves to GHC's current design. The one thing that gives me pause is the proposal's treatment of core-to-core plugins, which return a list of CoreTodos. Tying recompilation to this function is admittedly a bit odd. Cheers, - Ben

Also +1 from me in principle - with the caveat that I haven't written any plugins, but I am quite familiar with how recompilation avoidance works. It's an important issue to address, and I didn't spot any obvious problems with the design. Cheers Simon On 2 March 2018 at 17:49, Iavor Diatchki wrote:

Hello,

I've been really busy at work so I haven't had time to look at this in detail, but I think the proposal address an important problem that I've actually encountered, so I think it is great that we are doing something about it, so I am +1 in principle.

-Iavor

On Fri, Mar 2, 2018 at 9:05 AM Ben Gamari wrote:

...

Ben Gamari writes:

...

Hello everyone,

I have been assigned as the shepard for proposal #108. This proposes a change in GHC's plugin interface allowing plugins greater control over recompilation, significantly improving compilation times for projects relying on compiler plugins.

Summary:

The proposal suggests that the existing fields of the `ghc` package's Plugin.Plugin type be wrapped with a type capturing a function computing the plugin's recompilation desired behavior,

data PluginRecompile = ForceRecompile | NoForceRecompile | MaybeRecompile Fingerprint

GHC can then use this recompilation information to decide whether evaluating the plugin (and consequently invalidating any existing compilation artifacts) is necessary.

Opinion & recommendation:

The proposal addresses a long-standing limitation (#7414) of the GHC plugin interface which has become increasingly visible to users in recent years. While the particular approach proposed breaks existing plugin users, it is expressive enough to allow GHC to perform more aggressive recompilation avoidance in the future [1]. Moreover, the smart constructors proposed should make for a reasonably smooth migration path.

Given that the plugins appears to have buy-in from a few notable plugin authors, I recommend that we accept this proposal.

I've pinged a number of prominent plugin authors and heard no reply. Therefore, I would again recommend that we accept.

Cheers,

- Ben _______________________________________________ ghc-steering-committee mailing list ghc-steering-committee@haskell.org https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee

_______________________________________________ ghc-steering-committee mailing list ghc-steering-committee@haskell.org https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-steering-committee

Joachim Breitner writes:

Hi,

Sorry for the late response; this was buried in my inbox.

we are all in agreement that we want plugins the ability to avoid recompilation. The open question is: Should this be on a per module basis? Or a per module-and-pass bases?

The former has a simpler and backward-compatible API, the latter sets the stage for some potential future where the compiler might typecheck a program, but then abort and not code generate.

… seriously, I still don’t see how that future can be. Can someone please give a concrete example of circumstances where that might happen?

Ben hints at sub-module granularity, but that slices it orthogonal to the current proposal. He also write

...

Lumping together all of the recompilation checks into a single hash would expose a relatively ugly facet of GHC's implementation to plugin authors and force authors to tie together logic from conceptually-distinct compilation phases.

but that is also not quite true. Note that the hash is not generated _during_ the passes, but needs to be calculated before. The type signature of the recalculation check, namely

pluginRecompile :: [CommandLineOption] -> IfG PluginRecompile

clearly indicates that this is a per-module check (otherwise, you’d see an ModGuts input, or such).

And even if, in the future, we have ability to provide a finer grained mechanism, we can add the necessary functions _then_ when we know how it looks.

Your argument is pretty compelling. I'll admit that part of my dislike of the the simple approach that you advocate is purely aesthetic; it just feels terribly ad-hoc. That being said, you do raise a valid point that we really don't know enough to be able to say what a more precise interface would look like. Moreover, there is something to be said for the nice backwards-compatibility story that the simple approach allows. I would be fine accepting the simple approach. Either way, I think we should move this proposal forward to unblock our contributor. Cheers, - Ben

Hi Ben, Am Donnerstag, den 22.03.2018, 19:59 -0400 schrieb Ben Gamari:

Does anyone object to accepting the simple approach as suggested by Joachim?

I think you can go ahead with this one. Cheers, Joachim -- Joachim Breitner mail@joachim-breitner.de http://www.joachim-breitner.de/

Hi, Am Samstag, den 14.04.2018, 12:42 -0400 schrieb Ben Gamari:

Joachim Breitner writes:

...

Hi Ben,

Am Donnerstag, den 22.03.2018, 19:59 -0400 schrieb Ben Gamari:

...

Does anyone object to accepting the simple approach as suggested by Joachim?

I think you can go ahead with this one.

Sounds good. Let's accept.

I guess someone (maybe you as the shepherd) needs to either update the proposal with the results from this discussion before you merge it, or you need to request this from the author. Cheers, Joachim -- Joachim Breitner mail@joachim-breitner.de http://www.joachim-breitner.de/