Thanks to everyone who has contributed to this thread. Lots of good, civil discussion, along with robust disagreement.
I have made lots of (mostly small) edits in response. Some more reflections below
Simon
Here are some of my personal thoughts
- Beautiful code. Yes, every enhancement to GHC comes with a complexity cost; we have to make a judgement about whether the benefit pays for the (permanent) cost in terms of complexity. I have updated the principle to say this briefly.
- Agnostic. Some thoughtful people believe that we should have a blanket ban on LLMs. Others, equally thoughtful, believe that they have a helpful role to play in software development. I do not believe that we have anything close to a consensus on this, so I have framed the draft policy so that it is explicitly agnostic about whether LLMs are, in themselves, good or bad. That will please no one, but the alternatives seem worse to me.
- Principles. In contrast there seems to be universal agreement about the principles of (a) a human conversation among developers and (b) a high-quality beautiful code base. No one at all is arguing for blurting large quantities of machine-generated slop into GHC's code base or documentation, even if it passes the regression tests and superficial sniff-testing.
- I personally do not want to express or institutionalise a second-class status for LLM-assisted contributions. So I take Moritz's concerns on this score seriously.
But I do want to express a very strong desire for human judgement, responsibility, effort, and responsibility in GHC contributions. (And Moritz does too.)
How best to express that pair of desires? It seems difficult to find language that is acceptable to all, despite our agreement on the principles this language expresses. My proposal is to - Require full responsibility. For now at least I have settled on "human authorship" as a term to express this idea strongly -- adding language to say that (in this context) "authorship" does not mean "typed every word". But it does mean reviewing every word and almost certainly revising many of them.
- Recommend writing it yourself. I am worried that, even with the best of intentions, I might submit code that I had not forensically reviewed in the above way. Time is short, I want to get this done, honestly it looks pretty good etc. I think that the discipline of writing it out myself would force me to think. Is this duplication necessary? Why does this function need its third parameter?
But that's just me. Mindful that others may work differently, I have framed it as a recommendation only.
I have adjusted the language a bit in the light of the conversation of the last few days.
- LLM-only reviews. The policy explicitly says that if an LLM helps you to find a bug, with a repro case, it's fine to report the bug even if you don't understand its cause or how to fix it; that is the case with many human-reported bugs. But I am personally much more sceptical about LLM-generated reviews (of MRs) where no human has forensically checked the review. There may be good content in it; or it may just be distracting.
Perhaps in due course we'll want a channel for LLM-only reviews; but in the meantime the proposal asks that reviews follow (P2).
- Attribution. I continue to think that (P3) is important: we should be clear if an LLM has written any part of the material we are asking someone to review. That is transparent, gives us helpful data, gives the reviewer a bit more context, and helps those who want to avoid LLMs altogether.