Perhaps I can add my perspective.
This thread is largely about LLM generated tickets. Mikolaj in particular has been posting a series of a couple of dozen such tickets -- so many in fact that I started collecting them on a tab of my personal WIP spreadsheet.
You can see it here (look at the Mikolaj tab). From this I have learned
- All of them that I have investigated so far have proved genuine -- they expose genuine shortcomings in GHC.
- It's pretty overwhelming, even discouraging, to get so many in a short time. And yet, and yet.... I would rather know than not-know.
- The explanations are often long, but I zoom in on the reproducer, and only then refer back to the explanation. As Mikolaj says, these reproducers are far, far, far better than the gigantic reproducers he used to offer. Provided the ticket has a short statement of what is wrong, and a reproducer, I don't mind if there is AI-generated goop to accompany it.
Indeed, once I can reproduce it, the goop is often somewhat helpful.
- I say "somewhat" because
once I have understood the problem, I can often come up with a much crisper summary myself Here's a concrete example, with my summary belo.
- It would be absurd to expect Mikolaj to write every word. He doesn't know enough about GHC's internals. If I insisted on that, he just wouldn't submit them, and that would be a shame.
- The biggest shortcoming is this: I have no idea how important the ticket is. When all tickets were human generated, I knew that every single one was somewhat important to some human. I don't want to spend time fixing bugs that no one cares about, and I'm 100% against LLM's autonomously submitting tickets. Every ticket must have a human behind it. Meanwhile, I asked Mikolaj to prioritise his flood of tickets, which he has done as you'll see on the spreadsheet.
- One way in which the flood can feel exhausting is simply because it exposes so many embarrassing shortcomings in the compiler on which I have lavished so much love and care. How can I have been so stupid in so many different ways? Why is GHC so over-complicated? It is humbling. But in a good way, I think.
Finally, while tickets may have a lot of readers, they are ephemeral. What I really care about is the code and the Notes in the code. It will be read for decades, whereas no one in 10 years time is going to be reading today's tickets. (There are occasional exceptions.) For the code and Notes I want every line to be human written, or so well studied, audited and revised that it might as well be human written.
TL;DR: for me, provided a ticket has a human author who actively cares about, clear signposting and attribution, and a concrete reproducer, I'm pretty relaxed about accompanying slop.
Relatedly, let me add that I have also been requesting LLM reviews for my MRs, through Simon Jacobi's good offices, and I have been astonished how good they are. They often identify genuine issues with my lovingly crafted MRs, and ones which both I and human reviewers have missed. At the time it's a bit "Oh no, I wanted to land this and now I can't", but again it must be better to know about the bugs than not to know. Again it's a humbling experience.
So thus far, I have had a pretty positive experience of LLM-generated bug reports and reviews. But as I say above, I am much, much more cautious about contributions to GHC's code base and documentation.
Simon