Thanks for all the feedback, everyone! Regarding the choice of channel, how about I upload the reviews as snippets to e.g. https://gitlab.haskell.org/sjakobi/llm-reviews/-/snippets/? The advantage of snippets is that people can comment on them, so I can easily collect feedback on the quality of the reviews. It's important to me though that there is some visible connection from the MR to the review: 1. The information in the review is intended for all people who are involved in an MR: Both authors and reviewers. 2. Similarly, I'd really love to get feedback from everyone who cares to read the reviews. This way I can improve the service more quickly. 3. I want some visibility, so more people are likely to give the service a try. In case you're worried about your AI agents ingesting the generated reviews, I think it shouldn't be too hard to block access to that repo for them. In other news, Anthropic has generously accepted my application for their open source program (https://claude.com/contact-sales/claude-for-oss), so for the next 6 months, I do have a decent amount of usage that I can spend on reviews (including access to Fable 5!). Consequently, I have started vibe-coding a primitive "mini-sashiko" for GHC: GlaHiCo (https://gitlab.haskell.org/sjakobi/GlaHiCo/-/merge_requests/1). Right now there's just a single "finder" agent, and each finding gets checked separately by a "verifier" agent. The code is a mess already of course, but it's slowly getting useful too. If you have a feature request or want to contribute in other ways, please let me know! :) Cheers, Simon Am Mo., 27. Juli 2026 um 19:38 Uhr schrieb Simon Jakobi <simon.jakobi@gmail.com>:
Hi devs!
Inspired by the recent discussion on the LLM policy, I'd like to try introducing LLM-generated code reviews in GHC, as an opt-in service.
My motivation:
* Catch issues before human reviewers spend time on them. * Reduce the number of bugs merged.
What this is — and what it isn't --------------------------------
This is tool output: requested on the MR, produced by a model, posted verbatim and clearly marked as machine-generated. This is not a review in the sense of draft LLM policy.
I'm not vouching for any of it. Think of it as a linter the author could have run themselves, except on my LLM budget. Nobody is required to act on it even read it.
I mention this explicitly because the policy asks reviewers to take full responsibility for each line of their review. I can't do that here — it would mean checking every finding myself, which defeats the purpose and would restrict the service to areas of the compiler I already know well.
How to request one ------------------
* Ping me on the MR with a comment like "@sjakobi llm-review please". * Feel free to request particular aspects the review should cover. * Authors can request a review for their own MRs. Maintainers can request one on any MR. * If I have enough usage left on my plan, I'll confirm that I'm on it. * I'll then prompt the model, and post the result on the MR. * Feedback on the quality of the review is of course very welcome.
What I promise --------------
* Every review is labelled as machine-generated, with model, effort level and the exact prompt used. * Reviews are posted verbatim. I don't edit or curate them, so what you see is exactly what the model produced. * Only the public MR diff and public repository context go to the model. Nothing else. * I'll keep working on the prompts and the choice of model to improve the signal-to-noise ratio. * If the signal-to-noise ratio stays inacceptably bad, we can simply stop this service.
Review coverage ---------------
I'll probably start with a fairly basic review prompt, primarily aimed at correctness issues.
Possible extensions:
* Documentation consistency: check that documentation stays consistent, including Notes elsewhere that reference the changed code. * Performance: check for performance issues — potentially including checking the -ddump-simpl output for the changed code for unnecessary allocation etc. * Feel free to suggest anything else!
Format ------
Initially a single comment per review. Once we reach a good signal-to-noise ratio, we can consider inline comments on the diff.
Caveats -------
This is a volunteer service I intend to provide in my spare time. If I'm AFK or on vacation, reviews will take longer. If I can't keep up with requests, I will prioritise which MRs I run. I might take a break entirely.
Getting involved ----------------
Is anyone interested in joining this effort? If so we could form a team (@llm-reviewers?), and share prompting techniques, model choices, etc.
If the service turns out to be genuinely useful, we can look into automating it and – if necessary – possible funding.
Cheers, Simon