Last time I argued that AI pre-review quietly drained the politics out of the pull request, and that the toxic reviewer did not retire so much as relocate. I promised to write about the new arena when it emerged.
It has emerged. It is smaller than the old one, and it does not look like an arena at all.
It looks like a pull request description.
Picture a three-line change. A rename, a nudged default, nothing anyone would slow down for. The AI reviewer picks it up, thinks for forty seconds, and posts: no issues found. A human skims the green check and merges.
Now picture the same pull request carrying one extra line the human never saw, because GitHub does not render it, sitting in an HTML comment above the diff.
The reviewer saw them. The reviewer did what they said.
The Territory Is Whatever the Model Reads
Here is the thing about moving review from people to a service: you also move the attack surface.
A human reviewer reads rendered markdown. An AI reviewer reads raw text. Those are not the same document. Everything GitHub politely hides from you - HTML comments, collapsed <details> blocks, zero-width characters, white-on-white styling - arrives at the model as instructions with exactly the same weight as the maintainer's own prompt.
The model has no way to tell them apart. There is no channel separation in a prompt. There is just text, and text does not carry a provenance flag.
So the question "who controls this review" stops being about seniority and starts being about who can write into the context window. Anyone who can open a pull request can. That includes a first-time contributor from an account created ten minutes ago.
The old arena was the comment thread, and you had to be present in it to win. The new arena is the input, and you win before anyone shows up.
Three Disclosures, One Mechanism
This is not a thought experiment. Through 2025, security researchers landed the same finding against three different products. Two of the three came from one team, which tells you how repeatable it was.
| Product | Injection vector | Result | Disclosed |
|---|---|---|---|
| GitLab Duo | hidden text in MR descriptions, commit messages, issue comments | private source code theft, HTML injection into other users' responses | Legit Security, Feb 2025 (published May 2025) |
| GitHub Copilot Chat | a hidden comment in a PR description | private repo contents and secrets exfiltrated (CVSS 9.6) | Legit Security, Jun 2025 (fixed Aug 2025, published Oct 2025) |
| CodeRabbit | a malicious .rubocop.yml in a pull request | RCE on production servers, leaked tokens, read/write on roughly a million repositories | Kudelski Security, Jan 2025 (published Aug 2025) |
Read the middle column again. Two of the three needed nothing but prose. Not a dependency, not a build step, not a compromised maintainer account. A comment.
The GitLab Duo work is worth reading in full, because the researchers did not stop at plain text. They smuggled payloads through Unicode, Base16 encoding, and KaTeX rendered in white. Duo read the whole project context, so a poisoned commit message from a year ago was still live ammunition.
CamoLeak, found by Omer Mayraz at Legit Security, is the more elegant one. Copilot Chat could not be made to call out to an attacker's server, because the content security policy forbade it. So the researcher pre-generated a dictionary of URLs through GitHub's own Camo image proxy, one per character, and instructed Copilot to render the stolen data as ASCII art built entirely from images. Each 1x1 transparent pixel fetched in sequence spelled out a secret on the attacker's server, carried there by GitHub's infrastructure. The fix was to stop rendering images in Copilot Chat.
That is not a fix for prompt injection. That is closing one exit.
And CodeRabbit is the one that should make anyone running a review bot uncomfortable, because it did not involve the model at all. A pull request supplied a linter config; the linter loaded a Ruby file the pull request also supplied; that code ran on production infrastructure holding tokens with write access to about a million repositories. The AI was incidental. The architecture was the vulnerability.
To be fair to all three vendors: every one of them patched, and CodeRabbit's response was genuinely fast - Rubocop disabled within hours, credentials rotated, execution moved into a sandbox. This is not a story about negligent companies. It is a story about a shape of system that keeps producing the same bug.
The Shape Is the Problem
Simon Willison named it in June 2025: the lethal trifecta. An agent is dangerous when it has all three of private data access, exposure to untrusted content, and a way to communicate outward. Any two are survivable. All three is an exfiltration engine.
A GitHub App code reviewer has all three by construction.
It reads your private repository, because that is the job. It ingests pull request prose from strangers, because that is the job. And it holds a write token so it can post its review, which is both an outward channel and a set of hands.
OWASP has been blunt about the consequences. Prompt injection sits at LLM01, first on their list for LLM applications, and their 2026 Top 10 for Agentic Applications - built from incidents in real systems rather than projections - puts goal hijacking through injected content near the top of the agentic risks too. What is telling is the mitigation they recommend. Not a better filter. Limit tool privileges. Require human approval for high-impact actions. Treat natural language input as untrusted.
That is architecture advice, because the prompt filter you were hoping for does not exist.
I sat with that for a while and concluded the honest version is uncomfortable. If prompt injection is unsolved, and my reviewer needs untrusted input to do its job, then the only lever I actually control is what the reviewer is allowed to do and where the code goes.
So I rebuilt mine around removing capabilities instead of adding filters.
What I Built Instead
It is a shell script called prr. It reviews a pull request on my laptop, using a model running in Ollama on localhost, and it has no code path that writes to GitHub. Not a disabled one. Not one behind a flag. There is no function that posts.
Four stages:
Read. gh pr view and gh pr diff. Read-only calls.
Scan the prose before any model sees it. A local script walks the title, body, branch name, every comment and every review, looking for three things: characters that are invisible to a human but not to a tokenizer (zero-width marks, bidi overrides, Unicode tag characters, soft hyphens), markup that hides text (HTML comments, instruction-bearing <details> blocks, styles that render text at zero size or in white), and phrasing that only makes sense if the intended reader is a model. Fake system: role markers. Chat template tokens. "Ignore all previous instructions." "Do not report." "There are no security issues." Text is NFKC-normalized first, so homoglyph padding does not walk past it. A hostile verdict stops the run before the model is invoked at all.
Run something that cannot be argued with. Semgrep, locally, --metrics=off. A deterministic scanner has no opinion to change and no instructions to follow. It is the one finding source in the pipeline that is immune to everything above.
Then the model. pr-agent pointed at a local Qwen3-Coder, with publish_output false, so the review prints to my terminal and lands in a file.
The properties that matter are not in the model. They are in what is missing.
| GitHub App reviewer | prr | |
|---|---|---|
| Where the diff goes | vendor infrastructure | localhost |
| GitHub write access | required, to comment | none |
| Who triggers a review | a webhook, on any PR | me, on a PR I name |
| Untrusted PR prose | goes straight into the prompt | scanned first; hostile input stops the run |
| Deterministic pass | usually none | Semgrep, before the model |
There is no write token, so there are no hands. There is no egress, so there is no channel. Two legs of the trifecta are gone, and the third stops mattering as much when the worst case is a wrong opinion printed in my own terminal.
The Bill
I want to be specific about what this costs, because "run it locally" is usually said by people who have not.
The model is nineteen gigabytes. On a 24 GB machine that is most of the memory. A 19,000-token diff took two minutes and forty-nine seconds on the first run and thirty-five seconds once the weights were resident, which is fine, and a 30B model finds less than a frontier model does, which is the actual cost. That is the trade, made deliberately: this is the pass that runs on code I am not willing to send anywhere, and a smaller model reading your real diff beats a larger one you were never going to be allowed to call.
The setup was not clean either. pr-agent would not install on the Python 3.14 my machine had, because two of its dependencies have no wheels for it yet; it needed pinning to 3.12. My first script used mapfile, which macOS's bash 3.2 does not have. Neither is interesting, and both are why "just self-host it" is a two-hour errand rather than a one-liner.
My favorite one is more instructive. Setting publish_output=false does stop pr-agent posting the review. It also throws the review away: the only copy goes to a debug log line as a structured field, and the console log format it ships with explicitly does not print structured fields. So the safe configuration is the one where you never see the output. I ended up calling the reviewer class directly and printing the markdown myself. Read that again as a design signal: not publishing was not a supported way to use the tool.
The instructive failure was smaller and worse. On my first real run against one of my own pull requests, Semgrep reported no findings across seventeen files. It also emitted one warning I was not printing: it had failed to parse a TypeScript file and skipped part of it. "No findings" and "did not look" render identically if you only count results.
So the report now prints a coverage line - files scanned out of files changed - and a "coverage gaps" section listing anything the scanner could not read. An empty result is a state, not a verdict. That distinction is the entire difference between a tool and a green check mark.
One more limit, stated plainly: pr-agent reads pull requests through my existing gh token, which is broader than read-only. A fine-grained token with read scope is the tighter setup. The tool still cannot write, because it has nothing that writes, but I would rather say that than imply the credential is minimal when it is not.
Three Things This Does Not Fix
Injection detection is a blocklist, and blocklists lose. My scanner catches the published tricks. It will not catch the next encoding, and pattern matching on natural-language instructions is not a solvable problem. It is a speed bump that raises cost. The architecture is what actually holds; the scanner is just the part that tells me someone tried.
Local does not mean the model is trustworthy. A model running on my laptop is exactly as steerable by a poisoned diff as one running in a data center. What changed is the blast radius, not the obedience. Local review turns a data breach into a bad review comment. That is a large improvement and not a cure.
Somebody will notice this is a gate. Which brings me back to where the last piece ended. The reviewer that runs on my machine, with my rules, producing findings only I have seen, is a position. "My local pass caught this and yours did not" is available to exactly the same person who used to own the comment thread. Whoever writes the rule set decides what counts as a finding, and that is authority with a config file instead of a career behind it.
The arena did not close. It got smaller, more technical, and considerably harder to see from the outside.
That is still progress. It is still not a solution.
---Sources. Remote Prompt Injection in GitLab Duo (Legit Security) · CamoLeak (Legit Security) · How We Exploited CodeRabbit (Kudelski Security) · CodeRabbit's response · The lethal trifecta for AI agents (Simon Willison) · OWASP Top 10 for LLM Applications · PR-Agent · Semgrep
Comments
No comments yet. Be the first to share your thoughts.