AI code review
6 posts carrying this tag.
We Measured Grep on Real Codebases. Here's What Actually Breaks.
We measured how coding agents search with grep on Home Assistant and Saleor: about 150 model runs and 13 million tokens. Grep finds the files, then buries them in noise, and it can't follow inheritance or code that never says the name. A bigger model doesn't fix it.
A Team of Agents Built a System in a Week and Left €15M of Unmaintainable Code
The number that gets quoted from that case study is the token bill, somewhere between €10M and €15M for one week of autonomous agent work. The number that should get quoted is the second half of the sentence, which is that the resulting code was described as nearly unmaintainable. The bill was paid once. The code has to be lived with.
The Diff Is the Wrong Unit of Review, and CodeRabbit Is Built on It
A pull request changes 11 lines, the tests pass, every automated reviewer approves, and three weeks later a reporting service is quietly returning wrong revenue. Nothing about the diff was incorrect. What was wrong lived in another repository nobody in the pipeline had read. This is where diff scoped review has nothing to say, and it is now the majority of what breaks in a large organisation.
Greptile Indexes the Codebase, Which Is Necessary and Not Sufficient
Of every commercial tool in automated code review, Greptile made the closest architectural call to ours: index the whole repository first, then review the change against it. That decision is why it catches 82% of bugs against CodeRabbit's 44%. It also arrives with about 11 false positives per run, and the shape of those false positives tells you exactly what an index of symbols can and cannot know about intent.
Everyone Open Sourced the Hands This Year. Nobody Open Sourced the Check
Agent runtimes got very good very fast. The thing that decides whether a change is correct across your other repositories did not get built at all. On cross repository blast radius, why AI code review misses it, and what a check actually needs.