← All posts

Augment Code Measured Our Thesis for Us, Then Stopped One Step Short

When Augment exposed its context engine over MCP in February 2026, Claude Code on Opus 4.5 gained around 80% on quality and Cursor gained around 71%, with code completeness up 60% and correctness improved fivefold. Somebody else measured that, on somebody else's product, and it is the strongest evidence anyone has produced that context architecture matters more than model selection. Here is where our architecture diverges from theirs.

Augment Code Measured Our Thesis for Us, Then Stopped One Step Short

Augment Code Measured Our Thesis for Us, Then Stopped One Step Short

In February 2026, Augment Code exposed its context engine over the Model Context Protocol, which meant other tools could read Augment’s index instead of building their own. The published results were startling. Claude Code running on Opus 4.5 gained around 80% on quality. Cursor gained around 71% on the same measure. Code completeness rose 60% and correctness improved fivefold.

Those numbers were measured by somebody else, on somebody else’s product, and published independently of us. They are the strongest evidence anyone has produced that context architecture matters as much as model selection, and possibly more. A research survey of codebase intelligence published in April 2026 arrived at the same conclusion, naming index first systems that build persistent graphs before the agent begins work as one of three competing architectural philosophies.

We had been arguing that position since October 2025 without external proof. Now it is measured, and Augment is the company that measured it.

So this is not a piece explaining why a competitor is wrong. Augment is right about the most important thing in this category. The interesting question is what they built and where it stops.

What the context engine does

Augment’s engine indexes a large codebase and serves relevant context to a model on demand, in real time, keeping pace with a repository that is changing underneath it. The retrieval is good, the scale is real, and the MCP exposure was the right product decision, because it let the value show up inside tools people already use rather than requiring them to switch.

The mechanism is context assembly. When a question arrives, the engine works out which parts of the codebase are relevant and hands them to the model in a form that fits the window. The model then does the reasoning about what that code means and what ought to change.

That is a substantial improvement over an agent reading files at random, and the measured gains show it clearly. It is also the thing we do differently.

Assembly at query time against reasoning at index time

The distinction sounds small when written down, and it decides both the cost curve and the accuracy ceiling.

In a context assembly architecture, the understanding happens at query time. Every question requires a frontier model to work out what the retrieved code means before it can answer anything. That is accurate and it is expensive, and the expense recurs on every question forever. It also means the quality of the answer depends on the model getting a long chain of dependent inferences right, while the growing context degrades per step accuracy at the same time.

In our architecture, the understanding happens once. Cheap open source models read every file at indexing time and produce, for each one, what it does, what it promises, its preconditions, the contracts it provides, its edge cases, the business consequence if it fails, and every cross repository dependency that touches it. That is stored as a specification layer over a graph, in plain English, with the file and the line kept on every claim.

When a question arrives, the model is not being handed code to interpret. It is being handed meaning that was already established, carefully, before any question existed. Frontier models are better at running the inference chain, and they are not better at a chain they never have to run.

Why that inverts the cost structure

A context assembly architecture needs the most expensive model at query time, because the architecture makes the model do the understanding. We need a cheap model at index time and a cheap model at query time, because the understanding is already stored.

The pricing gap between those two positions is now extreme. DeepSeek V4 Flash charges around 0.28permillionoutputtokensagainstGPT5.5atroughly0.28 per million output tokens against GPT-5.5 at roughly30, and premium tiers stretch that further. Meanwhile the quality objection has largely collapsed. Five models now sit within 0.4 points of each other on SWE-bench Verified, between 80.2% and 80.6%, a set that includes DeepSeek V4 Pro Max, Qwen3.7 Max, Kimi K2.6, MiniMax M3 and Gemini 3.1 Pro.

Put those two facts together and the conclusion becomes uncomfortable for anyone whose economics depend on frontier access. Switching from the fifth best coding model to the best one buys about 0.4 points. Our internal measurement is that the verification layer adds somewhere between 20 and 30 points of accuracy on top of any model above roughly 500 billion parameters, which puts the layer at something close to 50 times the value of the model choice.

Our own figures follow from that. Indexing runs about 13per1,000files,once,withonlychangedfilesrederivedoneachcommit.Onatracedrun,tracingachangeendtoendacross32impactedfilesin4repositoriestook4minutes9secondsandcost13 per 1,000 files, once, with only changed files re-derived on each commit. On a traced run, tracing a change end to end across 32 impacted files in 4 repositories took 4 minutes 9 seconds and cost0.89, against roughly 30withoutthelayer.Afulltestgenerationandreviewpasscameto87,001tokensat30 without the layer. A full test generation and review pass came to 87,001 tokens at0.11. Accuracy of code holds above 93% while token spend falls by roughly 80%, which is how the layer manages to decrease AI cost and improve correctness in the same pass.

The second divergence, which is verification

Context assembly is good at answering questions. It does not check anything.

Once the layer holds what each file promised, a change can be compared against that promise rather than judged on its own terms. We trace what the change touches, generate tests for it, run them, and then review each changed file against what that file committed to at the indexed base. Collateral damage is checked across the other repositories that read the same helper before anyone approves.

That reverses the direction of the whole product. Retrieval runs from question to code. Verification runs from proposed change back to intent, which is the only way to catch a hallucination before it merges rather than after it ships.

The third divergence, which decides who can buy

A hosted context engine cannot serve a bank, an insurer, a healthcare payer or a defence contractor, because those organisations are frequently barred by law from sending source code to a vendor cloud. We run hosted, on dedicated hardware, or fully inside your own network, with read only access in all three and no source leaving your infrastructure on the second two. You bring your own model keys, including Anthropic, Gemini, OpenAI Codex, DeepSeek, Qwen, Z.ai or anything reachable through OpenRouter.

The honest limitation

Real time freshness is genuinely hard, and Augment has invested more in it than we have. Our index re-derives on every commit, which is fast, and there is still a window after a push where a claim can lag the branch. If your workflow depends on sub second index currency during an active editing session, test that specific behaviour rather than taking our word for it.

We also do not support every language. Mojo is not indexed today, and the remaining gaps are the new and the rare ones.

This is what ByteBell builds

The most useful thing Augment did for this category was publish a measurement nobody could dismiss as vendor marketing. Context architecture beats model selection. The open question is whether that context should be assembled fresh on every question or reasoned about once and stored, and the cost curves of those two answers diverge further with every quarter that open weight models get cheaper.

ByteBell is the verifiable context layer for code. We run the LLM compiler pattern, a one time pass where a model reads every file and lowers it into a verifiable code IR capturing purpose, business context and cross repository relationships, on your own infrastructure through Docker, with your source never leaving your environment. Every engineer, on any copilot, queries the same representation through a single MCP url, and every agent edit is checked against it before it lands, using per file SHA-256 diffing so only what actually changed gets examined again. Across 46 Kubernetes ecosystem repositories and 150,000 files we measured roughly 10% higher accuracy at 70% lower cost, on about a fifth of the tokens.

www.bytebell.ai

All posts