Waiting for a Bigger Context Window Is a Bet Against Four Orders of Magnitude
The most common response to everything written about codebase context is that the labs will fix it. Every quarter a model ships with a larger advertised window, so the failure looks temporary rather than structural.
That is a reasonable prior rather than a lazy one. Almost every capability complaint about language models over the last four years has been answered by the next release, and people who bet against that pattern have mostly been wrong.
This one looks different, and the argument is arithmetic rather than opinion.
The gap, stated plainly
At a conservative 1,000 tokens per source file, 10 million files is roughly 10 billion tokens of code.
| Context window | Share of a 10 million file codebase it holds |
|---|---|
| 200,000 tokens | 0.002% |
| 1,000,000 tokens | 0.01% |
| 10,000,000 tokens | 0.1% |
A 10 million token window, which is well beyond anything generally available, holds one thousandth of the codebase. Closing that gap by scaling requires something like a 10,000x increase rather than a 10x one. That is not one model generation away and it is not two.
Compression does not rescue it either. The best context packing tools cut token count by around 70% using tree sitter based stripping, which takes 10 billion tokens down to 3 billion. The window that held 0.01% now holds 0.03%. A constant factor applied to a problem that is off by orders of magnitude leaves you off by orders of magnitude.
The part people expect to go the other way
You might reasonably say that partial coverage is fine if retrieval picks the right fraction. That would be true if quality per token held steady as the window grew, and the evidence says it does not.
The lost in the middle finding is the well known version. Models retrieve reliably from the beginning and end of a long context and systematically underweight what sits in the middle, so information buried in a large window is not equally available.
The distractor finding is the more damaging one. When a needle is semantically distinct from its surroundings, models retrieve it reliably. When distractors are semantically similar to the answer, accuracy drops sharply, and the drop widens as input grows. A bigger window admits more near matches, which makes the retrieval problem harder rather than easier.
There is a stranger result in the same family. Chroma compared two versions of the same haystack, one preserving the natural flow of ideas within each excerpt and one where sentences were randomly reordered to hold the topic without logical continuity. They expected coherence to win, on the reasoning that a needle dropped into flowing prose would disrupt that flow and stand out. The result went the other way, with structural coherence consistently hurting performance and removing local coherence consistently improving it, holding across all 18 models tested.
The reading with most support behind it connects back to distractors, since logical document flow produces more plausible seeming near matches. Applied to this argument, a well written architecture document is coherent prose about your codebase, which makes it a distractor factory by construction. A graph slice returning a dozen facts with a file and a line attached carries almost no narrative coherence, which places it nearer the condition that measured better.
The shuffled advantage was described as slight though consistent, the experiment ran on essays and academic papers rather than on source code, and nobody has replicated it on either. It undercuts the instinct behind stuffing a tidy document into a large window rather than proving the case, and it is better to say that than to overclaim it.
What the benchmarks are already showing
The collapse is visible without any of this theory. Top models score around 53% on the SWE-Bench Pro public set against more than 70% on SWE-bench Verified. The harder benchmark exists specifically because it draws on diverse and complex real codebases including consumer applications, business services and developer tools.
The delta between those two numbers is what happens when a model meets a codebase rather than a well scoped repository task, and it has not closed as windows have grown.
There is also a session level version of the same decay. One 2026 study across 4,416 trials at six conversation depths found compliance with an agent’s own explicit constraints dropping from 73% at turn 5 to 33% at turn 16 with no memory mitigation. More room in the window has not produced more consistency inside a session.
The alternative is to stop putting code in the window
ByteBell is a verification layer generated on top of a codebase of any size below 10 million files, and it works by making the window irrelevant to the size of the codebase.
Cheap open source models read every file once at indexing time. An agent detects the boundaries of each language element, stores them separately, then analyses each one in the context of its file, its module, its repository and across every repository. For each file the layer records what it does, why it exists in business terms, what users lose if it fails, the capabilities it provides, its edge cases, the contracts it provides, its preconditions, and the specific places where the behaviour is genuinely undecided. Everything in plain English with the file and the line kept on every claim.
A question then touches a handful of pre-digested graph nodes rather than a slice of raw code. The window never has to approach the codebase, because the codebase was already read, once, by something cheap.
Why that improves accuracy rather than trading it away
The usual assumption is that reading less means knowing less. That holds when the model has to establish meaning at question time, because then accuracy is bought by reading more files.
The layer breaks the relationship. A model without it runs a long chain of dependent steps to work out what the code means before it can decide anything, and each step can go wrong while the growing context degrades per step accuracy at the same time. A model reading the layer skips the entire chain, because the meaning was established once, carefully, before any question existed.
Frontier models are better at running that chain, and they are not better at a chain they never have to run. Our internal measurement is that the layer adds somewhere between 20 and 30 points of accuracy on top of any sufficiently large model, while five leading models currently sit within 0.4 points of each other on SWE-bench Verified, which puts the layer at roughly 50 times the value of the model choice.
The numbers
Indexing runs about 0.89, against roughly 0.11.
Accuracy of code holds above 93% while token spend drops by roughly 80%. Because providers charge per token, brute force gets more expensive with every question, while cost per question against a prebuilt layer stays close to flat.
What a bigger window is genuinely good for
Long single documents, large transcripts, extended reasoning within one task, and holding a working set that genuinely fits. Those are real gains and every window increase delivers them. If your entire codebase is 100,000 tokens, put it in the window and skip everything above.
The argument here is narrower than it sounds. It is only that scaling the window is the wrong instrument for a 10 million file estate, because the gap is four orders of magnitude and quality per token moves against you while you close it.
The honest limitation
Our approach trades a window problem for an index problem. The first pass on a large estate costs real money and real time, the index has to be maintained, and a model derived layer can be wrong about what code means in ways a raw file read cannot. Every claim carries a file and a line so any statement can be checked in about 10 seconds, and the index re-derives on every commit, though there is a window after a push where a claim can lag the branch.
We also do not cover every language. Mojo is not indexed today.
This is what ByteBell builds
The calculation to run on your own codebase takes a minute. Count your files, multiply by 1,000, and divide by the window of whatever model you use. If the answer is a fraction of 1%, the strategy of waiting now has a number attached to it.
ByteBell is the verifiable context layer for code. We run the LLM compiler pattern, a one time pass where a model reads every file and lowers it into a verifiable code IR capturing purpose, business context and cross repository relationships, on your own infrastructure through Docker, with your source never leaving your environment. Every engineer, on any copilot, queries the same representation through a single MCP url, and every agent edit is checked against it before it lands, using per file SHA-256 diffing so only what actually changed gets examined again. Across 46 Kubernetes ecosystem repositories and 150,000 files we measured roughly 10% higher accuracy at 70% lower cost, on about a fifth of the tokens.