Context windows grew roughly a hundredfold in three years, and the folk conclusion was that context management had become someone else's problem. The measurements say otherwise, and the most useful finding in them is the one most often quoted backwards.
What Chroma actually measured
Chroma's context rot research is the most systematic public work on this. It evaluated 18 models across roughly 194,480 LLM calls, sweeping 8 input lengths against 11 needle positions, plus controlled variations in distractor content and haystack structure.
The design matters. Standard needle-in-a-haystack evaluations use lexically obvious targets in filler text, which is close to the easiest possible retrieval task — and models score near-perfectly on it, which is where the confidence in long contexts came from. Chroma varied the things that make real work harder: the semantic distance between question and answer, the presence of plausible-but-wrong material, and the internal ordering of the surrounding text.
The headline result is that performance degrades consistently as input length grows, and it does so across all 18 models tested. This is not a weakness of one architecture or one vendor. It is a property of the current approach.
Degradation is not smooth, which is why it surprises people
If long-context performance decayed gracefully, it would be easy to manage: you would notice quality sliding and trim. What the research found instead is that models hit cliffs. Behaviour holds up, holds up, and then falls off sharply — different models falling at different points, with no advance warning in the output.
This has a direct operational consequence. You cannot tune context budgets by feel. A prompt that worked reliably at 40,000 tokens can fail at 60,000 in a way that produces confident, fluent, wrong output rather than an obvious error. There is no degradation signal to watch for; the failure looks exactly like success.
Position compounds it. Accuracy drops by more than 30% in some configurations when the needed information sits in the middle of a long document rather than near the beginning or end. So the same fact, in the same context, produces different reliability depending on where it landed — and in an agent loop where context is assembled dynamically from tool results, nobody controls where anything lands.
Length alone costs you something
The research separates two effects that usually get conflated: harm from length itself, and harm from what the extra length contains.
Input length alone accounted for a loss of roughly 7.9%. That is the tax on padding — the cost of a longer window even when the additional material is inert. It is not catastrophic, but it is not free either, and it undercuts the idea that unused context is harmless.
The larger effect is what you fill it with. Semantically similar but irrelevant distractors degraded performance more than random noise did. Filler the model can dismiss is comparatively cheap. Material that looks like the answer but is not is expensive, because distinguishing it requires exactly the discrimination the model is least able to sustain at length.
Read that against a real repository. The most similar-but-wrong documents you own are your superseded designs, your abandoned proposals, and your older instruction files — all of which are topical, detailed, and confident. Loading "everything relevant" loads precisely the worst class of distractor.
The shuffled-haystack finding, stated carefully
The most counter-intuitive result: models performed better when the haystack was shuffled than when it was arranged in its original logical order.
This has been repeated as evidence that organising your documentation is pointless. It is not that, and the distinction is worth being exact about.
The finding concerns the internal ordering of a large block of text that has already been placed in the context window. A coherently ordered haystack invites the model to follow the narrative — to read it as a document with an argument, tracking themes across it. That is expensive attention spent on structure that is irrelevant to the question. Shuffled text offers no narrative to follow, so more attention goes to the local matching that the task actually requires.
Two things it does not mean. It does not mean structure is harmful to readers — the experiment measured retrieval from a monolithic blob, not navigation of a well-organised corpus. And it does not mean you should scramble your repository, which would make it harder for both humans and agents to find anything.
The lesson is about volume, not order: the problem is that the whole haystack was loaded at all. Structure earns its keep before the window, by determining what gets loaded. Once material is in the window, the model does not benefit from its arrangement — so the win is in loading less, not in ordering less.
Agent loops make the curve worse
The measurements above come from single-turn evaluations: one prompt, one long input, one answer. Coding agents do not work that way, and the difference runs in the unhelpful direction.
An agent session accumulates. Each file read, each command output, each failed attempt and its stack trace stays in the context for the remainder of the session. A task that starts at 5,000 tokens is at 60,000 by the twentieth tool call, without anyone deciding to load anything. The context length is an emergent property of how much fumbling happened, not a designed budget.
Two things follow. First, quality degrades over the course of a session in a way that is easy to misread as the task being hard — the agent seemed sharp at the start and vague later, which looks like a difficult problem and is partly a length effect. Second, everything the agent read and discarded as irrelevant is still in the window, still occupying attention, and much of it is topically similar to the task, which is exactly the distractor profile Chroma found most harmful.
This is the strongest practical argument for narrow reads. A tool call that returns a 4,000-line file to extract one function signature has not cost you one read; it has raised the floor for every subsequent turn in that session. Restarting a session when it starts to wander is not superstition — it is the only available way to clear accumulated distractors.
What this implies for how agents read a repository
The practical translation is a navigation discipline rather than a context-assembly strategy.
- Route, then read. An agent should read a small index that describes what exists and where, then open the one or two leaves it needs. It should not bulk-read a domain to find out what is in it.
- Make indexes small and boring. An index exists to be cheap. If it acquires explanatory prose, it becomes another thing to load in full and the discipline collapses.
- Retire distractors explicitly. A superseded design that is marked superseded can be skipped. The same file with no marking is the highest-quality distractor in your corpus.
- Treat context budget as a budget. Every token loaded speculatively is spent against a measured degradation curve. "It might be useful" is not a justification.
Why a bigger window does not fix any of this
Capacity was never the binding constraint. Even if a model could hold a repository perfectly, it would still have no way to tell a decision from a proposal, or a current runbook from a superseded one, because that information is not in the text. A larger window changes how much undifferentiated material fits; it does not differentiate it. That argument is developed in context is not knowledge.
This is the reasoning behind index-first navigation as a rule rather than a preference, and it is the discipline Codee3's governed layer is built around: a folder index that routes, leaf documents that carry declared authority, and guards that fail CI when a folder loses its index or the reference graph breaks. The point is not that the structure makes the model smarter. It is that it lets the agent load less and know more about what it loaded.