When content creators learn to optimize for a fixed ranking signal, the ecosystem does not just get more efficient at pleasing the ranker. In CHASE, it steadily moves away from independently judged document quality, even though the content becomes more aligned with the features that predict winning under the ranking model.
That matters because this is not a one-off prompt experiment. The paper runs the same loop over and over: rank, identify which features predict success, rewrite the non-winners, and evaluate again. The ranker stays fixed across rounds, so any drift comes from creator adaptation, not from the evaluator changing its mind.
The result is a controlled picture of what happens when visibility is mediated by an LLM ranker and everyone optimizes toward it. Across six domains, the quality signal and the ranking signal pull apart.
20 rounds
What CHASE models — and what it leaves out
CHASE is best understood as a simulation of incentives, not a full model of the web. The authors use ranking as a proxy for source visibility, then ask what happens when creators repeatedly adapt documents toward that ranking signal.
The pipeline is intentionally staged. For each query, the ranker produces a forced ordering of candidate documents. CHASE then identifies the features that distinguish winners from non-winners, rewrites a subset of the non-winning documents, and sends the new set back through evaluation. That loop repeats for 20 rounds.
A few design choices matter for interpretation. The ranker stays fixed across rounds. Creators are modeled as myopic and non-strategic. And quality is assessed by an independent LLM judge on factual accuracy, completeness, and usefulness, each on an anchored 1–5 scale, with quality taken as the mean.
So the paper is not claiming to capture every moving part of a real search or recommendation system. It is isolating one thing: what happens when visibility is tied to a ranker and creators keep adapting to whatever wins.
Quality–ranking alignment falls over time
The headline result is simple: quality–ranking alignment decreases in all six domains.
That decline shows up over the full 20-round horizon. The paper reports that proximity to the ranking-derived feature profile becomes less aligned with independently judged document quality from the beginning to the end of optimization. In other words, the documents do become more rank-aligned at the feature level, but that does not translate into better quality as measured by the separate judge.
The authors also validate ranking as a source-visibility proxy against citations in grounded generated responses, which makes the setup more plausible as a model of visibility pressure. But they are careful not to overstate the result: the alignment measure is associational, not a causal estimate of what optimization does to quality in the real world.
The important practical point is that the system can reward documents for learning the ranker’s preferences even while the independent quality signal moves the other way.
The controls show this is not just rewriting noise
The strongest diagnostic in the paper is that the divergence survives controls. Random-target and no-rewrite controls indicate that the drift is not explained by rewriting alone.
That distinction matters. If the only effect were that rewriting sometimes garbled text, then the result would be a generic degradation story. Instead, the paper’s control logic suggests something more specific: repeated adaptation toward features inferred from ranking success changes the ecosystem in a direction that is not reducible to mechanical rewriting side effects.
The authors also report directional negative associations between ranking and some evidentiary features. They treat those as observational signals, not as evidence of a causal citation penalty. That restraint is appropriate. It is one thing to observe that certain features correlate negatively with rank under the observed distribution; it is another to infer that the ranker is actively penalizing evidence in a universal sense.
This is the core methodological contribution of CHASE: it separates rank-driven dynamics from a simpler “rewriting makes things worse” story.
The effect is real, but it is not uniform across domains
CHASE does not produce one universal ecosystem trajectory. The resulting dynamics are strongly domain-dependent.
That is not a minor caveat. The study spans six domains across two task types: Retail, Video Games, and Books for recommendation, and Web, News, and Debate for question answering. Each domain has its own content structure, evidentiary norms, and room for surface-level adaptation, so the same incentive can reshape them differently.
The paper uses separate model families for ranking, rewriting, and quality evaluation, plus five random seeds in the primary experiments. It also initializes the simulation with C-SEO Bench, sampling 20 queries per domain from 100 available queries per domain, with each sampled query containing 5–10 candidate documents.
So the result is not “rankers always destroy quality.” It is more specific than that. The same optimization pressure can create different failure patterns depending on the domain, the task, and the content distribution.
What to do about it
If visibility is controlled by a model, improving the model alone is not enough. The paper’s main warning is that a rank-only incentive can make content more aligned with the ranker’s features while making it less aligned with independently judged quality.
That means the obvious fix — better ranking — is incomplete on its own. Systems need guardrails that account for adaptation. In practice, that points to evaluation setups that look beyond rank score, especially when creators can repeatedly optimize against the same signal. A visibility system should be judged not only by whether it orders documents well today, but by whether it encourages the ecosystem to stay high-quality over time.
The paper also suggests a design lesson for practitioners building AI-search or content-ranking flows: watch for feature drift, not just metric drift. If the content starts converging on the ranker’s preferences, ask whether the quality signal is holding up under repeated optimization. If not, the system may be selecting for what is easiest to win, not what people would actually call good.
In CHASE, creators repeatedly adapt documents toward features that predict winning under a fixed LLM ranker, while an independent judge evaluates factual quality/completeness/usefulness across rounds.
If visibility is mediated by a ranker and creators optimize for it, content can become increasingly “rank-aligned” at the feature level while simultaneously drifting away from independently judged quality—so rank-only incentives need guardrails that go beyond improving the ranker.
Qianwen Gao, Zichang Su, Yiwen Hou, Arlen Kumar, Leanid Palkhouski (2026). CHASE: How Content Ecosystems Are Reshaped When Ranking Is the Only Target. arXiv:2608.30466