The practical brief
The finding
Comparing Generative Engine Optimization (GEO) scores without matching editing permissions can confuse a stronger optimization algorithm with greater freedom to rewrite source content. [c1] [c3] [c4] [c5] [c6] [c8]
Why it matters
Buyers evaluating competing GEO tools or tactics need to know whether a reported visibility gain came from under the same editing and resource constraints, whether it preserved answer quality, and whether failed runs were excluded.
This paper is a conceptual evaluation proposal with no new datasets, model experiments, or empirical evidence that adopting its standard improves visibility or user-task performance.
Why Editing Freedom Confounds Visibility Scores
When vendors or agencies pitch Generative Engine Optimization (GEO) services, they often point to benchmark scores showing how much their approach increased source visibility inside an AI ranker or answer engine. A methodological paper by Qianfeng Wen and colleagues argues that these scores are not fairly comparable when the tested methods operate under different editing permissions.
Tactics can differ in where they alter a source, what editing operation they perform, and which cue—such as apparent authority or the prominence of evidence—they try to influence. In practice, one method might be limited to appending a short text snippet while another is permitted to rewrite an entire page. Even when both are judged by the same ranker, a higher score may simply reflect greater freedom to alter the page rather than a better optimization algorithm.
For a business comparing competing optimizers, a fair test has to hold those surrounding conditions constant. That means evaluating tools on the same tasks and target system, under identical allowed strategies, editing limits, feedback access, and compute budgets—including any prior training alongside per-task computation.
What Visibility Scores Leave Out: Answer Quality and Failed Attempts
Even when editing rules match, higher source visibility does not automatically mean the AI system gave the user a better answer or recommendation. Pushing an edited page into an answer can coincide with better, unchanged, or worse performance on the user's actual request. To catch gains that come at the user's expense, the authors propose measuring task performance—such as answer accuracy or recommendation suitability—before and after optimization, using reference answers or verified candidate information that are independent of the edited source.
How a report handles failed attempts matters just as much. A vendor benchmark can make an optimizer look more reliable than it is by counting only valid outputs and dropping cases where the tool produced invalid edits or ran out of budget. A complete evaluation keeps failed and unresolved attempts in view, comparing performance across valid outputs with performance across all attempted tasks—and acknowledging when unresolved cases prevent a clean overall score.
- Ask whether competing tools operated under identical editing permissions, feedback access, and total compute budgets.
- Check whether the evaluation verified answer correctness or recommendation suitability using evidence independent of the edited page.
- Look for how many attempted edits failed, broke formatting rules, or remained unresolved rather than judging a tool solely on its successful runs.
What This Proposal Cannot Tell You About Which Tactics Work
Buyers should treat this work as an evaluation lens for questioning vendor claims, not as proof of which GEO tactics succeed. The authors report no human-subject research, no new data, no model experiments, and no new optimizer, nor do they show empirically that adopting their standard improves visibility or task performance.
Isolating why a specific edit worked also remains difficult in practice. Changes across a document are often entangled: altering page structure, for example, can force simultaneous changes to wording or underlying information, which prevents neat attribution to a single adjustment.
Finally, the authors note that systematizing optimization tactics carries a dual-use tension. Mapping out how edits target system cues can help auditors test whether AI systems are easily manipulated, but it can also help content providers steer those systems. While clarifying a source can genuinely help users, manufacturing apparent authority or adding unsupported claims can mislead—making independent verification of answer quality essential when judging any reported GEO win.
The source and evidence
Humanity's Last Choice: A Common Evaluation Standard for Generative Engine Optimization
Study limitations and disclosures
- The paper is a conceptual proposal with no new datasets, model experiments, human-subject studies, or released optimizers, and it does not empirically demonstrate that adopting its standard improves visibility optimization or user-task performance.
- Source edits are often coupled—changing document structure can require changes to wording or content—making it difficult to attribute visibility gains to a single type of edit without case-specific analysis.
Evidence behind this briefing
[c1] The paper argues that GEO scores are not fairly comparable when methods have different editing permissions. A higher score could reflect permission to rewrite more content rather than a better optimization algorithm; this is a methodological argument, not a measured effect.
Section 1, Introduction · Read the study
[c2] The proposed framework describes GEO through three dimensions: where a source changes, what editing operation is performed, and which cue the edit targets. It offers a vocabulary for specifying and comparing tactics, not a ranking of tactics by effectiveness.
Section 3, A Meta-Taxonomy of GEO Strategies · Read the study
[c3] For businesses evaluating competing GEO tools or services, the proposed optimizer comparison requires shared tasks, target system, permitted strategies, editing constraints, feedback access, and resource limits. Resource reporting includes prior training as well as per-task computation.
Section 5.2, Comparing Strategies, Optimizers, and Models — Optimizers · Read the study
[c4] Increased visibility is not equivalent to greater answer accuracy or better recommendations. The protocol requires measuring original task performance before and after editing, using reference evidence independent of the optimized source.
Section 5.3, Evaluation Metrics — Original task performance · Read the study
[c5] The reporting proposal guards against selective success claims: declare permitted combinations in advance, distinguish attempted from validated coverage, retain failed and unresolved cases, and report an unconditional task-level result alongside valid-output effectiveness. If unresolved validation prevents that aggregate, report bounds or withhold it. Model comparisons must also measure usefulness rather than reward blanket refusal.
Appendix D, Comparison and Reporting Requirements — final two paragraphs · Read the study
[c6] The paper does not establish that adopting its standard improves visibility optimization or user-task performance. Source-layer changes can be coupled, complicating attribution to individual layers. It also acknowledges that a fixed taxonomy could miss domain differences and new strategies, so comparisons remain bounded by each evaluation release.
Limitations, including Alternative view · Read the study
[c8] The authors disclose that this work contains no human-subject research, new data, model experiments, new optimization method, or released optimizer. They also identify a dual-use risk: the same framework can help evaluate source influence or help content providers steer systems. Apparent authority and unsupported claims should not be mistaken for credible evidence.
Ethical Considerations · Read the study