A supplier says its AI research tool has guardrails. That sounds reassuring, but it leaves the useful question unanswered: guardrails against what?

A filter that detects prohibited content is doing a different job from a check that identifies a false factual claim. A source can look ordinary, avoid explicit malicious instructions and still contain a statement the answer should not repeat.

For a team using AI to summarize external material, the test needs to include both the false claim and the useful work that should continue after it is rejected.

A polished false source can pass a general filter

Counter-GEO-Bench tested 247 English queries in a controlled answer pipeline. Researchers changed one of five source documents and left the others intact. Three open-weight models produced answers from those packets.

The main attack measure gave full success a score of 1, partial success 0.5 and failure 0. Its average of 55.7% without a defense is therefore a weighted score, not the percentage of fully successful attacks.

General safety filters changed that score relatively little. The authors' purpose-built detector reduced it to 29.2%. That is useful experimental evidence for distinguishing defense tasks, but it is not certification of a commercial search product. Substantial distortion remained.

The study's relevance, completeness and clarity scores also need their own labels. A clear answer is not necessarily a correct answer.

Refusing everything can make a defense look effective

One tested defense refused 98.4% of clean-condition answers on one model. That cell was excluded from its main average. A system can stop repeating an attacker's claim by becoming almost unusable.

Even a more selective defense can lose valid information. In a transfer test using a different rewriting model, the specialized detector reduced attacks while lowering accuracy on fact-preserving inputs by 4.2 percentage points. The paper reports the transfer trade-off separately.

For a buyer, an attack score alone is therefore incomplete. Ask what happened to ordinary requests, legitimate source material and answers that should have remained available.

The appropriate balance depends on the task. A refusal may be preferable for an unsupported consequential claim. It may be unnecessary for a routine question with adequate evidence elsewhere in the source packet. The evaluation needs to show which occurred.

A different source is not automatically a better source

A separate controlled source-selection study illustrates another distinction. Its intervention reduced selection of an injected target, but the original, uninjected model choice served as the reference rather than a verified best source.

Avoiding the attacker's preferred option and choosing a good alternative are different outcomes. Agreement with the original answer is different again: the original answer may itself be wrong.

This is why a safeguard report should name its success condition. “The target was suppressed” is a precise result. “Source quality improved” requires evidence about the source that replaced it.

Request a paired evaluation

For a pilot of an internal research or document-answering tool, I would ask for a small set of paired cases using source material the team is authorized to test. One version contains the original evidence; the other contains a clearly documented factual distortion. Keep the legitimate question unchanged.

Review the complete output under both conditions. Record whether the false claim was repeated, whether the system refused, whether valid evidence was discarded, and whether the remaining answer is supported.

Also include ordinary difficult material: a genuine qualification, a disagreement between sources, or an unusual but correct statement. A detector that treats unfamiliarity as falsehood can produce tidy answers by removing the very evidence the task needs.

This proposed evaluation is broader than counting blocked passages. It asks whether the defense preserves the useful path to an answer.

The cited experiments are limited to controlled tasks and model configurations. They do not establish protection against every coordinated, multilingual or adaptive attack. Their practical contribution is a sharper buying question: show which failure the safeguard catches, then show what the user can still accomplish when it is switched on.