A neutral multiple-choice prompt and a prompt that says “another source answered X” are not behaving like the same task. In this audit, instruction-following models often switch from a correct neutral answer to a fixed wrong option when the prompt is framed as a source-attributed claim, which means wording around provenance can act like a control signal rather than harmless context.
That matters because a lot of production workflows now pass along answers with a source tag, a reliability note, or a “this came from another system” wrapper. The paper’s point is not that models cannot follow those cues. It is that they follow them even when the cue steers them away from the answer they had just given correctly.
Across 220,000 outputs.
What the audit measures, and why the prompt design matters
The paper is built to test one thing: whether claims about what another source answered destabilize multiple-choice QA. To isolate that effect, each base question is first run under a neutral prompt and then under 10 controlled cue variants, with the cue inserted immediately before the final answer instruction.
That setup matters because the wrong answer is fixed for each item. For a given question, the authors sample one wrong option and reuse it across the misleading conditions, so they are not constantly changing the target the model is being nudged toward. The result is easier to interpret: if the model flips, it is flipping toward the same wrong choice, not chasing a moving target.
The key metric is NC-MCAR. It only counts a cue adoption when the model first selected the gold answer under the neutral prompt and later selected the fixed cued option on a valid cued trial. In other words, this is not a raw error rate. It is a measure of whether the cue can override an answer the model already got right in the neutral setting.
The paper evaluates four instruction-following models on MMLU-Pro and IndicMMLU-Pro in English, Hindi, Bengali, Tamil, and Telugu. Each base example expands into 11 prompt conditions, and the runs are done once per condition at temperature 0, with optional extended reasoning disabled where available.
The main result: misleading source cues reliably change the answer
The headline number is straightforward: the expert template reaches 41.1% aggregate NC-MCAR, compared with 12.5% for the majority template. That is the clearest sign that source-attributed wording can move the model away from its neutral-correct answer and onto the cued wrong option at a substantial rate.
This is not confined to a single system. All four models show nonzero neutral-conditioned misleading cue adoption, so the effect is broad enough to show up across the evaluated stack rather than looking like a one-off failure mode.
The model-level numbers tell the same story. Pooled over wrong-cue templates, NC-MCAR is 10.1 for GPT-5.4, 14.2 for Claude Sonnet 4.6, 25.2 for Gemma-4-31B-it, and 27.5 for Qwen3-32B. The exact rates differ, but the direction does not: every model is vulnerable to source-framed answer shifts.
The expert-wrong cue is the highest NC-MCAR template for every model, which is why the paper treats it as the sharpest descriptive contrast. But the authors are careful not to overclaim what that means. The expert-versus-majority comparison is useful because the wrong option is fixed and the English forms use the same verb, yet it still does not isolate source identity from every wording choice.
Correct source cues stay strong, so this is about cue weight, not cue comprehension
The models are not confused by source attribution in general. When the cue is aligned with the gold answer, valid-response accuracy is high: 73.8% for majority-correct cues and 84.9% for expert-correct cues.
That distinction is the practical crux. The same mechanism that lets the model comply with a correct source cue also lets it comply with a misleading one. So the issue is not that the model cannot parse “someone said X.” It can. The issue is that it appears to treat that phrase as a high-weight instruction about which option to choose.
There is also a useful asymmetry in the results. Under expert-wrong cues, valid-response accuracy is 30.0, which shows the cue is not a magic override that always wins. But the fact that it wins often enough to matter means downstream systems should not treat source phrasing as cosmetic.
If your pipeline ever says “another source answered B,” this paper suggests the wording is doing more than annotating provenance. It is participating in answer selection.
What you can generalize, and what you can’t
The strongest claim the paper supports is local, not universal: source-attributed wrong cues can destabilize neutral-correct answers in multiple-choice QA. That is a benchmark-level finding, and the authors say so directly.
The multilingual results should be read the same way. The paper evaluates English plus Hindi, Bengali, Tamil, and Telugu, but the English and Indic benchmarks are not parallel translations, so the multilingual analysis is treated as a benchmark audit rather than evidence of a common Indic effect. The pooled numbers are informative, but they are not causal proof that language itself is driving the differences.
The same caution applies to the metric. NC-MCAR measures answer instability conditional on neutral correctness. It does not prove the model “knew” the answer under neutral prompting, and it does not prove that all deference is irrational. It only shows that a source-attributed wrong cue can pull a model off a correct neutral response.
There are also plain operational limits. NC-MCAR excludes invalid cued outputs from the denominator, the current tables do not report exact effective denominator counts, and temperature 0 with one response per prompt does not guarantee deterministic hosted endpoints. The findings may also fail to extend to explicit reasoning, repeated sampling, free-form conversation, abstention-enabled policies, open-ended tasks, or other models and benchmarks.
What to do about it in practice
Treat source-attributed wording as a potentially adversarial control channel. If a downstream system receives an answer plus language like “this came from an expert” or “another model answered X,” do not assume that wrapper is neutral metadata. The paper suggests it can measurably steer a model’s final choice even when the neutral answer was correct.
The safest move is to separate provenance from selection. Let the model answer the question first, then attach source labels, confidence, or retrieval metadata after the choice is made. If you have to pass source claims into the prompt, treat them like an input that needs auditing, not a benign comment.
This is especially relevant for QA layers that do reconciliation across agents, rerankers, or external sources. If the system’s instructions blur “what was said” with “what should be chosen,” the model may follow the wrong part very faithfully.
The models reliably follow source-attributed cues in multiple-choice QA, but those same cues can push them off the neutral-correct answer onto a fixed wrong option.
If your downstream system uses “other sources said X” wording, treat it as an adversarial control channel: it can measurably override a model’s correct neutral choice even when the neutral response was correct.
Manikandan Ravikiran, Siddharth Vohra (2026). When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA. arXiv:2609.08934