Established brands can vanish from AI buying shortlists

The decision: are you absent from AI consideration sets?

For a business owner, the immediate question is simple: when a buyer asks an LLM for brands in your category, do you appear at all? This paper argues that LLMs build a generated set—a short recommendation list assembled at answer time rather than chosen from a visible catalog. If your brand is omitted, you may never make the buyer’s consideration set.

That omission risk was not limited to obscure brands. In the category-only tests, the authors report that many large, established brands received no recommendations at all. They also show why one screenshot is the wrong way to judge performance: the same prompt can return different brands and different ordering, so visibility has to be estimated from repeated runs.

• Baseline test: 6 commercial LLMs across 5 categories.
• Each category-only prompt was run 40 times per model in independent fresh sessions.
• That produced 1,200 category-only recommendation lists in total.

Evidence [c1] [c2]

What the study measured, in business terms

The paper separates two questions marketers often blur together. BRP@5, or Brand Recommendation Probability at five, measures prevalence: the share of recommendation lists where a brand appears in the top five. MRR@5, or Mean Reciprocal Rank at five, measures prominence: it gives a brand 1 when ranked first, 1/2 when second, 1/3 when third, and 0 when absent, then averages that across repeated lists.

That distinction matters operationally. A brand can show up often but usually low in the list, or appear less often but near the top when it does surface. The authors’ framework is built to measure both because LLM recommendations are stochastic, meaning variable across identical requests.

• BRP@5 answers: how often do we get named?
• MRR@5 answers: how high do we tend to appear when named?
• Repeated sampling is necessary because identical prompts can yield different outputs.

Evidence [c1]

Fame alone did not explain who got recommended

A common assumption is that LLMs mostly echo the biggest brands. This study finds only limited evidence for that cleaner popularity-bias story. Some prominent brands were omitted, and recommendation prominence did not reliably track conventional salience measures across categories.

In the authors’ exploratory visibility analysis, search interest had the strongest predictive relationship with MRR. Online brand conversation ranked behind search interest in the ridge model, but this part of the paper is especially important to read cautiously: the predictors were highly correlated, the analysis was observational, and the authors explicitly warn against treating these relationships as causal proof for spending changes.

For marketers, the practical read is narrower than 'optimize search volume.' AI recommendation visibility may depend on whether your brand is broadly visible and associated with the category in the surrounding information environment, but this paper does not show which intervention would cause improvement.

• The paper reports only partial support for standard popularity bias.
• Search interest was the strongest predictor of recommendation prominence in the exploratory model.
• Any role for online conversation should be treated as suggestive, not decisive, because of multicollinearity and non-causal design.

Evidence [c3] [c4] [c5]

Needs-based prompts can recover some missing brands

The most useful managerial idea here is diagnostic, not magical. The authors took two brands that never appeared in category-only recommendations—Craftsman drills and L.L.Bean hiking jackets—and asked whether richer consumer context would help. Needs-based prompts described plausible goals and constraints without naming the brand.

The result: both brands became more retrievable, but unevenly. Craftsman moved from complete omission to BRP@5 of 35.4% and MRR of .108. L.L.Bean improved much less, reaching BRP@5 of 5.4% and MRR of .025. When the authors then used diagnostic positioning probes with unusually distinctive cues drawn from the brands’ own positioning, BRP@5 jumped above 80% for both brands. That suggests omission was not simply a lack of brand knowledge inside the model.

Our interpretation is that this supports a practical audit sequence: test category prompts, then realistic needs-based prompts, then diagnostic probes if a brand stays missing. But it does not prove that rewriting site copy, buying more media, or planting positioning phrases will reproduce the same gains in live consumer interfaces.

• Category-only baseline for both brands: BRP@5 = 0%, MRR = 0.
• Detailed needs-based prompts helped Craftsman more than L.L.Bean.
• Diagnostic probes test conditional retrievability, not normal market performance.

Evidence [c6] [c7] [c8]

What this study cannot settle for your brand

The boundary conditions matter. These tests were run through provider APIs, in fresh stateless sessions, with no prior user history and web search or other external retrieval disabled. That makes the paper strong for measuring baseline model behavior, but weaker as a direct proxy for what a logged-in consumer may see in a retrieval-enabled product.

The brand-level diagnostic results are also illustrative, not a validated general playbook. They come from two brands and a small handwritten prompt set, and the authors explicitly say the three-stage diagnostic approach needs larger-scale validation across a randomly selected brand sample before managers should treat it as a general tool.

• Do not generalize these results directly to personalized or browsing-enabled chat experiences.
• Do not read search-interest correlations as proof that more spend there will raise recommendation rates.
• Do use the paper as a measurement template for your own repeated brand audit.

Evidence [c1] [c5] [c8]