Why does AI choose one source over another?
We follow the research on it, and help you become the source AI picks.
See how AI picks its sources
Explainer
How AI picks its sources
When you ask ChatGPT, Perplexity or Google’s AI Mode a question, the answer can come from what the model learned in training, from pages it searches for in that moment, or from both. This walks through the search part, with the research behind each step.
Read it as text
How AI picks its sources
Where an answer comes from
Say someone asks an AI, “Who can paint a portrait of my cat?” It can answer from what its model learned in training, from what’s in the conversation, such as a photo of the cat, or from pages it looks up while it answers. Often it mixes them. A model stops learning when its training ends, so new pages reach its answers through search. That’s the part the steps below follow.
Research: Search Mode Changes the Local Businesses an AI Recommends
The question becomes several searches
Before searching, the engine often rewrites a question into a few shorter searches, such as “custom cat portrait from photo”, “pet portrait artist” and “best pet portrait artists”. Google calls this query fan-out. It’s one reason a page can match the words someone typed and still not appear, and how a “best of” list can end up in the answer.
Research: Reworded Buying Questions Produced Different AI Brand Lists
Each search pulls a shortlist
Each search returns a shortlist of pages from an index. It doesn’t always match Google’s first page: Google’s own AI Overviews cite sites from further down too, so a ranking report shows only part of the picture.
Research: Track AI Overview Citations Alongside Google Rankings
The search crawler has to reach the page
A page the engine’s search crawler can’t reach never makes the list, however good it is. AI companies run several crawlers with different jobs. OpenAI’s OAI-SearchBot finds pages for ChatGPT search. GPTBot collects pages to train OpenAI’s models. ChatGPT-User visits a page when a person asks it to, and robots.txt rules may not apply to it. Google-Extended decides whether Google may use your pages for Gemini, and doesn’t affect Google Search. So a studio can block those training crawlers and still be found by ChatGPT search. Know which crawler does what before you change anything.
Research: Before Paying for AI Page Formatting, Check How Sources Get Found · Glossary: robots.txt and crawler rules
The engine keeps passages
From each page on the shortlist, the engine keeps the passages that answer part of the question. A page can appear in the source list while its name, address or ordering details never reach the answer.
Research: When AI Cites a Page, Which Details Reach the Answer?
It writes the answer and links its sources
The model writes the answer from those passages and links its sources. A link makes a claim easier to check; it doesn’t prove the claim, so open the source.
Research: What an AI Citation Actually Tells You About Your Business
Being linked and being named are different
An answer can use a page and link it without ever naming the business behind it. Available, retrieved, mentioned, cited and recommended are separate outcomes, and none of them guarantees the next.
- In the example, a small studio that paints pets as Renaissance portraits had its page available, retrieved and cited.
- Its idea reached the answer: “some artists paint pets as Renaissance portraits”.
- The answer still named a marketplace and a list, and never mentioned the studio.
Research: Which Step Is Missing Between Your Page and an AI Mention?
Why the marketplace was named
A large study of Chinese-language AI search found that a business mentioned in many of an answer’s sources is more likely to be named in that answer. In the example, the studio’s name appeared on one source, its own page; the marketplace’s appeared on three. The study measured an association, and its authors call for controlled tests of cause.
Research: When AI Cites a Page, Which Details Reach the Answer?
One answer is one observation
Ask the same question tomorrow, from another city or in another language, and the answer can change. One screenshot shows one answer, so it can’t show that an edit worked. Keep a small set of real customer questions, note how you asked them, and repeat them over time.
Research: What Did That GEO Experiment Actually Improve? · Separate Language from Location in a Local AI Search Audit
What this means for your site
Before you block or allow an AI crawler, find out what it does. Then ask AI your customers’ questions from time to time, and check whether the pages it links to mention you.
The studio, the marketplace, the list, the painting and the answers in this example are made up: it’s an illustrative run.
Featured
Before Paying for AI Page Formatting, Check How Sources Get Found
A two-market study of AI citations. In ChatGPT’s answers, whether the engine’s own search retrieved a page explained 32% of the citation differences between pages on the same site. FAQ blocks, schema and other page features explained 2.8%.
Illustrative runexample.com
Done in 14 seconds
| Crawler | robots.txt | Server | Raw HTML | Indexing |
|---|---|---|---|---|
| GPTBot | ✓ Allowed | 200 OK | 41% of text | Indexable |
| OAI-SearchBot | ✓ Allowed | 200 OK | 41% of text | Indexable |
| ClaudeBot | ✓ Allowed | 200 OK | 41% of text | Indexable |
| PerplexityBot | × Blocked | 200 OK | 41% of text | Indexable |
PerplexityBot is blocked by a robots.txt rule. 59% of the page’s text appears only after JavaScript runs, so crawlers that read raw HTML see less than half of it.
Free AI Crawler Checker
Can AI crawlers read your page?
Our free checker reads one public page as GPTBot, ClaudeBot and PerplexityBot would. It checks robots.txt rules and live server responses for each crawler, compares the raw HTML with the rendered page, and reads the indexing signals. Add a competitor’s page to compare the two side by side.
Free, no signup, usually 10 to 20 seconds.
Run a free checkReadable is the first step. To see whether AI answers name you, and what else holds your site back, start with a free mini-audit.
Get a free mini-auditSEO and GEO audit
When people ask AI about what you do, does it name you or a competitor?
Start with a free mini-audit: three specific findings about your website, with the evidence behind each, within two working days.
Get a free mini-audit
Example audit
A full audit, start to finish
An independent audit of a diagnostic imaging centre in Istanbul, set against a competitor found in about 30 times as many Google searches.
Read the example audit