How we measure
What we do, how we calculate it, and where the numbers stop being reliable.
Why we ask every question more than once
AI answers are not fixed: the same question can produce a different answer each time. We ask every question several times on every engine, compute the mention rate separately for each sampling round, and report the average together with the minimum–maximum range.
Search sources by engine
Web search runs on each provider's side. ChatGPT and Gemini use their providers' own search; Perplexity's search is built into the model; DeepSeek uses third-party search (Exa). Yuanbao also uses third-party search (Exa). Doubao is currently measured without web search, so its results reflect what the model learned in training and cannot be compared directly with engines that search the web. Both are asked in Chinese only.
Results retrieved through an API are not identical to what a person sees in the provider's app. We measure the engines as they answer through their APIs.
Models measured: ChatGPT — GPT-6 Luna; Gemini — 3.6 Flash; Perplexity — Sonar; DeepSeek — V4.1 Flash; Doubao — Seed 2.1 Turbo, without web search (the app's free tier uses 2.1 Auto / Lite, which the international API does not offer); Yuanbao — Hy3. Models are adjusted as providers update; the model stated in each report applies.
Regional differences
Because search runs on the provider's side, results do not depend on where our servers are. Regional differences come from the places and languages written into the questions.
How brands are recognised
We combine AI entity extraction — an AI model lists the businesses named in each answer — with matching against an alias list for each brand, which handles Simplified and Traditional Chinese and company suffixes.
Facilities operated inside a brand's own site (for example a water park inside a resort) are treated as the same brand. Names of destinations that cover many operators are not.
How the language comparison is calculated
For a parallel test we write one scenario once per language, with the same place, budget and situation. For each scenario and language we take the businesses the AI recommended and keep those that appear in at least 2 answers.
For every pair of languages we then compute the overlap: the number of businesses on both lists divided by the number on either list. The headline figure is the median across all language pairs. Businesses are merged by their official English name.
Known limitations
- The samples are modest: in a lite diagnostic each language and scenario has 12 answers. Differences of a few points are noise.
- AI models are updated often. A result describes one model version on one date and can change.
- API results differ from what people see in the apps.
- Some engines cite sources less consistently, so citation statistics are incomplete for them.
- The overlap score depends on how many businesses clear the 2-answer threshold. Where answers vary a lot (restaurants, for example) the score is lower and less stable.
- We measure whether AI mentions and recommends a business, not whether customers then act on it.
- The questions are written by us to be realistic. They are not a sample of everything customers ask.
Last updated: 3 November 2026