AI Brand Recommendations Vary Significantly Upon Re-Query
A new analysis finds top-ranked brands change in nearly 75% of repeated AI queries, complicating their use in research.
Updated on Oct. 27, 2026 in Artificial Intelligence
Live Poll
Do you trust AI search tools to provide consistent and objective product recommendations?
Large language models including ChatGPT, Claude, and Gemini demonstrate significant inconsistency in brand recommendations when asked the same question repeatedly. A new study found that the top-ranked brand changes in 74.6% of answer pairs.
Why it matters
The findings challenge the reliability of using generative AI for market research or consumer decision-making. High volatility in output suggests that model randomness or internal optimization processes heavily influence outputs that users may perceive as objective data.
Across 172 product categories, researchers found a 77.3% median rate of top-brand turnover. In longitudinal tracking, only 85 of 480 question series maintained a consistent leader over a 15-week period.
The players
BusySeed
A digital marketing agency that conducted the analysis of AI brand ranking reliability.
OpenAI
The developer of the GPT model family and the primary interface utilized for this study.
The details
The study utilized OpenAI's developer interface to query models while standardizing version numbers, web search settings, and prompt syntax. By bypassing user-specific browser cookies and account-based personalization, the researchers isolated the model's output generation logic. The fluctuation indicates that even with identical inputs, the probabilistic nature of the underlying transformer architecture leads to divergent top-tier rankings.
Timeline
April 27, 2026: Data collection began for the BusySeed study.
October 16, 2026: Data collection concluded for the study.
The Tech Race
This study underscores the divergence between traditional search benchmarks and generative AI, which prioritizes probabilistic fluidity over historical stability. It highlights a critical research gap for developers attempting to standardize AI behavior for enterprise-grade analytics.
Users should not rely on generative AI as a primary tool for definitive brand ranking or objective product comparison. Expect results to fluctuate significantly if the query is re-run, even when maintaining consistent prompt phrasing.
The takeaway
The high volatility of AI outputs suggests that brand rankings provided by models are stochastic rather than factual. Readers should verify AI-generated comparisons against third-party market data before basing any commercial decisions on them.
Further reading
For broader trends in model behavior, explore the latest research in Artificial Intelligence.
More information
Review the complete study methodology and data for more details on the category-specific findings.
Source note: This article includes information reported by BNO News.
Live Poll
Do you trust AI search tools to provide consistent and objective product recommendations?
