# AI Search Engines Disagree by 2× on Brand Recommendations: Inside Treyci's Empirical Study # AI Search Engines Disagree by 2× on Brand Recommendations: Inside Treyci's Empirical Study A landmark empirical research study published on September 3, 2026, by AI visibility analytics firm Treyci reveals massive, structural discrepancies in how major generative search engines recommend commercial brands. In one representative B2B enterprise software category, one leading AI engine recommended tracked vendors in 81% of generated responses, while a competing engine referenced the exact same brands in only 43% of answers—representing a staggering 2x visibility disparity between platforms that enterprise buyers use interchangeably. The study, led by Treyci founder Keith Schilling, delivers a sharp rebuke to the emerging practice of manual screenshot-based AI auditing. Schilling noted that because large language models are probabilistic engines, asking the identical buying query in separate sessions routinely returns completely different vendor shortlists. "Marketing teams are making decisions about AI search based on one screenshot of one answer from one engine," Schilling warned. "The data says that's a coin flip wearing a suit." ``` +-----------------------------------------------------------------------------+ | TREYCI EMPIRICAL STUDY: BRAND VISIBILITY VARIANCE | +-----------------------------------------------------------------------------+ | | | ENGINE MENTION RATES ACROSS IDENTICAL B2B SOFTWARE PROMPTS | | | | Engine A (Perplexity / Gemini) [██████████████████████████████] 81% | | Engine B (ChatGPT / Copilot) [██████████████] 43% | | | | ----------------------------------------------------------------------- | | | | PRIMARY SOURCE CITATION BREAKDOWN | | | | Third-Party Review Hubs (G2, Capterra, Trustpilot) [████████████] 54.2% | | Industry Publications & Editorial Comparisons [████████] 31.6% | | Official Company Websites & Documentation [███] 14.2% | | | | ----------------------------------------------------------------------- | | Implementation Metric: 41 of 100 SaaS brands publish llms.txt, yet <10% | | can correlate presence with increased recommendation probability. | +-----------------------------------------------------------------------------+ ``` ## Fast Facts - **Publication Date:** September 3, 2026. - **Visibility Spread:** Up to an 81% vs. 43% recommendation rate disparity between competing AI search engines on identical prompt sets. - **Session Volatility:** A brand recommended in Run #1 has as little as a 48% probability of appearing in Run #2 of the identical query on the same engine. - **Dominant Citations:** Over 85% of citations generated in commercial buying queries point to third-party review platforms and editorial trade journals, with brand websites accounting for only 14.2%. - **The `llms.txt` Adoption Reality:** 41 out of 100 surveyed B2B SaaS companies have published an `llms.txt` file, but fewer than 10% possess tools to track whether it altered LLM recommendation rates. - **Core Takeaway:** Single-prompt checks provide anecdotal illusions; enterprise GEO requires automated, multi-engine, probabilistic sampling. ## Technical & Strategic Deep Dive Generative Engine Optimization (GEO) has rapidly transitioned from an experimental digital marketing tactic into a multi-million dollar corporate budget line item. However, Treyci's findings reveal that most organizations are applying deterministic SEO measurement mentalities to non-deterministic systems. ### 1. The Probabilistic Dilemma and Temperature Drift Traditional search engines operate on deterministic indexing. While personalization and geolocation cause localized SERP differences, a company ranking #1 for "enterprise CRM software" generally retains that position across multiple browser sessions on the same day. In contrast, generative search engines synthesize responses through temperature-adjusted probabilistic decoding. When an enterprise buyer asks, \*"What are the top three data observability platforms for Snowflake?"\*, the underlying model calculates token probabilities across its latent space and retrieved context documents: $$ ext{P}( ext{Vendor A}) = 0.38, \\quad ext{P}( ext{Vendor B}) = 0.35, \\quad ext{P}( ext{Vendor C}) = 0.31, \\quad ext{P}( ext{Vendor D}) = 0.28$$ Depending on the random seed, temperature parameter (typically 0.2 to 0.7 in commercial search interfaces), and real-time retrieval rankings, Vendor D may appear in one answer and completely vanish in the next. Relying on a single manual query to evaluate brand visibility leads to wildly inaccurate executive reporting. ### 2. The Third-Party Citation Hegemony Perhaps the most disruptive finding for content marketers is where AI engines look when evaluating product superiority. While brands spend millions optimizing their corporate websites, product landing pages, and corporate blogs, AI engines exhibit strong algorithmic skepticism toward self-authored marketing copy. When answering buying-intent queries, models heavily weight consensus data retrieved from: - Aggregated user reviews (G2, Trustpilot, Capterra, Gartner Peer Insights). - Independent editorial comparisons published on authoritative media domains. - Technical discussion forums (Reddit, Stack Overflow, GitHub Issues). Brand websites are primarily cited only for factual specifications (pricing tiers, supported protocols, documentation schemas), whereas qualitative recommendations ("Which tool is best for...") are almost universally backed by third-party citations. ### 3. The `llms.txt` Measurement Disconnect The study observed surging adoption of `llms.txt`—a proposed standard designed to provide AI web scrapers with clean, markdown-formatted summaries of site structure and product features. While 41% of top SaaS providers have deployed the file to their web root, Treyci discovered zero statistically significant correlation between having an `llms.txt` and achieving higher commercial recommendation rates. Without continuous tracking across engines, marketing teams are executing technical optimizations in a vacuum, unable to differentiate between effective visibility engineering and useless overhead. ## Real-World Utility & Limitations ### Implications for Modern Marketing - **Budget Reallocation:** Enterprise marketing teams must redirect budget away from isolated corporate blogs toward verified third-party review profiles and digital PR. - **Statistical Sampling:** Marketing reporting must shift from "current rank" to "win rate probability" calculated across at least 50 automated query executions per week. ### Limitations of Current GEO Metrics - **Black-Box Retrieval:** Because OpenAI, Perplexity, and Google frequently alter their retrieval-augmented generation (RAG) pipelines without public notice, visibility metrics can fluctuate dramatically due to underlying infrastructure changes rather than competitor action. - **High API Monitoring Costs:** Continuously polling multiple proprietary AI engines across hundreds of prompt permutations requires substantial API and proxy spend. ## Actionable Takeaways 1. **Stop Relying on Manual Screenshots:** Immediately ban single-screenshot reporting for executive AI visibility decks. Require minimum 30-run sample sets to establish baseline statistical confidence. 2. **Audit Third-Party Review Footprints:** Conduct an immediate audit of your brand presence on G2, Capterra, and Trustpilot. Ensure pricing, feature checklists, and integration details are 100% accurate and actively maintained. 3. **Implement Multi-Engine Monitoring:** Ensure your GEO tracking stack monitors at least five engines (ChatGPT, Claude, Perplexity, Google Gemini, and Microsoft Copilot) rather than focusing exclusively on ChatGPT. 4. **Deploy FAQ Schema with Direct Answers:** Structure product comparison content with clear, factual question-and-answer pairs to increase citation capture during real-time retrieval passes.