AEOsim blog
Conversation analytics vs prompt tracking vs GA4 referrals
Table of contents
Three tools, three different questions
Ecommerce teams now have three ways to look at AI search: prompt tracking, conversation analytics, and GA4 AI-referral reports. They are not competing versions of one metric. Each sees a different slice of the buyer's path, and each is blind where the others can see.
- Prompt tracking asks: does the engine mention us when a buyer opens with a category question?
- Conversation analytics asks: are we still recommended when the buyer adds constraints and asks the engine to pick?
- GA4 referrals ask: what did buyers do after they clicked through from an AI surface?
The mistake is using one to answer another's question. A high first-response mention rate does not mean you win the decision. A low AI-referral count does not mean AI is irrelevant to your sales. Most AI-influenced research ends without a click, so it never reaches GA4 at all (Ishita Kejriwal, AEOsim).
What each source can and cannot tell you
| Prompt tracking | Conversation analytics | GA4 AI referrals | |
|---|---|---|---|
| Unit measured | One prompt, one response | A full multi-turn buying chat | A session after a click |
| Where it looks | Turn 1 (discovery) | Turns 1 to 5, through the pick | Your website |
| Data type | Simulated prompts | Simulated conversations | Observed traffic |
| Answers well | Are we in the opening shortlist? | Where do we drop, and to whom? | What do AI visitors do on-site? |
| Blind to | Constraints, objections, the final pick | Real traffic volume and revenue | Every chat that ended without a click |
| Typical failure | High score, lost decisions | Over-trusting one engine's result | Reading low referrals as low influence |
Prompt tracking
Prompt tracking is a reasonable entry signal. It tells you whether the engine considers your brand part of the category. Its limit is structural: it measures the opening turn, and the opening turn is where the decision is least made. In AEOsim's constraint experiment, the discovery leader won 100% of neutral chats and 9% of constrained ones (Chat Is the New Funnel).
Sampling is the second limit. Many tools run each prompt once per day. In a 50-prompt study, half the prompts had not stabilised after 20 runs (AEOsim tool comparison). A single daily run is weather, not climate.
Conversation analytics
Conversation analytics replays a buying chat turn by turn and records which brands are in the set at each step. It produces the numbers prompt tracking cannot: the drop-off turn, Share of Recommendation at the pick, survival curves, and which constraint removed you. That is what makes it diagnostic. It tells a content team which objection to answer and which page to fix.
Its limit is that it is still simulation. It describes how engines behave on a designed set of conversations. It does not count real buyers or revenue on its own.
GA4 AI referrals
GA4 shows sessions arriving from chatgpt.com, perplexity.ai, gemini.google.com and similar sources, and what those visitors do next. That is real, observed behaviour, and it matters. AEOsim measured AI-referred sessions adding to cart at 2.5x to 5.1x the rate of Google-direct sessions across pharmacy, insurance and marketplace categories.
The limit is coverage. GA4 only sees the buyers who clicked. Everyone who got their answer inside the chat, or who later typed your URL directly, shows up as direct traffic or not at all. Referral counts are a floor on AI influence, not a measure of it.
How to use all three together
The three sources stack in the order a buyer moves: entry, decision, visit. Use each for the question it answers, and read them side by side.
- Prompt tracking for coverage. Are we in the opening set across our main categories and engines? Run it repeatedly, not once a day.
- Conversation analytics for diagnosis. For the categories that matter most, where do we drop, at which turn, and on which constraint? This produces the content and PDP brief.
- GA4 for outcomes. After a fix ships, do AI-referred sessions grow, and do they convert at least as well as before?
A hypothetical example. A luggage brand sees a 70% first-response mention rate (prompt tracking: fine). Conversation replays show it drops at the constraint turn whenever a buyer mentions a cabin-size limit (conversation analytics: diagnosis). The PDP has dimensions only inside an image. The team adds a cabin-size attribute block and an FAQ entry, then watches AI-referred add-to-carts on that product line (GA4: outcome).
Read the gaps between them as well:
- High mentions, low Share of Recommendation: you are a discovery brand. Fix the constraint and objection turns.
- Good Share of Recommendation, low referrals: buyers may be deciding in-chat and arriving direct or via marketplaces. Check branded search and marketplace sales.
- Rising referrals, flat conversion: the landing page does not match what the chat promised. Align the PDP with the claims engines repeat.
Related reading: Chat Is the New Funnel · Why single-prompt tracking overstates your AI visibility · Why AI visibility tools disagree
See all three views on your own category. Book an Analytika demo.