Building a prompt set that represents real buyers

Table of contents

The questions decide the answer

An AI visibility study can only prove what its prompt set lets it prove. If the prompts are narrow, repetitive, or unlike how buyers actually talk, no amount of analysis downstream will rescue the result. Strong statistics on weak prompts produce precise numbers about the wrong thing.

Most prompt sets start from a keyword list. Someone exports the top search terms for a category, turns each into a question, and runs them. The keywords are real; the questions are not. People do not talk to a chat assistant the way they type into a search box, and the mapping between the two is an assumption nobody measured.

This post walks through how to build a prompt set that represents real buyers instead: from defining who you are measuring, to structuring buying journeys, to weighting what matters. It draws on the method AEOsim and Kojable published in Synthetic Data in LLM Visibility Tools, Part 1.

Five steps to a representative set

1. Define the population before writing a single prompt

Decide who the set is meant to represent. For an Indian D2C skincare brand, that might be English and Hinglish-speaking buyers in metro and tier-2 cities, shopping for themselves, on mobile, with budgets under ₹1,500. Write it down. A set that looks varied on paper but does not resemble that population will mislead you.

Separate two goals while you are at it: prompts that mirror typical buyer behaviour, and prompts that deliberately stress-test edge cases. Both are useful. Mixing them without labels makes the result unreadable.

2. Script buying journeys, not single questions

Buyers do not decide in one prompt. Each entry in the set should be a short conversation: an opening question, a comparison, a constraint, an objection, and a request to pick. The decision sits at the end of that ladder, so a set of opening questions alone never reaches it.

3. Build prompt families around each task

Start from one canonical buying task, then add controlled variations:

• Variations that should not change the answer: short and long phrasing, casual and formal, a typo, Hinglish.
• Variations that should: a different budget, city, size, skin type, or a single word like "not".

Families let you check that the engine treats true paraphrases consistently, while still catching cases where a small change should flip the pick.

4. Mix sources, and label each one

AEOsim and Kojable suggest a corpus budget such as 60% de-identified real questions (from consented support or search data), 20% task-elicited questions (from moderated sessions with real users), and 20% synthetic stress tests for rare or risky cases. Treat that as a disclosure format, not a purity rule. The right mix differs by category.

For every prompt, store more than the text: its source, intent, persona, language, geography, funnel stage, and family.

5. Weight prompts by what they are worth

A generated set has a flat distribution by default. Real importance is not flat. Weight each prompt on at least three things:

• Frequency: how often buyers plausibly ask it.
• Proximity to purchase: a question asked right before checkout counts for more than idle browsing.
• Risk: a rare question with a compliance or trust consequence can outweigh a common one.

Then check information value. Prompts your brand always wins, or never wins, tell you little. The most informative prompts are the ones you win about half the time (Play the Games You Might Lose).

Check the set before you trust it

Before a prompt batch is used, run a short pre-flight:

• Target volume and balance across personas, stages and categories are met
• Prompts are unique, and impossible combinations are removed
• Branded and unbranded prompts are tagged and will be scored separately
• At least two reviewers checked a sample for plausibility, with agreement recorded
• Some families are held out, so later gains are not just gains against the test
• A fixed anchor block is set aside to keep results comparable across refreshes
• The set can be regenerated from a recorded seed

Keep an immutable manifest for each batch: generator version, templates, seed, target counts, any model used to write prompts, and which families were held out. If a score moves later, you need to know whether the brand changed or the instrument did.

Questions to ask before you trust the numbers

• What grounded this prompt set, and can I inspect it?
• How is each prompt weighted, and does the weighting reflect my business or only the language?
• Which prompts currently separate me from my competitors?
• What is held out, and what carries across a refresh?

If your AI visibility tool can answer those, the work has been done. "Our prompts are real" answers none of them.

Related reading: Chat Is the New Funnel · Syntheticity is a spectrum, not a flaw · Ground prompts in business data, not GSC keywords

Want a prompt set built from your buyers? Book an Analytika demo.

Sources

• Synthetic Data in LLM Visibility Tools, Part 1, AEOsim and Kojable
• Bridge The Real2Sim Gap, AEOsim and Kojable
• Play the Games You Might Lose, AEOsim and Kojable