AI Visibility Audit: Run the Shortlist Test in an Afternoon

An AI visibility audit costs one afternoon, needs nothing but a browser and a spreadsheet, and tells you something the dashboard will not: whether you win at the point buyers decide. Most platforms get bought before anyone has worked out what they are looking for.

This is the AI visibility audit we run ourselves, written out in full. Twenty-five prompts, four engines, six columns, and a scoring approach at the end. Copy it, change the brand names, run it yourself.

This is the practical half of AI visibility, and it will not replace tooling at scale. Manual auditing stops being practical past 10 to 15 queries per run, which is why tracking platforms exist. What it will do is show you what to ask those platforms for, and stop you paying $499 a month to measure the wrong funnel stage.

What this AI visibility audit measures, and what it does not

This audit answers one question: when a buyer already knows your name and is choosing between you and someone else, what does the machine say?

It does not measure awareness. If nobody has heard of you, this is the wrong test and a category prompt set is the right one. It does not measure traffic, rankings or conversions. And it is not a substitute for the on-site work covered in the AI visibility checklist, which looks at structure, crawlability and formatting rather than outcomes.

What it gives you is a baseline across the four shortlist prompts: comparison, alternatives, pricing and migration. Those are the questions asked during evaluation, and they behave differently from category questions, which is the argument in why your AI visibility score measures the wrong stage.

Run the AI visibility audit once and you have a snapshot. Run it monthly and you have the only thing that matters, which is a trend. That trend is also what feeds our wider citation research.

Before you start: four decisions

Pick your pairs. Your brand against the three competitors you actually lose to. The pillar’s prioritization method covers how to pull that list properly. Pull that from closed-lost data, not from the positioning deck. The two lists are rarely the same.

Pick your engines. An AI visibility audit on one engine is not an audit. ChatGPT, Google AI Mode and one other, usually Perplexity or Claude. They retrieve and cite very differently, as our engine comparison sets out, so results diverge sharply between them.

Fix your conditions. Logged out, one country, browser rather than app. Answers shift with each of these, part of the wider change in search behaviour. Write down which. Answers change across locations and interfaces, and an audit run under mixed conditions cannot be compared to itself next month.

Decide who runs it. The same person every time. Judgement calls on what counts as a mention need to be consistent, and they will not be across two people.

Thirty minutes of setup, and it is the difference between a baseline and a pile of screenshots.

Step one: build the twenty-five prompts

The prompt list is fixed from here. Changing it changes the denominator and breaks the comparison.

Comparison, six prompts. Your brand against each of three rivals, in both word orders. Buyers type both, and the answers differ.

Alternatives, four prompts. “Best alternative to [rival]” for each of the three, plus “best alternative to [your brand]”. That last one is uncomfortable and it is the most informative prompt in the set.

Pricing, nine prompts. “How much does [brand] cost”, “[brand] pricing”, and “is [brand] worth the money”, for you and your two closest rivals.

Migration, four prompts. “How to migrate from [rival] to [you]” and the reverse, for your two closest rivals.

Category control, two prompts. Two straightforward category questions, for contrast. These are the ones keyword tools and dashboards already cover, and where listicles dominate. These are the prompts your existing dashboard already measures, so they tell you how far apart the two pictures are.

Type them bare. No context, no qualifiers, no “we are a 50-person company”. Adding context does the engine’s fan-out work for it and produces an answer no real buyer would see.

Step two: run the audit properly

Rules that keep the results usable.

One prompt per conversation. Never two in the same chat. The second answer inherits the sources and framing of the first, which is a mistake we made in our own comparison study and disclosed as a limitation.

Logged out, or at minimum with memory and custom instructions off. A personal account leaks your history into the answer. In our run, ChatGPT referenced the account holder’s own work when recommending a tool. That is memory shaping an answer meant to be generic.

Expand every source panel. Google AI Mode hides most citations behind “Show all”. Claude puts them inline. If you log only what is visible by default, you will undercount badly.

Screenshot the whole answer. Not the summary. You will want the exact wording later, especially for anything about your pricing.

Run the whole set in one sitting. Answers drift over days. A set spread across a week is not one measurement.

Twenty-five prompts across three engines is 75 runs, which is the whole AI visibility audit. At roughly two minutes each, that is your afternoon.

Step three: what to log

Six columns. Any more and you will stop doing it.

Prompt. Exact text, so next month is identical.

Engine and date. Obvious, routinely forgotten.

Sources cited. Every domain, in order. If nothing was cited, write “none”. That column turns out to be the most interesting one.

Are we in it. Named, cited with a link, or absent. Three states, not two. Being named without a link still shapes the buyer.

What it said about us. One line. This is where you catch a wrong price or a competitor’s framing repeated as fact, which is the accuracy problem nobody scores.

Action. What this row implies. Leave it blank on the first pass and fill it after scoring.

Keep it in a sheet, one row per run. Seventy-five rows a month is manageable. A doc full of screenshots is not, which is what most teams produce and then abandon by run two.

Step four: score it

Four numbers, from the metrics set out here.

Shortlist presence. Of your 23 evaluation prompts, how many named you at all. Expressed as a percentage.

Source ownership. Of the runs that cited anything, how many included a page you own.

Answer accuracy. Of the runs that said something factual about you, how many were correct. Pricing errors count as wrong, not as close enough.

Silence rate. How many runs cited nothing at all. In our comparison study this was two thirds, and it is the number that tells you whether this quarter’s publishing can move anything.

Score each engine separately, then look at the spread. One overall figure hides the case where you are strong in one engine and invisible in another, which is common and calls for different work.

Write the four numbers at the top of the sheet with the date. That is your baseline.

Step five: read it honestly

High silence rate, low presence. The models are answering from memory and you are not in that memory. Publishing this month will not fix it. This is a long game of third-party presence, reviews and earned mentions.

Low silence rate, low source ownership. Engines are citing, and citing other people. This is the most fixable state you can be in. The pages exist to be written.

Good presence, poor accuracy. The dangerous one. You are visible and being described wrongly, usually on price. Fix the source page first, then the third-party pages carrying the stale number.

Strong on category control, weak everywhere else. The classic split. Your dashboard says you are fine and your evaluation results say otherwise. This is the exact gap this audit exists to expose.

Whatever the shape, resist explaining a single run. One audit is a snapshot. Three runs is a trend, and only a trend justifies a change of plan.

What to fix first

Match the finding to the work.

Missing from comparison prompts. Check who holds those results organically first, then build the comparison page for your biggest rival, written as an assessment rather than a pitch. A page that concedes nothing gets treated as promotional by both readers and engines.

Absent from alternatives lists. You need an alternatives page and third-party presence, including on G2 and Capterra and in the forums and video these answers draw on. This one is slower because half the fix is not on your site.

Wrong pricing quoted. Put the numbers in text on your pricing page, date them, and then chase the third-party pages repeating the old figure. Our outreach tools roundup found four of ten prices wrong across the roundups that list them, so this is common.

Nothing on migration. Write the switching guide. Almost nobody has one, which makes it the cheapest win in the set, and the customers who switched are your best case study material.

Everything looks fine. Then the AI visibility audit was too easy. Check the prompt set. If your brand is in every answer, add the two rivals you have been avoiding.

Six mistakes that invalidate the audit

Running prompts in one long conversation. Every answer after the first is contaminated.

Using a logged-in personal account. Memory and custom instructions change the output.

Changing the prompt set between runs. The comparison to last month becomes meaningless.

Logging only the default visible sources. You will systematically undercount AI Mode.

Running across several days. Answers drift, so the set is no longer one measurement.

Treating one run as evidence. Variance across runs is large enough that a single result proves nothing, which is why we publish run dates and sample sizes on everything in this cluster.

None of these are exotic. They are the ordinary ways a manual audit quietly stops being data.

FAQ

Do I need a tool to run an AI visibility audit? Not to start. Manual is better for the first two or three runs because you see the raw answers. Past 10 to 15 queries per run it stops scaling, and that is the point to buy something.

How long does an AI visibility audit take? Setup is 30 minutes, the runs are two to three hours, scoring is 30 minutes. One afternoon for the first, faster after that.

How is this different from an SEO audit? An SEO audit checks whether pages can rank. This checks what an answer says about you, which is a different question, covered in SEO versus AI search.

Can I use fewer prompts? Yes. Cut pricing to three and migration to two for a lighter version. Keep both comparison word orders, because the difference between them is often the finding.

What if a competitor runs the same audit on us? They will get roughly the same numbers, which is fine. The advantage is not the data, it is doing something about it first.

Should we share the results externally? Only if you are prepared to re-run and publish when the numbers get worse. Published benchmarks that stop after one run advertise that you stopped.

A platform tells you a number. An AI visibility audit run by hand tells you why.

That is the job we do. See how we approach AI visibility and how we write comparison pages, look at our work, check pricing.