One screenshot from ChatGPT is not a measurement. A real AI visibility report runs a fixed set of prompts for your trade and city, repeats some of them, logs the date and login state, and reports how often you were named, assistant by assistant, as “named in X of N.” If a report cannot show you the prompts and the repeats, you cannot tell progress from noise.

Update, October 2026

Search Console now has a Generative AI performance report that shows whether your pages appear in Google’s AI features. It is first-party data, so read it next to any prompt test. And Google’s guide to optimizing for generative AI features says no special AI file or markup is needed to appear, so a report that sells one as the fix is selling the wrong thing.

This page is for owners who were just sent an AI visibility report, or are about to pay for one. Below is a real test we ran on October 14, 2026, laid out the way a monthly report should look, with notes on what each part means and what to be suspicious of. If you want to know what gets a business named in the first place, that is a different question, covered in our guide to AI search visibility for local service businesses.

Why One Screenshot Proves Nothing

AI assistants do not keep a fixed ranking the way a map pack roughly does. Ask the same question twice in two fresh chats and the list of names can change. Ask a different assistant and you may get a different set of companies entirely. So a screenshot of your business inside one ChatGPT answer tells you that it happened once, on one day, to one session. It does not tell you how often it happens to the homeowner searching tonight.

Our own test below shows both effects. Repeated prompts came back with different tails, and two of the five most named companies were never named by Gemini at all. That is why the only useful number is a rate over many answers, split by assistant.

The Sample Report: A Real Scottsdale Pool Remodeling Test

We ran this for the market “pool remodeling company in Scottsdale, Arizona.” It was not run for a client, so there is no “you” in it. Read it as if you were one of the companies listed. Names are hidden on purpose: the five most named businesses appear as Company A through Company E, and everyone else is “other local companies.”

Real test, Oct 14, 2026. Business names anonymized.

1. Run log

FieldWhat we logged
Date and timeWednesday, October 14, 2026, prompts sent between about 10:22 and 10:35 AM ET
Login stateLogged out on every assistant. One new chat per prompt.
AssistantsChatGPT (logged out, temporary chat), Gemini (logged out), Google Search AI Overview (logged out, not AI Mode)
LocationCity written into every prompt
Answers24 primary answers (3 assistants, 8 prompts each), plus 2 ChatGPT repeats

What it means: anyone can rerun this next month under the same conditions and compare. Red flag: a report with no date, no login state and no location. Signed-in answers are shaped by the account’s history, so they say more about the person testing than about the market.

2. Prompt panel

Every prompt, exactly as sent. None of them names a business.

  1. Who are the best pool remodeling companies in Scottsdale, AZ?
  2. I want to resurface my pool in Scottsdale. Which companies should I call?
  3. Pool remodel contractors near Scottsdale Arizona with good reviews
  4. Who does pebble tec pool resurfacing in Scottsdale?
  5. Best company to remodel an old pool and add a baja shelf in Scottsdale AZ
  6. How much does a pool remodel cost in Scottsdale, and who should I get quotes from?
  7. Recommend a pool renovation company in North Scottsdale for new tile and coping
  8. Which Scottsdale pool remodelers are licensed and insured?

What it means: these are written the way buyers ask: by service, finish brand, neighborhood, cost and license. Red flag: a panel made of branded prompts like “Is [your company] good?” An assistant will usually talk about you when you put your name in the question. That proves nothing about whether it names you when a homeowner asks for options.

3. Mention rate: named in X of 24

Each cell counts primary answers that named the company in the answer itself. Companies that only showed up in a source link were not counted.

BusinessTotal (of 24)ChatGPT (of 8)Gemini (of 8)Google AI Overview (of 8)
Company A14374
Company B14365
Company C7601
Company D7601
Company E6015

Behind them, other local companies were named in 3 or 4 of the 24 answers. Adding the two ChatGPT repeats does not change the order of the top five: A 15, B 14, C 9, D 9, E 6.

What it means: “named in 14 of 24” is a number you can track month to month. Red flag: a single “visibility score” with no N behind it. If you cannot see how many answers the score came from, you cannot tell 1 of 2 from 50 of 100.

4. How the assistants differed

What it means: “AI” is not one channel. If Company C got a report with one blended number, it would look fine and never learn that it is missing from Gemini entirely. Red flag: results reported for “AI” as a whole, with no split by assistant.

5. What changed on reruns

What it means: in this test the names at the top held fairly steady while the bottom of each list moved around. We only repeated two prompts, so we are not putting a repeat rate on it. Red flag: a report built from one run per prompt that celebrates a new name near the bottom of a list. That may be gone tomorrow.

6. Sources the answers pulled from

What it means: this row tells you where to earn a listing, not just whether you were named. For a pool company here, that points to an active license record, review profiles, a manufacturer locator listing and service pages that say city, finish and license. For why an assistant can quote a tiny directory review count while you have hundreds on Google, see where your reviews actually show up. Red flag: a report that shows names but never the sources behind them, because then there is nothing to act on.

7. Competitors named next to you

In a client report, this row lists which companies sit beside the client in each assistant’s answers. Here, with no client, it is the table above: if you were Company E, your real rivals in Google’s AI Overview are not the same as your rivals in ChatGPT.

8. Google AI impressions by page

Not part of this test, because there was no client website. In a client report, this row comes from Search Console’s Generative AI performance report: [impressions by page, for the month]. Red flag: a report that claims AI queries or AI clicks the data source does not provide.

Method notes for this test

How We Run a Prompt Panel Each Month

The Scottsdale test is one run of the method we use for clients. In a client report, the setup stays fixed so this month can be compared with last month:

As run in Scottsdale, the test covered three assistants (Perplexity would not answer logged out) and repeated two prompts on ChatGPT. A full client panel repeats more of the set.

Three Checks Before You Trust This Month’s Numbers

Added October 19, 2026. Two things happened in the week after we published that change how an October report should be read, and one check belongs at the start of every panel.

1. ChatGPT traffic “dropped” in October? Check the tag first.

After GPT-6 rolled out (October 7 for paid plans, October 8 for free), several analytics vendors reported ChatGPT sessions in GA4 falling by roughly half while Direct traffic rose. The reported cause: links inside ChatGPT’s answer text stopped carrying the utm_source=chatgpt.com tag, and the visits landed as Direct. Adobe’s Cedric Huesler says OpenAI confirmed to him on October 14 that it was a bug and that tagged clicks were back to roughly normal by the morning of October 15 (Search Engine Journal, October 15, 2026). As of October 19, OpenAI has made no public statement, so treat that as reported, not confirmed.

2. Score the business, not the exact page.

A research paper posted October 11 (a preprint, not peer reviewed, with data collected before GPT-6) watched which pages ChatGPT actually cited and compared them with two stand-ins you can check without opening ChatGPT: Google’s results and OpenAI’s API. Together, the stand-ins found only 26.2% of the pages ChatGPT cited in the GPT-5.5 Instant tests, but 48.7% of the sites. And 44.5% of the distinct pages for a question showed up in just one of 20 runs (Search Engine Journal, October 18, 2026). That is why our panel is run in the assistants themselves, repeated, and scored as “named in X of N” at the business level. A report that tracks one URL’s exact citation, through an API, is measuring the noisiest thing available.

3. Make sure the AI knows which business you are.

Before counting mentions, starting with our next panel, we ask each assistant to describe the business by name and check what comes back: the right trade, the right city, the right phone and address. If an assistant mixes you up with a similar name, or splits one company into several names, mention counts for that assistant are not reliable until the business details are fixed. Consistent name, address, and phone across your site, Business Profile, and directories is the first repair.

Red Flags in Agency AEO Reports

Our rule: if a report fails two of these, treat its trend line as noise until the method is fixed.

Questions to Send Before You Pay for One

  1. Can I see every prompt you run, word for word?
  2. Are the answers collected logged out, and where is the location set?
  3. How many times do you repeat each prompt, and will you show me what changed?
  4. Will you report “named in X of N” for each assistant separately?
  5. Which sources did the answers cite, and which of them are we missing from?

A good agency answers all five without hesitating. Still choosing the agency itself? See the full sales call script for choosing an agency. For how this looks in a single trade, see how AI names pool companies in ChatGPT and law firms in ChatGPT.

Want Your Own Prompt Panel?

We are a Google Partner agency. Book a call and we will run a logged-out prompt panel for your trade and city, show you who gets named in each assistant and which sources come up, and tell you where to start. See how our AI search visibility work runs.

Book a Strategy Call

Sources, read October 14, 2026, with the October 19 update read that day. Ostmen Marketing test log, Scottsdale pool remodeling, October 14, 2026 (ChatGPT, Gemini and Google AI Overview, logged out). Search Console Help: Generative AI performance report (Search). Google Search Central: Optimizing your website for generative AI features on Google Search. Search Engine Journal: ChatGPT traffic drop appears to be a tracking problem (October 15, 2026) and OpenAI’s API only partly matched ChatGPT’s citations (October 18, 2026), on the arXiv preprint 2610.15382.