Why We Benchmark Before We Optimize

The Signal Harbor Journal

Research and perspectives from Signal Harbor

By Sebastian Miller
Co-Founder, Signal Harbor

Benchmark AI Visibility Before You Optimize for It

AI search has created a crowded market of dashboards, audits, content services, SEO projects, and promises to “optimize for ChatGPT.”

Those services can address different parts of a real problem. Before a company spends money changing its website or publishing more material, it needs to know which buyer decision the work is supposed to improve.

That is why Signal Harbor benchmarks before optimizing.

A Visibility Score Is a Starting Point

An AI visibility score summarizes how often a company appears in a defined set of tested answers. Brand mentions count appearances; a share of voice metric compares a brand with tracked competitors according to a particular tool’s method. These measures can be useful, but they do not represent every AI conversation in the market.

A company may appear often and still be described inaccurately. It may show up for a broad category question but disappear when the buyer adds a technical requirement, budget, location, or integration. It may be recommended for the wrong customer while a competitor appears for the work it most wants to win.

We examine inclusion, accuracy, recommendation, and competitive position separately. Each points toward a different decision.

Define the Market Before You Choose the Prompts

A benchmark begins with the customer segments, use cases, competitors, buying criteria, and services that matter to the business.

That step is prompt research. A prompt is the question we test with an AI platform. A prompt library is the organized set of questions used in the benchmark. We include branded prompts, which mention a company by name, and unbranded prompts, which ask for options before a buyer knows who to consider.

The questions follow the buying process: discovery, comparison, validation, and purchase requirements. They are researched tests, informed by product information, sales conversations, buyer objections, and available customer and search data. They are not a log of every question real buyers have typed into AI.

For a private equity firm comparing several portfolio companies, this discipline matters. Each company has a different market and buying process. A generic prompt list may produce a tidy cross-portfolio chart while missing the decisions that create qualified opportunities for each business.

Test the Answers and Check the Claims

AI answers can vary by platform and over time. We run comparable tests repeatedly to establish a baseline and look for patterns rather than treating one screenshot as the market’s verdict.

Then we inspect the answer. Is the company absent, inaccurately described, or recommended for the wrong use case? Did AI present a competitor as a better fit? Is that comparison supported by verifiable facts?

We distinguish three kinds of evidence:

- Company-verified facts: What the business can confirm about its products, capabilities, service areas, policies, and customers.

- Public evidence: What its pages, documents, reviews, directories, partners, and independent sources actually say.

- Buyer fit: Whether the available facts support recommending it for the buyer’s stated requirements.

That prevents an unsupported company claim from becoming “proof,” and it prevents an AI answer from being treated as ground truth.

Source Mapping Finds Evidence Gaps

Source mapping, also called source analysis or citation analysis, looks at the pages and publishers cited in the tested answers. It shows which information is appearing alongside a response and where competitors have stronger public support.

A citation does not prove that a source caused a brand recommendation. It is one piece of evidence to investigate.

We ask whether an important capability is clear on the company’s own pages, whether a third-party description is outdated, or whether the competitor has credible proof that the company has not made public. We also check whether the cited page actually supports the claim attached to it.

This is how the benchmark becomes useful to a marketing team. It identifies the information that needs attention, not just the brand that appeared more often.

Optimization Begins With a Diagnosis

SEO helps traditional search engines find and understand a site. AEO and GEO are market terms for work intended to improve how a company appears in AI answers. The practical work may overlap: accurate pages, clear comparisons, specific buyer answers, and credible evidence.

Signal Harbor identifies the gaps tied to valuable buyer decisions and helps shape priorities for the client’s team or agency to publish. One finding may call for a clearer service page. Another may require updated documentation, a corrected listing, or proof that a specialist capability is real. Some findings need monitoring rather than a new campaign.

After changes go live, we use the benchmark to remeasure the relevant questions. Did the company enter more answers? Did the description become more accurate? Did a competitor’s advantage narrow? A change in tested answers is measurable; a future recommendation is never guaranteed.

Where business data is available, we examine AI referrals, qualified inquiries, opportunities, and revenue signals alongside those results. Visibility alone does not establish that the work caused a sale.

The sequence is simple: define the buyer decision, measure the baseline, diagnose the gap, improve the evidence, and test again.

That is what Signal Harbor does. We give marketing teams a defensible place to start and a way to see whether their work changed what buyers hear from AI.

Sebastian Miller
Co-Founder, Signal Harbor
signalharborai.com
sebastian.miller@signalharborconsulting.com
402-306-2213