Signal Harbor Weekly
A citation makes an AI-generated answer look trustworthy.
A bracketed number, a recognizable publication, or a link to an official document creates the impression that the claim has been verified.
But the presence of a citation does not prove that the source supports what the AI actually said.
A source can be genuine, credible, and relevant while still failing to justify the exact language presented to the user. It may support only part of the statement. It may describe a future plan rather than a current capability. It may report results from one customer that the AI then generalizes to an entire market.
For companies increasingly being researched and evaluated through generative AI, that distinction matters.
The question is no longer simply:
Was our company cited?
It is:
What did the AI claim, and is the cited evidence strong enough to justify what the buyer saw?
Citation presence is not the same as citation support
Many AI systems use retrieval and search methods to incorporate external information into their answers. This can make responses more current and easier to inspect.
It can also create false confidence.
A system may successfully retrieve a relevant document while still misunderstanding, oversimplifying, or overstating what the document says.
Researchers introduced the ALCE benchmark to evaluate how well large language models produce answers with citations. The benchmark separates several issues that are often treated as one:
Is the answer correct?
Are the important claims supported?
Are the citations attached to the appropriate statements?
Do the cited passages actually contain the necessary evidence?
On one long-form question benchmark, even the strongest systems failed to provide complete citation support for roughly half of their answers.
That does not mean every citation was false. It means an answer could appear well-sourced while still containing claims that were not adequately supported.
For a business user, that gap can be easy to miss.
A competitive-intelligence team might ask an AI system to summarize a competitor’s financial filings. The answer may link to a real SEC document, but the AI can still draw conclusions that the filing itself never makes.
For example, a filing may report declining revenue, while the AI concludes that the company is losing market share. The source is legitimate, but the interpretation is not necessarily supported by it.
The link may be legitimate. The interpretation may still be wrong.
Three ways cited AI answers can fail
Most citation problems fall into three broad categories.
1. Important claims are left unsupported
An answer may include citations beside some sentences while making additional claims without clear evidence.
Because the response looks researched overall, users may assume the entire answer has been verified.
This is especially risky in long-form answers where factual statements, interpretations, and recommendations are blended together.
A company may be accurately described in the first paragraph, while later claims about its pricing, market position, customer fit, or performance receive little or no support.
2. The citation is relevant but does not establish the claim
A source can discuss the correct company, product, or topic without proving the specific statement attached to it.
Consider a product marketer researching a competitor.
The AI says:
Platform X currently includes predictive analytics.
The cited source is real and discusses predictive analytics. But the underlying document is a roadmap stating that the feature is planned for the following year.
The citation is relevant. The claim is still inaccurate.
This is the difference between finding a document related to the subject and finding evidence that actually supports the sentence.
Recent peer-reviewed work such as VeriCite addresses this problem by separating answer generation from evidence verification. Rather than attaching sources to a completed response and assuming the job is done, the system checks individual claims, identifies passages that genuinely support them, and refines the answer around that validated evidence.
The broader lesson is simple:
Citation generation and citation verification are not the same task.
3. The AI makes a stronger claim than the evidence allows
This may be the most subtle failure.
A source can support a narrow or qualified statement, while the AI rewrites it as something broader, more certain, more current, or more dramatic.
A recent preprint describes one version of this problem as “citation laundering”: a legitimate and topically relevant source gives an overstated claim the appearance of credibility.
This can happen in several ways.
A future plan becomes a current capability
A company announcement says a feature is expected next year.
The AI says the company currently offers it.
A single case study becomes a universal outcome
A customer case study reports that one business reduced costs by up to 30% under specific conditions.
The AI says:
Customers reduce costs by 30%.
The qualifiers disappear. A narrow result becomes a general product promise.
Correlation becomes causation
An industry report notes that a company lowered its prices during the same period that sales increased.
The AI concludes:
The price reduction caused the increase in sales.
The source may document both events without establishing that one caused the other.
Old evidence becomes current positioning
A 2022 press release describes a company as hardware-focused.
The company later shifts toward software, but the AI continues to repeat the older positioning because the outdated source remains prominent and retrievable.
Tentative language becomes certainty
A leadership letter says the company may gain market share.
The AI says the company will dominate the market.
Each version preserves enough of the source’s original meaning to sound plausible. That is exactly what makes these errors difficult to detect.
In the FORCEBENCH experiments, model-based evaluators using ordinary citation-support instructions frequently failed to distinguish properly calibrated claims from subtly overstated versions. More explicit instructions improved performance, but did not fully solve the problem.
A source can therefore be real, relevant, and still too weak to support the exact wording used.
Correctness is not the same as faithfulness
There is another, more technical concern.
A citation may support a statement without proving that the model genuinely relied on that source when producing the answer.
Researchers describe this distinction as citation correctness versus citation faithfulness.
Citation correctness asks:
Is the cited source compatible with the claim?
Citation faithfulness asks:
Did the cited evidence actually influence how the model reached the answer?
The difference matters because an AI system may generate a conclusion using patterns already embedded in the model and then attach a retrieved source that appears to support it afterward.
This is sometimes described as post-rationalization.
Under some experimental conditions, researchers found that up to 57% of citations lacked faithfulness. That result should not be generalized to every citation produced by ChatGPT, Gemini, Perplexity, or other commercial platforms.
But it illustrates an important risk:
A citation can make an answer look evidence-driven even when the connection between the evidence and the conclusion is weaker than it appears.
For most companies, proving the internal causal process of a model may not be practical. The immediate operational priority is simpler: determine whether the evidence actually supports the claims buyers are seeing.
Why this matters for companies
Citation problems are not merely academic.
They can affect how a company is understood before a buyer ever visits its website or speaks with its sales team.
An AI system might incorrectly describe:
which customer segment the company serves;
whether a product includes a particular feature;
how pricing works;
whether the company operates in a certain region;
which industries it specializes in;
how it compares with a competitor;
whether a performance claim applies broadly;
whether an old limitation still exists.
Some inaccuracies will be obviously wrong.
The more dangerous ones will be plausible.
A potential customer may never notice that a source was outdated, that a case study was generalized, or that a roadmap item was presented as a live capability.
They may simply remove the company from consideration.
This creates a new kind of brand and competitive risk. A company can be highly visible in AI-generated answers while still being represented inaccurately or supported by weak evidence.
Visibility alone is therefore not success.
What organizations should measure
A serious AI recommendation audit should go beyond counting citations.
It should examine:
Factual accuracy
Is the company, product, leadership, pricing, geography, and customer fit described correctly?
Citation completeness
Are all important factual claims supported, or only selected parts of the answer?
Claim-to-source alignment
Does the cited passage actually establish the statement attached to it?
Source freshness
Is the AI relying on information that still reflects the company’s current product and positioning?
Scope and certainty
Has a narrow, tentative, or conditional finding been rewritten as a broad and definitive claim?
Numerical consistency
Were ranges, estimates, and “up to” figures converted into precise or universal outcomes?
Repetition
Does the same unsupported or misleading claim appear repeatedly across platforms, prompts, and testing periods?
Business severity
Would the error materially affect trust, product evaluation, shortlisting, or purchasing decisions?
Not every citation problem deserves an emergency response.
A risk-based system should prioritize claims involving:
pricing;
product capabilities;
customer suitability;
market leadership;
performance outcomes;
security;
compliance;
regulatory status;
major competitive comparisons.
A single unsupported statement may be random model variation.
A claim repeated across several AI platforms or measurement periods becomes a more meaningful reputation and decision risk.
A citation is the beginning of verification
Citations remain valuable.
They give users a starting point for inspection. They can make AI answers more transparent. They can help companies identify which sources are shaping their public representation.
But they should not be mistaken for proof.
The next generation of AI visibility measurement cannot stop at asking whether a company was cited.
It must ask:
What did the AI claim?
Which evidence was attached?
Does that evidence support the full statement?
Is the source current?
Has the wording become broader or more certain than the source allows?
Is the same issue appearing repeatedly?
A citation may make an answer look trustworthy.
Verification determines whether that trust is deserved.
At Signal Harbor, this is why we look beyond citation counts. We repeatedly test how companies are described and recommended across AI platforms, then examine whether the claims are accurate, current, and genuinely supported by the sources attached to them. Visibility matters, but only when the representation behind it can be trusted.
Sebastian Miller
Co-Founder, Signal Harbor
signalharborconsulting.com
sebastian.miller@signalharborconsulting.com
402-306-2213
Research Notes
This newsletter is based on research examining citation quality, source attribution, retrieval-augmented generation, and the difference between attaching a source to an AI-generated statement and verifying that the source actually supports it.
The goal is not to claim that citations are useless or that all AI-generated citations are unreliable. Citations can improve transparency and give users a way to inspect an answer.
The narrower argument is this:
A real and relevant citation does not automatically prove that every claim in an AI-generated answer is complete, accurate, or justified by the underlying evidence.
Core research foundation
Gao, T., Yen, H., Yu, J., & Chen, D. (2023). Enabling Large Language Models to Generate Text with Citations. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.
This peer-reviewed paper introduced ALCE, a benchmark designed to evaluate large language models that produce answers with citations.
The benchmark separates three issues that are often incorrectly treated as the same:
the quality and correctness of the answer;
whether important claims receive citation support;
whether the cited passages actually support the attached claims.
On the ELI5 long-form question-answering benchmark, the authors found that even the strongest evaluated systems lacked complete citation support approximately 50% of the time.
This does not mean half of all individual citations were fabricated or incorrect. It means that an answer could contain citations and still include important claims that were not fully supported.
This supports the newsletter’s central argument that citation presence should not be treated as proof of complete evidentiary support.
Claim-level citation verification
Qian, H., Fan, Y., Guo, J., Zhang, R., Chen, Q., Yin, D., & Cheng, X. (2025). VeriCite: Towards Reliable Citations in Retrieval-Augmented Generation via Rigorous Verification. [Preprint]
VeriCite proposes a structured process for improving citation quality in retrieval-augmented generation.
Instead of simply generating an answer and attaching citations afterward, the framework:
generates an initial answer;
verifies the individual claims;
evaluates which retrieved documents provide useful evidence;
extracts supporting passages;
refines the final answer around the validated evidence.
The authors tested the framework across five open-source language models and four datasets. They reported improvements in citation quality while maintaining answer correctness.
This supports the newsletter’s distinction between citation generation and citation verification.
A system can retrieve a relevant source and still need an additional verification step to determine whether the source supports the precise statement being made.
Citation correctness and citation faithfulness
Wallat, J., Heuss, M., de Rijke, M., & Anand, A. (2024). Correctness Is Not Faithfulness in RAG Attributions [Preprint].
This paper distinguishes between two forms of citation quality.
Citation correctness asks whether a cited document supports or is compatible with the generated statement.
Citation faithfulness asks whether the language model genuinely relied on the cited information when producing its answer.
The authors examine a problem they call post-rationalization. In this situation, a model may produce an answer based on information or beliefs already represented internally, then attach a retrieved source that appears consistent with the conclusion afterward.
Under some of the paper’s experimental conditions, up to 57% of citations lacked faithfulness.
This statistic should be used carefully.
It does not establish that 57% of citations produced by ChatGPT, Gemini, Perplexity, Claude, or other commercial AI platforms are unfaithful. The result comes from the researchers’ particular experimental settings.
The paper supports a more limited claim:
A citation can be compatible with an answer without necessarily proving that the cited evidence genuinely drove the model’s conclusion.
For most businesses, directly reconstructing a model’s internal reasoning is not practical. The more immediate task is determining whether the source adequately supports the claim shown to the buyer.
Evidence strength and “citation laundering”
Qian, P., Wang, S., Wang, X., Chen, Y., Xu, W., Yu, Q., Lin, S., Zhang, S., You, J., & Wei, X. (2026). Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG [Preprint].
This recent preprint studies cases where an AI-generated claim includes a real and topically relevant citation, but the evidence is not strong enough to justify the exact wording.
The authors describe this problem as “citation laundering.”
They introduce FORCEBENCH, a diagnostic benchmark that compares a properly evidence-calibrated claim with a subtly overstated version while keeping the cited passage unchanged.
The benchmark evaluates changes across five dimensions:
Relation: correlation or association becomes causation.
Modality: possibility or uncertainty becomes certainty.
Scope: evidence about a limited group becomes a broad generalization.
Temporal validity: past or planned information becomes a current claim.
Numeric specificity: estimates or ranges become overly precise figures.
Across four model-based evaluators, ordinary citation-support prompting produced a 47.2% aggregate monotonicity violation rate. In practical terms, the evaluators frequently failed to rank the properly supported claim above a subtly overstated version. More explicit instructions about evidentiary strength reduced the violation rate to 24.5%, but did not eliminate the problem.
This should not be presented as a general real-world error rate for commercial AI systems.
It supports the narrower conclusion that standard citation checks may struggle to detect subtle differences between what a source says and how strongly an AI system presents it.
Because this is a recent preprint, its findings should be described as emerging research rather than settled scientific consensus.
Conclusion
Citations improve transparency, but they should be treated as the beginning of verification rather than the end of it.
For companies, meaningful AI recommendation measurement should examine not only whether a source appears but also whether the source is current, whether it supports the full claim, whether qualifiers have been preserved, and whether the same unsupported representation appears repeatedly across platforms and testing periods.
This article first appeared in Signal Harbor Weekly, the Signal Harbor newsletter on AI visibility.