The claims agencies make that do not survive checking

None of these spread through dishonesty. They spread because a number gets repeated, then attributed to a more credible publisher than the one that produced it, and after enough repetitions nobody checks. The people quoting them at you usually believe them.

We removed one of our own case studies for a version of the same failure. It said applications had tripled, and nobody had recorded the starting number. The change was real and the claim was not defensible, so it came down. Every rule below applies to us first.

Why do false numbers spread so easily?

Because a statistic gets separated from its source on the first retelling. Someone publishes a small survey. A blog cites it without the sample size. A deck cites the blog and adds a famous publisher's name, because the number feels like something they would have said. From then on it travels with borrowed authority.

Check who benefits from the number being true. A missed-call statistic published by a company selling answering services is not disqualified by that fact, but it changes how much weight it can carry alone.

Which claims are real research wearing the wrong name?

The most persuasive failure, because the underlying work is genuine.

The best known is that calling a lead within five minutes makes you 21x more likely to qualify it, credited to Harvard Business Review. HBR contains no such figure. It comes from the Lead Response Management Study, presented by InsideSales.com at a MarketingSherpa summit in October 2007. Six companies, a conference deck, never peer reviewed, data never released. The domain that hosted it now carries paid links to casino review sites.

There is real HBR research here and it is better than the folklore. Oldroyd, McElheran and Elkington, in the March 2011 issue, audited 2,241 US companies and found 37% responded to a lead within an hour and 23% never responded at all. A separate dataset of 1.25 million leads across 42 companies found firms contacting within an hour were nearly 7x more likely to qualify a lead than those trying an hour later, and more than 60x more likely than those waiting a day. One caveat travels with it: a co-author founded the company whose platform produced the data.

The same shape shows up elsewhere. The $5,864 cost to replace a restaurant employee is real Cornell Hospitality Report research, but the sample was 33 US hotels in 2006 dollars. The claim that 88% of local searches end in a call or visit corrupts a Google and Ipsos MediaCT study from May 2014, which reported 50% of smartphone local searchers visiting a store within a day and 34% for computer and tablet.

Real research, wrong publisher, wrong industry, or wrong decade. Ask which one.

Which claims are a vendor blog dressed as research?

The category with the most volume, and the phone statistics are the worst offenders.

The claim that 62% of business calls go unanswered traces to a post published by 411 Locals on 18 January 2016. It is still live. The post says the company monitored 85 businesses across 58 industries for 30 days and found 37.8% of calls answered, 37.8% sent to voicemail, and 24.3% getting no response. It then pivots into recommending you hire a virtual secretary, which is what the publisher sells.

It is now attributed to Forbes, to Ruby Receptionists, and in one place to a ServiceTitan analysis of 50,000 contractor phone lines. None of those attributions is real, and three incompatible parents is itself the tell. The date is routinely reported as 2023 or later. It is 2016.

What holds instead is unglamorous and more useful. CallRail's January 2025 benchmark, built on 1.1 million de-identified conversations, puts missed call rates at 32% in health care, 28% in legal, 14% in home services, and 9% in real estate. Patient Prism, scoring 11,552,668 calls across 8,280 dental locations in 2025, found 31 of every 100 dental calls abandoned before reaching an agent. The spread from 9% to 32% is the actual finding. Both vendors sell into the category, so label them.

The llms.txt file belongs here too, and it is the cleanest one to settle. Ahrefs examined every domain in its web analytics with traffic in May 2026, 137,210 of them, and looked at every request to an llms.txt path by user agent. 28% published the file. 97% of those files received zero requests that month, from any bot or human. AI retrieval bots were 1.1% of an already tiny pool. Slackbot fetched llms.txt files more often than PerplexityBot did. No AI bot requested a file that did not exist, so publishing one puts you on no list and skipping it removes you from none. Google states Search ignores them.

Ahrefs sells SEO software and would benefit commercially from llms.txt being a real ranking input. They published the study saying it is not. That is what a credible vendor study looks like.

Which claims cite a study that does not exist?

This is the one that changes how you read a deck, and local search is where it lives.

Local SEO pitches often carry a table of click-through rates by local pack position. Position one gets some percentage, position two less, position three less again. Three of the four figure sets in circulation cannot be traced to anything.

One set, 17.8% for position one, 13.6% for two, 10.4% for three, is credited to a "BrightLocal Local Pack CTR Study." That study does not exist. The URL returns a 404 and no such report appears in BrightLocal's research index.

A second set, 24.4%, 13.3% and 8.6%, is credited to a "Moz Local CTR Study." Also nonexistent. Two contradictory figure sets, two studies never published, both circulated by the same page family.

A third set, 17.6%, 15.4% and 15.1%, comes from FirstPageSage, an SEO agency, which describes its own method as a meta-analysis combining research on click-through rates. No sample size, no method, and its source list includes Backlinko, which now cites FirstPageSage in return.

The only traceable study of this shape is weak, and everyone should know why. BrightLocal ran a Mechanical Turk test in October 2018 in which 5,500 paid testers looked at static screenshots of five search results pages, all San Francisco, all service-area categories, desktop only. It found local pack position one at 16.1% of clicks, position two at 10.4%, position three at 2.3%, and the whole three-result pack at 32.3% when Local Services Ads were absent. The author wrote in a public forum that nothing is as accurate as Google's own data and this was the best she could do.

Which claims are asking for something nobody can measure?

The structural category, and the most useful one, because it settles the argument permanently.

Nobody can publish local pack click-through rate by position, because Google exposes no local pack position dimension anywhere. Google Business Profile performance reports publish views, calls, website clicks, direction requests, messages and bookings. No rank. No position. Search Console has no local pack dimension either. A metric the platform does not expose cannot be aggregated by anyone.

The same logic disposes of several AI search claims. There is no rank inside an AI answer. Research from the University of St. Gallen found that repeated runs of the same prompt on the same day shared as little as 23% of their cited sources on ChatGPT, and that ChatGPT activated web search on only 42.2% of runs. A visibility score that moved five points in a week is inside the noise band of the measurement.

Google's own documentation supplies the sentence to keep on hand: no third-party tool has access to its internal ranking or AI systems. The tools in that category differ in what they can honestly claim, and the vendor's own limitations page is usually the most informative thing they publish.

Which claims are right about the rule and wrong about the regulator?

The subtlest failure, and getting it right marks you as someone who reads primary sources.

You will hear that the FTC banned review gating in 2024. It did not, and the imprecision gets your whole argument dismissed by the one person in the room who has read the rule.

Google bans gating explicitly. Its Maps content policy states that merchants may not discourage or prohibit negative reviews, or selectively solicit positive reviews from customers. One clause, no ambiguity.

The FTC bans the conduct around gating. 16 CFR Part 465, effective 21 October 2024, does not name gating. Section 465.4 prohibits compensation conditioned expressly or by implication on a review expressing a particular sentiment. Section 465.7(b) prohibits representing that displayed reviews are all or most of those submitted when reviews have been suppressed by rating or sentiment. Section 465.2(d) carves out the safe path: reviews resulting from generalized solicitations to purchasers.

An agency proposing a gated funnel is proposing your policy exposure, not theirs. Say it precisely: Google bans gating, and the FTC bans the adjacent conduct.

How do I check a claim in ten minutes?

Four steps, and the third one is the whole thing.

Search the exact number in quotation marks and sort by oldest. You are looking for the first appearance, not the most authoritative looking one.

When you find the origin, look for three things: sample size, method, and date. Any claim missing all three is not a finding.

Then divide the sample by the number of segments it is broken into. This is the arithmetic that ends most arguments, and you can run it on anything.

Take the 62% missed-call figure. Sample of 85 businesses, reported across 58 industries. That is about 1.5 businesses per industry. A per-industry claim built on one and a half businesses is not a benchmark.

Take the 2018 local pack test. 5,500 testers sounds large, and it is the wrong denominator. Five queries, one city, one device type. The effective sample is five results pages. That is why position three collapsing to 2.3% is almost certainly an artifact rather than a law, and the study's own data shows it: in one test the second local pack result, rated 5.0 stars, outperformed the first, rated 4.3.

Finally, ask the agency where the number came from. That is the whole test. The answer separates people who read sources from people who inherited a slide, and it is the same discipline behind filtering candidates properly in the first place.

What to do this week

Take the last proposal or deck you received and circle every number in it. Write the source next to each. Whatever you cannot source becomes one email.

Run the sample-per-cell division on the two biggest numbers. A minute each, and it is the fastest way to see whether a statistic can bear the weight it is carrying.

Then apply the same treatment to your own marketing. Any statistic on your website should survive the check, and most businesses have one or two inherited from a template.

If your current agency reports numbers you cannot trace to a tool you can log into, ask for the source there too. Unverifiable reporting is one of the earliest signs that an account has stopped being actively worked.

And notice which claims never need this treatment. Nobody has to launder a source to tell you that your primary category selection does real work in local ranking. The claims that need borrowed authority usually have the least behind them, which is also why a promise about rankings should make you more careful rather than less.

Questions about a number you were quoted?

Email me at eric@seod.com with any statistic from a pitch, deck, or proposal, and I will trace it as far as it goes and tell you the sample size and who published it. If it holds up, you get that answer too, which happens more often than this article suggests. No pitch and no message afterward.

There is more on hiring and working with agencies alongside this one.