Why most AI voice agents fail in the first week

Coval, a company that sells evaluation tooling for voice agents and runs roughly 3.1 million evaluation metrics a week, reports that about 95% of voice agents work in a demo and roughly 62% survive their first week live. That is a vendor figure and should be read as one. It also matches what I watch happen.

The number that should worry you is not the failure rate. It is that the failure is almost always discovered by a customer rather than by the business.

What actually breaks in week one?

Five things, in rough order of frequency.

Interruption handling. People talk over a phone agent constantly. If the system cannot stop mid sentence and listen, every call turns into two people speaking at once and the caller hangs up.

Names and addresses. "Kaneohe." "Villalobos." "Twenty two forty one." Confirmation of spelling and digits is the single most skipped step in a rushed buildout, and it produces callbacks to numbers that do not exist.

Questions outside the tree. The script covers booking, hours, and pricing. The caller asks whether you take their insurance, whether you can come today, or whether the dish contains sesame. What the agent does next is the entire product.

Escalation that goes nowhere. The agent offers a transfer, the transfer rings a phone nobody is holding, and the caller lands in the voicemail you were trying to escape.

Volume surprises. The plan covered 300 minutes. A spam wave and one busy Monday burn through it by day nine, and calls start failing silently. How call volume and overage pricing actually work is worth understanding before launch rather than after the first invoice.

Why does interruption handling break, specifically?

Because the system has to decide when you have stopped talking, and that decision is tuned by a setting almost nobody touches.

Speech recognition services expose an endpointing parameter, which is how long a silence has to run before the system treats your turn as finished. Deepgram's endpointing parameter defaults to 10 milliseconds. Production deployments typically run 300 to 500 milliseconds. A deployment left at the default will cut callers off mid sentence every time they pause to think, which is what people do on the phone. If your vendor cannot tell you what value they set, they did not set one.

The other half of the problem is total delay. Human conversation has a very tight rhythm: Stivers and colleagues, publishing in the Proceedings of the National Academy of Sciences in 2009, measured a modal gap between speakers of roughly 200 milliseconds across ten languages. That is the standard the caller's ear is holding your agent to.

Nothing over a phone line hits it. Retell publishes end to end latency targets of 800 milliseconds at the median, 1,200 at the ninetieth percentile, 1,500 at the ninety fifth, and 2,500 at the ninety ninth. Cekura's cross vendor benchmarking puts real world median turn latency at 1.5 to 2.5 seconds, which is roughly eight to 12x the human baseline.

Read the tail, not the median. A median of 800 milliseconds sounds fine. The ninety ninth percentile of 2,500 milliseconds is where the hangups live, and in a hundred call week that is one call a week going badly for a reason no summary report will show you. Ask your vendor for p95 and p99, not average.

One more caution on latency marketing. Vapi states plainly in its own documentation that its displayed latency metric excludes endpointing and transport time, and that endpointing can account for a meaningful share of what the caller experiences. A published half second is not what your customer hears.

Why does the demo work when the real thing does not?

Because the demo is a sales call and you are a cooperative caller.

You speak clearly. You wait your turn. You ask the questions the salesperson steered you toward. You are in a quiet room. You want it to work.

Your customer is none of those things. They are in a parking lot with a kid in the back seat, they have called two competitors already, and they are prepared to spend about fifteen seconds finding out whether you are worth the trouble.

A demo tests whether the system can talk. Week one tests whether it can be interrupted, corrected, and confused without falling apart. Those are different products, and only one of them is being sold to you.

The largest deployments in the industry make the point better than any benchmark. McDonald's ran IBM voice ordering in more than 100 US drive thrus and confirmed on 17 June 2024 that it was ending the partnership, with the technology off by 26 July 2024. The failures that circulated were long tail order modifications: bacon added to ice cream, hundreds of nuggets on one order, items piling on after the customer said stop. Taco Bell reached roughly 500 locations and about two million AI orders before Yum publicly slowed the rollout in 2025, most memorably after a customer ordered 18,000 cups of water.

Neither of those was a model capability problem. One failed on the long tail and one failed on the adversarial tail, and both are design problems. Your agent will be probed by somebody within days of launch, which means it needs quantity ceilings, absurdity detection, rate limits, escalation on repeat failure, and a policy for recorded abuse.

Wendy's is the counterexample worth studying. It reports 86% of transactions completed without employee intervention across more than 500 locations, which is a company reported and unaudited figure. The reason to take it seriously is the 14% it concedes. Wendy's is not claiming full automation, and the deployments that survive are the ones that route a deliberate share of calls to a person.

What happens when the script hits a question it cannot answer?

This is where failures stop being annoying and start being dangerous.

An agent that improvises an answer is worse than one that says it does not know. In several industries an improvised answer is a compliance event, not a customer service problem.

A restaurant agent asked whether a dish is safe for a peanut allergy must never answer that question. Not carefully, not with a disclaimer. The FDA Food Code puts allergen knowledge with the designated person in charge, and its notification duty is satisfied by written notice, which a voice agent is not. The hard line on allergen questions is the shortest and most important script rule a restaurant will write.

The same logic runs through every regulated category. There is a specific list of things an AI phone agent should never say, and it is not a matter of taste. Write it before you write the greeting, and have whoever carries the license sign it.

The correct behavior for an unanswerable question is short and unglamorous. Acknowledge, do not guess, hand to a human, and log it. Every logged unknown becomes next week's script improvement.

How do you know it is failing before a customer tells you?

What to do this week

If you are already live, pull every transcript from the last seven days and read them in one sitting. Then run the five step count above.

If you are not live yet, write the unknowns list first. Every question your front desk gets that the agent must not answer, in plain language, with the exact handoff sentence you want used.

Ask your vendor two technical questions in writing: what is the endpointing value, and what are the p95 and p99 end to end latencies on your account.

Then have four people who do not work for you call the number. Ask them to be difficult on purpose: interrupt, change their mind, give a hard to spell name, ask something off script, and try to order 18,000 of something.

Fix what breaks. Then launch, on a small share of traffic, with a human watching the log daily for the first week.

Questions about a script before it goes live?

Email me at eric@seod.com with the script or prompt your vendor gave you, or the greeting and first three responses if that is all you have. I will mark the specific lines that will break in week one and the questions the agent is going to answer that it should not.

I do this myself and it takes me about twenty minutes. No sequence, no pitch, and if the script is solid I will tell you it is solid.

More on the phone side sits in the AI phone and lead response library.