AI Voice Agents Are Replacing Call Centre Teams. Here's How.

We've spent eighteen months replacing parts of call centres with AI voice agents. The marketing position is 'AI is replacing human agents'. The operational reality is more nuanced: AI voice agents have decisively won in three call-centre categories and are still losing in three others, and the line between them is sharper than the industry coverage suggests. Knowing where the line is determines whether your voice-AI programme delivers a 60% cost-per-contact reduction or becomes the customer-experience disaster that ends up on social media with a screenshot of the AI telling a grieving widow that her late husband's account couldn't be closed because she didn't have the right password. The technology is ready; the deployment discipline is what separates the wins from the public failures.

Where voice agents have actually replaced human teams

Three categories have flipped. Outbound collections — particularly first-bucket reminders for credit cards and personal loans, where the conversation is structured and the goal is single (payment commitment) — now run at 70-85% AI handling rates in most of our deployments. Appointment booking, rescheduling, and confirmation across healthcare, retail services, and field service has gone almost fully automated for the booking transaction itself, with humans handling only the cases the agent flags. Inbound Tier-1 support — password resets, balance enquiries, transaction-status lookups, basic account servicing — runs at 60-75% containment rates with no human handoff. These three categories together can account for 50-65% of a typical retail-banking or telco call centre's volume. That's where the cost-per-contact reduction comes from.

The voice agent technology stack

Production voice agents are stitched from four components and a real-time orchestration layer that ties them together with sub-second budgets. Automatic Speech Recognition is the most settled part — Whisper Large V3 self-hosted for sovereign deployments, AssemblyAI or Deepgram for cloud-permitted, with custom acoustic-model adaptation for accent handling (this matters more than vendors admit; a generic ASR model achieves 88% word accuracy on Gulf-accented English, an adapted model 96%, and the eight-point difference materially changes how many calls succeed versus need human takeover). Text-to-Speech has moved from robotic to indistinguishable — ElevenLabs and Coqui are our defaults, with custom voice cloning for brand-consistent agent personas where the customer wants their voice agent to sound like their brand rather than like a generic voice-over artist. The LLM 'brain' is typically an 8-13B model fine-tuned on redacted call-centre transcripts plus the customer's product knowledge base, running on vLLM with prompt caching aggressive enough that repeated context (the customer's product catalogue, the agent persona, the operational policies) doesn't get re-processed every turn. Telephony integration is the underrated hard part — Twilio for cloud-permissive, Asterisk or FreeSWITCH self-hosted, with SIP trunking into the customer's existing carrier infrastructure or Genesys/Avaya/NICE contact-centre platform. The orchestration layer manages turn-taking, barge-in handling, endpoint detection, and the latency budget — under 800ms end-to-end for the conversation to feel natural, which is where most engineering effort goes in the first three months of any deployment.

The latency budget is where most implementations fail

A human conversation expects a sub-second response. If your voice agent's end-to-end loop — ASR finalisation, LLM call, TTS generation, audio playback — exceeds 1.2 seconds, the conversation feels broken. The user starts talking over the agent, the agent gets confused, the call quality collapses. Most teams shipping voice agents discover this in production. The fixes are non-obvious: streaming ASR with partial results, LLM responses that start streaming before the user finishes speaking (using endpoint detection), TTS that begins playback on the first generated sentence rather than waiting for completion, and aggressive prompt caching to avoid re-processing context every turn. We target a P95 end-to-end latency of 750ms; we miss it more often than we'd like and improve every quarter.

The economics

A fully-loaded human contact-centre agent in a tier-1 outsourced market costs $5-8 per handled call when you include training, attrition, supervision, infrastructure, idle time, and shrinkage. A voice-agent handled call, on our deployed customers' infrastructure, costs $0.30-1.20 depending on average handle time, inference hardware utilisation, and the per-call burden of the eval and observability infrastructure that you can't skip. The headline 60% cost reduction is conservative on the right workloads — we've seen 70-80% on outbound collections where the call durations are short and the topic surface is narrow. The reduction is much smaller (20-30%) on complex inbound where the agent has to hand off to a human on the long tail; the cost there is mostly in the human, and the AI's role is to compress average handle time rather than eliminate the human entirely. The capital cost of the infrastructure matters too — a deployment supporting 50 concurrent voice channels needs roughly 4 GPUs (one for the LLM, one for TTS, two for the ASR depending on language coverage), plus CPU for orchestration; the all-in capex amortises to well under $1 per call at typical utilisation, but only if you actually achieve the utilisation. Sized incorrectly, idle GPU time will eat the economics.

What still goes to humans, and should

Three categories should not be automated end-to-end with current technology. Complaint handling — particularly any complaint with emotional content — needs a human, both for empathy and because the regulatory exposure of an AI mishandling a vulnerable customer is unacceptable. Retention and save-the-account conversations require negotiation latitude that we don't trust LLMs with in 2026; the cost of a wrongly-offered discount or a poorly-handled cancellation is greater than the labour saving. Complex troubleshooting where the agent needs to direct the customer through a multi-step diagnostic flow with unpredictable branching is still better in human hands; the agent's role here is to triage the call to the right specialist team, not to attempt the resolution itself.

Why on-premise voice agents matter for regulated industries

Voice contains more identifiable signal than any other customer-communication channel — voiceprint biometrics that can re-identify a customer across calls, accent and location indicators, emotional state, background-audio cues that hint at personal circumstances. Regulatory regimes in BFSI and healthcare treat voice recordings and transcripts as sensitive personal data subject to localisation and access-control requirements that most cloud voice-AI vendors cannot meet, particularly under the SAMA, RBI, and central-bank-of-UAE frameworks that all require voice data to remain within national borders for the regulated entity's customers. We deploy the entire voice stack on-prem for our regulated customers: GPU servers for the LLM and TTS, CPU servers for ASR and orchestration, all of it integrated with the customer's existing PBX or contact-centre platform (Genesys, Avaya, NICE inContact, Cisco Unified) via SIP, with the call recording and transcription pipeline routing into the customer's existing compliance recording infrastructure rather than a vendor cloud. The deployment overhead is real — a typical on-prem voice deployment takes 6-8 weeks from kickoff to first production call — but for the regulated customer it's the only viable path, and the deployment workshop is also where the customer's contact-centre operations team learns the new operating model, which makes the overhead a feature rather than a bug.

The Gulf telecom case study

A tier-1 Gulf telecom deployed our voice agent stack across three call-centre categories: outbound collection reminders, prepaid balance and recharge enquiries, and SIM-replacement appointment booking. The deployment ran on-prem in their existing data centre, integrated with their Genesys platform via SIP, with the LLM fine-tuned on twelve months of redacted call transcripts plus their product knowledge base. Six months after go-live: 58% reduction in cost-per-contact across the three categories, 81% containment rate on prepaid enquiries with no human handoff, average handle time on outbound collections compressed from 4.2 minutes to 2.1 minutes, and customer-satisfaction scores on AI-handled calls at 4.3/5 versus 4.1/5 on the human baseline. The bigger win was capacity reallocation — the human-agent team that was previously handling these categories is now focused on retention and complex troubleshooting, where their NPS contribution is meaningfully higher.

Voice as front end, and practical recommendations for 2026

The most under-appreciated implication of voice agents at scale: voice becomes the primary front end to every other AI capability the enterprise has built. The customer-onboarding flow that previously required an app or branch visit becomes a voice conversation. The internal IT support that previously required a portal becomes a phone call to an internal voice agent. The field-service technician who previously had to navigate three apps to log an incident now talks to a voice agent that fills the forms behind the scenes. This is the multiplier that the cost-per-contact reduction story under-sells — the enterprise that has invested in voice infrastructure can wire every other automation behind a phone number, which is a UX that every customer and employee already knows how to use. Concrete recommendations: if you're considering a voice-agent deployment, do not start with the highest-volume call-centre category — start with the most contained. Outbound collections, appointment confirmation, or a single inbound enquiry type. Build the operating model — escalation paths, supervisor review, quality scoring of AI calls, ongoing prompt-tuning — before you scale. Insist on on-prem or sovereign deployment if you operate in BFSI or healthcare; the regulatory cost of getting voice data residency wrong is greater than the deployment-overhead cost of doing it right. And benchmark against your human baseline on the same set of calls, not against the vendor's case studies. The cases where voice AI wins are real; the cases where it loses are also real, and which is which depends entirely on your specific workflow. Pilot for eight weeks, measure ruthlessly, expand only on the categories where the numbers hold up. The category-by-category expansion model has consistently outperformed the big-bang call-centre-replacement model in every customer we've supported.