What Is Automatic Speech Recognition (ASR)?
Automatic speech recognition (ASR) turns spoken audio into text. Learn how ASR works on phone calls, why phone audio is hard, and how accuracy is scored.
Quick answer
Automatic speech recognition (ASR) is technology that turns spoken audio into text. A recognizer takes an audio stream, applies a model of speech and language, and returns a transcription. On phone calls it is harder than on clean recordings because of narrowband audio, noise and varied accents.
ASR is the listening half of a voice assistant and the engine behind call transcripts. Everything an AI agent does with a caller's words starts with ASR getting those words right.
What is automatic speech recognition?
ASR converts speech to text. "Speech-to-text" is the everyday name, and the two terms are used for the same thing. The W3C Speech Recognition Grammar Specification describes a speech recognizer as a user agent that takes in one or more grammars, which describe the words and patterns to listen for, plus an audio stream, and produces descriptions of the speech it detected. It notes that most practical recognizers include at least a transcription of the words detected.
How does ASR work on a phone call?
- Audio is captured. The call's audio arrives as a stream, often one channel per speaker.
- The audio is prepared. It is cut into short frames and turned into features the model can read. Silence can be set aside with voice activity detection.
- A model predicts words. It combines how the sound looks with what words are likely to follow each other in the language.
- Text is returned. Often as a running transcript while the speaker is still talking, then finalised at the end of an utterance.
- Software acts on the text. It may be stored as a transcript, searched, summarised, or passed to an AI agent that decides what to say next.
Why is phone audio harder for ASR?
- Narrowband audio. Telephone audio carries less detail than studio sound, so similar sounds are easier to confuse.
- Noise. Traffic, wind, and background voices on a mobile call.
- Accents and dialects. A model trained on one variety may stumble on another. See the post on Arabic speech recognition across dialects.
- Code-switching. Speakers who mix two languages in one sentence.
- Numbers and names. Short, unpredictable words with no context to help.
How is ASR accuracy measured?
The usual score is word error rate (WER): the share of words that were substituted, deleted or inserted compared with a correct reference transcript. NIST publishes the Speech Recognition Scoring Toolkit (SCTK), which includes the SCLITE scoring tool, for evaluating recognizer output. The WER entry and the guide to measuring call transcription accuracy show how to score your own calls. A WER from someone else's test set says little about your callers, so measure on your own audio.
How does ASR differ from text-to-speech and voice activity detection?
| ASR | Text-to-speech | Voice activity detection | |
|---|---|---|---|
| Input | Speech audio | Text | Audio |
| Output | Text | Synthetic speech | A decision: speech or not speech |
| Role in a voice agent | Hears the caller | Speaks to the caller | Tells when the caller starts and stops |
| Learn more | This page | Text-to-speech | Voice activity detection |
Where does DialerBee use ASR?
Two places on the site's feature pages. Call transcription converts calls into searchable text, with the interface, answering machine detection and transcription covering 11 languages. The AI voice agent holds the call itself, and leaves behind a recording, a transcript and an outcome. The voice agent has run on live calls in Arabic and English. For other languages, test it on a demo first.
How can you evaluate ASR for your own calls?
- Collect a sample of real calls that covers your languages, accents and noise levels.
- Have people write correct transcripts for them.
- Run the recognizer and score its output against those transcripts as word error rate.
- Look at where it fails: names, numbers, one dialect, noisy lines.
- Repeat on fresh calls from time to time, as your callers and lines change.
Frequently Asked Questions
What does ASR stand for?
ASR stands for automatic speech recognition, technology that converts spoken audio into text.
Is ASR the same as speech-to-text?
In everyday use, yes. Both describe turning speech into text. ASR is the technical term.
How accurate is speech recognition on phone calls?
It varies with the language, accent, audio quality and noise. Phone audio is usually harder than clean recordings, so measure word error rate on your own calls rather than relying on a general figure.
Why does ASR struggle with names and numbers?
They are short, and there is little surrounding context to help the model guess. They also vary widely between callers.
Does ASR work in a call centre?
Yes. It is used to transcribe calls for search and review, and to let an AI voice agent understand what a caller says.
Sources
- W3C Speech Recognition Grammar Specification Version 1.0, Recommendation, 16 March 2004 (source, checked 2026-11-19)
- NIST Information Technology Laboratory, evaluation tools (Speech Recognition Scoring Toolkit, SCTK) (source, checked 2026-11-19)
- NIST SCTK source repository (source, checked 2026-11-19)
Related reading: Call transcription · Word error rate · Text-to-speech · Voice activity detection · Measuring transcription accuracy · Arabic speech recognition and dialects · AI voice agent
Related reading: What Is a Large Language Model (LLM)? · What Is Prompt Injection?
Related terms
What Is Average Handle Time (AHT)?
Average handle time (AHT) is the average total time an agent spends on an interaction — talk, hold, and after-call work.
What Is Average Speed of Answer (ASA)?
ASA is the average time a contact waits in queue before an agent answers. Learn the formula, how it differs from service level, and why the average can mislead.
What Is Barge-In in Voice AI and IVR?
Barge-in lets a caller cut into an IVR prompt or AI voice agent and be heard. See how it works, when to turn it off, and how it differs from supervisor barge.
What Is Barge, Whisper and Listen?
Listen, whisper and barge are the three levels of live call monitoring a supervisor can use. What each one does, when to use it, and what agents should see.
What Is BYOC (Bring Your Own Carrier)?
BYOC (bring your own carrier) lets a business connect its own SIP trunks and telecom carrier to a communications platform instead of using bundled telephony.
What Is Call Blending?
Call blending lets the same agents take inbound calls and make outbound calls from one desktop. See how it works, when it helps, and what to watch.
Ready to see DialerBee in action?
Request a 15-minute live demo, or start a 14-day free trial — no slides, no commitment. Calling starts once your own SIP trunk or PBX is connected and approved.
14-day free trial · free AI credit · no card · 11 languages · BYOC · compliance-supporting controls