Short answer: Word error rate is the share of words a speech system gets wrong. It adds up substitutions, deletions and insertions, then divides that total by the number of words in a correct reference transcript. A 5 per cent WER means one error every twenty words. The figure only means something when you know which audio produced it.
A vendor quoting "99 per cent accurate" is quoting one number from one test. Accuracy is not a fixed property of a transcription engine. It is a property of the engine and the recording you feed it. The same model can score 4 per cent error on a read audiobook and 25 per cent on a meeting held around a speakerphone. Neither figure is dishonest. Both are close to useless without their conditions.
Behind nearly every published accuracy claim sits one metric: word error rate, usually shortened to WER. It has been the standard yardstick of automatic speech recognition since the DARPA benchmark evaluations of the late 1980s. It is cheap to compute, hard to argue with, and misread constantly. This guide shows how WER is calculated, what it cannot see, and how to measure it on your own recordings.
What word error rate counts
WER answers one narrow question. Start with a reference transcript: what was really said, written down carefully by people. How many word-level edits turn the machine output into that reference? A scorer lines the two texts up and counts three kinds of edit. Nothing else is counted at all.
The three error types
- Substitution: a wrong word sits in the right place. "Fifteen" written for "fifty" is one substitution.
- Deletion: a spoken word is missing from the output. "At half past ten" becomes "at half ten".
- Insertion: the system adds a word nobody said. "By Friday" becomes "by next Friday".
The formula, in plain arithmetic
WER equals substitutions plus deletions plus insertions, divided by the number of words in the reference. Multiply by 100 for a percentage. The "accuracy" figure in marketing copy is almost always just 100 minus WER. A 6 per cent error rate becomes "94 per cent accurate" on the pricing page.
The scorer finds the smallest set of edits that explains the difference. That alignment uses the same dynamic programming as Levenshtein edit distance, applied to words rather than characters. Because the cheapest alignment always wins, two people scoring the same pair of texts reach the same answer. That reproducibility is why the metric has survived four decades.
One consequence surprises people. WER can exceed 100 per cent. Insertions count as errors but replace nothing. A system that transcribes music or silence as speech can produce more edits than the reference has words. Speech researchers call that hallucinated output, and it behaves much like the confident inventions of a chatbot.
Score a sentence yourself
Take a ten-word reference: please send the signed contract to our office by Friday. The system returns: please send a signed contract to your office by Friday morning.
Read the output aloud and nothing sounds wrong. Now score it. "The" became "a" and "our" became "your", which is two substitutions. "Morning" appeared from nowhere, which is one insertion. Three edits against ten reference words is a word error rate of 30 per cent.
That is a poor score from a sentence most readers would wave straight through. It is also worse than the number suggests. One of those tiny edits sends a signed contract to the wrong party's office. Fluent output is not faithful output, and only the audio can tell you which one you are holding.
Deleting the word "not" costs exactly one edit. So does deleting an "um". Only one of them reverses what the speaker said.
What a single WER number hides
Every error costs the same
WER counts edits, not consequences. Dropping "not" from "we did not agree" reverses the meaning of the sentence. Dropping an "um" improves it. Each costs one deletion. A transcript can post a respectable score while mangling the two names and the one figure that made the recording worth keeping.
Normalisation decides what counts as wrong
Both texts are normalised before scoring. Case is stripped, punctuation is removed, and formats are standardised. Whether "OK" matches "okay", or "£20" matches "twenty pounds", or "don't" matches "do not" — each is a policy choice. Different choices produce different scores from identical output. Punctuation is usually excluded entirely, so WER says nothing at all about whether a transcript is readable.
The reference transcript is a human artefact
The truth that systems are scored against is written by people, and people disagree. Skilled transcribers given the same difficult recording differ on mumbles, crosstalk, contractions and dialect. That disagreement sets a practical floor. On genuinely hard audio, no engine can score meaningfully better than the humans who defined the answer.
Tip: When comparing two vendors, ask each for the alignment file rather than the score. Seeing exactly which words were substituted tells you more in five minutes than a headline percentage tells you in a year.
The metrics that sit alongside WER
Word error rate is one instrument on the bench, not the whole bench. Serious evaluation reports several numbers, because each answers a different question about the same transcript.
| Metric | What it counts | Best used for |
|---|---|---|
| Word error rate (WER) | Word-level substitutions, deletions and insertions | General transcription quality in space-separated languages |
| Character error rate (CER) | The same three edits, applied to single characters | Chinese, Japanese and other scripts written without spaces |
| Sentence error rate (SER) | The share of sentences containing any error at all | Voice commands, where one wrong word breaks the instruction |
| Diarisation error rate (DER) | Audio time attributed to the wrong speaker | Interviews, panels and meetings with several voices |
| Real-time factor (RTF) | Processing time divided by the length of the audio | Live captioning, where an RTF above 1.0 cannot keep up |
Entity error rate deserves a place on that list too. It scores only the words a reader cannot guess from context: names, organisations, drug doses, amounts and dates. In most working transcripts that small set of words carries nearly all of the value. A team running transcription across recurring meetings will feel entity errors long before it feels a two-point shift in WER.
How much accuracy do you actually need?
Chasing the lowest possible error rate is rarely the right goal. The useful question is what the transcript is for, and what one error costs there. Three broad tiers cover most work.
- Searchable record. Internal calls, team meetings, voice memos. An error that leaves the keywords intact costs almost nothing here. A rough draft you can search today beats a perfect transcript that arrives next week.
- Quotable source. Journalism, qualitative research, podcast show notes. Every published sentence has to be checked against the audio, so the engine's job is to reduce typing rather than to be trusted. Our step-by-step interview transcription workflow builds that check into the process.
- Legal or clinical record. Court proceedings, medical notes, regulated board meetings. Errors here have consequences outside the document, so a human review pass is a requirement rather than an upgrade.
Notice what actually changes between the tiers. It is not the engine. It is how much human attention arrives after the engine, and which words that attention is pointed at.
Why published accuracy figures disagree
A WER is inseparable from the audio it was measured on. Much public benchmarking uses LibriSpeech, a corpus built from public-domain audiobooks. That material is clean, single-speaker and carefully read. Point the same engine at a four-person meeting on a ceiling microphone and the error rate multiplies.
Conditions are not only acoustic. Accent, specialist vocabulary and regional idiom all move the number. An engine that shines on broadcast news can stumble in a cardiology clinic or a Glasgow call centre. Overlapping speech is harder still, because two voices on one channel hand the acoustic model behind speech-to-text a signal it was never trained to separate.
This is ordinary measurement discipline, not a quirk of speech technology. Every engineering field states its conditions alongside its figures, as we describe in how tolerance and testing control quality. A tolerance quoted without a temperature is meaningless. So is a WER quoted without its test set.
Three questions clear most of the fog around a headline number:
- Which audio? Read speech, telephone calls, meetings and field recordings are four different problems.
- Whose normalisation rules? Filler words, numbers and contractions can move a score by several points on their own.
- Measured when? Models are retrained continually, so last year's benchmark may describe a system that no longer exists.
Standards work solved this long ago by publishing the test alongside the claim. The US National Institute of Standards and Technology ran formal speech evaluations for decades. Its scoring toolkit, SCTK, is still a reference implementation for aligning and counting errors. A claim tied to a named test set starts to behave like a real specification. That is precisely what a published quality standard buys for anyone reading a vendor's number.
How to test accuracy on your own audio
The simplicity of WER makes self-service evaluation genuinely practical. One afternoon of work produces a number you can defend.
- Collect a representative sample. Take 30 to 60 minutes from real work: your speakers, your rooms, your jargon. A vendor's demo clip only tells you about the demo clip.
- Write a careful reference. Transcribe the sample precisely, with a second person checking the first pass. This is the expensive step, and the one that makes every later number mean anything.
- Fix the rules before you score. Decide how you will treat numbers, fillers, contractions, casing and punctuation. Apply identical rules to both texts.
- Score with standard tooling. SCTK or a maintained open-source scoring library will align the texts and count the edits for you.
- Read the alignment, not just the total. Ask which kinds of word failed. Fillers, proper nouns and negations point to completely different fixes.
- Re-run it every quarter. Engines change underneath you. The same sample catches improvement and regression alike.
Keep the sample honest. Five minutes of audio gives a noisy estimate, and a one-point gap between two engines on a short clip is probably chance. Thirty minutes across several speakers is a sensible floor, and an hour is better. If two systems land within a point of each other, treat them as tied on accuracy and decide on other grounds.
Those other grounds usually settle the purchase anyway: punctuation quality, speaker labels, turnaround time, export formats and data handling. Our own transcription tool, TRANSCRIPT.YOU, is built around that whole list rather than a single benchmark score. Whichever engine you choose, run the test above before you commit a workflow to it.
Tip: Score two samples, not one — your cleanest routine recording and your worst. The gap between those two numbers predicts your real experience far better than either number alone.
Key takeaways
- WER equals substitutions plus deletions plus insertions, divided by the length of the reference transcript.
- The marketed accuracy percentage is usually just 100 minus WER, measured on audio you never hear.
- Every error costs one edit, whether it flips a sentence's meaning or removes a filler word.
- Two figures are only comparable when the audio and the normalisation rules are the same.
- The strongest evidence is your own test: representative audio, a careful reference, standard scoring, and time spent reading the actual errors.
When WER stops being enough
Word error rate will not be replaced, because it is cheap, objective and reproducible. What is changing is the company it keeps. Transcripts are now inputs to search, summaries and analysis rather than final documents. That shifts the question from "how many words changed" to "did the meaning survive".
Practical teams now watch a small panel of measures instead of one. Entity accuracy covers names and numbers. Diarisation error rate covers who spoke. Punctuation and casing are scored separately, because they decide readability and they drive how cleanly a transcript converts into SRT or VTT captions. Where errors carry real cost, the fix is usually a review layer rather than a different engine. We weigh that trade-off in human versus AI transcription.
Treat WER the way an engineer treats any measurement. Quote it with its conditions. Respect what it cannot see. Trust your own bench over anyone's brochure. That habit runs through everything in our Voice & Transcription hub, and it makes you a considerably harder customer to impress.
