CER versus WER: align segmentation and normalization first
Compare character and word error rates, including Chinese segmentation, number formatting and insertions in transcription evaluation.
Choose an approach
CER compares characters and WER compares words; choose units appropriate to language and task. Chinese requires explicit segmentation or character-level evaluation, while English transcription commonly uses WER. Fix normalization and retain raw text. Neither metric establishes semantic correctness; check numbers and negations separately.
Error units change substitutions, insertions and deletions
Hugging Face CER and WER use edit differences with different units. Insertions can produce rates above 100%, so clipping scores does not improve fairness. Punctuation, case and number normalization change results and require predefined rules. Decide whether “twenty” and “20” differ for the task while retaining raw text. A low average can hide a critical amount error, so scores alone do not establish usability.
Inspect text before scoring the same transcripts
Use checked references and raw outputs with identical preprocessing. Add an insertion case to verify implementation behavior; document tokenization when reporting Chinese WER. Do not compare punctuation-stripped output against unnormalized output. List number and name errors to explain scores. No transcription service or dataset score was run here and no model accuracy is reported.
Report rules and task limitations with metrics
Keep scoring versions and preprocessing so changes reflect outputs rather than altered rules. Report CER and WER with units; do not average them into one accuracy. Include reference preparation and error review, especially names. Add field checks for extraction tasks and full comparisons for verbatim transcripts. State language, scope and gaps so metrics serve the actual use.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.