Exact match versus rubric grading: distinguish fixed answers from open expression
Compare exact text matches with explicit content rubrics, checking normalization, omissions and grader disagreement.
Choose an approach
Use exact match for fixed labels or forms and rubrics for multiple valid expressions. Combine field checks with explanation grading. Exact match may reject equivalent wording and rubrics may favor fluent errors. Define allowed variation and required facts instead of treating automated scores as inherently objective.
String equality differs from satisfying content requirements
Hugging Face exact match compares predictions with references, with normalization affecting decisions; Google evaluation supports rubric-based assessment. Equality fits fixed enums but not summaries with valid alternative wording. Conversely, semantic grading of amounts can miss numerical errors. Separate exact fields from flexible prose and define acceptable unknowns for incomplete sources.
Include equivalent wording and critical errors in one answer set
Use a correct reference, equivalent paraphrase and a version with one wrong number. Fix whitespace and case handling for exact match and required facts and deductions for rubrics, then check human agreement. Do not loosen rules afterward to favor a model. Reserve unseen samples and verify grader explanations against actual content. No grading run or model comparison occurred; these examples validate the scoring method.
Update references and rubrics together when standards change
Review references and rubrics when labels or requirements change and version them to explain historical scores. Record judge versions and review disagreements when models grade. Include standard-writing and boundary resolution in costs. Match deterministic fields programmatically and use calibrated rubrics for prose, reporting results separately. State output coverage rather than extending a format pass to complete factual acceptance.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.