Evaluation includes prices, versions or dates: how do you avoid grading with stale answers?
For time-sensitive facts such as pricing rules, versions and dates, specify an as-of time and scope and verify reference answers before grading. Disagreement between an old reference and a new response is not automatically model error.
Start troubleshooting here
For time-sensitive facts such as pricing rules, versions and dates, specify an as-of time and scope and verify reference answers before grading. Disagreement between an old reference and a new response is not automatically model error. Preserve evidence and checking times and separate historical from current questions, without treating a newer-looking response as automatically correct.
Give time-sensitive questions an as-of date and scope
This applies after pricing, software or event-date changes produce sudden score declines. Mark dynamic fields and specify region, plan and effective time. Current pages may no longer preserve history, so historical questions need period-appropriate records. Today’s page alone cannot establish that a past answer was wrong.
Check official evidence and valid versions of reference answers
Open original sources and inspect fact dates, page updates, units and scope. Determine whether the question concerns now or a historical point and whether grading demands exact values or an acceptable range. Crawl timestamps differ from event dates and do not establish that every fact is current.
Keep historical baselines separate from current-fact results
Keep traceable evidence and check records for dynamic fields and preserve old baselines when updating current tests. Mark stale or unverified items rather than silently replace references with the model’s latest output. Specify tools and temporal conditions for retrieval tasks and compare closed-book and online systems separately without undisclosed information advantages.
Recheck dynamic fields and historical-time judgments
Recheck changed and stable questions and record reference-staleness misgrades while preserving the question’s intended meaning. Report evaluation dates and validity scope and leave unverified answers ungraded where necessary. Success means timely traceable evidence, not accepting every fresh response. Updating a few items does not establish full temporal review.
This article ran no temporal evaluation and supplies no current dynamic-field values. Verify facts at task time; evaluation research does not replace specific sources. Historical pages may be unavailable and require disclosed evidence limits. Truth, accessible citations and source freshness are separate checks; a successful page response does not establish valid content.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.