hotai.news
← Back to AI WikiAI Wiki / Solve a problem

The structured result parses but its values are wrong: how do you check structure and facts separately?

For parseable objects with wrong values, separate syntax, Schema and factual support. A valid number does not establish the correct amount, and a string type does not establish a source-supported name.

HotAI.news · Sources checked with CodexPublished on 2026-10-20Sources checked on 2026-10-20
Our take

Start troubleshooting here

For parseable objects with wrong values, separate syntax, Schema and factual support. A valid number does not establish the correct amount, and a string type does not establish a source-supported name. Define provenance and unknown-value rules before checking extraction. Do not invent missing information to fill every field or replace accuracy review with structured-output success.

01

Evaluate structural validity separately from factual accuracy

This applies to correctly formatted amounts assigned to the wrong object, dates taken from another row and invented phone numbers. Preserve source and original object and identify fields requiring direct evidence. Define permitted calculation or normalization in advance rather than retrospectively authorize inference to rationalize the output.

02

Record each field’s source and unknown-value policy

Check JSON syntax and types first, then trace source positions, units and objects. Separate absent fields, explicit unknowns and wrong values rather than converting every empty string to zero. JSON Schema can allow extra properties by default; explicitly configure supported restrictions to reject them. Even an object without extra fields can contain false facts.

03

Validate types before tracing values to evidence

Keep evidence positions or traceable records for important values and validate numbers and identifiers. Use agreed unknown representations or explicit failure for absent material rather than model invention to satisfy required. Check business relationships programmatically or manually, including totals against line items. Verify valid source versions for external values.

04

Test false facts inside structurally valid objects

Test valid structure with false facts, true facts with invalid structure and missing source fields, recording outcomes separately. Neither class of failure may silently succeed. Preserve original outputs after repair. Acceptance needs both structural and factual requirements; field completion and parse success alone do not establish better extraction.

This article ran no model extraction and reports no accuracy. Faithful extraction differs from verifying reality when source material itself is wrong; define the boundary. Supported Schema subsets vary and complex rules still need application checks. Use risk-appropriate human confirmation for precise business data rather than only model self-assessment.

Checked sources

These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.

Back to AI Wiki ↗