Rule parsers versus LLM extraction: distinguish stable syntax from semantic variation
Compare regex or field rules with LLM extraction, checking format changes, unknown values and source locations.
Choose an approach
Start with rules for stable formats and clear boundaries and consider models for varied phrasing needing interpretation. Combine deterministic parsing with model-assisted hard cases. Schema conformity and successful matches do not prove correct fields. Preserve locations and verify meaning rather than accepting the absence of errors.
Pattern matching and semantic judgment rely on different conditions
Python regex matches patterns; structured model outputs constrain fields and types. A pattern can select an example identifier and a model can treat a rejected date as current. Rules are inspectable but format-sensitive; models handle variation but may guess. Define semantics and unknowns first. Verify upstream reading order so extraction comparisons do not conceal earlier parsing errors.
Add format variation and negation to one source set
Use varied phrasing with verified answers, missing fields and explicit negation. Preserve match locations or supporting sentences and inspect records rather than counting nonempty fields. Allow unmatched cases for review and unknown model values instead of forcing every record full. Check numbers and object relationships after parsing. No model or real dataset was processed and neither approach is ranked universally.
Maintain rules and examples around observed failures
Replay old formats after rule changes and difficult examples after model or prompt changes. Keep a stable evaluation set rather than endlessly adding exceptions or using costly processing for simple cases. Include review and correction, not parsing speed alone. Route uncertain records from rules to models where useful, then validate consistently against originals. Preserve versions and methods to retract errors and explain history.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.