Fixed versus semantic chunking: keep answer prerequisites together
Compare length-based and content-boundary chunking using cross-paragraph questions, and account for reindexing and duplicate passages.
Choose an approach
Start with fixed-length chunks when simple reproducible rules suffice. Evaluate structural or semantic boundaries when answers cross sentence boundaries. Semantic chunking has different implementations and does not guarantee better retrieval. Identify complete evidence for each question and check whether prerequisites and conclusions can be retrieved together.
Length limits and meaning boundaries may differ
Fixed chunking exposes length and overlap settings, and implementations may avoid splitting sentences. Semantic chunking groups related content but still faces input limits. Azure AI Search describes several methods and LlamaIndex offers multiple parsers; semantic chunking is not one universal switch. If units remain in an earlier chunk, retrieving a row of numbers may omit essential meaning. Overlap can preserve adjacent context but duplicates text, so compare it with retaining section context.
Check retrieved evidence with cross-paragraph questions
Use one document version with both local and cross-paragraph questions. Fix the embedding model and retrieval budget, vary only chunking and check whether all required evidence appears. Larger chunks also change total input; do not attribute the effect of more text solely to better boundaries. Where headings detach from body text, test attaching section metadata as a separate change. Judge evidence coverage before answer accuracy; fluent answers do not validate retrieval.
Recheck indexes when chunking rules change
New boundaries can change chunk identifiers and citation positions. Preserve source versions and offsets and ensure withdrawn documents can be removed. If semantic processing calls an additional model, count ingestion costs as well as query costs. Replay retained questions and check whether duplicates crowd out evidence. Keep fixed chunking as a baseline when it covers important tasks; changes should address identifiable gaps. This article checks documentation without building indexes or measuring accuracy.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.