Company knowledge
Document Chunking Decisions That Change Search Quality
If answers lose exceptions after documents are split, inspect the boundaries. A useful chunk keeps the conditions and evidence needed for the question together; it is not simply the shortest possible chunk.
In this article
Key takeaways
An embedding represents an item, such as text or an image, as a vector: a collection of numbers a model can process.
Compare paragraph-only chunks with chunks preserving headings, clauses, and exceptions using the same questions.
No fixed chunk length is universally optimal.
Retrieval may find the right sentence while leaving its exception in the next chunk. That is a risk of splitting by length alone. First ask which passages a reader needs to see together.
For the underlying concept, see What is an embedding? in the AI glossary.
Do not judge quality by chunk size alone
Azure AI Search documents several approaches, including fixed-size and structure-based splitting. Respecting model input limits and creating meaningful retrieval units are related but separate concerns. Do not assume one optimal token count applies to every document.
Keep rules and exceptions interpretable
Imagine eligibility rules on one page and exclusions on the next. Retrieving only the first can yield a faithful quotation but an incorrect application. Test preserving headings, scope, and notes or retrieving related passages together, while checking the cost of duplicated content.
Read extracted tables and procedures
Table extraction can detach a price from its product or unit. Check that row and column relationships survive in retrieved chunks. In procedures, retain prerequisites as well as numbered steps. Do not assume a different embedding model repairs information already damaged during extraction.
Select boundaries through a controlled comparison
Compare two chunking strategies on matching sources and questions. Record relevant-clause coverage, missed exceptions, provenance, and input length. Chunk-level contextual enrichment is another candidate, but a provider’s published experiment is not your deployment’s expected result.
A worked business example
In a fictional support policy, general refund terms occupy one paragraph and enterprise exceptions the next. A character-count split can separate the two conditions.
Compare paragraph-only chunks with chunks preserving headings, clauses, and exceptions using the same questions. Check whether the required exception remains retrievable, and keep source locations and versions.
Failure conditions to check
No fixed chunk length is universally optimal. Inspect conditions lost in small chunks alongside unrelated material introduced by larger ones.
A checklist for your workflow
Item | Review question |
|---|---|
Meaning | Can the rule, conditions, and exceptions be interpreted? |
Structure | Were tables, lists, and headings preserved? |
Validation | Were failures compared on matching questions? |
Frequently asked questions
Will more overlap solve it?
It can reduce boundary loss but adds duplication and cost. Test it on real questions.
Related AI concepts
Sources and editorial information
Anthropic: Contextual Retrieval
TENCONIC editorial team · Structure reviewed October 28, 2026. Official sources explain concepts and implementation examples; features vary by product. Scenarios and checks are editorial recommendations, not customer results or product promises.
For the approach to connecting company knowledge with AI, explore the TENCONIC Data Engine introduction.