Chunking Strategies
Learn how the platform creates searchable chunks from tables, pages, document structure, and figures, including its fallback behavior.
A chunk is the unit that participates in embedding, BM25 indexing, and RAG retrieval. UnieAI Studio does not apply one fixed character window to every document. It selects an available path from the parsed structure, file type, and platform model configuration.
Users can configure rows per chunk for CSV and Excel files and choose whether Figures receive standalone chunks under the knowledge base's Settings → Parsing. Other chunking paths are selected automatically from the document and system-model settings.
Table chunks: CSV and Excel
CSV and Excel data preferentially use tabular_rows, grouping consecutive rows within each sheet. Sheet context and column headers are repeated in every chunk so rows do not lose their meaning after separation.
- The default is 50 rows per chunk; deployments can override this default.
- Before uploading a CSV or Excel file, the dialog accepts an integer greater than or equal to 1.
- Rows per chunk can also be changed for an existing table document. If rechunking is selected, the document returns to processing and is indexed again.
- A smaller value produces finer but more numerous chunks; a larger value includes more table context in each retrieved result.
Rechunking rebuilds that document's chunks and index. Wait until processing completes before evaluating retrieval again.
Per-page LLM chunks: line_based
When page content is available and an administrator has configured the KB Chunking Task Model, the system attempts to plan semantically coherent chunks as line ranges.
- Each page is processed independently, so a text chunk does not cross page boundaries.
- Headings, paragraphs, lists, and table blocks are kept together when possible.
- The embedding input can include page, section, and document-title location details when available.
- If the model is missing, times out, returns invalid ranges, or otherwise fails, processing continues with semantic or deterministic chunking.
line_based is therefore a preferred attempt when its prerequisites are available, not a mandatory step for every document.
Document detection and semantic chunks
If a table path or successful per-page LLM result is not used, the system scans the complete document for structural patterns:
chunkingStrategy | Detection clues | Main behavior |
|---|---|---|
legal_article | 第 X 條, Article N, §N | Split by article |
sop_procedure | SOP-001, Procedure N, step or procedure markers | Keep a complete procedure together; use major headings if procedure titles are absent |
faq_pair | Q:/A:, 問：/答： | Keep each question and answer together |
contract_clause | 甲方/乙方, 第 X 款, Clause N | Split by clause or section |
academic_section | Abstract, Methods, Results, Conclusion, 摘要, 結論, and similar headings | Split by academic section |
tabular | Most non-empty lines look like a Markdown table | Split by section while preserving table blocks; CSV and Excel normally use tabular_rows earlier |
general_semantic | No other type reaches the detection threshold | Split by headings, paragraphs, and natural sentence boundaries |
Detection runs over the complete document. For page-based content, the selected semantic strategy is still applied page by page so pageIndex and page-image metadata remain correct.
The deployment default semantic soft maximum is about 1,800 characters. Longer sections are split at paragraph and then natural sentence boundaries. This is a soft target: when no safe boundary exists, preserving content takes priority.
Deterministic fallback
Simpler deterministic chunking is used when:
- Semantic processing produces no chunks.
- An integrity check indicates that the semantic result may have lost source text.
- The task model is unavailable or does not produce valid output.
The deployment defaults target about 900 characters and use 0 chunk overlap. Administrators can override these values, so a particular deployment may differ.
Standalone Figure chunks
PDF, image, and other visual parsing results can contain charts, photos, and diagrams:
- Figures are first written into page text chunks with usable asset links so they can be retrieved with surrounding content.
- Create an independent Chunk for each Figure is off by default. When enabled, every Figure with a cropped image also receives a standalone chunk with
chunkingStrategy: "figure". - Figure chunks retain descriptions, visible text, nearby paragraphs, page location, and the corresponding cropped image.
- The switch affects new uploads and re-index only. Re-index an existing document to apply a changed setting.
Leave it off for documents primarily searched by page text. Enable it when questions about a particular image or chart should retrieve the crop directly instead of only returning the complete page preview.
Metadata and downstream processing
Depending on strategy and source, a chunk can contain:
chunkId,documentId,pageIndex, andchunkIndexsectionTitle,pageImageKey, andfigureImageKeymetadata.chunkingStrategy- Page, document-title, and parser metadata
Fallback chunks do not necessarily carry chunkingStrategy, so that field should not be treated as a complete source classifier. After chunking, the platform can attempt Contextual Retrieval, then creates embeddings and writes the result to Milvus. A generated contextual prefix can replace the earlier locator stored in chunkContext.
Inspecting and organizing chunks
After indexing completes, open the document's Chunks page to:
- Inspect chunk content, page location, strategy, and image data.
- Edit text chunks and save their versions.
- View version history or restore an individual chunk version.
- Remove chunks that should not participate in retrieval.
After making changes, use the knowledge base Search page with representative questions to evaluate retrieval again.
Related documentation
Uploading Documents
Contextual Retrieval
Figure Extraction
RAG Architecture
RAG Architecture
Follow the complete path from document parsing and indexing to hybrid retrieval and Customize Model responses.
Contextual Retrieval
Learn how the platform generates document-locating context for text chunks before indexing, including model requirements, fallbacks, cost, and data limits.