Contextual Retrieval

After a document is split into chunks, an individual passage can lose its document name, section, time period, or subject. Contextual Retrieval uses an LLM before indexing to generate a short locator for each text chunk, then prepends it to the original content so both Dense Search and BM25 can use those additional clues.

Processing flow

For each non-figure chunk, the platform sends one OpenAI-compatible POST /chat/completions request containing:

• The document title, when available
• Up to the first 120,000 characters of document text
• The original chunk being situated

The model is instructed to answer in the chunk's language with a short locator of 2–3 sentences and roughly 50–100 tokens. The request uses temperature: 0 and does not stream. Any <think>...</think> reasoning emitted by the model is removed before storage.

Example

Original chunk:

Revenue grew by 3% over the previous quarter.

Possible generated context:

This passage is from ACME Corp's Q2 2025 financial report and discusses quarterly operating results and revenue changes.

Indexed content:

This passage is from ACME Corp's Q2 2025 financial report and discusses quarterly operating results and revenue changes.

Revenue grew by 3% over the previous quarter.

Generated context is a model-produced locator, not a quotation from the source document, so its accuracy should still be sampled and reviewed.

Data written to the chunk and index

Context is accepted only when its trimmed length is greater than 10 characters. On success, the platform writes:

Both content and embeddingInput are capped at 65,535 characters. Excess text is truncated from the end, so an unusually large chunk can lose trailing content. Fix oversized chunks through chunking rather than relying on contextualization to compensate.

Model prerequisite

Contextual Retrieval shares the platform KB Chunking Task Model with chunk planning. It does not currently have an independent enable switch.

An administrator selects a Model Zoo model under Admin panel → System Models → KB Chunking Task Model. The platform calls that model's Base URL and API key through /chat/completions.

Limits and deployment defaults

These are deployment defaults and can be overridden by platform administrators through environment configuration.

Best-effort behavior and fallback

One failed chunk does not stop the entire document:

• Figure chunks already contain visual descriptions and nearby information, so they are skipped without a model call.
• A non-success response, timeout, empty output, short output, or invalid result leaves that chunk unchanged.
• If the model, Base URL, or document text cannot be resolved, the lower-level process skips context generation.
• Contextualized and unchanged chunks can coexist in the same document.

If an administrator adds or changes the KB Chunking Task Model after a document has been indexed, existing chunks are not updated automatically. Re-index the document, or rechunk a supported document, to generate new context.

Progress and verification

During processing, the UI can show Contextualizing chunks or Generating chunk context, together with completed and total chunk counts. Percentage bands vary between standard, page, parallel, and rechunk flows; do not use one fixed percentage range as proof of success.

After processing:

1. Open the document's Chunks page and verify that the source text remains correct. The editor prioritizes metadata.originalContent so generated context is not mistaken for editable source text.
2. Inspect chunk metadata for chunkContext and originalContent. Only successfully contextualized chunks have those fields.
3. Use the knowledge-base Search page with questions that require document-location clues and review both matches and sources.
4. If context is inaccurate, check the source and chunk boundary before re-indexing. Generated context should not be treated as an authoritative fact.

Cost, latency, and data privacy

• Each non-figure chunk can add one model call. More chunks increase indexing time and model usage.
• Higher concurrency can reduce elapsed time but increases burst load and rate-limit risk at the model endpoint.
• Document text and chunk content are sent to the configured KB Chunking Task Model endpoint. Before using an external model service, review data residency, retention, confidentiality, and personal-data requirements.
• temperature: 0 reduces randomness but does not guarantee that generated context is correct or free of inference.

The percentage improvements often associated with the original Contextual Retrieval research are external experimental results, not guaranteed UnieAI Studio metrics. Measure effectiveness with your own documents and representative queries.

Related documentation