Text chunks
Extracting spans of a document's text by offset, with surrounding sentences — the retrieval step of a RAG pipeline.
A search gives you documents and passages; a text chunk gives you the actual text at a given position, plus as many surrounding sentences as you ask for. It is what you send to a language model, and what you show when a snippet needs context.
const { chunks } = await client.api.preview.textChunks(
record.id,
[{ offset: 1200, length: 300 }], // where
["contract", "renewal"], // terms to highlight
query,
1, // sentences before
1, // sentences after
);The parameters, in order
The signature is positional and long, so it is worth naming each part:
Prop
Type
Each returned TextChunk carries its offset, length, the text itself, and a sql field that is
informational — the statement the server used.
Widening a passage into context
The natural pairing: a search returns passages with locations, and chunks turn one of them into readable text.
import { createAtomicClient } from "@sinequa/atomic";
import type { Query, TextChunk } from "@sinequa/atomic";
const client = createAtomicClient({
app: "<your-app>",
backendUrl: "https://<your-sinequa-server>",
});
await client.initializeConfig();
await client.auth.login();
const query: Query = { name: "<your-query>", text: "contract renewal terms", pageSize: 5 };
const result = await client.api.query.search(query);
const passage = result.topPassages?.passages?.[0];
if (passage) {
// A passage's `location` is a [start, end] pair of offsets into the document.
const [start, end] = passage.location;
const { chunks } = await client.api.preview.textChunks(
passage.recordId,
[{ offset: start, length: end - start }],
// Highlight the terms the user actually searched for.
(query.text ?? "").split(/\s+/).filter(Boolean),
query,
1,
1,
);
for (const chunk of chunks) {
console.log(quote(chunk));
}
}
function quote(chunk: TextChunk) {
return `…${chunk.text.trim()}… (offset ${chunk.offset})`;
}import { Injectable, inject } from "@angular/core";
import type { Query, TextChunk, TopPassage } from "@sinequa/atomic";
import { SINEQUA } from "./sinequa";
@Injectable({ providedIn: "root" })
export class ChunkService {
private readonly client = inject(SINEQUA);
/**
* Turns a passage into readable text with a sentence of context on each side.
* A passage's `location` is a [start, end] pair of document offsets.
*/
async contextFor(passage: TopPassage, query: Query, sentences = 1): Promise<TextChunk[]> {
const [start, end] = passage.location;
const { chunks } = await this.client.api.preview.textChunks(
passage.recordId,
[{ offset: start, length: end - start }],
(query.text ?? "").split(/\s+/).filter(Boolean),
query,
sentences,
sentences,
);
return chunks;
}
}import type { Query, TextChunk, TopPassage } from "@sinequa/atomic";
import { useCallback } from "react";
import { ready, sinequa } from "./sinequa";
export function useChunkContext() {
/**
* Turns a passage into readable text with a sentence of context on each side.
* A passage's `location` is a [start, end] pair of document offsets.
*/
return useCallback(
async (passage: TopPassage, query: Query, sentences = 1): Promise<TextChunk[]> => {
const [start, end] = passage.location;
await ready;
const { chunks } = await sinequa.api.preview.textChunks(
passage.recordId,
[{ offset: start, length: end - start }],
(query.text ?? "").split(/\s+/).filter(Boolean),
query,
sentences,
sentences,
);
return chunks;
},
[],
);
}Assembling context for a model
For retrieval-augmented generation the pattern is: search, take the best passages, widen each into a chunk, and concatenate with attribution. Requesting several locations in one call keeps it to one round trip per document.
const locations = passages
.filter((passage) => passage.recordId === record.id)
.map(({ location: [start, end] }) => ({ offset: start, length: end - start }));
const { chunks } = await client.api.preview.textChunks(
record.id,
locations,
terms,
query,
2,
2,
);Keep the offset with the text: it is what lets you cite a position, and what lets a preview scroll to it.