AxilJS RAG Pipeline
Build RAG apps with AxilJS: document chunking, embeddings, vector search with minScore filtering, sourced answers, streaming, and separate embedding providers.
On this page14
AxilJS provides a Retrieval-Augmented Generation (RAG) pipeline for building AI applications that retrieve relevant information from indexed documents and use that information to generate contextual answers.
The RAGPipeline combines an AI provider with a VectorStore, allowing applications to index text, perform retrieval-based queries, return source documents, and stream RAG responses.
What Is RAG?
Retrieval-Augmented Generation combines two stages:
- Retrieval — Find relevant information from indexed documents using vector search.
- Generation — Pass the retrieved information to an AI provider to generate an answer.
A typical AxilJS RAG workflow looks like this:
RAGPipeline.index()chunks the documents, embeds each chunk and stores the vectors in theVectorStore.- The user query enters at
RAGPipeline.query(), which embeds the query and searches the store (topK,minScore). - The query returns the relevant sources alongside the context passed to the AI provider.
This approach allows an AI application to answer questions using information retrieved from a specific document collection.
Setup
Import RAGPipeline, VectorStore, and loadFromText from @axiljs/ai.
1import {
2 OpenAIProvider,
3 RAGPipeline,
4 VectorStore,
5 loadFromText
6} from '@axiljs/ai'
7
8const ai = new OpenAIProvider({ apiKey: process.env.OPENAI_API_KEY! })
9
10const rag = new RAGPipeline({
11 provider: ai,
12 vectorStore: new VectorStore()
13})The provider generates answers (and, by default, the embeddings), while vectorStore stores the vectors used during retrieval. If vectorStore is omitted, the pipeline creates an in-memory VectorStore for you.
Pipeline Options
All RAGPipeline options and their defaults:
| Option | Default | Description |
|---|---|---|
provider | required | Provider used to generate answers (and embeddings, unless embeddingProvider is set) |
embeddingProvider | provider | Provider used to embed chunks and queries |
vectorStore | new in-memory VectorStore | Where chunk embeddings are stored |
generationModel | provider's default model | Model used to generate answers |
embeddingModel | provider's default embedding model | Model used for embeddings (index and query) |
topK | 5 | Number of chunks retrieved per query |
minScore | 0.3 | Minimum similarity score a chunk needs to be used as context |
chunker.chunkSize | 800 | Maximum chunk size in characters |
chunker.overlap | 150 | Characters of overlap between consecutive chunks |
chunker.minChunkSize | 50 | Chunks shorter than this are dropped (see below) |
logger | coloured console output | false to silence, or an object with info(msg) and warn(msg) |
systemPrompt | built-in grounding prompt | System prompt used when generating answers |
1import { AnthropicProvider, OpenAIProvider, RAGPipeline } from '@axiljs/ai'
2
3const rag = new RAGPipeline({
4 provider: new AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY! }), // answers
5 embeddingProvider: new OpenAIProvider({ apiKey: process.env.OPENAI_API_KEY! }), // embeddings
6 generationModel: 'claude-sonnet-5-5',
7 topK: 5,
8 minScore: 0.3,
9 chunker: { chunkSize: 800, overlap: 150, minChunkSize: 50 },
10 logger: false
11})Note
Changed in 0.3.3: embeddingProvider, chunker.minChunkSize, and logger are new options.
Separate Embedding Provider
Answer generation and embeddings can come from different providers. This is required when the chat provider has no embeddings API: AnthropicProvider.embed() always throws, so a pipeline that uses Claude for answers needs another provider, such as OpenAI, Gemini, or a local Ollama model, for embeddings.
1import { AnthropicProvider, OllamaProvider, RAGPipeline } from '@axiljs/ai'
2
3const rag = new RAGPipeline({
4 provider: new AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY! }),
5 generationModel: 'claude-opus-5-5',
6 embeddingProvider: new OllamaProvider(), // local embeddings (nomic-embed-text by default)
7 embeddingModel: 'nomic-embed-text'
8})embeddingModel is passed to the embeddingProvider, and generationModel to the provider. The same embedding provider and model are used for index(), query(), and queryStream(), so stored and query vectors always have matching dimensions.
Index Documents
Before querying the RAG pipeline, index your documents.
Use loadFromText() to create documents from text and pass them to rag.index().
1const stats = await rag.index(
2 loadFromText([
3 'AxilJS supports PostgreSQL, MySQL and SQLite through @axiljs/orm.'
4 ])
5)
6
7stats.documentsIndexed // 1
8stats.chunksCreated // 1index() splits each document into chunks, embeds the chunks in batches of 20, and stores them in the vector store. Each stored chunk keeps the document's metadata plus documentId and chunkIndex.
Other loaders are available for different sources: loadFromString(content, id?, metadata?), loadFromFiles(paths), and loadFromUrls(urls) (async).
Chunking
Documents are split recursively (paragraphs, then lines, sentences, words) into chunks of at most chunker.chunkSize characters, with chunker.overlap characters carried over between chunks. Chunks shorter than chunker.minChunkSize are discarded.
A short document is never lost: a document that fits in one chunk becomes a single chunk, and a non-empty document shorter than minChunkSize is still indexed as one chunk.
Note
Changed in 0.3.3: previously a document shorter than the minimum chunk size (50 characters) produced no chunks and was silently skipped. It is now indexed as a single chunk, and the threshold is configurable with chunker.minChunkSize.
Query the RAG Pipeline
Use rag.query() to ask a question against the indexed knowledge base.
1const result = await rag.query(
2 'What databases are supported?'
3)
4
5result.response // generated answer
6result.sources // [{ content, score, metadata }]
7result.usage // { retrievalEmbeddingTokens, generationTokens }The result contains both the generated answer and the sources used to produce it.
result.responsecontains the AI-generated answer.result.sourcescontains the retrieved chunks, each with its similarityscoreandmetadata.result.usagereports the tokens used to embed the query and to generate the answer.
For multi-turn conversations, pass the previous messages as the second argument:
1const result = await rag.query('And which one is the default?', [
2 { role: 'user', content: 'What databases are supported?' },
3 { role: 'assistant', content: 'PostgreSQL, MySQL and SQLite.' }
4])If no chunk reaches minScore, the pipeline does not call the model. It returns a fixed "not enough information" response with an empty sources array.
Similarity Scores and minScore
Retrieval scores use the vector store's metric. With the default cosine metric, score is the raw cosine similarity in the range -1 to 1: 1 means the same direction, 0 means unrelated, and negative values mean opposite. minScore uses the same scale, and its default of 0.3 keeps related chunks while filtering out unrelated ones.
Warning
Changed in 0.3.3: cosine scores are now plain cosine similarity (-1 to 1). Up to 0.3.2 they were rescaled to (cos + 1) / 2, where 0.5 meant "unrelated", so the default minScore of 0.3 filtered almost nothing. If you set a custom threshold t on the old scale, the equivalent value now is 2t - 1 (for example, an old 0.75 becomes 0.5). Code that displays or compares sources[].score values should be checked as well.
See Embeddings for the score scale of the other metrics.
Stream RAG Responses
AxilJS also supports streaming RAG queries with queryStream().
1for await (
2 const event of rag.queryStream('Tell me about auth')
3) {
4 if (event.type === 'sources') {
5 console.log(event.data)
6 }
7
8 if (event.type === 'delta') {
9 process.stdout.write(event.data as string)
10 }
11
12 if (event.type === 'done') {
13 console.log('\nfinished')
14 }
15}The streaming API emits three event types:
sources— Emitted first, with the retrieved sources (content previews of up to 200 characters,score, andmetadata).delta— Incremental text of the generated answer.done— Emitted last;dataholds the provider's final completion response (ornullwhen no sources were found).
queryStream() accepts the same optional conversation history as query().
This is useful for interactive RAG applications where the client needs to display retrieved sources and AI-generated content progressively.
Query vs Streaming
| API | Output | Typical use |
|---|---|---|
rag.query() | Complete result | Standard RAG requests |
rag.queryStream() | Incremental events | Interactive RAG interfaces |
Logging and Stats
By default the pipeline prints coloured progress messages (and an in-place embedding progress line) to the console. Pass logger: false to silence it, or route messages to your own logger:
1const rag = new RAGPipeline({
2 provider: ai,
3 logger: {
4 info: msg => appLogger.info(msg),
5 warn: msg => appLogger.warn(msg)
6 }
7})
8
9rag.getStats() // { documentCount, embeddingTokensUsed }Typical RAG Architecture
A production RAG application generally follows an indexing and query lifecycle:
The separation between indexing and querying allows documents to be prepared before users begin asking questions. To keep an index between restarts, persist the vector store with save() and load() (see Embeddings).
Common RAG Use Cases
The AxilJS RAG pipeline can be used for applications such as:
- Document question answering
- Knowledge-base assistants
- Technical documentation search
- Semantic document retrieval
- Internal knowledge systems
- AI-powered search
- Context-aware AI assistants
- Retrieval-based chat applications
RAG Workflow
A basic AxilJS RAG implementation follows these steps:
- Configure an AI provider (and an
embeddingProviderif the chat provider has no embeddings API). - Create a
VectorStore. - Create a
RAGPipeline. - Load documents with
loadFromText(). - Index the documents with
rag.index(). - Query the indexed content with
rag.query(). - Use
result.responsefor the generated answer. - Use
result.sourcesto expose the retrieved sources. - Use
rag.queryStream()when streaming is required.
Was this page helpful?