AI Platform

AxilJS RAG Pipeline

Build RAG apps with AxilJS: document chunking, embeddings, vector search with minScore filtering, sourced answers, streaming, and separate embedding providers.

8 min readUpdated DocumentationEdit this page
On this page14

AxilJS provides a Retrieval-Augmented Generation (RAG) pipeline for building AI applications that retrieve relevant information from indexed documents and use that information to generate contextual answers.

The RAGPipeline combines an AI provider with a VectorStore, allowing applications to index text, perform retrieval-based queries, return source documents, and stream RAG responses.

What Is RAG?

Retrieval-Augmented Generation combines two stages:

  1. Retrieval — Find relevant information from indexed documents using vector search.
  2. Generation — Pass the retrieved information to an AI provider to generate an answer.

A typical AxilJS RAG workflow looks like this:

Documents
loadFromText()
RAGPipeline.index()
VectorStore
RAGPipeline.query()
AI Provider
Generated Response
  • RAGPipeline.index() chunks the documents, embeds each chunk and stores the vectors in the VectorStore.
  • The user query enters at RAGPipeline.query(), which embeds the query and searches the store (topK, minScore).
  • The query returns the relevant sources alongside the context passed to the AI provider.

This approach allows an AI application to answer questions using information retrieved from a specific document collection.

Setup

Import RAGPipeline, VectorStore, and loadFromText from @axiljs/ai.

TypeScript
TS
1import {
2  OpenAIProvider,
3  RAGPipeline,
4  VectorStore,
5  loadFromText
6} from '@axiljs/ai'
7
8const ai = new OpenAIProvider({ apiKey: process.env.OPENAI_API_KEY! })
9
10const rag = new RAGPipeline({
11  provider: ai,
12  vectorStore: new VectorStore()
13})

The provider generates answers (and, by default, the embeddings), while vectorStore stores the vectors used during retrieval. If vectorStore is omitted, the pipeline creates an in-memory VectorStore for you.

Pipeline Options

All RAGPipeline options and their defaults:

OptionDefaultDescription
providerrequiredProvider used to generate answers (and embeddings, unless embeddingProvider is set)
embeddingProviderproviderProvider used to embed chunks and queries
vectorStorenew in-memory VectorStoreWhere chunk embeddings are stored
generationModelprovider's default modelModel used to generate answers
embeddingModelprovider's default embedding modelModel used for embeddings (index and query)
topK5Number of chunks retrieved per query
minScore0.3Minimum similarity score a chunk needs to be used as context
chunker.chunkSize800Maximum chunk size in characters
chunker.overlap150Characters of overlap between consecutive chunks
chunker.minChunkSize50Chunks shorter than this are dropped (see below)
loggercoloured console outputfalse to silence, or an object with info(msg) and warn(msg)
systemPromptbuilt-in grounding promptSystem prompt used when generating answers
TypeScript
TS
1import { AnthropicProvider, OpenAIProvider, RAGPipeline } from '@axiljs/ai'
2
3const rag = new RAGPipeline({
4  provider: new AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY! }),     // answers
5  embeddingProvider: new OpenAIProvider({ apiKey: process.env.OPENAI_API_KEY! }),  // embeddings
6  generationModel: 'claude-sonnet-5-5',
7  topK: 5,
8  minScore: 0.3,
9  chunker: { chunkSize: 800, overlap: 150, minChunkSize: 50 },
10  logger: false
11})

Note

Changed in 0.3.3: embeddingProvider, chunker.minChunkSize, and logger are new options.

Separate Embedding Provider

Answer generation and embeddings can come from different providers. This is required when the chat provider has no embeddings API: AnthropicProvider.embed() always throws, so a pipeline that uses Claude for answers needs another provider, such as OpenAI, Gemini, or a local Ollama model, for embeddings.

TypeScript
TS
1import { AnthropicProvider, OllamaProvider, RAGPipeline } from '@axiljs/ai'
2
3const rag = new RAGPipeline({
4  provider: new AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY! }),
5  generationModel: 'claude-opus-5-5',
6  embeddingProvider: new OllamaProvider(),   // local embeddings (nomic-embed-text by default)
7  embeddingModel: 'nomic-embed-text'
8})

embeddingModel is passed to the embeddingProvider, and generationModel to the provider. The same embedding provider and model are used for index(), query(), and queryStream(), so stored and query vectors always have matching dimensions.

Index Documents

Before querying the RAG pipeline, index your documents.

Use loadFromText() to create documents from text and pass them to rag.index().

TypeScript
TS
1const stats = await rag.index(
2  loadFromText([
3    'AxilJS supports PostgreSQL, MySQL and SQLite through @axiljs/orm.'
4  ])
5)
6
7stats.documentsIndexed  // 1
8stats.chunksCreated     // 1

index() splits each document into chunks, embeds the chunks in batches of 20, and stores them in the vector store. Each stored chunk keeps the document's metadata plus documentId and chunkIndex.

Other loaders are available for different sources: loadFromString(content, id?, metadata?), loadFromFiles(paths), and loadFromUrls(urls) (async).

Chunking

Documents are split recursively (paragraphs, then lines, sentences, words) into chunks of at most chunker.chunkSize characters, with chunker.overlap characters carried over between chunks. Chunks shorter than chunker.minChunkSize are discarded.

A short document is never lost: a document that fits in one chunk becomes a single chunk, and a non-empty document shorter than minChunkSize is still indexed as one chunk.

Note

Changed in 0.3.3: previously a document shorter than the minimum chunk size (50 characters) produced no chunks and was silently skipped. It is now indexed as a single chunk, and the threshold is configurable with chunker.minChunkSize.

Query the RAG Pipeline

Use rag.query() to ask a question against the indexed knowledge base.

TypeScript
TS
1const result = await rag.query(
2  'What databases are supported?'
3)
4
5result.response  // generated answer
6result.sources   // [{ content, score, metadata }]
7result.usage     // { retrievalEmbeddingTokens, generationTokens }

The result contains both the generated answer and the sources used to produce it.

  • result.response contains the AI-generated answer.
  • result.sources contains the retrieved chunks, each with its similarity score and metadata.
  • result.usage reports the tokens used to embed the query and to generate the answer.

For multi-turn conversations, pass the previous messages as the second argument:

TypeScript
TS
1const result = await rag.query('And which one is the default?', [
2  { role: 'user', content: 'What databases are supported?' },
3  { role: 'assistant', content: 'PostgreSQL, MySQL and SQLite.' }
4])

If no chunk reaches minScore, the pipeline does not call the model. It returns a fixed "not enough information" response with an empty sources array.

Similarity Scores and minScore

Retrieval scores use the vector store's metric. With the default cosine metric, score is the raw cosine similarity in the range -1 to 1: 1 means the same direction, 0 means unrelated, and negative values mean opposite. minScore uses the same scale, and its default of 0.3 keeps related chunks while filtering out unrelated ones.

Warning

Changed in 0.3.3: cosine scores are now plain cosine similarity (-1 to 1). Up to 0.3.2 they were rescaled to (cos + 1) / 2, where 0.5 meant "unrelated", so the default minScore of 0.3 filtered almost nothing. If you set a custom threshold t on the old scale, the equivalent value now is 2t - 1 (for example, an old 0.75 becomes 0.5). Code that displays or compares sources[].score values should be checked as well.

See Embeddings for the score scale of the other metrics.

Stream RAG Responses

AxilJS also supports streaming RAG queries with queryStream().

TypeScript
TS
1for await (
2  const event of rag.queryStream('Tell me about auth')
3) {
4  if (event.type === 'sources') {
5    console.log(event.data)
6  }
7
8  if (event.type === 'delta') {
9    process.stdout.write(event.data as string)
10  }
11
12  if (event.type === 'done') {
13    console.log('\nfinished')
14  }
15}

The streaming API emits three event types:

  • sources — Emitted first, with the retrieved sources (content previews of up to 200 characters, score, and metadata).
  • delta — Incremental text of the generated answer.
  • done — Emitted last; data holds the provider's final completion response (or null when no sources were found).

queryStream() accepts the same optional conversation history as query().

This is useful for interactive RAG applications where the client needs to display retrieved sources and AI-generated content progressively.

Query vs Streaming

APIOutputTypical use
rag.query()Complete resultStandard RAG requests
rag.queryStream()Incremental eventsInteractive RAG interfaces

Logging and Stats

By default the pipeline prints coloured progress messages (and an in-place embedding progress line) to the console. Pass logger: false to silence it, or route messages to your own logger:

TypeScript
TS
1const rag = new RAGPipeline({
2  provider: ai,
3  logger: {
4    info: msg => appLogger.info(msg),
5    warn: msg => appLogger.warn(msg)
6  }
7})
8
9rag.getStats()  // { documentCount, embeddingTokensUsed }

Typical RAG Architecture

A production RAG application generally follows an indexing and query lifecycle:

Source Documents
loadFromText()
RAGPipeline.index()
VectorStore
Indexing
User Question
RAGPipeline.query()
Relevant SourcesAI Provider
Generated Answer
Query: searches the VectorStore built during indexing

The separation between indexing and querying allows documents to be prepared before users begin asking questions. To keep an index between restarts, persist the vector store with save() and load() (see Embeddings).

Common RAG Use Cases

The AxilJS RAG pipeline can be used for applications such as:

  • Document question answering
  • Knowledge-base assistants
  • Technical documentation search
  • Semantic document retrieval
  • Internal knowledge systems
  • AI-powered search
  • Context-aware AI assistants
  • Retrieval-based chat applications

RAG Workflow

A basic AxilJS RAG implementation follows these steps:

  1. Configure an AI provider (and an embeddingProvider if the chat provider has no embeddings API).
  2. Create a VectorStore.
  3. Create a RAGPipeline.
  4. Load documents with loadFromText().
  5. Index the documents with rag.index().
  6. Query the indexed content with rag.query().
  7. Use result.response for the generated answer.
  8. Use result.sources to expose the retrieved sources.
  9. Use rag.queryStream() when streaming is required.

Was this page helpful?

Help improve the documentation

AxilJS is open source and documentation improvements are welcome.

AxilJS DocumentationMIT License · Built by SyntaxilitY