Chat Completions and Streaming

AxilJS provides APIs for building AI chat applications in TypeScript and Node.js, including standard chat completions, streamed AI responses, system instructions, and HTTP endpoints.

Use ai.complete() when you need the complete response at once, or ai.stream() when your application needs to process the AI response incrementally.

Chat Completions

Use ai.complete() to send a conversation to an AI provider and wait for the complete response.

typescript

const res = await ai.complete({
  messages: [
    {
      role: 'user',
      content: 'Hello!'
    }
  ],
  system: 'You are helpful.',
  temperature: 0.7
})

The messages array contains the conversation history, while system provides instructions that guide the AI's behavior.

The temperature option controls the variability of generated responses.

This approach is suitable for APIs and applications where the client can wait for the complete AI response before displaying or processing it.

Streaming AI Responses

For interactive applications, AxilJS supports streaming responses through ai.stream().

typescript

for await (const chunk of ai.stream({ messages })) {
  if (chunk.delta) process.stdout.write(chunk.delta)
 
  if (chunk.done) console.log(chunk.response)
}

The async iterator yields response chunks as the AI generates them.

You can use:

chunk.delta to process incremental response content.
chunk.done to determine when generation has completed.
chunk.response to access the completed response.

Streaming is particularly useful for AI chat interfaces because users can see generated content progressively instead of waiting for the entire completion.

HTTP Chat Endpoints

AxilJS also provides HTTP handlers for exposing AI chat functionality through your application API.

typescript

import { aiChat, aiChatStream } from '@axiljs/ai'
 
app.post('/chat', aiChat({ provider: ai }))
 
app.post(
  '/chat/stream',
  aiChatStream({ provider: ai })
)

The aiChat() handler provides a standard chat endpoint, while aiChatStream() exposes a streaming endpoint.

This allows an AxilJS application to expose AI functionality through conventional HTTP routes.

Chat vs Streaming

Choose the execution model based on how your application consumes AI responses.

API	Response model	Typical use
`ai.complete()`	Complete response	Standard API requests
`ai.stream()`	Incremental chunks	Interactive AI interfaces
`aiChat()`	HTTP completion endpoint	Backend chat APIs
`aiChatStream()`	HTTP streaming endpoint	Streaming chat APIs

Typical AI Chat Architecture

A common AxilJS implementation can connect an HTTP endpoint to an AI provider and return either a complete or streamed response.

text

Client
  │
  ▼
AxilJS HTTP Endpoint
  │
  ├──► aiChat()
  │       │
  │       ▼
  │   Complete Response
  │
  └──► aiChatStream()
          │
          ▼
      Stream Chunks
          │
          ▼
        Client

When to Use Streaming

Use streaming when response latency and user experience are important. It is a good fit for:

AI chat interfaces
Real-time AI assistants
Long-form AI generation
Interactive developer tools
Applications that display tokens or response chunks progressively

For simpler backend operations where the complete result is required before continuing, ai.complete() provides a more straightforward execution model.

Chat Completions and Streaming