Cluster
Use serve() from @axiljs/scale to fork one worker per core, restart crashed workers with backoff and drain gracefully on SIGTERM with node:cluster.
On this page12
serve() runs an app on every core of a machine with node:cluster, keeps the workers alive, and shuts them down without dropping requests.
1import { Application } from '@axiljs/core'
2import { serve } from '@axiljs/scale'
3
4serve(() => {
5 const app = new Application()
6 app.get('/', (_req, res) => res.success({ ok: true, pid: process.pid }))
7 return app
8}, { port: 3000 })The first argument is a factory: a function that builds and returns the app (or a Promise of it). serve() calls it in each worker, never in the supervising primary process.
Roles
serve() resolves the scaling config (the scaling option, else axil.scaling.json in the working directory, else defaults, with AXIL_WORKERS on top), then behaves according to the worker count and the current process:
| Role | When | What happens |
|---|---|---|
primary | workers > 1, top-level process | Prints host-check warnings, forks one worker per slot, supervises them. Never calls the factory |
worker | A process forked by the primary | Calls the factory, listens, applies limits, drains on request |
single | workers is 1 | Prints host-check warnings, calls the factory and listens in-process, with the same graceful shutdown |
All workers listen on the same port; node:cluster distributes incoming connections between them.
serve() returns a ServeHandle:
| Field | Type | Description |
|---|---|---|
role | 'primary' | 'worker' | 'single' | Which of the above this process is |
workers | number | Configured worker count (1 for single) |
scaling | ResolvedScaling | The resolved config |
findings | Finding[] | Startup host-check findings (empty in workers) |
shutdown() | () => Promise<void> | Drain and stop. On the primary, drains every worker |
On the primary, the returned promise resolves once every worker is listening.
Options
| Option | Type | Default | Description |
|---|---|---|---|
port | number | the app's default | Port passed to app.listen() |
host | string | the app's default | Host passed to app.listen() |
scaling | ScalingConfig | axil.scaling.json, else defaults | Scaling config in code |
maxRestarts | number | 5 | Crashes allowed per minute before the primary gives up |
checkOnStart | boolean | true | Print host-check warnings at startup |
log | (level, message) => void | console with an [axil:scale] prefix | Logger for supervisor messages; level is 'info', 'warn' or 'error' |
onPrimary | () => void | (() => void) | Runs once in the primary before workers are forked. See below |
Worker count
vertical.workers decides how many workers the primary forks: "auto" (default) means one per core as reported by os.availableParallelism(). Override it per environment with AXIL_WORKERS:
Each worker gets a stable slot number from 1 to workers, exposed as the AXIL_WORKER_ID environment variable. A restarted worker reuses its slot. Only worker 1 prints @axiljs/core's startup banner; the others are silenced while they start listening.
Per-worker limits
Every worker applies the vertical settings to its HTTP server:
| Setting | Applied as |
|---|---|
memoryPerWorkerMb | --max-old-space-size=<memoryPerWorkerMb> on each forked worker. Skipped if you already pass --max-old-space-size to the primary |
maxConnectionsPerWorker | server.maxConnections |
keepAliveTimeoutMs (when set) | server.keepAliveTimeout, and server.headersTimeout raised to keepAliveTimeoutMs + 1000 if it is not already higher |
Note
The heap flag is added when the primary forks workers. In single mode, nothing is forked, so pass --max-old-space-size to node yourself if you need it.
Crash restart
When a worker dies unexpectedly, the primary forks a replacement in the same slot after a backoff:
delay = min(10 000 ms, 250 ms × 2^(crashes in the last minute − 1))
250 ms, 500 ms, 1 s, 2 s, 4 s, ...If more than maxRestarts crashes (default 5) happen within one minute, the primary treats it as a crash loop: it logs an error, drains the remaining workers and exits with code 1. Your process manager (systemd, Kubernetes) then decides what to do, instead of the primary restarting a broken build forever.
A worker that exits during startup (a throwing factory, a port already in use) also stops everything: serve() drains the other workers and rejects with a worker exited during startup (code N); see its error above.
Graceful shutdown
On SIGTERM or SIGINT, the primary drains every worker:
- Sends each worker a shutdown message.
- Each worker stops accepting new connections and closes idle keep-alive sockets.
- In-flight requests finish. While draining, the worker closes keep-alive sockets as soon as they become idle (checked every 50 ms), so finished connections do not linger until their keep-alive timeout.
- After
vertical.shutdownTimeoutMs(default 10 s), any connection still open is force-closed. - Each worker exits with code 0 once its server has closed.
- Workers still alive
shutdownTimeoutMs + 2000ms after the signal are killed withSIGKILL. - The primary exits.
In single mode, steps 2 to 5 run in-process on SIGTERM/SIGINT.
Workers also drain if the primary disappears (IPC disconnect) or if they receive SIGTERM directly. They ignore SIGINT, so pressing Ctrl+C in a terminal (which signals the whole process group) is coordinated by the primary.
If the app has a close() method (as @axiljs/core's Application does), the drain calls it; otherwise it closes the HTTP server returned by getServer().
Tip
In Kubernetes, set terminationGracePeriodSeconds above shutdownTimeoutMs so the pod is not killed mid-drain. axil deploy sets a grace period that covers the drain. Also make sure the container's entrypoint is node itself (or forwards signals) so SIGTERM reaches the primary.
Shutting down from code
1const handle = await serve(() => createApp(), { port: 3000 })
2
3// later, e.g. from your own admin hook or test teardown
4await handle.shutdown()shutdown() is idempotent: calling it twice returns the same promise. It drains and stops the server (or every worker, on the primary) but does not call process.exit(); the signal handlers do that.
Primary-side services: onPrimary
Note
New in 0.3.4: the onPrimary option.
Each worker is a separate process, so in-memory state is per worker. onPrimary runs once in the primary, before the first worker is forked, so services that workers talk to over IPC are already listening when the workers start. Use it to install the primary side of cluster rate limiting and the cluster event relay:
1import { serve } from '@axiljs/scale'
2import { installClusterRateLimitPrimary } from '@axiljs/security'
3import { installClusterEventRelay } from '@axiljs/events'
4
5serve(createApp, {
6 port: 3000,
7 onPrimary: () => {
8 const stopRateLimits = installClusterRateLimitPrimary()
9 const stopRelay = installClusterEventRelay()
10 return () => { stopRateLimits(); stopRelay() }
11 }
12})Inside the app, use rateLimit({ store: new ClusterStore() }) and new EventBus({ transport: new ClusterTransport() }). Both behave like their in-process versions when there is only one worker.
onPrimaryruns only in cluster mode (primaryrole). It is not called in workers or insinglemode.- If it returns a function, that function is called during shutdown, after every worker has stopped.
- These services share state between the workers of one machine. For several servers, see Running multiple instances.
Startup host checks
Before forking, the primary (or single process) runs hostChecks() and prints every warning and failure, with its fix:
workers: "auto"inside a container whose CPU quota is smaller than the visible cores- more workers than CPUs (oversubscription)
workers × memoryPerWorkerMbabove 85% of the machine's (or container's) memory- an open-file limit (
ulimit -n) belowmaxConnectionsPerWorker + 256, on Linux
They are also returned in handle.findings. Turn the printing off with checkOnStart: false.
The Listenable interface
serve() does not require @axiljs/core. The factory may return anything with this shape:
1interface Listenable {
2 listen(port?: number, host?: string): Promise<unknown>
3 close?(): Promise<void>
4 getServer?(): import('node:http').Server | null
5}listen()is required.getServer()letsserve()applymaxConnectionsand keep-alive settings and force-close connections at the drain deadline. Without it, those limits are not applied.close()is preferred for draining; without it,serve()closes the server fromgetServer().
Project layout
Keep the factory free of side effects at import time, so tests can build the app without forking:
1import { Application } from '@axiljs/core'
2
3export function createApp(): Application {
4 const app = new Application()
5 // routes, middleware ...
6 return app
7}1import { serve } from '@axiljs/scale'
2import { createApp } from './app.js'
3
4await serve(() => createApp(), { port: Number(process.env.PORT ?? 3000) })1// src/app.test.ts: single process, the cluster is never forked
2import { createTestClient } from '@axiljs/testing'
3import { createApp } from './app.js'
4
5const client = createTestClient(createApp())Everything created inside the factory exists once per worker. Anything kept in memory there (caches, rate-limit counters, schedulers) is multiplied by the worker count. See readiness, and Running multiple instances for sharing that state.
Related
Was this page helpful?