Configuration
Every axil.scaling.json field with type, default and validation rule: target, vertical workers and limits, horizontal instances, headroom, AXIL_WORKERS.
On this page12
Scaling is described in one file, axil.scaling.json, at the project root. It is read by serve(), axil capacity and axil deploy, so the process that runs, the capacity check and the deployment manifest all agree.
1{
2 "target": { "rps": 1500000, "concurrentConnections": 2000000, "p99Ms": 50 },
3 "vertical": { "workers": "auto", "maxConnectionsPerWorker": 20000, "memoryPerWorkerMb": 512,
4 "shutdownTimeoutMs": 10000, "keepAliveTimeoutMs": 65000 },
5 "horizontal": { "instances": { "min": 6, "max": 18 }, "instanceSize": { "cpus": 8, "memoryGb": 16 } },
6 "headroom": 0.3
7}Every field is optional. A missing file means "all defaults".
target
What the whole deployment must sustain at peak. Used by the capacity planner only; serve() ignores it.
| Field | Type | Default | Description |
|---|---|---|---|
rps | number | not set | Peak requests per second across all instances |
concurrentConnections | number | not set | Peak open connections (roughly concurrent users with keep-alive or WebSockets) |
p99Ms | number | not set | Latency budget for the 99th percentile, ms |
Each value must be a positive number when present. Without target.rps, axil capacity reports what the deployment can carry but gives no pass/fail verdict on throughput.
vertical
How each instance uses its cores. Applied by serve() to every worker.
| Field | Type | Default | Description |
|---|---|---|---|
workers | positive integer or "auto" | "auto" | Worker processes per instance. "auto" is one per CPU core |
maxConnectionsPerWorker | positive integer | 10000 | Open connections one worker accepts (server.maxConnections) |
memoryPerWorkerMb | positive number | 512 | V8 heap limit per worker (--max-old-space-size) |
shutdownTimeoutMs | positive integer | 10000 | Grace period for in-flight requests on shutdown |
keepAliveTimeoutMs | positive integer | Node's default (5000) | Idle keep-alive timeout (server.keepAliveTimeout) |
workers and AXIL_WORKERS
The AXIL_WORKERS environment variable overrides vertical.workers, so one file works across machine sizes:
An empty AXIL_WORKERS is ignored. Any other value that is not auto or a positive integer fails validation.
"auto" uses os.availableParallelism(): the cores this process may run on. It does not see container CPU quotas (cgroup limits). In a container limited to 2 CPUs on a 32-core node, "auto" starts 32 workers that compete for 2 CPUs. serve() and axil capacity warn about this at startup. Set AXIL_WORKERS to the container's CPU count; axil deploy does this for you by pinning AXIL_WORKERS to the pod's CPUs.
keepAliveTimeoutMs behind a load balancer
Load balancers keep idle connections to your instances open and reuse them. If Node.js closes an idle socket at the same moment the balancer sends a new request on it, the balancer answers the client with a 502. These show up as rare, random 502s under load.
The fix is to keep sockets open longer than the balancer does:
| Load balancer | Default idle timeout | Suggested keepAliveTimeoutMs |
|---|---|---|
| AWS Application Load Balancer | 60 s | 65000 |
Node's default keep-alive timeout is 5 s, far below that. When keepAliveTimeoutMs is set, serve() also raises server.headersTimeout to at least keepAliveTimeoutMs + 1000, because Node may drop reused sockets when the headers timeout is not above the keep-alive timeout.
horizontal
How many instances run, and how big each is. Used by the capacity planner and axil deploy; serve() only runs one instance.
| Field | Type | Default | Description |
|---|---|---|---|
instances | positive integer, or { "min": n, "max": n } | 1 | A fixed count, or autoscaling bounds |
instanceSize.cpus | positive number | this machine's cores | CPUs per instance (VM or pod) |
instanceSize.memoryGb | positive number | this machine's memory | Memory per instance in GB |
- With a fixed number,
minandmaxare both that number. - With an object,
mindefaults to1,maxdefaults tomin, both must be positive integers, andmaxmust be>= min. - When
instanceSizeis omitted, the planner assumes instances like the machine it runs on and says so in an info finding. WheninstanceSizeis present but one of its fields is missing, that field defaults to1.
axil deploy uses instances.min as the replica count, instances.max as the autoscaler ceiling and instanceSize for pod resources.
headroom
| Field | Type | Default | Rule |
|---|---|---|---|
headroom | number | 0.3 | 0 up to (not including) 5 |
Spare capacity kept above target.rps: the planner sizes for target.rps × (1 + headroom). The default 30% covers traffic spikes, rolling deploys and losing an instance. At saturation latency climbs steeply, so running instances cooler is what keeps p99 inside budget.
Validation
Values are validated when the config is resolved (at serve() startup and in axil capacity). A bad value throws with the field name:
[@axiljs/scale] vertical.workers must be a positive integer, got 0
[@axiljs/scale] horizontal.instances.max must be >= min
[@axiljs/scale] headroom must be between 0 and 5
[@axiljs/scale] axil.scaling.json is not valid JSON: Unexpected token ...Warning
0 is not "unset". "target": { "rps": 0 } fails validation; leave the field out instead. JSON also has no comments: put notes in your README, not in axil.scaling.json.
Configuration in code
All of this is also available programmatically:
1import { defineScaling, detectSystem, loadScalingConfig, resolveScaling, serve } from '@axiljs/scale'
2
3// Typed config in code (identity helper)
4const scaling = defineScaling({
5 vertical: { workers: 'auto', keepAliveTimeoutMs: 65000 },
6 horizontal: { instances: { min: 2, max: 8 }, instanceSize: { cpus: 4, memoryGb: 8 } }
7})
8
9serve(() => createApp(), { port: 3000, scaling }) // instead of axil.scaling.json
10
11// Read and resolve the file yourself
12const raw = loadScalingConfig() // undefined when the file is absent
13const resolved = resolveScaling(raw ?? {}, detectSystem())
14resolved.vertical.workers // a number; 'auto' already resolved| Export | Description |
|---|---|
defineScaling(config) | Returns config unchanged, typed as ScalingConfig |
loadScalingConfig(dir?) | Reads axil.scaling.json from dir (default process.cwd()). Returns undefined when absent, throws on invalid JSON |
resolveScaling(config?, machine?) | Fills defaults, applies AXIL_WORKERS, validates, returns ResolvedScaling. machine is { cores, memoryBytes }; pass detectSystem() |
SCALING_FILE | 'axil.scaling.json' |
SCALING_TEMPLATE | The starter config written by axil capacity --init |
Note
Pass detectSystem() as the second argument to resolveScaling(). Without it, the default instance memory falls back to 1 GB, because the built-in default machine reports no memory.
ResolvedScaling
1{
2 target: { rps: number; concurrentConnections: number; p99Ms: number } // 0 = not set
3 vertical: {
4 workers: number
5 workersAuto: boolean
6 maxConnectionsPerWorker: number
7 memoryPerWorkerMb: number
8 shutdownTimeoutMs: number
9 keepAliveTimeoutMs?: number
10 }
11 horizontal: {
12 min: number
13 max: number
14 instanceSize: { cpus: number; memoryGb: number }
15 instanceSizeDefaulted: boolean
16 }
17 headroom: number
18}Starter template
SCALING_TEMPLATE, as written by axil capacity --init:
1{
2 "target": { "rps": 10000, "concurrentConnections": 20000, "p99Ms": 100 },
3 "vertical": { "workers": "auto", "maxConnectionsPerWorker": 10000, "memoryPerWorkerMb": 512 },
4 "horizontal": { "instances": { "min": 2, "max": 6 }, "instanceSize": { "cpus": 4, "memoryGb": 8 } },
5 "headroom": 0.3
6}Docker images
serve() reads axil.scaling.json from the working directory. If your image copies only dist/, also copy the file, otherwise containers run with defaults (no keepAliveTimeoutMs, default limits):
COPY dist ./dist
COPY axil.scaling.json ./Related
- Cluster: how
serve()applies theverticalsettings - Capacity planning
axil capacity
Was this page helpful?