Runtime

Resilience and Concurrency

How @axiljs/runtime executes tCircuitBreaker, tFallback, tBulkhead, tConcurrency, tMutex, tLock, tDedupe, tHedge, tCache, tRateLimit, tThrottle and tDebounce: options, defaults, key specs, errors and order.

14 min readUpdated DocumentationEdit this page
On this page24

@axiljs/runtime executes the reliability, concurrency, caching and rate-limit decorators from @axiljs/decorator. Put them on a method, wire() the class, and the behaviour applies to every call.

Note

New in 0.3.4: the interceptors on this page. Together with tAudit, tRetry, tTimeout and the intent decorators, the runtime enforces 26 decorators, up from 3. New in 0.3.5: setRateLimitStore().

TypeScript
TS
1import { tCache, tCircuitBreaker, tFallback, tRetry, tTimeout } from '@axiljs/decorator'
2import { wire } from '@axiljs/runtime'
3
4class Pricing {
5  @tCache({ key: 'price:{0}', ttl: 60_000, tags: ['prices'] })
6  @tFallback({ method: 'cachedPrice' })
7  @tRetry({ attempts: 3, delay: 100, retryOn: ['ECONNRESET'] })
8  @tCircuitBreaker({ threshold: 5, resetTimeout: 30_000 })
9  @tTimeout(800)
10  async price(sku: string) {
11    return this.upstream.getPrice(sku)
12  }
13
14  async cachedPrice(sku: string, error: unknown) {
15    return this.lastKnown.get(sku)
16  }
17}
18
19wire(Pricing)

The order in which you write the decorators does not matter; the interceptor order does. For price() above: a cache hit returns immediately; on a miss, up to three attempts each pass the circuit breaker and get their own 800 ms timeout; if all fail, cachedPrice(sku, error) supplies the answer.

State is per process

Circuit states, bulkheads, concurrency limits, mutexes, deduplication and debouncing live in the memory of one process. Rate-limit counters, cache entries, locks and idempotency results also default to in-memory, but have pluggable stores. With several workers or instances, install shared stores with useStores().

Errors

All errors below extend axilError from @axiljs/common, so @axiljs/core's default error handler answers with their status and code.

ErrorStatusCodeThrown by
RateLimitExceededError429RATE_LIMITEDtRateLimit, tThrottle
CircuitOpenError503CIRCUIT_OPENtCircuitBreaker
CapacityExceededError503CAPACITY_EXCEEDEDtBulkhead, tConcurrency
LockTimeoutError409LOCK_TIMEOUTtLock
DeadlineExceededError410DEADLINE_EXCEEDEDtDeadline (intent)
IntentViolationError500INTENT_VIOLATIONIntent checks in enforce mode

Every error has a target property (Class.method). RateLimitExceededError also has retryAfterMs, CapacityExceededError has resource, and LockTimeoutError has key and waitMs.

Note

TimeoutError from tTimeout is a plain Error with code: 'ETIMEDOUT', not an axilError, so it reaches the client as a 500 unless your error handler maps it (the high-scale-api template maps it to 504). See Interceptors.

Failure handling

tCircuitBreaker({ threshold, resetTimeout, halfOpenAttempts })

OptionTypeDefaultDescription
thresholdnumberrequiredConsecutive failures that open the circuit
resetTimeoutnumber30000Time the circuit stays open before probe calls are allowed, ms
halfOpenAttemptsnumber1Probe calls allowed at the same time while half-open
  • Closed: calls run; a success resets the failure count. After threshold consecutive failures the circuit opens.
  • Open: calls fail immediately with CircuitOpenError until resetTimeout has passed.
  • Half-open: up to halfOpenAttempts probe calls run; further calls get CircuitOpenError. A successful probe closes the circuit, a failed one opens it again.

There is one circuit per method (Class.method), shared by all instances in the process. Each transition emits runtime.circuit with { target, from, to }, and getCircuitStates() returns { 'Pricing.price': 'CLOSED' | 'OPEN' | 'HALF_OPEN' } for every method that has been called.

Because the breaker (order 250) runs inside tRetry (200), every retry attempt counts as a failure: with attempts: 3 and threshold: 5, two failed calls are enough to open it.

tFallback({ value }) / tFallback({ method })

Replaces a failure with a value, or with the result of another method of the same instance. The method receives the original arguments followed by the error:

TypeScript
TS
1class Recommendations {
2  @tFallback({ value: [] })
3  async forUser(id: string) { /* ... */ }
4
5  @tFallback({ method: 'popular' })
6  async personalised(id: string) { /* ... */ }
7
8  async popular(id: string, error: unknown) { return this.top10 }
9}

If method does not name a function, the original error is re-thrown. Each fallback emits runtime.fallback with { target, error } (the error message). tFallback (150) runs outside tRetry and tCircuitBreaker, so it handles the final failure, including CircuitOpenError.

tHedge({ delay, maxAttempts })

Starts another attempt when the first is slower than delay ms; the first success wins.

OptionTypeDefaultDescription
delaynumberrequiredWait before starting the next attempt, ms
maxAttemptsnumber2Total attempts, including the first

If every attempt started so far has failed, the next one starts immediately. The call rejects with the last error when all maxAttempts have failed. Each extra attempt emits runtime.hedge with { target, attempt }. Slower attempts are not cancelled.

Warning

Hedging executes the method more than once. Use it only on reads and other operations that are safe to repeat.

Concurrency

tBulkhead({ name, concurrency, queue })

A named capacity shared by every method that uses the same name, so one slow dependency cannot use up the whole process.

OptionTypeDefaultDescription
namestringrequiredBulkhead name; methods with the same name share it
concurrencynumberrequiredCalls running at the same time
queuenumber0Calls allowed to wait for a free slot

When all slots and the queue are full, the call fails with CapacityExceededError (503). The bulkhead is created on first use with that call's concurrency and queue.

TypeScript
TS
1class Reports {
2  @tBulkhead({ name: 'warehouse', concurrency: 4, queue: 20 })
3  async daily() { /* ... */ }
4
5  @tBulkhead({ name: 'warehouse', concurrency: 4, queue: 20 })
6  async monthly() { /* ... */ }   // shares the same 4 slots with daily()
7}

tConcurrency({ limit, strategy, key })

A limit per method, or per key.

OptionTypeDefaultDescription
limitnumberrequiredCalls running at the same time
strategy'queue' | 'reject' | 'drop-oldest''queue'What happens to calls over the limit
keykey specnone (one limit for the method)Separate limit per key, e.g. 'tenant'
  • queue: excess calls wait, without a bound.
  • reject: excess calls fail at once with CapacityExceededError.
  • drop-oldest: up to limit calls wait; when the queue is full, the oldest waiting call fails with CapacityExceededError and the new one takes its place.

tMutex({ key })

One call at a time per method, or per key (key is a string key spec). Other calls wait in order.

TypeScript
TS
1class Accounts {
2  @tMutex({ key: 'account:{0}' })
3  async transfer(accountId: string, amount: number) { /* ... */ }
4}

tMutex is per process. For a lock that spans processes, use tLock with a shared provider.

tLock({ key, ttl, wait })

A keyed lock with a wait timeout and automatic release.

OptionTypeDefaultDescription
keykey specrequiredLock key; when it cannot be resolved, the method name is used
ttlnumber30000Maximum time the lock is held, ms; released automatically after it
waitnumber10000Maximum time to wait for the lock, ms

When the lock cannot be acquired within wait, the call fails with LockTimeoutError (409). Lock keys are not scoped to the method: two methods that resolve to the same key share the lock. The default provider locks within the process; useStores({ lock }) or setLockProvider() installs a cross-process provider such as the SQL lease or PostgreSQL advisory lock providers from @axiljs/stores.

Warning

The lock is released after ttl even if the method is still running, so a hung holder cannot block forever. Choose a ttl longer than the slowest normal execution.

tDedupe({ key, window })

Calls with the same key while one is running share that one execution.

OptionTypeDefaultDescription
keykey specrequiredCalls with the same key are coalesced
windownumber0Also share a successful result for this many ms after it completes

Keys are scoped to the method. Failures are never shared after they settle. When the key cannot be resolved, the call runs normally. Unlike tIdempotent, nothing is stored, so tDedupe is for collapsing bursts of identical reads.

Caching

tCache({ key, ttl, tags, condition })

OptionTypeDefaultDescription
keykey specrequiredCache key
ttlnumberrequiredTime to live, ms
tagsstring[][]Tags for group invalidation
condition(result) => booleancache everythingStore the result only when it returns true

A hit returns the stored value without calling the method. Errors are never cached.

Cache keys. A plain string without {, : or an args. prefix is treated as a name, and the key becomes name: followed by the JSON of the arguments, so @tCache({ key: 'revenue', ttl: 60_000 }) caches one entry per argument list. Templates ('price:{0}'), args. paths, header: specs and functions resolve as described under key specs. Cache keys are not scoped to the method, which is what lets tCachePut and tCacheEvict on other methods address the same entries.

tCachePut, tCacheEvict, tCacheInvalidate

DecoratorAfter a successful call
tCachePut({ key, ttl, tags })Always runs the method and stores its result under key
tCacheEvict({ key, tags, allEntries })Deletes key and every entry tagged with tags; with allEntries: true, clears the whole cache
tCacheInvalidate(...tags)Deletes every entry tagged with any of tags

Nothing is evicted when the method throws.

TypeScript
TS
1class Products {
2  @tCache({ key: 'product:{0}', ttl: 300_000, tags: ['products'] })
3  async get(id: string) { return this.repo.findById(id) }
4
5  @tCachePut({ key: 'product:{0}', ttl: 300_000, tags: ['products'] })
6  async update(id: string, patch: Partial<Product>) { return this.repo.update(id, patch) }
7
8  @tCacheEvict({ key: 'product:{0}' })
9  async remove(id: string) { await this.repo.delete(id) }
10
11  @tCacheInvalidate('products')
12  async importCatalog(file: string) { /* ... */ }
13}

The default store is MemoryCacheStore (up to 10,000 entries per process; the oldest entry is dropped when full). getCacheStore() returns the current store. For a cache shared by every instance, use useStores({ cache }) or setCacheStore().

tCache reports a cache.read side effect, which a tReadOnly method may perform but a tPure method may not. See Intent verification.

Rate limiting

tRateLimit({ limit, window, key })

A fixed-window limit per key.

OptionTypeDefaultDescription
limitnumberrequiredCalls allowed per window
windowstringrequiredWindow length: '500ms', '10s', '1m', '1h', '1d'
keystringone shared counter'ip', 'user', 'tenant', or another key spec
TypeScript
TS
1class Exports {
2  @tRateLimit({ limit: 5, window: '1m', key: 'user' })
3  async generate(format: string) { /* ... */ }
4}

Over the limit, the call fails with RateLimitExceededError (429), whose message says when to retry (Exports.generate: rate limit exceeded, retry in 42s) and whose retryAfterMs gives the exact time. Each rejection emits runtime.rate_limited with { target, key, retryAfterMs }. Without key, all callers share one counter; when the key cannot be resolved (no user, for example), the caller counts as anonymous. Counters are per method.

tThrottle(calls, periodMs)

At most calls executions per periodMs for the method, across all callers. Calls over the limit are rejected with RateLimitExceededError, not delayed.

TypeScript
TS
1class Sync {
2  @tThrottle(10, 1000)    // at most 10 per second
3  async push(batch: unknown[]) { /* ... */ }
4}

Both decorators count in memory by default. With several workers or instances each one counts separately, so the effective limit is multiplied. Install a shared counter store with useStores({ rateLimit }) or setRateLimitStore(); the ClusterStore from @axiljs/security and the SQL store from @axiljs/stores both fit.

tDebounce(ms)

Collapses a burst of calls into one. Each call restarts the timer; when no call has arrived for ms, the method runs once with the arguments of the last call, and every caller in the burst receives that result (or that error). Bursts are tracked per instance and method.

TypeScript
TS
1class SearchIndex {
2  @tDebounce(250)
3  async rebuild(reason: string) { /* ... */ }
4}

Key specs

Keyed decorators (tIdempotent, tCache, tCachePut, tCacheEvict, tDedupe, tLock, tMutex, tConcurrency, tRateLimit) take a key spec. It is either a function or a string:

SpecResolves to
(...args) => stringThe function's return value, called with the method arguments (and the instance as this)
'request.id'The request ID from the request context
'header:<name>'A request header, e.g. 'header:idempotency-key'
'user'The authenticated user's ID
'tenant'The current tenant ID
'ip'The client IP
'args.0.id'A path into the arguments
'order:{0.id}'A template; each {…} is a path into the arguments
'global'One shared key for every call
any other stringUsed literally

'user.id', 'tenant.id', 'tenantId' and 'requestId' are accepted as aliases. A spec cannot be resolved when the value is missing (no such header, no user, a template path that is undefined); what happens then depends on the decorator: tIdempotent records a violation, tRateLimit counts the caller as anonymous, tLock uses the method name, tConcurrency and tMutex fall back to one limit for the method, and tCache and tDedupe simply run the method. Remember the tCache naming rule for plain strings.

resolveKey(spec, invocation) and parseDuration(value) are exported for custom interceptors.

Interceptor order

Every runtime interceptor has a fixed order. Lower runs outside higher, regardless of the order the decorators are written in.

OrderDecoratorPage
50tDeprecatedIntent
60tSloIntent
70tPure, tReadOnlyIntent
75tSideEffectIntent
80tTenantIsolatedIntent
90tIdempotentIntent
100tAuditInterceptors
110tRateLimitthis page
115tThrottlethis page
120tDeadlineIntent
125tDebouncethis page
130tCachethis page
135tCachePut, tCacheEvict, tCacheInvalidatethis page
140tDedupethis page
150tFallbackthis page
200tRetryInterceptors
250tCircuitBreakerthis page
260tBulkheadthis page
265tConcurrencythis page
270tMutexthis page
275tLockthis page
290tHedgethis page
300tTimeoutInterceptors

Consequences worth knowing:

  • An idempotent replay (90) returns before rate limits, the cache, retries or locks run.
  • A cache hit (130) returns before retries, the breaker and locks, but has already been counted by tRateLimit (110).
  • tFallback catches the final error after all retries.
  • Each retry attempt passes the circuit breaker, bulkhead, concurrency limit and lock again, and gets its own timeout.
  • Each hedged attempt (290) gets its own tTimeout (300).

enforcedDecorators() returns the registered interceptors in this order, outermost first, including any you register yourself:

TypeScript
TS
1import { enforcedDecorators } from '@axiljs/runtime'
2
3enforcedDecorators()
4// [{ decorator: 'tDeprecated', order: 50 }, { decorator: 'tSlo', order: 60 }, ..., { decorator: 'tTimeout', order: 300 }]

axil verify uses this list to tell enforced decorators from metadata-only ones.

Events

EventPayload
runtime.circuit{ target, from, to }
runtime.fallback{ target, error }
runtime.hedge{ target, attempt }
runtime.rate_limited{ target, key, retryAfterMs }

Events go to the global event bus from @axiljs/events and never break the business call.

Testing

resetResilienceState() clears all in-process resilience state (rate-limit windows, deduplication, circuits, bulkheads, limits, mutexes) and restores the default lock provider, cache store and rate-limit counters. Call it between tests that share decorated classes.

The interceptors are exported individually (circuitBreakerInterceptor, bulkheadInterceptor, cacheInterceptor, …), as are Semaphore and MemoryCacheStore, so you can replace one with registerInterceptor() or reuse its building blocks.

Was this page helpful?

Help improve the documentation

AxilJS is open source and documentation improvements are welcome.

AxilJS DocumentationMIT License · Built by SyntaxilitY