Runtime

Shared Stores

Make tIdempotent, tCache, tLock, tRateLimit and tThrottle correct behind a load balancer with useStores(): the store interfaces, the individual setters and custom backends.

6 min readUpdated DocumentationEdit this page
On this page12

Several runtime decorators keep state between calls: tIdempotent remembers results, tCache keeps entries, tLock holds leases, and tRateLimit / tThrottle count calls. By default that state lives in the memory of one process.

Note

New in 0.3.6: useStores() installs all four stores in one call. New in 0.3.5: setRateLimitStore(), and SQL implementations of every store in @axiljs/stores.

Why in-memory state breaks with several instances

In-memory state is correct for one process and wrong as soon as you run several, whether as cluster workers on one machine or as instances behind a load balancer:

DecoratorWith per-process state
tIdempotentA retried payment reaches a process that has not seen its idempotency key and charges again
tLockTwo processes each hold "the" lock for the same key at the same time
tRateLimit, tThrottleA limit of 100 per minute becomes 100 per minute per process
tCacheEach process has its own cache; an eviction in one process leaves stale entries in the others

A shared store fixes this by keeping the state where every process can see it.

useStores()

Install shared stores once at startup, before the first decorated call:

TypeScript
TS
1import { createConnection } from '@axiljs/orm'
2import { createSqlStores, fromConnection } from '@axiljs/stores'
3import { useStores } from '@axiljs/runtime'
4
5const db = await createConnection({ driver: 'postgres', url: process.env.DATABASE_URL })
6
7const stores = createSqlStores(fromConnection(db))
8await stores.ensureSchema()
9useStores(stores)        // tRateLimit, tThrottle, tIdempotent, tCache*, tLock

useStores() accepts an object with any of these members and ignores everything else:

MemberUsed byInterface
rateLimittRateLimit, tThrottleRateLimitCounterStore
idempotencytIdempotentIdempotencyStore
cachetCache, tCachePut, tCacheEvict, tCacheInvalidateCacheStore
locktLockLockProvider

Members you leave out keep their current implementation (in-process by default). The function returns the names of the stores it installed, for example ['rateLimit', 'idempotency', 'cache', 'lock'].

Because the stores are matched by shape, not by class, a Redis client wrapper, a call to another service or a test double plugs in the same way:

TypeScript
TS
1useStores({
2  cache: myRedisCache,          // implements CacheStore
3  lock: myRedisLocks            // implements LockProvider
4})

The SQL implementations for PostgreSQL, MySQL 8 and SQLite, createSqlStores() and its options are documented under Stores.

Individual setters

useStores() calls these; they remain available:

SetterStore
setRateLimitStore(store | undefined)Rate-limit counters; undefined restores in-process counting
setIdempotencyStore(store)Idempotency results
setCacheStore(store)Cache entries (getCacheStore() returns the current one)
setLockProvider(provider)Locks
TypeScript
TS
1import { setCacheStore, setIdempotencyStore } from '@axiljs/runtime'
2import { SqlCacheStore, SqlIdempotencyStore } from '@axiljs/stores'
3
4setIdempotencyStore(new SqlIdempotencyStore(exec))
5setCacheStore(new SqlCacheStore(exec))

Warning

Stores are read when a decorated method is called, not when it is wired, so the order of wire() and useStores() does not matter. But a call made before useStores() uses the in-process store, and its state is not copied over. Install stores before the app starts serving requests.

Store interfaces

Every method may return a value or a promise. The runtime awaits them.

RateLimitCounterStore

TypeScript
TS
interface RateLimitCounterStore {
  increment(key: string, windowMs: number): { count: number; resetAt: number } | Promise<{ count: number; resetAt: number }>
}

Count one hit for key and return the count in the current window and when it resets (epoch ms). If no window is active, start one of windowMs. The shape matches RateLimitStore from @axiljs/security, so its ClusterStore (one host, several workers) and the SQL store can be used directly.

IdempotencyStore

TypeScript
TS
1interface IdempotencyStore {
2  get(key: string): unknown | Promise<unknown>
3  set(key: string, value: { result: unknown }, ttlMs: number): void | Promise<void>
4  delete?(key: string): void | Promise<void>
5}

set stores a completed result as { result } for ttlMs; get returns that object or undefined when the key is unknown or expired. Keys look like Payments.charge:<idempotency key>. Results must be JSON-serializable for a shared store. Failed calls are never stored. The default is MemoryIdempotencyStore.

CacheStore

TypeScript
TS
1interface CacheStore {
2  get(key: string): { value: unknown } | undefined | Promise<{ value: unknown } | undefined>
3  set(key: string, value: unknown, ttlMs: number, tags: string[]): void | Promise<void>
4  delete(key: string): void | Promise<void>
5  deleteByTag(tag: string): void | Promise<void>
6  clear(): void | Promise<void>
7}

get wraps the value in { value }, so a cached undefined or null is still a hit. deleteByTag backs tCacheInvalidate and tCacheEvict({ tags }); clear backs tCacheEvict({ allEntries: true }). The default is MemoryCacheStore (10,000 entries per process).

LockProvider

TypeScript
TS
interface LockProvider {
  acquire(key: string, options: { ttlMs: number; waitMs: number }): Promise<() => void | Promise<void>>
}

Resolve with a release function once the lock for key is held, or reject when waitMs elapses (the runtime turns any rejection into LockTimeoutError, 409). The lock should expire after ttlMs even if it is never released. Keys are passed as lock:<key>. The default provider locks within one process.

Writes made by stores

A database-backed store writes rows: a cache entry, an idempotency record, a lock lease. Those writes happen on behalf of the decorator, not the business method, so they must not count as the method's side effects inside tReadOnly or tPure. The SQL stores from @axiljs/stores already run every statement inside runWithoutSideEffectReports(). Do the same in a custom store that writes through @axiljs/orm:

TypeScript
TS
1import { runWithoutSideEffectReports } from '@axiljs/runtime'
2
3const cache: CacheStore = {
4  async set(key, value, ttlMs, tags) {
5    await runWithoutSideEffectReports(() => db.query('INSERT INTO app_cache ...', [key, JSON.stringify(value)]))
6  },
7  // ...
8}

What stays per process

Some state has no store and is always per process: circuit-breaker states, bulkhead and concurrency slots, tMutex, tDedupe, tDebounce, tSlo measurements, recorded intent violations, and joining a tIdempotent call that is still running. Completed idempotency results are shared once a store is installed.

In the templates

Since 0.3.6 the saas-api and high-scale-api templates open a DATABASE_URL connection (SQLite by default, PostgreSQL or MySQL by URL), install the SQL stores with useStores(), call wireAll() and close everything on shutdown:

Terminal
❯axil create my-api --template saas-api

Was this page helpful?

Help improve the documentation

AxilJS is open source and documentation improvements are welcome.

AxilJS DocumentationMIT License · Built by SyntaxilitY