Shared Stores
Make tIdempotent, tCache, tLock, tRateLimit and tThrottle correct behind a load balancer with useStores(): the store interfaces, the individual setters and custom backends.
On this page12
Several runtime decorators keep state between calls: tIdempotent remembers results, tCache keeps entries, tLock holds leases, and tRateLimit / tThrottle count calls. By default that state lives in the memory of one process.
Note
New in 0.3.6: useStores() installs all four stores in one call. New in 0.3.5: setRateLimitStore(), and SQL implementations of every store in @axiljs/stores.
Why in-memory state breaks with several instances
In-memory state is correct for one process and wrong as soon as you run several, whether as cluster workers on one machine or as instances behind a load balancer:
| Decorator | With per-process state |
|---|---|
tIdempotent | A retried payment reaches a process that has not seen its idempotency key and charges again |
tLock | Two processes each hold "the" lock for the same key at the same time |
tRateLimit, tThrottle | A limit of 100 per minute becomes 100 per minute per process |
tCache | Each process has its own cache; an eviction in one process leaves stale entries in the others |
A shared store fixes this by keeping the state where every process can see it.
useStores()
Install shared stores once at startup, before the first decorated call:
1import { createConnection } from '@axiljs/orm'
2import { createSqlStores, fromConnection } from '@axiljs/stores'
3import { useStores } from '@axiljs/runtime'
4
5const db = await createConnection({ driver: 'postgres', url: process.env.DATABASE_URL })
6
7const stores = createSqlStores(fromConnection(db))
8await stores.ensureSchema()
9useStores(stores) // tRateLimit, tThrottle, tIdempotent, tCache*, tLockuseStores() accepts an object with any of these members and ignores everything else:
| Member | Used by | Interface |
|---|---|---|
rateLimit | tRateLimit, tThrottle | RateLimitCounterStore |
idempotency | tIdempotent | IdempotencyStore |
cache | tCache, tCachePut, tCacheEvict, tCacheInvalidate | CacheStore |
lock | tLock | LockProvider |
Members you leave out keep their current implementation (in-process by default). The function returns the names of the stores it installed, for example ['rateLimit', 'idempotency', 'cache', 'lock'].
Because the stores are matched by shape, not by class, a Redis client wrapper, a call to another service or a test double plugs in the same way:
1useStores({
2 cache: myRedisCache, // implements CacheStore
3 lock: myRedisLocks // implements LockProvider
4})The SQL implementations for PostgreSQL, MySQL 8 and SQLite, createSqlStores() and its options are documented under Stores.
Individual setters
useStores() calls these; they remain available:
| Setter | Store |
|---|---|
setRateLimitStore(store | undefined) | Rate-limit counters; undefined restores in-process counting |
setIdempotencyStore(store) | Idempotency results |
setCacheStore(store) | Cache entries (getCacheStore() returns the current one) |
setLockProvider(provider) | Locks |
1import { setCacheStore, setIdempotencyStore } from '@axiljs/runtime'
2import { SqlCacheStore, SqlIdempotencyStore } from '@axiljs/stores'
3
4setIdempotencyStore(new SqlIdempotencyStore(exec))
5setCacheStore(new SqlCacheStore(exec))Warning
Stores are read when a decorated method is called, not when it is wired, so the order of wire() and useStores() does not matter. But a call made before useStores() uses the in-process store, and its state is not copied over. Install stores before the app starts serving requests.
Store interfaces
Every method may return a value or a promise. The runtime awaits them.
RateLimitCounterStore
interface RateLimitCounterStore {
increment(key: string, windowMs: number): { count: number; resetAt: number } | Promise<{ count: number; resetAt: number }>
}Count one hit for key and return the count in the current window and when it resets (epoch ms). If no window is active, start one of windowMs. The shape matches RateLimitStore from @axiljs/security, so its ClusterStore (one host, several workers) and the SQL store can be used directly.
IdempotencyStore
1interface IdempotencyStore {
2 get(key: string): unknown | Promise<unknown>
3 set(key: string, value: { result: unknown }, ttlMs: number): void | Promise<void>
4 delete?(key: string): void | Promise<void>
5}set stores a completed result as { result } for ttlMs; get returns that object or undefined when the key is unknown or expired. Keys look like Payments.charge:<idempotency key>. Results must be JSON-serializable for a shared store. Failed calls are never stored. The default is MemoryIdempotencyStore.
CacheStore
1interface CacheStore {
2 get(key: string): { value: unknown } | undefined | Promise<{ value: unknown } | undefined>
3 set(key: string, value: unknown, ttlMs: number, tags: string[]): void | Promise<void>
4 delete(key: string): void | Promise<void>
5 deleteByTag(tag: string): void | Promise<void>
6 clear(): void | Promise<void>
7}get wraps the value in { value }, so a cached undefined or null is still a hit. deleteByTag backs tCacheInvalidate and tCacheEvict({ tags }); clear backs tCacheEvict({ allEntries: true }). The default is MemoryCacheStore (10,000 entries per process).
LockProvider
interface LockProvider {
acquire(key: string, options: { ttlMs: number; waitMs: number }): Promise<() => void | Promise<void>>
}Resolve with a release function once the lock for key is held, or reject when waitMs elapses (the runtime turns any rejection into LockTimeoutError, 409). The lock should expire after ttlMs even if it is never released. Keys are passed as lock:<key>. The default provider locks within one process.
Writes made by stores
A database-backed store writes rows: a cache entry, an idempotency record, a lock lease. Those writes happen on behalf of the decorator, not the business method, so they must not count as the method's side effects inside tReadOnly or tPure. The SQL stores from @axiljs/stores already run every statement inside runWithoutSideEffectReports(). Do the same in a custom store that writes through @axiljs/orm:
1import { runWithoutSideEffectReports } from '@axiljs/runtime'
2
3const cache: CacheStore = {
4 async set(key, value, ttlMs, tags) {
5 await runWithoutSideEffectReports(() => db.query('INSERT INTO app_cache ...', [key, JSON.stringify(value)]))
6 },
7 // ...
8}What stays per process
Some state has no store and is always per process: circuit-breaker states, bulkhead and concurrency slots, tMutex, tDedupe, tDebounce, tSlo measurements, recorded intent violations, and joining a tIdempotent call that is still running. Completed idempotency results are shared once a store is installed.
In the templates
Since 0.3.6 the saas-api and high-scale-api templates open a DATABASE_URL connection (SQLite by default, PostgreSQL or MySQL by URL), install the SQL stores with useStores(), call wireAll() and close everything on shutdown:
Related
- Stores: the SQL implementations and
createSqlStores() - Resilience:
tCache,tLock,tRateLimitandtThrottle - Intent verification:
tIdempotent - Multi-process readiness
Was this page helpful?