Testing

Contract Testing

Turn declared intent into HTTP-level tests with expectIdempotent, expectTenantIsolated and expectNoIntentViolations from @axiljs/testing, with options, reports and failure messages.

7 min readUpdated DocumentationEdit this page
On this page11

Overview

Contract helpers turn declared intent into executable checks. They run over HTTP through createTestClient(app), so they test the same pipeline your users hit: middleware, authentication, validation, the handler and the database.

Note

New in 0.3.5: expectIdempotent, expectTenantIsolated and expectNoIntentViolations.

TypeScript
TS
1import {
2  createTestClient,
3  expectIdempotent,
4  expectNoIntentViolations,
5  expectTenantIsolated
6} from '@axiljs/testing'
7
8const client = createTestClient(app)

Requests are described as data, so the helpers can replay them:

TypeScript
TS
1interface RequestSpec {
2  method?: 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE'   // case-insensitive, default 'GET'
3  path: string
4  headers?: Record<string, string>
5  body?: unknown
6}

Every helper resolves with a report object. On failure it throws a ContractViolationError, a subclass of AssertionError (its name is 'AssertionError'), whose message explains what diverged. The error carries contract ('idempotent', 'tenant-isolated' or 'no-intent-violations') and the report as error.report.

expectIdempotent

Asserts that replaying a request yields the same status, the same body (minus volatile fields) and, optionally, the same server-side state.

TypeScript
TS
1await expectIdempotent(client, { method: 'POST', path: '/orders', body: { sku: 'A1' } }, {
2  times: 3,                                  // default 2
3  concurrent: true,                          // first call alone, the rest in parallel
4  ignore: ['timestamp', 'data.*.updatedAt'], // volatile fields
5  snapshot: () => db.query('SELECT count(*) AS n FROM orders')
6})

The same request is sent times times with one generated Idempotency-Key header. Every response must have the status and body of the first call. With snapshot, the state observed after the first call must equal the state after the last.

Options

OptionTypeDefaultDescription
timesnumber2Total number of identical requests (minimum 2)
concurrentbooleanfalseSend the first request alone, then the remaining times - 1 at the same time, so the replays race each other
ignorestring[]['timestamp', 'requestId', 'meta.timestamp']Dotted body paths left out of the comparison
snapshot() => unknownnoneObserve server-side state (for example a row count); called after the first request and after the last, and both results must be deep-equal
idempotencyKeyboolean | stringtruetrue generates a UUID, a string uses that key, false sends no key

If the request already has an Idempotency-Key header (any case), the helper keeps it and adds none.

Ignored paths. * matches any key or array index, so data.*.updatedAt ignores updatedAt in every element of data. For an AxilJS envelope ({ status, code, message, data }) each path is applied both from the root and inside data, so the default timestamp also matches data.timestamp. The default list is exported as DEFAULT_IDEMPOTENCY_IGNORE.

Failure messages

Output
TXT
AssertionError: Expected POST /counter (2x, sequential) to be idempotent:
call #2 body differs from call #1 (ignoring timestamp, requestId, meta.timestamp):
  - data.count: 1 != 2

Up to five differing paths are listed. The other failures read:

Output
TXT
Expected POST /orders (3x, concurrent) to be idempotent: call #2 returned status 409, call #1 returned 201
Expected POST /orders (2x, sequential) to be idempotent: state changed after the first call: snapshot [{"n":1}] after call #1, [{"n":2}] after call #2

Report

TypeScript
TS
1const report = await expectIdempotent(client, { method: 'POST', path: '/orders', body: { sku: 'A1' } })
2
3report.idempotencyKey   // the key that was sent
4report.responses        // [{ status, body }, ...] in order
5report.snapshot         // { afterFirst, afterAll } when snapshot was given

The report also has request, times, concurrent and ignored.

Tip

On the server, tIdempotent({ key: 'header:idempotency-key' }) is the decorator this helper checks. Use concurrent: true to test that duplicates arriving while the first call is still running are handled too.

expectTenantIsolated

Asserts that each tenant's listing returns only that tenant's rows, and that cross-tenant requests are refused.

TypeScript
TS
1await expectTenantIsolated(client, {
2  seed: [
3    { tenant: 'acme',   as: { authorization: `Bearer ${acmeToken}` } },
4    { tenant: 'globex', as: { headers: { authorization: `Bearer ${globexToken}` } } }
5  ],
6  read: { path: '/invoices' },
7  tenantOf: row => row.tenantId,
8  forbidden: [{ path: `/invoices/${globexInvoiceId}`, as: { authorization: `Bearer ${acmeToken}` } }]
9})

The helper does not create data; seed each tenant's rows before calling it.

Options

OptionTypeDescription
seed{ tenant, as }[]Required, at least one. tenant is the expected owner ID, as the credentials that act as that tenant
readRequestSpecThe listing request, performed once per tenant
tenantOf(row) => string | undefined | nullReturns the tenant that owns a row
rows(body) => unknown[]Extracts rows from the response body (default below)
forbiddenForbiddenProbe[]Cross-tenant requests that must be refused

as is either a plain header map ({ authorization: 'Bearer …' }) or a partial RequestSpec merged over the base request ({ headers: { … }, path: '/other' }).

For each tenant, read must succeed with a 2xx status, and tenantOf(row) must equal that tenant for every returned row. Rows are found in a plain array, { data: [...] }, or the paginated { data: { metadata: [...] } } shape; pass rows for anything else. The default extraction is exported as extractRows(body).

Each probe in forbidden has path, as (the tenant that must not see the resource), an optional method (default GET) and an optional expect list of acceptable statuses (default [403, 404]).

Failure messages

Output
TXT
Expected tenant isolation: GET /invoices as tenant "acme" returned 2 of 5 row(s) owned by "globex":
  - {"id":7,"tenantId":"globex","total":120}
  - {"id":9,"tenantId":"globex","total":80}

Up to three foreign rows are shown. The other failures read:

Output
TXT
Expected tenant isolation: GET /invoices as tenant "acme" returned 401 (expected 2xx): {...}
Expected tenant isolation: could not find rows in the response to GET /invoices as tenant "acme"; pass options.rows(body). Body: {...}
Expected tenant isolation: cross-tenant GET /invoices/42 returned 200 (expected 403 or 404): {...}

The report has tenants ({ tenant, status, rows, foreign } per tenant) and forbidden ({ request, status } per probe).

Tip

tTenantIsolated() checks the same promise inside the service layer on every call. The two complement each other: the decorator catches leaks in any code path, the contract test proves the HTTP API as a whole.

expectNoIntentViolations

Runs a function and fails if @axiljs/runtime recorded new intent violations while it ran.

TypeScript
TS
await expectNoIntentViolations(async () => {
  await client.post('/reports').send({ range: 'month' })
})

The helper reads the runtime's report from globalThis[Symbol.for('axiljs.runtime.report')] (exported as RUNTIME_REPORT_KEY), compares the violations before and after, and fails with a list of the new ones:

Output
TXT
AssertionError: Expected no intent violations, but 1 were recorded:
  - Reports.generate violates tReadOnly: performed side effect db.write (INSERT INTO report_runs ...)
  • It also fails when the function threw and violations were recorded; the message then ends with (the function also threw: <message>).
  • When no new violations were recorded and the function threw, the original error is re-thrown.
  • It resolves with the function's result.
  • Without @axiljs/runtime loaded, it simply runs the function and returns its result. @axiljs/testing does not depend on the runtime.

This matters in warn mode, where a violation does not throw: a request can succeed with a 200 and still have broken a promise. In enforce mode the violation usually surfaces as a 500 response too.

Note

The runtime caps its violation list at 500 entries, so the helper compares the lists by content rather than by length.

In CI

axil verify runs your test suite with AXIL_INTENT_MODE=enforce and lists any intent violations by test name, next to a static check of which decorators are enforced.

Next steps

Was this page helpful?

Help improve the documentation

AxilJS is open source and documentation improvements are welcome.

AxilJS DocumentationMIT License · Built by SyntaxilitY