Before you ask an agent to change code, pin down its current behaviour with characterization tests, and describe the critical paths — payment, lead forms, access rules — with scenario tests backed by an explicit table of expectations. These tests fail when the agent's next edit "simplifies" a check or silently changes a result. To keep them from being bypassed quietly, run them in pre-commit and in CI, and let the linter catch skipped and empty tests.
After this you will be able to
- write a characterization test with
toMatchInlineSnapshotand read the snapshot as a list of decisions for the product owner - describe a critical path (payment, order cancellation) with scenario tests that check side effects and the failure path
- build a "role × owner × status" permission matrix that fails when the check is "simplified"
- make the run mandatory:
npm run checkin a husky pre-commit hook and in GitHub Actions, plus lint rules against.skip,.onlyand tests withoutexpect
Know beforehand
Verified with
Node.js 24 LTS (24.18)Vitest 5.0.4TypeScript 6.0.3ESLint 10.12, typescript-eslint 8.71, @vitest/eslint-plugin 1.6.27husky 9.1.7actionlint 1.7.12
Checked against docs: October 12, 2026
Contents
Why you need tests before the agent's next edit
An assistant changes code quickly and confidently, but it only knows what it sees in the current session. An edit in one place easily breaks a rule somewhere else: a discount, an access check, how a failed payment is handled. Neither you nor the agent keeps all of those rules in mind, and without tests the regression shows up in front of users.
The Claude Code documentation opens its advice the same way: give the agent a check it can run itself — tests, a build, a linter. Without one, the only signal is "looks done", and you become the verification loop. With one, the agent runs the check, reads the result and iterates until it passes. A check only helps, though, if it checks the right things and can't be quietly weakened. That is what this lesson is about.
Characterization
- What it pins down
- The function's current behaviour on a set of inputs, whatever it is
- What it catches
- Any unnoticed change in output, including quirks "fixed" along the way
Scenario
- What it pins down
- A business promise: the customer is charged once, a cancelled order can't be paid
- What it catches
- An edit after which the path still works but breaks the rule
Guard test
- What it pins down
- The full table of allowed combinations, written by hand
- What it catches
- A "simplified" condition in an access check
The practice module is the orders part of a small online shop, the way an assistant might have written it: a total with a promo code, an order cancellation rule and payment through a gateway. In the tests the gateway is replaced with vi.fn(); there are no real payments or keys.
npm init -y
npm pkg set type=module
npm install -D vitest@5.0.4 typescript@6.0.3 @types/node@24.19.2 \
eslint@10.12.0 typescript-eslint@8.71.1 @vitest/eslint-plugin@1.6.27 husky@9.1.7export type Role = "guest" | "client" | "manager" | "admin";
export type User = { id: string; role: Role };
export type OrderStatus = "new" | "paid" | "cancelled";
export type Order = { id: string; ownerId: string; status: OrderStatus; totalKzt: number };
export type Item = { priceKzt: number; qty: number };
const PROMOS: Record<string, number> = { WELCOME10: 10, VIP25: 25 };
export function calcTotal(items: Item[], promo?: string): number {
const sum = items.reduce((acc, item) => acc + item.priceKzt * item.qty, 0);
const percent = promo ? (PROMOS[promo.trim().toUpperCase()] ?? 0) : 0;
return Math.round((sum * (100 - percent)) / 100);
}
export function canCancel(user: User, order: Order): boolean {
if (order.status !== "new") return false;
if (user.role === "admin" || user.role === "manager") return true;
return user.role === "client" && user.id === order.ownerId;
}import type { Order } from "./orders.js";
export interface PaymentGateway {
charge(orderId: string, amountKzt: number, idempotencyKey: string): Promise<{ paymentId: string }>;
}
export type Deps = { orders: Map<string, Order>; gateway: PaymentGateway };
export async function payOrder(orderId: string, deps: Deps): Promise<Order> {
const order = deps.orders.get(orderId);
if (!order) throw new Error("ORDER_NOT_FOUND");
if (order.status === "paid") return order;
if (order.status !== "new") throw new Error("ORDER_NOT_PAYABLE");
if (order.totalKzt <= 0) throw new Error("INVALID_AMOUNT");
await deps.gateway.charge(order.id, order.totalKzt, `order-${order.id}`);
const paid: Order = { ...order, status: "paid" };
deps.orders.set(order.id, paid);
return paid;
}A characterization test: a snapshot of what is there
A characterization test (the term comes from Michael Feathers's book on working with legacy code) doesn't ask whether the code is right. It records how the code behaves now. For someone else's or generated code this is an honest starting point: you don't know all of the author's intentions yet, but you can already notice any change.
An inline snapshot is the easiest way to write one. Collect interesting inputs — normal, empty, edge, odd — into one object and call toMatchInlineSnapshot() with no argument:
import { expect, it } from "vitest";
import { calcTotal } from "../../src/orders.js";
// Characterization test: records what calcTotal does today,
// including behaviour nobody has decided on yet.
it("calcTotal: current behaviour", () => {
const cases = {
"one item": calcTotal([{ priceKzt: 5000, qty: 1 }]),
"two items": calcTotal([{ priceKzt: 5000, qty: 2 }, { priceKzt: 1500, qty: 1 }]),
"empty cart": calcTotal([]),
"promo WELCOME10": calcTotal([{ priceKzt: 5000, qty: 1 }], "WELCOME10"),
"promo lower case + spaces": calcTotal([{ priceKzt: 5000, qty: 1 }], " vip25 "),
"unknown promo": calcTotal([{ priceKzt: 5000, qty: 1 }], "FREE100"),
"rounding 0.5": calcTotal([{ priceKzt: 5, qty: 1 }], "WELCOME10"),
"negative qty": calcTotal([{ priceKzt: 5000, qty: -1 }]),
};
expect(cases).toMatchInlineSnapshot();
});$ npx vitest run
Snapshots 1 written
Test Files 1 passed (1)
Tests 1 passed (1)import { expect, it } from "vitest";
import { calcTotal } from "../../src/orders.js";
// Characterization test: records what calcTotal does today,
// including behaviour nobody has decided on yet.
it("calcTotal: current behaviour", () => {
const cases = {
"one item": calcTotal([{ priceKzt: 5000, qty: 1 }]),
"two items": calcTotal([{ priceKzt: 5000, qty: 2 }, { priceKzt: 1500, qty: 1 }]),
"empty cart": calcTotal([]),
"promo WELCOME10": calcTotal([{ priceKzt: 5000, qty: 1 }], "WELCOME10"),
"promo lower case + spaces": calcTotal([{ priceKzt: 5000, qty: 1 }], " vip25 "),
"unknown promo": calcTotal([{ priceKzt: 5000, qty: 1 }], "FREE100"),
"rounding 0.5": calcTotal([{ priceKzt: 5, qty: 1 }], "WELCOME10"),
"negative qty": calcTotal([{ priceKzt: 5000, qty: -1 }]),
};
expect(cases).toMatchInlineSnapshot(`
{
"empty cart": 0,
"negative qty": -5000,
"one item": 5000,
"promo WELCOME10": 4500,
"promo lower case + spaces": 3750,
"rounding 0.5": 5,
"two items": 11500,
"unknown promo": 5000,
}
`);
});Now read the snapshot as a list of decisions someone made for you. A negative quantity gives a negative total. An unknown promo code is silently ignored. Half a tenge is rounded up. All of that may be correct or not, but now it's visible and a person makes the call. Write the quirks down as separate tasks rather than fixing them "while you're at it".
$ CI=true npx vitest run
FAIL tests/characterization/calc-total.test.ts > calcTotal: current behaviour
Error: Snapshot `calcTotal: current behaviour 1` mismatched
❯ tests/characterization/calc-total.test.ts:17:17
17| expect(cases).toMatchInlineSnapshot();
| ^
Snapshots 1 failed
Test Files 1 failed (1)Let's make sure the test does its job. Say the agent was asked to "clean up" the calculation and banned negative quantities along the way:
@@ -7,7 +7,7 @@ export type Item = { priceKzt: number; qty: number };
const PROMOS: Record<string, number> = { WELCOME10: 10, VIP25: 25 };
export function calcTotal(items: Item[], promo?: string): number {
- const sum = items.reduce((acc, item) => acc + item.priceKzt * item.qty, 0);
+ const sum = items.reduce((acc, item) => acc + item.priceKzt * Math.max(0, item.qty), 0);
const percent = promo ? (PROMOS[promo.trim().toUpperCase()] ?? 0) : 0;
return Math.round((sum * (100 - percent)) / 100);
}$ npx vitest run tests/characterization
FAIL tests/characterization/calc-total.test.ts > calcTotal: current behaviour
Error: Snapshot `calcTotal: current behaviour 1` mismatched
- Expected
+ Received
@@ -1,8 +1,8 @@
{
"empty cart": 0,
- "negative qty": -5000,
+ "negative qty": 0,
"one item": 5000,
Snapshots 1 failed
Test Files 1 failed (1)The test doesn't say the new behaviour is wrong. It says behaviour changed and stops the edit until a person decides. If the change is intended, update the snapshot with npx vitest run -u in the same commit and explain why in the pull request. If not, revert the edit.
Scenario tests for critical paths
A snapshot records values but doesn't say which of them matter. For paths where a mistake costs money or access — payment, lead forms, permissions — you need tests named in business language: "a repeated payment doesn't charge twice", "the order stays unpaid if the gateway fails". Such a test reads like a requirement, and nobody should rewrite it just to get a pass.
import { beforeEach, describe, expect, it, vi } from "vitest";
import type { Order } from "../../src/orders.js";
import { payOrder, type Deps } from "../../src/payments.js";
function setup(order: Order) {
const charge = vi.fn(async () => ({ paymentId: "pay_test_1" }));
const deps: Deps = { orders: new Map([[order.id, order]]), gateway: { charge } };
return { deps, charge };
}
describe("payment of an order", () => {
let order: Order;
beforeEach(() => {
order = { id: "o-1", ownerId: "u-1", status: "new", totalKzt: 4500 };
});
it("charges the exact total once and marks the order as paid", async () => {
const { deps, charge } = setup(order);
const paid = await payOrder("o-1", deps);
expect(paid.status).toBe("paid");
expect(charge).toHaveBeenCalledTimes(1);
expect(charge).toHaveBeenCalledWith("o-1", 4500, "order-o-1");
});
it("a repeated payment does not charge the customer twice", async () => {
const { deps, charge } = setup(order);
await payOrder("o-1", deps);
await payOrder("o-1", deps);
expect(charge).toHaveBeenCalledTimes(1);
});
it("a cancelled order cannot be paid", async () => {
const { deps, charge } = setup({ ...order, status: "cancelled" });
await expect(payOrder("o-1", deps)).rejects.toThrow("ORDER_NOT_PAYABLE");
expect(charge).not.toHaveBeenCalled();
});
it("an order with a zero or negative total is never sent to the gateway", async () => {
const { deps, charge } = setup({ ...order, totalKzt: -5000 });
await expect(payOrder("o-1", deps)).rejects.toThrow("INVALID_AMOUNT");
expect(charge).not.toHaveBeenCalled();
});
it("the order stays unpaid if the gateway fails", async () => {
const { deps, charge } = setup(order);
charge.mockRejectedValueOnce(new Error("GATEWAY_TIMEOUT"));
await expect(payOrder("o-1", deps)).rejects.toThrow("GATEWAY_TIMEOUT");
expect(deps.orders.get("o-1")?.status).toBe("new");
});
});- Check side effects, not just the return value: how many times the gateway was called and with what.
- Always cover the failure path: a cancelled order, a zero total, an external service error. Generated code goes wrong there most often.
- The idempotency key (
order-o-1) is part of the contract too: the gateway uses it to tell a retry from a new payment. - Each test prepares its own data:
setupcreates a fresh order and its own fake gateway.
A typical assistant edit that the last test catches: asked to "make payment more resilient", it wraps the gateway error in try/catch and logs it. The code looks sturdier, but after a failure the order is marked as paid with no money received:
@@ -13,7 +13,11 @@ export async function payOrder(orderId: string, deps: Deps): Promise<Order> {
if (order.status !== "new") throw new Error("ORDER_NOT_PAYABLE");
if (order.totalKzt <= 0) throw new Error("INVALID_AMOUNT");
- await deps.gateway.charge(order.id, order.totalKzt, `order-${order.id}`);
+ try {
+ await deps.gateway.charge(order.id, order.totalKzt, `order-${order.id}`);
+ } catch (error) {
+ console.warn("payment failed, will retry later", error);
+ }
const paid: Order = { ...order, status: "paid" };
deps.orders.set(order.id, paid);
return paid;$ npx vitest run tests/critical
❯ tests/critical/payment.test.ts (5 tests | 1 failed)
× the order stays unpaid if the gateway fails
FAIL tests/critical/payment.test.ts > payment of an order > the order stays unpaid if the gateway fails
AssertionError: promise resolved "{ id: 'o-1', ownerId: 'u-1', …(2) }" instead of rejecting
+ "status": "paid",
Test Files 1 failed | 1 passed (2)
Tests 1 failed | 29 passed (30)A guard test: the full permission matrix
Access checks are a favourite target for "simplification": three conditions look redundant and beg to be folded into one. The defence is to go through every combination of role, ownership and status and compare each answer with a table written by hand.
import { describe, expect, it } from "vitest";
import { canCancel, type Order, type OrderStatus, type Role } from "../../src/orders.js";
// The rule written out by hand, not derived from the code:
// only these combinations may cancel an order. Everything else is forbidden.
const ALLOWED = new Set([
"client/own/new",
"manager/own/new",
"manager/foreign/new",
"admin/own/new",
"admin/foreign/new",
]);
const roles: Role[] = ["guest", "client", "manager", "admin"];
const statuses: OrderStatus[] = ["new", "paid", "cancelled"];
const matrix = roles.flatMap((role) =>
(["own", "foreign"] as const).flatMap((owner) =>
statuses.map((status) => ({ role, owner, status, allowed: ALLOWED.has(`${role}/${owner}/${status}`) })),
),
);
describe("who can cancel an order", () => {
it("the matrix covers every combination", () => {
expect(matrix).toHaveLength(roles.length * 2 * statuses.length);
});
it.each(matrix)("$role, $owner order, $status → $allowed", ({ role, owner, status, allowed }) => {
const order: Order = { id: "o-1", ownerId: "u-1", status, totalKzt: 5000 };
const user = { id: owner === "own" ? "u-1" : "u-2", role };
expect(canCancel(user, order)).toBe(allowed);
});
});An important detail: the ALLOWED expectation isn't computed with the same logic as canCancel. If you copy the condition from the code into the test, the test agrees with any bug — it checks the code against itself. The list of allowed combinations is a requirement, and the product owner signs it off.
Now a "simplification" that is easy to miss in review: the condition is shorter and looks logical.
@@ -13,7 +13,5 @@ export function calcTotal(items: Item[], promo?: string): number {
}
export function canCancel(user: User, order: Order): boolean {
- if (order.status !== "new") return false;
- if (user.role === "admin" || user.role === "manager") return true;
- return user.role === "client" && user.id === order.ownerId;
+ return order.status === "new" && user.role !== "guest";
}$ npx vitest run tests/critical
❯ tests/critical/permissions.test.ts (25 tests | 1 failed)
❯ who can cancel an order (25)
× client, foreign order, new → false
FAIL tests/critical/permissions.test.ts > who can cancel an order > client, foreign order, new → false
AssertionError: expected true to be false // Object.is equality
Test Files 1 failed | 1 passed (2)
Tests 1 failed | 29 passed (30)The failing test's name already describes the hole: "client, foreign order, status new". Without the matrix this edit passes both a manual check and happy-path tests: clients can still cancel their own orders and guests still can't.
Keeping tests from being quietly weakened
When a check fails, an agent, like a tired person, has a shortcut: skip the test, drop the inconvenient expect, or run only "its" test with .only. The run is green afterwards:
@@ -25,12 +25,10 @@ describe("payment of an order", () => {
});
it("a repeated payment does not charge the customer twice", async () => {
- const { deps, charge } = setup(order);
+ const { deps } = setup(order);
await payOrder("o-1", deps);
await payOrder("o-1", deps);
-
- expect(charge).toHaveBeenCalledTimes(1);
});
it("a cancelled order cannot be paid", async () => {
@@ -47,7 +45,7 @@ describe("payment of an order", () => {
expect(charge).not.toHaveBeenCalled();
});
- it("the order stays unpaid if the gateway fails", async () => {
+ it.skip("the order stays unpaid if the gateway fails", async () => {
const { deps, charge } = setup(order);
charge.mockRejectedValueOnce(new Error("GATEWAY_TIMEOUT"));$ npm test
Test Files 3 passed (3)
Tests 30 passed | 1 skipped (31)
$ npm run lint
tests/critical/payment.test.ts
27:3 error Test has no assertions vitest/expect-expect
48:6 error Disabled test - if you want to skip a test temporarily, use .todo() instead vitest/no-disabled-tests
✖ 2 problems (2 errors, 0 warnings)Three mechanisms catch this. @vitest/eslint-plugin rules forbid skipped tests, .only and tests without a single assertion. Setting allowOnly: false in the Vitest config fails the run on .only locally as well, although by default .only is only rejected in CI.
import { defineConfig } from "eslint/config";
import tseslint from "typescript-eslint";
import vitest from "@vitest/eslint-plugin";
export default defineConfig(
tseslint.configs.recommended,
{
files: ["tests/**/*.ts"],
plugins: { vitest },
rules: {
...vitest.configs.recommended.rules,
// a skipped or focused test must not reach the main branch
"vitest/no-disabled-tests": "error",
"vitest/no-focused-tests": "error",
// a test without a single expect proves nothing
"vitest/expect-expect": "error",
},
},
);import { defineConfig } from "vitest/config";
export default defineConfig({
test: {
include: ["tests/**/*.test.ts"],
allowOnly: false,
},
});$ npx vitest run
FAIL tests/critical/payment.test.ts > payment of an order > a cancelled order cannot be paid
Error: [Vitest] Unexpected .only modifier. Remove it or pass --allowOnly argument to bypass this error
Test Files 1 failed | 2 passed (3)The third mechanism is organisational. Rules for the agent go into the project instruction file (how to set one up is covered in the article on AI assistants in development), and changes to critical tests need approval from someone responsible: GitHub automatically requests review from the path owners listed in CODEOWNERS, and the branch protection setting "Require review from Code Owners" makes that approval mandatory.
# AGENTS.md
## Tests
- Run `npm run check` before reporting a task as done and show the output.
- For a bug, first write a test that fails, then fix the code.
- Tests in `tests/critical/` describe payment and access rules.
Do not edit, skip or delete them without asking.
- A failing snapshot means behaviour changed. Report the diff;
never run `vitest -u` to make the run green.
- Never add `.skip`, `.only` or remove `expect` calls to pass a check.# Payment and access rules need a human review
/tests/critical/ @your-org/maintainers
/tests/characterization/ @your-org/maintainersA mandatory run: pre-commit and CI
Tests that run "when someone remembers" don't protect anything. Put every check behind one command and run it in two places: before a commit on the developer's machine and in CI on every pull request.
{
"scripts": {
"test": "vitest run",
"typecheck": "tsc --noEmit",
"lint": "eslint .",
"check": "npm run typecheck && npm run lint && npm test",
"prepare": "husky"
}
}npm run check$ git commit -m "Simplify canCancel"
> vitest run
❯ tests/critical/permissions.test.ts (25 tests | 1 failed)
× client, foreign order, new → false
Test Files 1 failed | 2 passed (3)
Tests 1 failed | 30 passed (31)
husky - pre-commit script failed (code 1)
exit code: 1
$ git commit --no-verify -m "Simplify canCancel"
[main 48bc214] Simplify canCancel
1 file changed, 1 insertion(+), 3 deletions(-)
exit code: 0The second commit shows where pre-commit ends: according to the Git docs the hook is skipped with --no-verify, and husky is turned off with HUSKY=0. A hook saves time and catches mistakes early, but it can't be the mandatory gate. CI is:
name: check
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
jobs:
check:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: 24
cache: npm
- run: npm ci
- run: npm run check$ docker run --rm -v "$PWD:/repo" -w /repo rhysd/actionlint:1.7.12 -verbose .github/workflows/check.yml
verbose: Found 0 parse errors in 0 ms for .github/workflows/check.yml
verbose: Found total 0 errors in 22 ms for .github/workflows/check.ymlProtect the branch
In the repository settings for
main, turn on "Require status checks to pass before merging" and select thecheckjob. According to the GitHub docs, changes can then be merged only after a successful check.Don't allow bypassing
By default the restrictions don't apply to administrators. Turn on "Do not allow bypassing the above settings" if one or two people run the project and the temptation to click "Merge" anyway is real.
Require owner review
"Require review from Code Owners" makes approval mandatory for changes in
tests/critical/. Workflow details, caching and version matrices are in the GitHub Actions CI/CD lesson.
Limits and exercises
- Tests check only what's written in them. The permission matrix won't catch a hole in another endpoint that has no check at all.
- Line coverage isn't correctness: a test without assertions covers code and proves nothing. That's why
expect-expectmatters more than the coverage percentage. - Large snapshots turn into noise, and people start updating them without reading. Snapshot compact values, not whole response objects.
- Tests written by the same agent as the code may repeat its mistake. The Claude Code docs suggest splitting the roles: one session writes tests, another writes code to pass them. A person reads critical tests either way.
- The next step is mutation testing (for example, StrykerJS): the tool makes small "simplifications" in your code and checks that the tests catch them.
How this was verified7 · passed1 · partial1 · not runchecked 12 October 2026
“Run” means the code was executed in a scratch project or container and the output in the article comes from that run. “Partial” means it was checked against stubs instead of external services. “Not run” means it needs a device, an account or a key; the steps were checked against the official documentation. The verification project files and run logs are kept in the InterLAB archive.