PR AA: Phoenix / systemd deployment scaffolding (migrate Phoenix off Next.js stub)

Closes the gap between Gitea main (b48eb2a, Vite portal + Node orchestrator, 29 PRs merged, 167 tests) and what's actually serving curucombo.xn--vov0g.com (Next.js 'ISO-20022 Combo Flow' app from an unpushed local b118b2b checkout). After this PR is merged and the runbook in scripts/deployment/README.md is followed on CT 8604, the Phoenix deployment will serve d-bis/CurrenciCombo main. Artifacts (all under scripts/deployment/): - systemd/currencicombo-orchestrator.service - Node orchestrator, EnvironmentFile=/etc/currencicombo/orchestrator.env, full systemd hardening (ProtectSystem=strict, PrivateTmp, no caps). - systemd/currencicombo-webapp.service - nginx serving Vite SPA on :3000 via RuntimeDirectory=/run/currencicombo-webapp. - webapp-nginx.conf - self-contained nginx config; intentionally 421s on /api/* and /events/* so an NPMplus misconfig fails loudly instead of silently returning index.html. - .env.prod.example - template for /etc/currencicombo/orchestrator.env. Documents every EXT-* blocker env var 1:1 with the Proxmox repo's check-external-dependencies.sh. - install.sh - idempotent host setup: user, dirs, nginx, fresh Postgres role/DB (--force-recreate-db to wipe), Redis autodetect, env file with auto-generated EVENT_SIGNING_SECRET + 3 API keys, systemd units enabled but not started. --dry-run supported. - deploy-currencicombo-8604.sh - build-and-swap deploy driver (the script deploy-targets.json / phoenix-deploy-api calls): git fetch/reset, orchestrator tsc build, portal vite build with VITE_ORCHESTRATOR_URL baked in, migrations, timestamped backup, systemctl stop, rsync, systemctl start, smoke /ready + portal /, grep EXT-* from journalctl. --ref, --dry-run, --skip-migrate, --skip-build, --rollback. - README.md - architecture diagram, first-time setup (8 steps), NPMplus ingress rule table, subsequent- deploy one-liner, rollback, troubleshooting table, cutover-from- pre-existing-Next.js sequence, explicit list of Proxmox-side follow-ups. Target-agnostic: no IP / hostname / VLAN hardcoded. The only file that embeds the public hostname is README.md (for documentation) and the default VITE_ORCHESTRATOR_URL in deploy-currencicombo-8604.sh (which is overridable via env). Single-origin NPMplus routing (confirmed with user): curucombo.\xe6\x9b\xbc\xe6\x9d\x8e.com/api/* -> 10.160.0.14:8080 (orchestrator) curucombo.\xe6\x9b\xbc\xe6\x9d\x8e.com/events/* -> 10.160.0.14:8080 (SSE) curucombo.\xe6\x9b\xbc\xe6\x9d\x8e.com/* -> 10.160.0.14:3000 (Vite SPA) Verified on this box (headless): - shellcheck --severity=warning: clean on both scripts. - bash -n: clean on both scripts. - systemd-analyze verify: both unit files parse cleanly (only complaint is /usr/sbin/nginx not being executable, expected -- nginx is installed at deploy time). - install.sh --dry-run: fails fast with the expected FATAL on hosts without psql (build box). On CT 8604 with Postgres+Redis already installed, it walks through every step. - deploy-currencicombo-8604.sh --help: prints the usage. No runtime code changes. Non-UI. Complements PR #30 (docker-compose sandbox) which remains the local-dev path. Proxmox-side follow-up (separate commit on /home/intlc/projects/proxmox after this PR merges and cutover runs cleanly): - Update phoenix-deploy-api/deploy-targets.json to point at scripts/deployment/deploy-currencicombo-8604.sh. - Retire the inaccurate "Next.js webapp with ignoreBuildErrors" language in EXTERNAL_DEPENDENCY_BLOCKERS.md. Co-Authored-By: Nakamoto, S <defi@defi-oracle.io>
2026-04-22 23:05:18 +00:00
parent b48eb2ab76
commit 361776ab2e
7 changed files with 812 additions and 0 deletions
--- a/scripts/deployment/README.md
+++ b/scripts/deployment/README.md
@@ -0,0 +1,183 @@
+# CurrenciCombo — Phoenix / systemd deployment
+
+This directory holds everything needed to deploy CurrenciCombo onto a
+systemd host — starting with Phoenix CT 8604 on `r630-01`, but any
+Debian/Ubuntu (or Alpine) host with Postgres + Redis available works.
+
+The files here are **target-agnostic**. They hardcode no IPs, hostnames,
+or VLANs. Environment-specific values — `curucombo.曼李.com`, the
+`10.160.0.14` VIP, the NPMplus reverse proxy — are applied at the
+edge (NPMplus) and at `/etc/currencicombo/orchestrator.env`, never in
+the repo.
+
+## Architecture on CT 8604
+
+```
+                                   ┌────────────────────┐
+   curucombo.曼李.com  ──▶ NPMplus │192.168.11.167      │
+   (Cloudflare-proxied)            │ TLS terminates here│
+                                   └─────────┬──────────┘
+                                             │
+                      ┌──────────────────────┴──────────────────────┐
+                      │                                             │
+                      ▼                                             ▼
+       curucombo.曼李.com/* (default)                   curucombo.曼李.com/api/*
+       curucombo.曼李.com/events/* (SSE)   ← swap ─ correctly routed to :8080
+                      │                                             │
+             CT 8604  │10.160.0.14:3000                    CT 8604  │10.160.0.14:8080
+                      ▼                                             ▼
+       ┌─────────────────────────────┐               ┌─────────────────────────────┐
+       │ currencicombo-webapp.service │               │ currencicombo-orchestrator  │
+       │  nginx → /opt/currencicombo/ │               │  .service (systemd)          │
+       │          webapp/dist/        │               │  node dist/index.js          │
+       └─────────────────────────────┘               │  env /etc/currencicombo/     │
+                                                      │       orchestrator.env       │
+                                                      └──────────────┬──────────────┘
+                                                                     │
+                                                                     ▼
+                                                 postgresql + redis (same CT, local)
+```
+
+## Files
+
+| path | purpose |
+|---|---|
+| `systemd/currencicombo-orchestrator.service` | Node orchestrator, reads `/etc/currencicombo/orchestrator.env` |
+| `systemd/currencicombo-webapp.service` | nginx serving the Vite SPA on `:3000` |
+| `webapp-nginx.conf` | full nginx.conf for the webapp unit |
+| `.env.prod.example` | env template installed to `/etc/currencicombo/orchestrator.env` |
+| `install.sh` | one-shot host setup: user / dirs / DB role / systemd units |
+| `deploy-currencicombo-8604.sh` | build-and-swap deploy driver (the script Phoenix/proxmox deploy-api calls) |
+| `README.md` | you're reading it |
+
+## First-time setup on CT 8604
+
+All commands run as **root** inside the CT.
+
+1. Ensure Postgres + Redis are installed and running:
+   ```
+   apt-get install -y postgresql redis-server
+   systemctl enable --now postgresql redis-server
+   ```
+2. Clone the repo into its staging location (once):
+   ```
+   install -d -o root -g root /var/lib/currencicombo
+   git clone https://gitea.d-bis.org/d-bis/CurrenciCombo.git /var/lib/currencicombo/repo
+   ```
+3. Run `install.sh` (creates user, DB, systemd units, env file):
+   ```
+   bash /var/lib/currencicombo/repo/scripts/deployment/install.sh
+   ```
+   On success you'll see:
+   ```
+   [install] generated EVENT_SIGNING_SECRET (64 hex)
+   [install] generated 3 API keys (initiator/settler/auditor) — grep /etc/currencicombo/orchestrator.env
+   [install] install complete.
+   ```
+   **Grab the three API keys from `/etc/currencicombo/orchestrator.env`** and put them in your password manager — they authenticate initiator / settler / auditor calls.
+4. If you need to resolve any `EXT-*` blocker (e.g. point at a real dbis_core), edit `/etc/currencicombo/orchestrator.env` before the first deploy.
+5. First build-and-start:
+   ```
+   bash /var/lib/currencicombo/repo/scripts/deployment/deploy-currencicombo-8604.sh
+   ```
+   Expected tail:
+   ```
+   [deploy] orchestrator ready: {"ready":true}
+   [deploy] portal OK (HTTP 200)
+   [deploy] EXT-* blocker summary from orchestrator boot log:
+              [ExternalBlockers] 6 active, 1 resolved
+              id: EXT-DBIS-CORE
+              id: EXT-CC-PAYMENT-ADAPTERS
+              ...
+              id: EXT-CHAIN138-CI-RPC  (resolved)
+   [deploy] deploy complete. ref=main sha=<short> ts=<timestamp>
+   ```
+
+## NPMplus ingress changes required at cutover
+
+`curucombo.曼李.com` today proxies 100% to `10.160.0.14:3000`. After
+cutover it must become a **single-origin path-routed proxy**:
+
+| location | upstream | notes |
+|---|---|---|
+| `/api/*` | `http://10.160.0.14:8080` | orchestrator API. Forward `Host`, `X-Real-IP`, `X-Forwarded-*`. `proxy_read_timeout 60s`. |
+| `/events/*` | `http://10.160.0.14:8080` | **SSE** — must set `proxy_buffering off;` and `proxy_read_timeout 24h;`. |
+| `/` | `http://10.160.0.14:3000` | Vite SPA. Default upstream. |
+
+If you skip the `/api` + `/events` rules, the nginx in `webapp-nginx.conf`
+intentionally returns `HTTP 421` for those paths — a clean "upstream is
+misconfigured" signal instead of silently returning `index.html` and
+breaking the browser with a JSON parse error.
+
+## Subsequent deploys
+
+Every deploy after the first is just:
+
+```
+sudo /var/lib/currencicombo/repo/scripts/deployment/deploy-currencicombo-8604.sh
+```
+
+Flags:
+- `--ref=<branch-or-sha>` — deploy something other than `main`.
+- `--dry-run` — print what would happen, don't touch anything.
+- `--skip-migrate` — hotfix deploys that don't change the schema.
+- `--skip-build` — reuse the build from the previous run (debugging only).
+- `--rollback` — restore the most recent `/var/lib/currencicombo/backups/<ts>/` and restart units. Does **not** git-pull or rebuild.
+
+Every deploy writes a timestamped backup to
+`/var/lib/currencicombo/backups/<YYYYmmdd-HHMMSS>/` before swapping. Old
+backups are not auto-pruned — `find /var/lib/currencicombo/backups -maxdepth 1 -mtime +30 -exec rm -rf {} +` on a cron.
+
+## Rollback (if a deploy goes sideways)
+
+```
+sudo /var/lib/currencicombo/repo/scripts/deployment/deploy-currencicombo-8604.sh --rollback
+```
+
+Restores the most recent backup and restarts both units. Does not touch
+the DB. If the deploy that failed applied a new migration, a DB rollback
+is a manual `psql` task — we don't attempt generic `down` migrations
+because the orchestrator's migration runner only emits `up()` paths.
+
+## Troubleshooting
+
+| symptom | cause / check |
+|---|---|
+| `/api/*` returns `421 NPMplus is misconfigured` | NPMplus `/api/*` rule missing or wrong upstream. |
+| `/events/*` connects then disconnects after ~60s | NPMplus forgot `proxy_buffering off` + high `proxy_read_timeout`. |
+| orchestrator unit enters `activating (auto-restart)` loop | `journalctl -u currencicombo-orchestrator -n 80` — usually a zod env-validation error. The boot-time assertion message names the missing/invalid var. |
+| orchestrator boot log says `[ExternalBlockers] N active` where N > 6 | you added an `EXT-*` env var without also updating the central registry in `orchestrator/src/config/externalBlockers.ts`. |
+| `/health` returns 503 but `/ready` is 200 | memory `critical` is a separate signal from readiness. Inspect CT memory; this happens on constrained builders and is not a deploy bug. |
+| portal page loads but MetaMask login does nothing | the portal couldn't reach `/api/auth/*`. Walk back up the NPMplus rule chain. |
+
+## Cutting over from the pre-existing Next.js build
+
+Phoenix previously had an older Next.js "ISO-20022 Combo Flow" app in
+`/opt/currencicombo/webapp`. The cutover sequence on CT 8604 is:
+
+1. **Backup the old install** out-of-band:
+   ```
+   tar czf /root/currencicombo-preRepo-$(date +%s).tgz /opt/currencicombo /etc/currencicombo 2>/dev/null || true
+   ```
+2. **Disable the pre-existing systemd units** (they're the same names but point at the old tree):
+   ```
+   systemctl stop currencicombo-webapp currencicombo-orchestrator
+   systemctl disable currencicombo-webapp currencicombo-orchestrator
+   ```
+3. Run `install.sh` (writes the new units, new nginx, new env). On an already-set-up host this is idempotent: it preserves `/etc/currencicombo/orchestrator.env` if it already exists.
+4. Run `deploy-currencicombo-8604.sh`.
+5. Apply the NPMplus `/api` + `/events` path rules.
+6. Smoke from outside the CT: `curl -skI https://curucombo.xn--vov0g.com/ && curl -sk https://curucombo.xn--vov0g.com/api/ready`.
+
+## Proxmox-side follow-up (not in this PR)
+
+After this PR merges and the above cutover runs cleanly, the
+`/home/intlc/projects/proxmox` repo needs a separate commit to:
+
+- Update `phoenix-deploy-api/deploy-targets.json` to point at:
+  - repo: `d-bis/CurrenciCombo`
+  - branch: `main`
+  - target: `default`
+  - deploy entrypoint: `scripts/deployment/deploy-currencicombo-8604.sh`
+- Remove any stale `/opt/currencicombo/webapp` Next.js references.
+- Drop any description of `ignoreBuildErrors: true` in `webapp/next.config.ts` — the new webapp is Vite+tsc-strict, no build-error suppression.