diff --git a/AGENTS.md b/AGENTS.md index 59751af6..f276ccb9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,7 +34,7 @@ Orchestration for Proxmox VE, Chain 138 (`smom-dbis-138/`), explorers, NPMplus, | Submodule + explorer remotes | `docs/00-meta/SUBMODULE_HYGIENE.md` — `mcp-proxmox` uses **Gitea** `https://gitea.d-bis.org/d-bis/mcp-proxmox.git` (not the old GitHub-only URL). `cross-chain-pmm-lps-publish` is a **worktree** of `cross-chain-pmm-lps`, not a submodule. | | smom-dbis-138 `.env` in bash scripts | Prefer `source smom-dbis-138/scripts/lib/deployment/dotenv.sh` + `load_deployment_env --repo-root "$PROJECT_ROOT"` (trims RPC URL line endings). From an interactive shell: `source smom-dbis-138/scripts/load-env.sh`. Proxmox root scripts: `source scripts/lib/load-project-env.sh` (also trims common RPC vars). | | Sankofa portal → CT 7801 (build + restart) | `./scripts/deployment/sync-sankofa-portal-7801.sh` (`--dry-run` first); sets `NEXTAUTH_URL` on CT via `sankofa-portal-ensure-nextauth-on-ct.sh` | -| Gov Portals (CT **7804** @ `IP_GOV_PORTALS_DEV`, r630-04): rebase from Gitea then build + sync | `./scripts/deployment/gov-portals-git-pull-rebase.sh` then `./scripts/deployment/sync-gov-portals-ct-7804-from-git.sh` — `GITEA_TOKEN` in `.env`; sync defaults to **pull --rebase** (use `--reset-hard` to mirror Gitea and discard local changes); optional `--skip-fetch`. See `docs/04-configuration/GOV_PORTALS_XOM_DEV_DEPLOYMENT.md`. | +| Gov Portals (CT **7804** @ `IP_GOV_PORTALS_DEV`, r630-04): rebase from Gitea then build + sync | `./scripts/deployment/gov-portals-git-pull-rebase.sh` then `./scripts/deployment/sync-gov-portals-ct-7804-from-git.sh` — `GITEA_TOKEN` in `.env`; sync defaults to **pull --rebase** (use `--reset-hard` to mirror Gitea and discard local changes); optional `--skip-fetch`. Source-of-truth and Phoenix paths: `docs/04-configuration/GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md`; ops template: `docs/04-configuration/GOV_PORTALS_XOM_DEV_DEPLOYMENT.md`. Optional Gitea tag: `scripts/deployment/gitea-tag-repo-release.sh`. | | CCIP relay (r630-01 host) | Unit: `config/systemd/ccip-relay.service` → `/etc/systemd/system/ccip-relay.service`; `systemctl enable --now ccip-relay` | | TsunamiSwap VM 5010 check | `./scripts/deployment/tsunamiswap-vm-5010-provision.sh` (inventory only until VM exists) | | Solana native SOL (robust JSON-RPC submit) | `scripts/lib/solana_jsonrpc.py` (stdlib `sendTransaction`), `./scripts/deployment/solana-transfer-native.py` (sign with `solders`). Install: `pip install -r scripts/lib/requirements-solana-ops.txt`. Avoids solana-py `SendTransactionResp` parse failures on RPCs that return only a signature string. Env: `SOLANA_RPC_URL`, `SOLANA_KEYPAIR_PATH` via `source scripts/lib/load-project-env.sh`. | diff --git a/config/gitea-workflow-templates/README.md b/config/gitea-workflow-templates/README.md index 319c8a77..b1cda364 100644 --- a/config/gitea-workflow-templates/README.md +++ b/config/gitea-workflow-templates/README.md @@ -1,6 +1,6 @@ # Gitea Actions workflow templates -Copy one of these into **your repo** as `.gitea/workflows/.yml`, then set repo **Secrets** in Gitea (`PHOENIX_DEPLOY_URL`, `PHOENIX_DEPLOY_TOKEN`). +Copy one of these into **your repo** as `.gitea/workflows/.yml`, then set repo **Secrets** in Gitea (`PHOENIX_DEPLOY_URL`, `PHOENIX_DEPLOY_TOKEN`, and **`GITEA_TOKEN`** when the workflow clones `gov-portals-monorepo` for portal CI). | Template | Use when | |----------|----------| diff --git a/config/gitea-workflow-templates/repos/README.md b/config/gitea-workflow-templates/repos/README.md index d20b2dbd..1bc68caa 100644 --- a/config/gitea-workflow-templates/repos/README.md +++ b/config/gitea-workflow-templates/repos/README.md @@ -4,7 +4,11 @@ Copy the matching file into **that** Gitea repo as `.gitea/workflows/.yml` | File | Gitea `repo` | `target` | Notes | |------|----------------|----------|--------| -| [`dbis-portal-live.yml`](dbis-portal-live.yml) | `Gov_Web_Portals/DBIS` | `dbis-portal-live` | CT 7804 portal | +| [`dbis-portal-ci-and-live.yml`](dbis-portal-ci-and-live.yml) | `Gov_Web_Portals/DBIS` | `dbis-portal-live` | Recommended: monorepo overlay CI + deploy; secrets include **`GITEA_TOKEN`** | +| [`iccc-portal-ci-and-live.yml`](iccc-portal-ci-and-live.yml) | `Gov_Web_Portals/ICCC` | `iccc-portal-live` | Same pattern as DBIS | +| [`omnl-portal-ci-and-live.yml`](omnl-portal-ci-and-live.yml) | `Gov_Web_Portals/OMNL` | `omnl-portal-live` | Same pattern | +| [`xom-portal-ci-and-live.yml`](xom-portal-ci-and-live.yml) | `Gov_Web_Portals/XOM` | `xom-portal-live` | Same pattern | +| [`dbis-portal-live.yml`](dbis-portal-live.yml) | `Gov_Web_Portals/DBIS` | `dbis-portal-live` | Deploy-only fallback (no `GITEA_TOKEN` CI) | | [`cromero-default.yml`](cromero-default.yml) | `d-bis/CROMERO` | `default` | NPM ecosystem build | | [`currencicombo-default.yml`](currencicombo-default.yml) | `d-bis/CurrenciCombo` | `default` | Phoenix CT 8604 | | — | `d-bis/explorer-monorepo` | `explorer-live` | Already in **explorer-monorepo** submodule: `.gitea/workflows/deploy-live.yml` | diff --git a/config/gitea-workflow-templates/repos/dbis-portal-ci-and-live.yml b/config/gitea-workflow-templates/repos/dbis-portal-ci-and-live.yml new file mode 100644 index 00000000..318b6d92 --- /dev/null +++ b/config/gitea-workflow-templates/repos/dbis-portal-ci-and-live.yml @@ -0,0 +1,61 @@ +# Copy to Gov_Web_Portals/DBIS → .gitea/workflows/deploy-portal-live.yml (or a second workflow file). +# Secrets: PHOENIX_DEPLOY_URL, PHOENIX_DEPLOY_TOKEN, GITEA_TOKEN (read access to gov-portals-monorepo for CI). +# +# verify: shallow-clone monorepo, overlay this repo into DBIS/, run lint + typecheck (workspace deps). +# deploy: POST Phoenix with full sha (optional: set PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH=1 on Phoenix host). +name: CI and deploy DBIS portal (Phoenix) + +concurrency: + group: gov-portal-dbis-phoenix-deploy + cancel-in-progress: false + +on: + push: + branches: [main] + pull_request: + workflow_dispatch: + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + path: portal-src + - uses: actions/setup-node@v4 + with: + node-version: "20" + - name: Enable pnpm + run: corepack enable && corepack prepare pnpm@8.15.0 --activate + - name: Clone monorepo and overlay DBIS + env: + GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} + run: | + set -euo pipefail + test -n "${GITEA_TOKEN:-}" + git clone --depth 1 --branch main "https://oauth2:${GITEA_TOKEN}@gitea.d-bis.org/Gov_Web_Portals/gov-portals-monorepo.git" /tmp/gwp + rm -rf /tmp/gwp/DBIS + mkdir -p /tmp/gwp/DBIS + rsync -a --delete portal-src/ /tmp/gwp/DBIS/ + - name: pnpm install, lint, typecheck + run: | + cd /tmp/gwp + pnpm install --frozen-lockfile + pnpm --filter portal-dbis lint + pnpm --filter portal-dbis run typecheck + + deploy: + needs: [verify] + if: github.event_name == 'push' && github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Trigger Phoenix deployment + run: | + SHA="$(git rev-parse HEAD)" + BRANCH="$(git rev-parse --abbrev-ref HEAD)" + curl -sSf --connect-timeout 10 --max-time 3600 \ + -X POST "${{ secrets.PHOENIX_DEPLOY_URL }}" \ + -H "Authorization: Bearer ${{ secrets.PHOENIX_DEPLOY_TOKEN }}" \ + -H "Content-Type: application/json" \ + -d "{\"repo\":\"Gov_Web_Portals/DBIS\",\"sha\":\"${SHA}\",\"branch\":\"${BRANCH}\",\"target\":\"dbis-portal-live\"}" diff --git a/config/gitea-workflow-templates/repos/dbis-portal-live.yml b/config/gitea-workflow-templates/repos/dbis-portal-live.yml index 84d59949..b7c8fa8f 100644 --- a/config/gitea-workflow-templates/repos/dbis-portal-live.yml +++ b/config/gitea-workflow-templates/repos/dbis-portal-live.yml @@ -1,3 +1,5 @@ +# Minimal deploy-only workflow (no monorepo CI). Prefer dbis-portal-ci-and-live.yml when the repo +# can store GITEA_TOKEN for cloning Gov_Web_Portals/gov-portals-monorepo in Actions. # Copy to Gov_Web_Portals/DBIS → .gitea/workflows/deploy-portal-live.yml # Secrets: PHOENIX_DEPLOY_URL, PHOENIX_DEPLOY_TOKEN name: Deploy DBIS portal (Phoenix) diff --git a/config/gitea-workflow-templates/repos/iccc-portal-ci-and-live.yml b/config/gitea-workflow-templates/repos/iccc-portal-ci-and-live.yml new file mode 100644 index 00000000..4b30dfbd --- /dev/null +++ b/config/gitea-workflow-templates/repos/iccc-portal-ci-and-live.yml @@ -0,0 +1,58 @@ +# Copy to Gov_Web_Portals/ICCC → .gitea/workflows/deploy-portal-live.yml +# Secrets: PHOENIX_DEPLOY_URL, PHOENIX_DEPLOY_TOKEN, GITEA_TOKEN +name: CI and deploy ICCC portal (Phoenix) + +concurrency: + group: gov-portal-iccc-phoenix-deploy + cancel-in-progress: false + +on: + push: + branches: [main] + pull_request: + workflow_dispatch: + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + path: portal-src + - uses: actions/setup-node@v4 + with: + node-version: "20" + - name: Enable pnpm + run: corepack enable && corepack prepare pnpm@8.15.0 --activate + - name: Clone monorepo and overlay ICCC + env: + GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} + run: | + set -euo pipefail + test -n "${GITEA_TOKEN:-}" + git clone --depth 1 --branch main "https://oauth2:${GITEA_TOKEN}@gitea.d-bis.org/Gov_Web_Portals/gov-portals-monorepo.git" /tmp/gwp + rm -rf /tmp/gwp/ICCC + mkdir -p /tmp/gwp/ICCC + rsync -a --delete portal-src/ /tmp/gwp/ICCC/ + - name: pnpm install, lint, typecheck + run: | + cd /tmp/gwp + pnpm install --frozen-lockfile + pnpm --filter portal-iccc lint + pnpm --filter portal-iccc run typecheck + + deploy: + needs: [verify] + if: github.event_name == 'push' && github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Trigger Phoenix deployment + run: | + SHA="$(git rev-parse HEAD)" + BRANCH="$(git rev-parse --abbrev-ref HEAD)" + curl -sSf --connect-timeout 10 --max-time 3600 \ + -X POST "${{ secrets.PHOENIX_DEPLOY_URL }}" \ + -H "Authorization: Bearer ${{ secrets.PHOENIX_DEPLOY_TOKEN }}" \ + -H "Content-Type: application/json" \ + -d "{\"repo\":\"Gov_Web_Portals/ICCC\",\"sha\":\"${SHA}\",\"branch\":\"${BRANCH}\",\"target\":\"iccc-portal-live\"}" diff --git a/config/gitea-workflow-templates/repos/omnl-portal-ci-and-live.yml b/config/gitea-workflow-templates/repos/omnl-portal-ci-and-live.yml new file mode 100644 index 00000000..49157377 --- /dev/null +++ b/config/gitea-workflow-templates/repos/omnl-portal-ci-and-live.yml @@ -0,0 +1,58 @@ +# Copy to Gov_Web_Portals/OMNL → .gitea/workflows/deploy-portal-live.yml +# Secrets: PHOENIX_DEPLOY_URL, PHOENIX_DEPLOY_TOKEN, GITEA_TOKEN +name: CI and deploy OMNL portal (Phoenix) + +concurrency: + group: gov-portal-omnl-phoenix-deploy + cancel-in-progress: false + +on: + push: + branches: [main] + pull_request: + workflow_dispatch: + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + path: portal-src + - uses: actions/setup-node@v4 + with: + node-version: "20" + - name: Enable pnpm + run: corepack enable && corepack prepare pnpm@8.15.0 --activate + - name: Clone monorepo and overlay OMNL + env: + GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} + run: | + set -euo pipefail + test -n "${GITEA_TOKEN:-}" + git clone --depth 1 --branch main "https://oauth2:${GITEA_TOKEN}@gitea.d-bis.org/Gov_Web_Portals/gov-portals-monorepo.git" /tmp/gwp + rm -rf /tmp/gwp/OMNL + mkdir -p /tmp/gwp/OMNL + rsync -a --delete portal-src/ /tmp/gwp/OMNL/ + - name: pnpm install, lint, typecheck + run: | + cd /tmp/gwp + pnpm install --frozen-lockfile + pnpm --filter portal-omnl lint + pnpm --filter portal-omnl run typecheck + + deploy: + needs: [verify] + if: github.event_name == 'push' && github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Trigger Phoenix deployment + run: | + SHA="$(git rev-parse HEAD)" + BRANCH="$(git rev-parse --abbrev-ref HEAD)" + curl -sSf --connect-timeout 10 --max-time 3600 \ + -X POST "${{ secrets.PHOENIX_DEPLOY_URL }}" \ + -H "Authorization: Bearer ${{ secrets.PHOENIX_DEPLOY_TOKEN }}" \ + -H "Content-Type: application/json" \ + -d "{\"repo\":\"Gov_Web_Portals/OMNL\",\"sha\":\"${SHA}\",\"branch\":\"${BRANCH}\",\"target\":\"omnl-portal-live\"}" diff --git a/config/gitea-workflow-templates/repos/xom-portal-ci-and-live.yml b/config/gitea-workflow-templates/repos/xom-portal-ci-and-live.yml new file mode 100644 index 00000000..db11c9e7 --- /dev/null +++ b/config/gitea-workflow-templates/repos/xom-portal-ci-and-live.yml @@ -0,0 +1,58 @@ +# Copy to Gov_Web_Portals/XOM → .gitea/workflows/deploy-portal-live.yml +# Secrets: PHOENIX_DEPLOY_URL, PHOENIX_DEPLOY_TOKEN, GITEA_TOKEN +name: CI and deploy XOM portal (Phoenix) + +concurrency: + group: gov-portal-xom-phoenix-deploy + cancel-in-progress: false + +on: + push: + branches: [main] + pull_request: + workflow_dispatch: + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + path: portal-src + - uses: actions/setup-node@v4 + with: + node-version: "20" + - name: Enable pnpm + run: corepack enable && corepack prepare pnpm@8.15.0 --activate + - name: Clone monorepo and overlay XOM + env: + GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} + run: | + set -euo pipefail + test -n "${GITEA_TOKEN:-}" + git clone --depth 1 --branch main "https://oauth2:${GITEA_TOKEN}@gitea.d-bis.org/Gov_Web_Portals/gov-portals-monorepo.git" /tmp/gwp + rm -rf /tmp/gwp/XOM + mkdir -p /tmp/gwp/XOM + rsync -a --delete portal-src/ /tmp/gwp/XOM/ + - name: pnpm install, lint, typecheck + run: | + cd /tmp/gwp + pnpm install --frozen-lockfile + pnpm --filter portal-xom lint + pnpm --filter portal-xom run typecheck + + deploy: + needs: [verify] + if: github.event_name == 'push' && github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Trigger Phoenix deployment + run: | + SHA="$(git rev-parse HEAD)" + BRANCH="$(git rev-parse --abbrev-ref HEAD)" + curl -sSf --connect-timeout 10 --max-time 3600 \ + -X POST "${{ secrets.PHOENIX_DEPLOY_URL }}" \ + -H "Authorization: Bearer ${{ secrets.PHOENIX_DEPLOY_TOKEN }}" \ + -H "Content-Type: application/json" \ + -d "{\"repo\":\"Gov_Web_Portals/XOM\",\"sha\":\"${SHA}\",\"branch\":\"${BRANCH}\",\"target\":\"xom-portal-live\"}" diff --git a/docs/00-meta/GITEA_CD_OPERATOR_CHECKLIST.md b/docs/00-meta/GITEA_CD_OPERATOR_CHECKLIST.md index 74a8c5cf..08d19cd8 100644 --- a/docs/00-meta/GITEA_CD_OPERATOR_CHECKLIST.md +++ b/docs/00-meta/GITEA_CD_OPERATOR_CHECKLIST.md @@ -8,6 +8,7 @@ Use this after changing **`phoenix-deploy-api/deploy-targets.json`** or adding w 2. **Secrets** on **that repo** (not only global): - **`PHOENIX_DEPLOY_URL`** — full URL for `POST` (same shape as **`d-bis/proxmox`** workflows use), typically `http://:4001/api/deploy` or HTTPS equivalent. - **`PHOENIX_DEPLOY_TOKEN`** — bearer token accepted by Phoenix deploy API. + - **`GITEA_TOKEN`** — required for **gov portal CI** workflows that clone `Gov_Web_Portals/gov-portals-monorepo` (read-only token is enough). 3. **Workflow file** in the repo: copy from [`config/gitea-workflow-templates/repos/README.md`](../config/gitea-workflow-templates/repos/README.md) or use the repo’s existing `.gitea/workflows/*.yml`. ## Phoenix deploy host (LAN) @@ -15,6 +16,7 @@ Use this after changing **`phoenix-deploy-api/deploy-targets.json`** or adding w 1. **`git pull`** **proxmox** so **`deploy-targets.json`** and **`scripts/deployment/phoenix-deploy-*.sh`** match Gitea **`d-bis/proxmox`** `master` / `main`. 2. Restart or reinstall **phoenix-deploy-api** if you manage it via systemd (see **`phoenix-deploy-api/scripts/install-systemd.sh`**). 3. **`GITEA_TOKEN`** on that host must allow archive fetch for repos you deploy. +4. Optional: set **`PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH=1`** on the Phoenix deploy API host so `POST /api/deploy` requests that include **`sha`** must resolve to a commit on the declared **`branch`** (see **`phoenix-deploy-api/README.md`**). ## Verify locally (proxmox clone) @@ -26,4 +28,5 @@ bash scripts/verify/report-gitea-cd-parity.sh ## Canonical references - [GITEA_REPO_VM_CD_CI_MATRIX.md](../04-configuration/GITEA_REPO_VM_CD_CI_MATRIX.md) +- [GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md](../04-configuration/GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md) - [config/gitea-workflow-templates/README.md](../../config/gitea-workflow-templates/README.md) diff --git a/docs/04-configuration/GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md b/docs/04-configuration/GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md new file mode 100644 index 00000000..95b0220d --- /dev/null +++ b/docs/04-configuration/GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md @@ -0,0 +1,65 @@ +# Gov Web Portals — Gitea source of truth and live deploy paths + +This document ties together **Gitea repos**, the **gov-portals-monorepo** umbrella, **Phoenix deploy API**, **CT 7804**, and **operator scripts** so there is a single place to answer “what is canonical?” and “how does it get live?”. + +## North star + +- **Authoritative application code** for each portal lives in its **Gitea repo** under `Gov_Web_Portals/` (DBIS, ICCC, OMNL, XOM), with **`main`** as the integration branch unless you have agreed otherwise. +- **Integration / shared layout** lives in **`Gov_Web_Portals/gov-portals-monorepo`** (workspace + `packages/shared`). Portal repos use `workspace:*` for `@public-web-portals/shared`, so **CI that runs `pnpm install` in the portal repo alone is insufficient** unless you vendor shared — the recommended Actions flow clones the monorepo and **overlays** the pushed portal tree (see templates below). +- **Live runtime** for the xom-dev stack is **LXC 7804** (`IP_GOV_PORTALS_DEV`, default `192.168.11.54`) with systemd units `gov-portal-DBIS` … `gov-portal-XOM` on ports **3001–3004**. Public hostnames are documented in [GOV_PORTALS_XOM_DEV_DEPLOYMENT.md](./GOV_PORTALS_XOM_DEV_DEPLOYMENT.md). + +## Two operator paths (pick deliberately) + +| Path | When to use | What updates live | Submodule pointers | +|------|-------------|---------------------|---------------------| +| **A — Portal repo push (Phoenix / Pattern A)** | You want **this portal’s `main`** to drive redeploys without touching the monorepo pointer first. | Gitea Actions in **`Gov_Web_Portals/`** POSTs Phoenix → `phoenix-deploy-gov-portal-live-from-workspace.sh` → full monorepo-shaped build on **7804** (replaces `/srv/gov-portals` for that deploy). | Monorepo **parent** on Gitea can lag until someone bumps submodules; live site can still match portal `main`. | +| **B — Monorepo-first (operator sync)** | You want **one tree** (recorded submodule SHAs) to be exactly what runs after `sync-gov-portals-ct-7804-from-git.sh`. | `scripts/deployment/sync-gov-portals-ct-7804-from-git.sh` (rebase by default) builds **all** portals on the CT. | **Canonical for “what shipped together”** when you rely on the monorepo commit. | + +Use **Path A** for fast single-portal iteration; use **Path B** when releasing a **coherent monorepo revision** (all four portals + shared in lockstep). + +## Wiring checklist (Path A) + +1. **`phoenix-deploy-api/deploy-targets.json`** (in **`d-bis/proxmox`**) contains a target for `Gov_Web_Portals/` / `main` / `default` and named target `*-portal-live` where applicable — validated by `bash scripts/validation/validate-phoenix-deploy-targets.sh`. +2. **Phoenix host** has pulled proxmox `main`, **`GITEA_TOKEN`** can archive-fetch all `Gov_Web_Portals/*` repos used, **`PHOENIX_REPO_ROOT`**, **`PHOENIX_DEPLOY_SECRET`**, optional **`PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH=1`** (see below). +3. **Each portal repo** on Gitea: Actions enabled; secrets **`PHOENIX_DEPLOY_URL`**, **`PHOENIX_DEPLOY_TOKEN`**; for CI templates also **`GITEA_TOKEN`** (read `gov-portals-monorepo`). +4. Copy the workflow template from [`config/gitea-workflow-templates/repos/README.md`](../../config/gitea-workflow-templates/repos/README.md) into **that repo** as `.gitea/workflows/deploy-portal-live.yml` (or split files if you prefer). + +## CI before deploy (recommended) + +Use **`dbis-portal-ci-and-live.yml`** (and the matching **ICCC / OMNL / XOM** templates): job **`verify`** runs on **pull_request** and **push**; job **`deploy`** runs only on **`push` to `main`** and **`needs: verify`**. + +The verify job clones **`Gov_Web_Portals/gov-portals-monorepo`** with `GITEA_TOKEN`, overlays the current repo into the correct subdirectory, then runs **`pnpm install --frozen-lockfile`** and **`pnpm --filter portal-*`** lint + typecheck. + +## Phoenix: verify commit belongs to branch + +When the systemd (or shell) environment for **phoenix-deploy-api** sets: + +`PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH=1` + +then a deploy request that includes **`sha`** must resolve in Gitea to a commit that is **on the requested `branch`** (branch tip or merge-base check via the Gitea compare API). This blocks accidental **`curl` with wrong branch + sha** pairs. Default is **off** so existing runners keep working until you opt in. + +## Health checks after deploy + +Each gov-portal target in **`deploy-targets.json`** defines a **`healthcheck`** URL (public xom-dev host or DBIS `d-bis.org` trust manifest). Phoenix runs these after the deploy script exits. Adjust URLs if DNS or NPMplus routing changes. + +## Rollback and tags + +- **CT rollback:** the deploy script swaps `/srv/gov-portals` with a timestamped `-previous-*` directory if the new build fails its local curl check. +- **Git rollback:** revert on `main` and push, or deploy an older **`sha`** via Phoenix **`POST /api/deploy`** with the same **`repo`**, **`branch`**, and **`target`**. +- **Release tags:** optional helper (requires `GITEA_TOKEN` with write access): `bash scripts/deployment/gitea-tag-repo-release.sh Gov_Web_Portals/DBIS v2026.05.12-dbis `. + +## Gitea branch protection (manual) + +In Gitea org/repo settings, enable **branch protection** on `main` (required reviews, block force-push, require passing Actions) per your governance. This cannot be enforced from the proxmox repo alone; document ownership in Gitea admin. + +## Related docs and scripts + +| Topic | Location | +|-------|-----------| +| CT 7804, DNS, NPM, troubleshooting | [GOV_PORTALS_XOM_DEV_DEPLOYMENT.md](./GOV_PORTALS_XOM_DEV_DEPLOYMENT.md) | +| Repo / VM / target matrix | [GITEA_REPO_VM_CD_CI_MATRIX.md](./GITEA_REPO_VM_CD_CI_MATRIX.md) | +| Operator checklist (secrets, pull Phoenix) | [GITEA_CD_OPERATOR_CHECKLIST.md](../00-meta/GITEA_CD_OPERATOR_CHECKLIST.md) | +| Rebase helper before sync | `scripts/deployment/gov-portals-git-pull-rebase.sh` | +| Sync monorepo to CT | `scripts/deployment/sync-gov-portals-ct-7804-from-git.sh` | +| Generic Phoenix portal deploy | `scripts/deployment/phoenix-deploy-gov-portal-live-from-workspace.sh` | +| Parity report (local) | `bash scripts/verify/report-gitea-cd-parity.sh` | diff --git a/docs/04-configuration/GITEA_REPO_VM_CD_CI_MATRIX.md b/docs/04-configuration/GITEA_REPO_VM_CD_CI_MATRIX.md index 0b78f04a..6be6ebaf 100644 --- a/docs/04-configuration/GITEA_REPO_VM_CD_CI_MATRIX.md +++ b/docs/04-configuration/GITEA_REPO_VM_CD_CI_MATRIX.md @@ -2,7 +2,7 @@ Each **application repo** should carry **its own** `.gitea/workflows/*.yml` so pushes trigger the right pipeline for **that** codebase. Deploy execution typically happens on the **designated LAN VM** (via **Phoenix deploy API** on the dev workspace host), not on the public Gitea runner alone. -**Canonical integration:** [Phoenix deploy API](../../phoenix-deploy-api/server.js) + [`deploy-targets.json`](../../phoenix-deploy-api/deploy-targets.json). +**Canonical integration:** [Phoenix deploy API](../../phoenix-deploy-api/server.js) + [`deploy-targets.json`](../../phoenix-deploy-api/deploy-targets.json). Gov portals two-path model: [GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md](./GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md). **Postgres / Prisma in Actions (self-hosted runners):** use **`...@postgres:5432`** (service hostname) and keep **`container.network`** empty on **`act_runner`** — [GITEA_ACT_RUNNER_SETUP.md](GITEA_ACT_RUNNER_SETUP.md) (troubleshooting **P1001**). @@ -16,7 +16,7 @@ Each **application repo** should carry **its own** `.gitea/workflows/*.yml` so p 2. Body includes `repo` (Gitea `owner/name`), `branch`, `sha`, `target` (matches `deploy-targets.json`). 3. Phoenix syncs the repo archive from Gitea, sets `PHOENIX_DEPLOY_WORKSPACE`, runs the target `command` with LAN access (SSH `pct`, rsync, etc.). -**Secrets (per repo in Gitea):** `PHOENIX_DEPLOY_URL`, `PHOENIX_DEPLOY_TOKEN` (same pattern as `d-bis/proxmox` workflows). +**Secrets (per repo in Gitea):** `PHOENIX_DEPLOY_URL`, `PHOENIX_DEPLOY_TOKEN` (same pattern as `d-bis/proxmox` workflows). Gov portal CI templates also need **`GITEA_TOKEN`** (read `Gov_Web_Portals/gov-portals-monorepo`). ## Pattern B — Monorepo-only (`d-bis/proxmox`) @@ -28,7 +28,10 @@ Multiple deploy jobs in one workflow ([`.gitea/workflows/deploy-to-phoenix.yml`] |------------|------------|--------------|-----------------------------|----------| | `d-bis/proxmox` | `main`, `master` | Phoenix deploy host + varies by job | `default`, `atomic-swap-dapp-live`, `portal-live`, `cloudflare-sync`, … | `.gitea/workflows/deploy-to-phoenix.yml`, `validate-on-pr.yml` | | `Gov_Web_Portals/CyberSecur-Global` | `main` | CT **7810** | `default` | In **CyberSecur-Global** repo: `.gitea/workflows/deploy-to-ct7810.yml` | -| `Gov_Web_Portals/DBIS` | `main` | CT **7804** | `dbis-portal-live` | Copy [`repos/dbis-portal-live.yml`](../../config/gitea-workflow-templates/repos/dbis-portal-live.yml) → DBIS repo | +| `Gov_Web_Portals/DBIS` | `main` | CT **7804** | `dbis-portal-live`, `default` | Copy [`repos/dbis-portal-ci-and-live.yml`](../../config/gitea-workflow-templates/repos/dbis-portal-ci-and-live.yml) (recommended) or deploy-only [`repos/dbis-portal-live.yml`](../../config/gitea-workflow-templates/repos/dbis-portal-live.yml) → DBIS repo | +| `Gov_Web_Portals/ICCC` | `main` | CT **7804** | `iccc-portal-live`, `default` | Copy [`repos/iccc-portal-ci-and-live.yml`](../../config/gitea-workflow-templates/repos/iccc-portal-ci-and-live.yml) → ICCC repo | +| `Gov_Web_Portals/OMNL` | `main` | CT **7804** | `omnl-portal-live`, `default` | Copy [`repos/omnl-portal-ci-and-live.yml`](../../config/gitea-workflow-templates/repos/omnl-portal-ci-and-live.yml) → OMNL repo | +| `Gov_Web_Portals/XOM` | `main` | CT **7804** | `xom-portal-live`, `default` | Copy [`repos/xom-portal-ci-and-live.yml`](../../config/gitea-workflow-templates/repos/xom-portal-ci-and-live.yml) → XOM repo | | `d-bis/explorer-monorepo` | `main`, `master` | VMID **5000** | `explorer-live` | Submodule: `.gitea/workflows/deploy-live.yml` | | `d-bis/CROMERO` | `main`, `master` | NPM ecosystem path | `default` | Copy [`repos/cromero-default.yml`](../../config/gitea-workflow-templates/repos/cromero-default.yml) → CROMERO repo | | `d-bis/CurrenciCombo` | `main`, `master` | Phoenix CT **8604** | `default` | Copy [`repos/currencicombo-default.yml`](../../config/gitea-workflow-templates/repos/currencicombo-default.yml) → CurrenciCombo repo | @@ -39,7 +42,7 @@ Multiple deploy jobs in one workflow ([`.gitea/workflows/deploy-to-phoenix.yml`] 1. Add rows to [`deploy-targets.json`](../../phoenix-deploy-api/deploy-targets.json) with `repo`, `branch`, `target`, `command`, `healthcheck`. 2. Implement or reuse a `scripts/deployment/phoenix-deploy-*-from-workspace.sh` wrapper if the deploy needs `PHOENIX_DEPLOY_WORKSPACE`. 3. Copy a template from [`config/gitea-workflow-templates/`](../../config/gitea-workflow-templates/README.md) into **that repo** as `.gitea/workflows/.yml`. -4. In Gitea → Repo → **Secrets**: `PHOENIX_DEPLOY_URL`, `PHOENIX_DEPLOY_TOKEN`. +4. In Gitea → Repo → **Secrets**: `PHOENIX_DEPLOY_URL`, `PHOENIX_DEPLOY_TOKEN` (and **`GITEA_TOKEN`** if the workflow clones `gov-portals-monorepo` for CI). 5. Document the VM / URL here. ## References diff --git a/docs/04-configuration/GOV_PORTALS_XOM_DEV_DEPLOYMENT.md b/docs/04-configuration/GOV_PORTALS_XOM_DEV_DEPLOYMENT.md index 1ad396af..477ef54d 100644 --- a/docs/04-configuration/GOV_PORTALS_XOM_DEV_DEPLOYMENT.md +++ b/docs/04-configuration/GOV_PORTALS_XOM_DEV_DEPLOYMENT.md @@ -1,5 +1,7 @@ # Gov Portals xom-dev.phoenix.sankofa.nexus Deployment +**Gitea vs monorepo vs Phoenix:** [GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md](./GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md) (two-path table, CI templates, optional sha-on-branch enforcement). + **Domains:** `dbis`, `iccc`, `omnl`, `xom` `.xom-dev.phoenix.sankofa.nexus` **VM:** LXC 7804 (gov-portals-dev) @ 192.168.11.54 **NPMplus:** Primary (192.168.11.167) — same as sankofa.nexus zone diff --git a/docs/MASTER_INDEX.md b/docs/MASTER_INDEX.md index ff1f5011..a1d9bde4 100644 --- a/docs/MASTER_INDEX.md +++ b/docs/MASTER_INDEX.md @@ -29,7 +29,7 @@ | **Chain 138 txpool incident recovery** | `bash scripts/fix-all-validators-and-txpool.sh` → `bash scripts/maintenance/apply-chain138-strict-future-tx-pool.sh` → `bash scripts/clear-all-transaction-pools.sh` → `bash scripts/monitoring/monitor-blockchain-health.sh` | | **Gitea TLS expiry check** | `bash scripts/verify/check-gitea-certificate-expiry.sh` — warns before `gitea.d-bis.org` cert expiry blocks HTTPS pushes | | **Gitea TLS expiry cron** | `bash scripts/maintenance/schedule-gitea-cert-check-cron.sh --install` — installs a daily warning check with `WARN_DAYS=30` | -| **Gitea repo ↔ VM CI/CD matrix** | [04-configuration/GITEA_REPO_VM_CD_CI_MATRIX.md](04-configuration/GITEA_REPO_VM_CD_CI_MATRIX.md) — per-repo workflows, Phoenix deploy targets, templates under `config/gitea-workflow-templates/` | +| **Gitea repo ↔ VM CI/CD matrix** | [04-configuration/GITEA_REPO_VM_CD_CI_MATRIX.md](04-configuration/GITEA_REPO_VM_CD_CI_MATRIX.md) — per-repo workflows, Phoenix deploy targets, templates under `config/gitea-workflow-templates/`; gov portals live paths: [GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md](04-configuration/GITEA_GOV_PORTALS_LIVE_SOURCE_OF_TRUTH.md) | | **Gitea CD operator checklist** | [00-meta/GITEA_CD_OPERATOR_CHECKLIST.md](00-meta/GITEA_CD_OPERATOR_CHECKLIST.md) — secrets, Phoenix host sync, `report-gitea-cd-parity.sh` | | **TsunamiSwap DEX plan** | [00-meta/AAVE_CHAIN138_AND_MARIONETTE_TSUNAMISWAP_PLAN.md](00-meta/AAVE_CHAIN138_AND_MARIONETTE_TSUNAMISWAP_PLAN.md) — canonical TsunamiSwap VM `5010` plan, current DEX link, and publish checklist | | **Required / optional / recommended (full plan)** | [00-meta/COMPLETE_REQUIRED_OPTIONAL_RECOMMENDED_INDEX.md](00-meta/COMPLETE_REQUIRED_OPTIONAL_RECOMMENDED_INDEX.md) | diff --git a/phoenix-deploy-api/README.md b/phoenix-deploy-api/README.md index 2d4a0fd2..07882035 100644 --- a/phoenix-deploy-api/README.md +++ b/phoenix-deploy-api/README.md @@ -55,6 +55,7 @@ Copy `.env.example` to `.env` and set `GITEA_TOKEN` (and optionally `PHOENIX_DEP | SERVER_FUNDS_SIDECAR_HEALTH_PATH | | Optional first path to try (e.g. `/actuator/health`); also tries `/actuator/health`, `/health`, `/api/health` | | PHOENIX_REPO_ROOT | | Proxmox repo root; loads `config/public-sector-program-manifest.json` if present | | DEPLOY_TARGETS_PATH | | Override deploy target file; default is `phoenix-deploy-api/deploy-targets.json` | +| PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH | 0 | Set to `1` or `true` to reject `POST /api/deploy` when `sha` is set but that commit is not on the requested `branch` (Gitea API: commit + compare merge-base). Requires `GITEA_TOKEN`. | **URA smoke (proxmox repo):** `bash scripts/verify/smoke-universal-resource-activation.sh` validates the manifest with JSON Schema; add `--http` and set `PHOENIX_BASE_URL` to also GET `/api/v1/universal-resource-activation/manifest`, `/api/v1/universal-resource-activation/policy-profiles`, and `/api/v1/universal-resource-activation/server-funds-sidecar-probe` (503 + `configured: false` is OK if `SERVER_FUNDS_SIDECAR_URL` is unset). See [UNIVERSAL_RESOURCE_WIRING.md](../docs/04-configuration/universal-resource-activation/UNIVERSAL_RESOURCE_WIRING.md) §2.1. diff --git a/phoenix-deploy-api/deploy-targets.json b/phoenix-deploy-api/deploy-targets.json index 619b0746..08ae167e 100644 --- a/phoenix-deploy-api/deploy-targets.json +++ b/phoenix-deploy-api/deploy-targets.json @@ -150,6 +150,156 @@ "timeout_ms": 15000 } }, + { + "repo": "Gov_Web_Portals/ICCC", + "branch": "main", + "target": "iccc-portal-live", + "description": "Redeploy the ICCC portal on CT 7804 from the staged ICCC checkout overlaid into the Gov Portals workspace.", + "cwd": "${PHOENIX_REPO_ROOT}", + "command": [ + "bash", + "scripts/deployment/phoenix-deploy-gov-portal-live-from-workspace.sh", + "ICCC" + ], + "required_env": [ + "PHOENIX_REPO_ROOT", + "PHOENIX_DEPLOY_WORKSPACE" + ], + "timeout_sec": 2400, + "healthcheck": { + "url": "https://iccc.xom-dev.phoenix.sankofa.nexus/", + "expect_status": 200, + "expect_body_includes": " k.trim()).filter(Boolean); const WEBHOOK_DEPLOY_ENABLED = process.env.PHOENIX_WEBHOOK_DEPLOY_ENABLED === '1' || process.env.PHOENIX_WEBHOOK_DEPLOY_ENABLED === 'true'; +const DEPLOY_VERIFY_SHA_ON_BRANCH = + process.env.PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH === '1' || + process.env.PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH === 'true'; const execFile = promisify(execFileCallback); function expandEnvTokens(value, env = process.env) { @@ -169,6 +172,111 @@ function findDeployTarget(repo, branch, requestedTarget) { return { config, match, wantedTarget }; } +async function giteaApiGetJson(apiPath) { + if (!GITEA_TOKEN) { + return { ok: false, status: 503, json: null, text: 'GITEA_TOKEN unset' }; + } + const pathPart = apiPath.startsWith('/') ? apiPath : `/${apiPath}`; + const url = `${GITEA_URL}${pathPart}`; + const res = await fetch(url, { headers: { Authorization: `token ${GITEA_TOKEN}` } }); + const text = await res.text(); + let json = null; + try { + json = text ? JSON.parse(text) : null; + } catch (_) { + json = null; + } + return { ok: res.ok, status: res.status, json, text }; +} + +function normalizeCommitSha(s) { + return String(s || '') + .toLowerCase() + .replace(/[^a-f0-9]/g, ''); +} + +/** + * When PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH=1, require commit sha to exist on the + * given branch (tip match or merge-base check via Gitea compare API). + */ +async function assertShaIsOnBranch(owner, repoName, branch, shaInput) { + if (!DEPLOY_VERIFY_SHA_ON_BRANCH || !GITEA_TOKEN) return; + const trimmed = String(shaInput || '').trim(); + if (!trimmed || trimmed === 'HEAD' || trimmed === 'local') return; + if (trimmed.length < 7) { + const err = new Error( + 'sha must be at least 7 characters when PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH is enabled', + ); + err.statusCode = 400; + err.payload = { error: err.message }; + throw err; + } + const enc = encodeURIComponent; + const base = `/api/v1/repos/${enc(owner)}/${enc(repoName)}`; + + const commitRes = await giteaApiGetJson(`${base}/commits/${enc(trimmed)}`); + if (!commitRes.ok) { + const err = new Error(`Gitea: commit not found for ${owner}/${repoName}`); + err.statusCode = 400; + err.payload = { + error: err.message, + gitea_status: commitRes.status, + branch, + sha: trimmed.slice(0, 12), + }; + throw err; + } + + const resolved = normalizeCommitSha( + commitRes.json?.sha || commitRes.json?.commit?.id || trimmed, + ); + if (!resolved || resolved.length < 7) { + const err = new Error('Gitea: could not resolve commit sha'); + err.statusCode = 400; + err.payload = { error: err.message }; + throw err; + } + + const branchRes = await giteaApiGetJson(`${base}/branches/${enc(branch)}`); + if (!branchRes.ok || !branchRes.json?.commit?.id) { + const err = new Error(`Gitea: branch not found: ${branch}`); + err.statusCode = 400; + err.payload = { error: err.message, gitea_status: branchRes.status }; + throw err; + } + const tip = normalizeCommitSha(branchRes.json.commit.id); + if (tip === resolved) return; + + const basehead = `${trimmed}...${branch}`; + const cmp = await giteaApiGetJson(`${base}/compare/${encodeURIComponent(basehead)}`); + if (!cmp.ok) { + const err = new Error(`Gitea: compare failed; sha may not belong to ${branch}`); + err.statusCode = 400; + err.payload = { error: err.message, gitea_status: cmp.status, compare: basehead }; + throw err; + } + const mergeBaseSha = normalizeCommitSha( + cmp.json?.merge_base_commit?.sha || + cmp.json?.merge_base_commit || + cmp.json?.base_commit?.sha || + '', + ); + if (mergeBaseSha && mergeBaseSha === resolved) return; + + const err = new Error( + `Deploy rejected: sha is not on branch ${branch} (disable with PHOENIX_DEPLOY_VERIFY_SHA_ON_BRANCH=0)`, + ); + err.statusCode = 400; + err.payload = { + error: err.message, + owner, + repo: repoName, + branch, + sha: trimmed.slice(0, 12), + }; + throw err; +} + async function sleep(ms) { await new Promise((resolve) => setTimeout(resolve, ms)); } @@ -427,6 +535,8 @@ async function executeDeploy({ repo, branch = 'main', target = 'default', sha = throw error; } + await assertShaIsOnBranch(owner, repoName, branch, commitSha); + if (commitSha && GITEA_TOKEN) { await setGiteaCommitStatus(owner, repoName, commitSha, 'pending', 'Phoenix deployment in progress'); } diff --git a/scripts/deployment/gitea-tag-repo-release.sh b/scripts/deployment/gitea-tag-repo-release.sh new file mode 100755 index 00000000..71849d14 --- /dev/null +++ b/scripts/deployment/gitea-tag-repo-release.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +# Create an annotated tag on a Gitea repo at an existing commit (optional release marker). +# +# Usage: +# source scripts/lib/load-project-env.sh # or export GITEA_TOKEN +# bash scripts/deployment/gitea-tag-repo-release.sh Gov_Web_Portals/DBIS v2026.05.12-dbis <40-char-sha> +# +# Requires: GITEA_TOKEN with write access to the repo; curl; jq. + +set -euo pipefail + +die() { echo "ERROR: $*" >&2; exit 1; } + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" +# shellcheck source=/dev/null +[ -f "$ROOT/scripts/lib/load-project-env.sh" ] && source "$ROOT/scripts/lib/load-project-env.sh" 2>/dev/null || true + +REPO_FULL="${1:-}" +TAG_NAME="${2:-}" +SHA="${3:-}" + +[[ -n "$REPO_FULL" ]] || die "usage: $0 " +[[ -n "$TAG_NAME" ]] || die "usage: $0 " +[[ -n "$SHA" ]] || die "usage: $0 " +[[ -n "${GITEA_TOKEN:-}" ]] || die "GITEA_TOKEN is required" + +GITEA_URL="${GITEA_URL:-https://gitea.d-bis.org}" +GITEA_URL="${GITEA_URL%/}" + +OWNER="${REPO_FULL%%/*}" +NAME="${REPO_FULL#*/}" +[[ "$OWNER" != "$REPO_FULL" ]] || die "repo must be owner/name" + +if ! command -v jq >/dev/null 2>&1; then + die "jq is required for JSON body" +fi + +# Gitea CreateTagOption: tag_name, target (commit sha or branch), message (optional). +BODY="$(jq -nc --arg tag "$TAG_NAME" --arg tgt "$SHA" --arg msg "release $TAG_NAME" \ + '{tag_name:$tag, target:$tgt, message:$msg}')" + +HTTP_CODE="$(curl -sS -o /tmp/gitea-tag-res.json -w '%{http_code}' \ + -X POST "${GITEA_URL}/api/v1/repos/${OWNER}/${NAME}/tags" \ + -H "Authorization: token ${GITEA_TOKEN}" \ + -H "Content-Type: application/json" \ + -d "$BODY")" + +if [[ "$HTTP_CODE" != "201" && "$HTTP_CODE" != "200" ]]; then + echo "Gitea API HTTP $HTTP_CODE" >&2 + cat /tmp/gitea-tag-res.json >&2 || true + exit 1 +fi + +echo "Tagged $REPO_FULL @ $SHA as $TAG_NAME (HTTP $HTTP_CODE)" +rm -f /tmp/gitea-tag-res.json diff --git a/scripts/deployment/phoenix-deploy-dbis-portal-live-from-workspace.sh b/scripts/deployment/phoenix-deploy-dbis-portal-live-from-workspace.sh index d4f15d27..830560ed 100755 --- a/scripts/deployment/phoenix-deploy-dbis-portal-live-from-workspace.sh +++ b/scripts/deployment/phoenix-deploy-dbis-portal-live-from-workspace.sh @@ -1,200 +1,7 @@ #!/usr/bin/env bash -# Deploy the DBIS public portal from a Phoenix Deploy API staged DBIS checkout. -# -# The DBIS repo is normally a submodule of Gov_Web_Portals/gov-portals-monorepo -# and depends on the parent workspace package @public-web-portals/shared. This -# wrapper builds a temporary monorepo-shaped workspace, overlays the staged DBIS -# source into it, syncs that tree to CT 7804, then rebuilds/restarts DBIS. +# Deploy the DBIS public portal from a Phoenix-staged DBIS checkout. +# Implementation: shared gov-portal driver (monorepo overlay + CT 7804). set -euo pipefail - -die() { - echo "ERROR: $*" >&2 - exit 1 -} - SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" - -source "$PROJECT_ROOT/config/ip-addresses.conf" 2>/dev/null || true -[ -f "$PROJECT_ROOT/.env" ] && set +u && source "$PROJECT_ROOT/.env" 2>/dev/null || true && set -u - -PHOENIX_REPO_ROOT="${PHOENIX_REPO_ROOT:-$PROJECT_ROOT}" -PHOENIX_DEPLOY_WORKSPACE="${PHOENIX_DEPLOY_WORKSPACE:-}" -GOV_PORTALS_REPO_URL="${GOV_PORTALS_REPO_URL:-https://gitea.d-bis.org/Gov_Web_Portals/gov-portals-monorepo.git}" -GOV_PORTALS_REF="${GOV_PORTALS_REF:-main}" - -VMID_GOV_PORTALS="${VMID_GOV_PORTALS:-7804}" -IP_GOV_PORTALS_DEV="${IP_GOV_PORTALS_DEV:-192.168.11.54}" -PROXMOX_HOST="${DBIS_PORTAL_PROXMOX_HOST:-${PROXMOX_HOST_GOV_PORTALS:-192.168.11.14}}" -CT_APP_DIR="${DBIS_PORTAL_CT_DIR:-/srv/gov-portals}" -SERVICE_NAME="${DBIS_PORTAL_SERVICE:-gov-portal-DBIS}" -DBIS_PORT="${DBIS_PORT:-3001}" - -[[ -d "$PHOENIX_REPO_ROOT" ]] || die "PHOENIX_REPO_ROOT does not exist: $PHOENIX_REPO_ROOT" -[[ -n "$PHOENIX_DEPLOY_WORKSPACE" ]] || die "PHOENIX_DEPLOY_WORKSPACE is required" -[[ -d "$PHOENIX_DEPLOY_WORKSPACE" ]] || die "staged DBIS workspace missing: $PHOENIX_DEPLOY_WORKSPACE" -[[ "$CT_APP_DIR" != "/" ]] || die "refusing to deploy into /" - -TMP_DIR="$(mktemp -d)" -BUILD_CONTEXT="$TMP_DIR/gov-portals" -ARCHIVE="$TMP_DIR/gov-portals-dbis-live.tgz" -REMOTE_ARCHIVE="/tmp/gov-portals-dbis-live-${PHOENIX_DEPLOY_SHA:-manual}-$$.tgz" -REMOTE_STAGE_DIR="/srv/gov-portals-new-${PHOENIX_DEPLOY_SHA:-manual}-$$" - -cleanup() { - rm -rf "$TMP_DIR" -} -trap cleanup EXIT - -echo "Preparing DBIS live deploy context" -echo " DBIS source: $PHOENIX_DEPLOY_WORKSPACE" -echo " parent repo: $GOV_PORTALS_REPO_URL#$GOV_PORTALS_REF" -echo " target: CT $VMID_GOV_PORTALS ($IP_GOV_PORTALS_DEV), service $SERVICE_NAME, port $DBIS_PORT" - -git_auth_args=() -if [[ -n "${GITEA_TOKEN:-}" ]]; then - git_auth_args=(-c "http.extraHeader=Authorization: token ${GITEA_TOKEN}") -fi - -git "${git_auth_args[@]}" clone --depth 1 --branch "$GOV_PORTALS_REF" "$GOV_PORTALS_REPO_URL" "$BUILD_CONTEXT" - -rm -rf "$BUILD_CONTEXT/DBIS" -mkdir -p "$BUILD_CONTEXT/DBIS" -tar \ - --exclude=.git \ - --exclude=node_modules \ - --exclude=.next \ - --exclude='*.tsbuildinfo' \ - -C "$PHOENIX_DEPLOY_WORKSPACE" \ - -cf - . | tar -C "$BUILD_CONTEXT/DBIS" -xf - - -tar \ - --exclude=.git \ - --exclude=node_modules \ - --exclude=.next \ - --exclude='*.tsbuildinfo' \ - -C "$BUILD_CONTEXT" \ - -czf "$ARCHIVE" . - -echo "Uploading deploy archive to Proxmox host $PROXMOX_HOST" -scp -q -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "$ARCHIVE" "root@$PROXMOX_HOST:$REMOTE_ARCHIVE" - -echo "Pushing archive into CT $VMID_GOV_PORTALS" -ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "root@$PROXMOX_HOST" \ - "pct push $VMID_GOV_PORTALS '$REMOTE_ARCHIVE' '$REMOTE_ARCHIVE'" - -echo "Extracting and building DBIS in a staged directory inside CT $VMID_GOV_PORTALS" -ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "root@$PROXMOX_HOST" \ - "pct exec $VMID_GOV_PORTALS -- bash -s" </dev/null 2>&1; then - apt-get update -qq - apt-get install -y -qq curl ca-certificates -fi - -NODE_MAJOR="\$(node -p 'process.versions.node.split(\".\")[0]' 2>/dev/null || echo 0)" -if [ "\$NODE_MAJOR" -lt 20 ]; then - curl -fsSL https://deb.nodesource.com/setup_20.x | bash - - apt-get install -y nodejs - hash -r -fi - -export PATH="/usr/local/bin:/usr/bin:/bin:\$PATH" -export NEXT_TELEMETRY_DISABLED=1 -export NODE_OPTIONS="\${NODE_OPTIONS:---max-old-space-size=4096}" -if ! command -v pnpm >/dev/null 2>&1; then - npm install -g pnpm@8.15.0 - hash -r -fi -PNPM_BIN="\$(command -v pnpm || true)" -if [ -z "\$PNPM_BIN" ]; then - for candidate in /usr/local/bin/pnpm /usr/bin/pnpm; do - if [ -x "\$candidate" ]; then - PNPM_BIN="\$candidate" - break - fi - done -fi -[ -n "\$PNPM_BIN" ] || { echo "pnpm is required but was not found after install" >&2; exit 1; } - -cd "\$REMOTE_STAGE_DIR" -"\$PNPM_BIN" install --frozen-lockfile -"\$PNPM_BIN" --filter portal-dbis build - -PREVIOUS_DIR="" -if [ -e "\$CT_APP_DIR" ]; then - PREVIOUS_DIR="\${CT_APP_DIR}-previous-\$(date +%Y%m%d%H%M%S)" - mv "\$CT_APP_DIR" "\$PREVIOUS_DIR" -fi -mv "\$REMOTE_STAGE_DIR" "\$CT_APP_DIR" - -cat > "/etc/systemd/system/\$SERVICE_NAME.service" </dev/null; then - if [ -n "\$PREVIOUS_DIR" ] && [ -d "\$PREVIOUS_DIR" ]; then - rm -rf "\$CT_APP_DIR" - mv "\$PREVIOUS_DIR" "\$CT_APP_DIR" - systemctl restart "\$SERVICE_NAME" || true - fi - exit 1 -fi -CT_SCRIPT - -ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "root@$PROXMOX_HOST" "rm -f '$REMOTE_ARCHIVE'" >/dev/null 2>&1 || true - -echo "DBIS live deployment complete." -echo "Local origin check: http://$IP_GOV_PORTALS_DEV:$DBIS_PORT/" +exec bash "$SCRIPT_DIR/phoenix-deploy-gov-portal-live-from-workspace.sh" DBIS diff --git a/scripts/deployment/phoenix-deploy-gov-portal-live-from-workspace.sh b/scripts/deployment/phoenix-deploy-gov-portal-live-from-workspace.sh new file mode 100755 index 00000000..1affc3fb --- /dev/null +++ b/scripts/deployment/phoenix-deploy-gov-portal-live-from-workspace.sh @@ -0,0 +1,239 @@ +#!/usr/bin/env bash +# Deploy one Gov Web Portals Next app (DBIS, ICCC, OMNL, XOM) from a Phoenix-staged +# checkout of that portal repo, overlaid into a shallow gov-portals-monorepo clone, +# then sync to CT 7804 and restart its systemd unit. +# +# Usage: +# bash scripts/deployment/phoenix-deploy-gov-portal-live-from-workspace.sh DBIS +# Or: export GOV_PORTAL_SLUG=ICCC +# +# Requires: PHOENIX_REPO_ROOT, PHOENIX_DEPLOY_WORKSPACE (staged single-portal tree). + +set -euo pipefail + +die() { + echo "ERROR: $*" >&2 + exit 1 +} + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" + +source "$PROJECT_ROOT/config/ip-addresses.conf" 2>/dev/null || true +[ -f "$PROJECT_ROOT/.env" ] && set +u && source "$PROJECT_ROOT/.env" 2>/dev/null || true && set -u + +GOV_PORTAL_SLUG="${GOV_PORTAL_SLUG:-${1:-}}" +[[ -n "$GOV_PORTAL_SLUG" ]] || die "GOV_PORTAL_SLUG or first arg required (DBIS|ICCC|OMNL|XOM)" + +case "$GOV_PORTAL_SLUG" in + DBIS) + PORTAL_SUBDIR="DBIS" + PNPM_FILTER="portal-dbis" + PORTAL_PORT="${DBIS_PORT:-3001}" + SERVICE_NAME="${DBIS_PORTAL_SERVICE:-gov-portal-DBIS}" + ;; + ICCC) + PORTAL_SUBDIR="ICCC" + PNPM_FILTER="portal-iccc" + PORTAL_PORT="${ICCC_PORT:-3002}" + SERVICE_NAME="${ICCC_PORTAL_SERVICE:-gov-portal-ICCC}" + ;; + OMNL) + PORTAL_SUBDIR="OMNL" + PNPM_FILTER="portal-omnl" + PORTAL_PORT="${OMNL_PORT:-3003}" + SERVICE_NAME="${OMNL_PORTAL_SERVICE:-gov-portal-OMNL}" + ;; + XOM) + PORTAL_SUBDIR="XOM" + PNPM_FILTER="portal-xom" + PORTAL_PORT="${XOM_PORT:-3004}" + SERVICE_NAME="${XOM_PORTAL_SERVICE:-gov-portal-XOM}" + ;; + *) + die "Unknown GOV_PORTAL_SLUG=$GOV_PORTAL_SLUG (expected DBIS|ICCC|OMNL|XOM)" + ;; +esac + +PHOENIX_REPO_ROOT="${PHOENIX_REPO_ROOT:-$PROJECT_ROOT}" +PHOENIX_DEPLOY_WORKSPACE="${PHOENIX_DEPLOY_WORKSPACE:-}" +GOV_PORTALS_REPO_URL="${GOV_PORTALS_REPO_URL:-https://gitea.d-bis.org/Gov_Web_Portals/gov-portals-monorepo.git}" +GOV_PORTALS_REF="${GOV_PORTALS_REF:-main}" + +VMID_GOV_PORTALS="${VMID_GOV_PORTALS:-7804}" +IP_GOV_PORTALS_DEV="${IP_GOV_PORTALS_DEV:-192.168.11.54}" +PROXMOX_HOST="${GOV_PORTAL_PROXMOX_HOST:-${DBIS_PORTAL_PROXMOX_HOST:-${PROXMOX_HOST_GOV_PORTALS:-192.168.11.14}}}" +CT_APP_DIR="${GOV_PORTAL_CT_DIR:-${DBIS_PORTAL_CT_DIR:-/srv/gov-portals}}" + +SLUG_LOWER="$(printf '%s' "$GOV_PORTAL_SLUG" | tr '[:upper:]' '[:lower:]')" + +[[ -d "$PHOENIX_REPO_ROOT" ]] || die "PHOENIX_REPO_ROOT does not exist: $PHOENIX_REPO_ROOT" +[[ -n "$PHOENIX_DEPLOY_WORKSPACE" ]] || die "PHOENIX_DEPLOY_WORKSPACE is required" +[[ -d "$PHOENIX_DEPLOY_WORKSPACE" ]] || die "staged workspace missing: $PHOENIX_DEPLOY_WORKSPACE" +[[ "$CT_APP_DIR" != "/" ]] || die "refusing to deploy into /" + +TMP_DIR="$(mktemp -d)" +BUILD_CONTEXT="$TMP_DIR/gov-portals" +ARCHIVE="$TMP_DIR/gov-portals-${SLUG_LOWER}-live.tgz" +REMOTE_ARCHIVE="/tmp/gov-portals-${SLUG_LOWER}-live-${PHOENIX_DEPLOY_SHA:-manual}-$$.tgz" +REMOTE_STAGE_DIR="/srv/gov-portals-new-${SLUG_LOWER}-${PHOENIX_DEPLOY_SHA:-manual}-$$" + +cleanup() { + rm -rf "$TMP_DIR" +} +trap cleanup EXIT + +echo "Preparing $GOV_PORTAL_SLUG live deploy context" +echo " Portal source: $PHOENIX_DEPLOY_WORKSPACE" +echo " parent repo: $GOV_PORTALS_REPO_URL#$GOV_PORTALS_REF" +echo " target: CT $VMID_GOV_PORTALS ($IP_GOV_PORTALS_DEV), service $SERVICE_NAME, port $PORTAL_PORT" + +git_auth_args=() +if [[ -n "${GITEA_TOKEN:-}" ]]; then + git_auth_args=(-c "http.extraHeader=Authorization: token ${GITEA_TOKEN}") +fi + +git "${git_auth_args[@]}" clone --depth 1 --branch "$GOV_PORTALS_REF" "$GOV_PORTALS_REPO_URL" "$BUILD_CONTEXT" + +rm -rf "$BUILD_CONTEXT/$PORTAL_SUBDIR" +mkdir -p "$BUILD_CONTEXT/$PORTAL_SUBDIR" +tar \ + --exclude=.git \ + --exclude=node_modules \ + --exclude=.next \ + --exclude='*.tsbuildinfo' \ + -C "$PHOENIX_DEPLOY_WORKSPACE" \ + -cf - . | tar -C "$BUILD_CONTEXT/$PORTAL_SUBDIR" -xf - + +tar \ + --exclude=.git \ + --exclude=node_modules \ + --exclude=.next \ + --exclude='*.tsbuildinfo' \ + -C "$BUILD_CONTEXT" \ + -czf "$ARCHIVE" . + +echo "Uploading deploy archive to Proxmox host $PROXMOX_HOST" +scp -q -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "$ARCHIVE" "root@$PROXMOX_HOST:$REMOTE_ARCHIVE" + +echo "Pushing archive into CT $VMID_GOV_PORTALS" +ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "root@$PROXMOX_HOST" \ + "pct push $VMID_GOV_PORTALS '$REMOTE_ARCHIVE' '$REMOTE_ARCHIVE'" + +echo "Extracting and building $GOV_PORTAL_SLUG in a staged directory inside CT $VMID_GOV_PORTALS" +ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "root@$PROXMOX_HOST" \ + "pct exec $VMID_GOV_PORTALS -- bash -s" </dev/null 2>&1; then + apt-get update -qq + apt-get install -y -qq curl ca-certificates +fi + +NODE_MAJOR="\$(node -p 'process.versions.node.split(\".\")[0]' 2>/dev/null || echo 0)" +if [ "\$NODE_MAJOR" -lt 20 ]; then + curl -fsSL https://deb.nodesource.com/setup_20.x | bash - + apt-get install -y nodejs + hash -r +fi + +export PATH="/usr/local/bin:/usr/bin:/bin:\$PATH" +export NEXT_TELEMETRY_DISABLED=1 +export NODE_OPTIONS="\${NODE_OPTIONS:---max-old-space-size=4096}" +if ! command -v pnpm >/dev/null 2>&1; then + npm install -g pnpm@8.15.0 + hash -r +fi +PNPM_BIN="\$(command -v pnpm || true)" +if [ -z "\$PNPM_BIN" ]; then + for candidate in /usr/local/bin/pnpm /usr/bin/pnpm; do + if [ -x "\$candidate" ]; then + PNPM_BIN="\$candidate" + break + fi + done +fi +[ -n "\$PNPM_BIN" ] || { echo "pnpm is required but was not found after install" >&2; exit 1; } + +cd "\$REMOTE_STAGE_DIR" +"\$PNPM_BIN" install --frozen-lockfile +"\$PNPM_BIN" --filter "\$PNPM_FILTER" build + +PREVIOUS_DIR="" +if [ -e "\$CT_APP_DIR" ]; then + PREVIOUS_DIR="\${CT_APP_DIR}-previous-\$(date +%Y%m%d%H%M%S)" + mv "\$CT_APP_DIR" "\$PREVIOUS_DIR" +fi +mv "\$REMOTE_STAGE_DIR" "\$CT_APP_DIR" + +cat > "/etc/systemd/system/\$SERVICE_NAME.service" </dev/null; then + if [ -n "\$PREVIOUS_DIR" ] && [ -d "\$PREVIOUS_DIR" ]; then + rm -rf "\$CT_APP_DIR" + mv "\$PREVIOUS_DIR" "\$CT_APP_DIR" + systemctl restart "\$SERVICE_NAME" || true + fi + exit 1 +fi +CT_SCRIPT + +ssh -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "root@$PROXMOX_HOST" "rm -f '$REMOTE_ARCHIVE'" >/dev/null 2>&1 || true + +echo "$GOV_PORTAL_SLUG live deployment complete." +echo "Local origin check: http://$IP_GOV_PORTALS_DEV:$PORTAL_PORT/" diff --git a/scripts/verify/report-gitea-cd-parity.sh b/scripts/verify/report-gitea-cd-parity.sh index 94974876..26e0a8a9 100755 --- a/scripts/verify/report-gitea-cd-parity.sh +++ b/scripts/verify/report-gitea-cd-parity.sh @@ -25,4 +25,10 @@ for d in explorer-monorepo cross-chain-pmm-lps; do ls -1 "$ROOT/$d/.gitea/workflows" 2>/dev/null || true fi done -echo "See config/gitea-workflow-templates/repos/ for copy-paste workflows for repos not submodules here." +echo "" +echo "== Workflow templates (config/gitea-workflow-templates/repos/) ==" +shopt -s nullglob +for f in "$ROOT/config/gitea-workflow-templates/repos/"*.yml; do + basename "$f" +done +shopt -u nullglob