d-bis/proxmox
4
0
Fork 0
Code Issues 2 Pull Requests 13 Actions Packages Projects Releases Wiki Activity
Files
b388cd1fbc65b4e0dcc2c2633e3efe6880b2b613
proxmox/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md
defiQUG 1f44a50a25 feat(it-ops): cluster live inventory + QEMU ipconfig LAN IPs 
Add scripts/it-ops export pipeline (collect_inventory_remote, compute_ipam_drift)
and proxmox_guest_lan_ips parser for ipconfig* and all net* interfaces.

Reconcile ALL_VMIDS, ip-addresses.conf, and operational template with live
VMID/IP data; Order portal env vars; DBIS node matrix; inventory helpers.

Track latest reports/status/live_inventory.json and drift.json (137 guests,
no duplicate LAN IPs). Document export in AGENTS.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 10:22:59 -07:00

44 KiB
Raw Blame History

Complete VMID and Endpoints Reference

Last Updated: 2026-05-11
Document Version: 1.4
Status: Active Documentation — Master (source of truth) for VMID, IP, port, and domain mapping. Use this with the live Besu fleet map in ../06-besu/BESU_NODE_CONFIGURATION_MAP_20260424.md and the cluster audit in ../../scripts/verify/check-cluster-besu-inventory.sh.

Operational template (hosts, peering, deployment gates, JSON): ../03-deployment/PROXMOX_VE_OPERATIONAL_DEPLOYMENT_TEMPLATE.md · config/proxmox-operational-template.json

Date: 2026-05-11
Status: Current Active Configuration (Reconciled)
Last Updated: 2026-05-11
Verification Status: ✅ Cluster-wide guest inventory — 137 running LXC/QEMU (regenerate: bash scripts/it-ops/export-live-inventory-and-drift.sh → reports/status/live_inventory.json; same collector runs on r630-01 as /opt/proxmox/...). Parses ipconfig* (QEMU) and net* (LXC). ml110 0 guests; primary counts on r630-01 (57), r630-02 (41), r630-03 (20), r630-04 (19). Besu fleet detail: host audit + ../../scripts/verify/check-cluster-besu-inventory.sh.

Quick Summary

  • Cluster (all nodes, LXC+QEMU) — running: 137 (2026-05-11 live inventory export); all were running in that pass.
  • Per Proxmox node (guests): r630-01 57, r630-02 41, r630-03 20, r630-04 19, ml110 0.
  • Documented VMID rows in this file: 50+ service entries (excl. deprecated); category rolls below are Besu / app taxonomy — reconcile exact Besu counts with check-cluster-besu-inventory.sh and the Besu map doc.
  • Infrastructure Services (sample category): 10
  • Blockchain Nodes: 37 canonical Besu nodes (Validators: 5, Sentries: 11, RPC: 21) — verify against live map
  • Application Services: 22 (category roll — verify)

Canonical-use guardrails

Infrastructure Services

Proxmox Infrastructure (mostly r630-01; Omada not present)

VMID IP Address Hostname Status Endpoints Purpose
100 192.168.11.32 proxmox-mail-gateway ✅ Running SMTP: 25, 587, 465 Email gateway
101 192.168.11.33 proxmox-datacenter-manager ✅ Running Web: 8006 Datacenter management
102 192.168.11.34 cloudflared ✅ Running Tunnel Cloudflare Tunnel (cloudflared)
104 192.168.11.31 gitea ✅ Running Web: 80, 443 Git repository
105 192.168.11.26 nginxproxymanager ✅ Running Web: 80, 81, 443 Nginx Proxy Manager (legacy)
130 192.168.11.27 monitoring-1 ✅ Running Web: 80, 443 Monitoring services

Not in live cluster inventory (2026-05-11): 103 (Omada). Prior doc row 192.168.11.30 omada is retired unless reprovisioned.

NPMplus (r630-01 / r630-02)

VMID IP Address Hostname Status Endpoints Purpose
10233 192.168.11.167 npmplus ✅ Running Web: 80, 81, 443 NPMplus reverse proxy (primary ingress)
10234 192.168.11.168 npmplus-secondary ✅ Running Web: 80, 81, 443 NPMplus secondary (HA standby); r630-02
10235 192.168.11.169 npmplus-alltra-hybx ✅ Running Web: 80, 81, 443 Third NPM — Alltra/HYBX / rpc-core-2 style paths
10236 192.168.11.170 npmplus-fourth ✅ Running Web: 80, 81, 443 Fourth NPM — dev / Codespaces / Gitea tunnel
10237 192.168.11.171 npmplus-mifos ✅ Running Web: 80, 81, 443 NPMplus (Mifos / Fineract path); r630-02

Live placement (reconcile with cluster): Run bash scripts/maintenance/npmplus-cluster-placement-status.sh. As of 2026-05, 10233, 10235, 10236 often run on r630-01; 10234 on r630-02. Target: redistribute for blast-radius — see NPMPLUS_MISSION_CRITICAL_DISTRIBUTION_AND_HA_PLAN.md.

Dual-homed (10233): net0 192.168.11.166/24; net1 192.168.11.167/24 (default route / gateway on .167). Use .167 for NPMplus API, public ingress, and runbook examples. live_inventory.json may list only the first net* address (.166).

Note: NPMplus primary is on VLAN 11 (192.168.11.167). Secondary NPMplus instance on r630-02 for HA configuration.

Operational note (2026-03-26): if 192.168.11.167:81 accepts TCP but hangs without returning HTTP, CT 10233 may be wedged even when networking looks healthy. Rebooting it from r630-01 with pct reboot 10233 restored the expected 301 on port 81 and unblocked the API updater.

RPC Translator Supporting Services

Status (2026-05-11): VMIDs 106–108 are not present in cluster live_inventory.json. Prior translator LXCs are retired unless reprovisioned.

VMID IP Address Hostname Status Endpoints Purpose
— — (historical) redis-rpc-translator Retired Redis: 6379 Was ~106 @ .110
— — (historical) web3signer-rpc-translator Retired Web3Signer: 9000 Was ~107 @ .111
— — (historical) vault-rpc-translator Retired Vault: 8200 Was ~108 @ .112

Live reassignment: .111 / .112 are used by 8811 (sankofa-proxmox-mcp) and 8812 (operator-services) on r630-04 — see Supplementary cluster inventory.

Blockchain Nodes - Validators (ChainID 138)

VMID IP Address Hostname Status Endpoints Purpose
1000 192.168.11.100 besu-validator-1 ✅ Running P2P: 30303, Metrics: 9545 Validator node 1
1001 192.168.11.101 besu-validator-2 ✅ Running P2P: 30303, Metrics: 9545 Validator node 2
1002 192.168.11.102 besu-validator-3 ✅ Running P2P: 30303, Metrics: 9545 Validator node 3
1003 192.168.11.103 besu-validator-4 ✅ Running P2P: 30303, Metrics: 9545 Validator node 4
1004 192.168.11.104 besu-validator-5 ✅ Running P2P: 30303, Metrics: 9545 Validator node 5

Blockchain Nodes - Sentries (ChainID 138)

VMID IP Address Hostname Status Endpoints Purpose
1500 192.168.11.150 besu-sentry-1 ✅ Running P2P: 30303, Metrics: 9545 Sentry node 1
1501 192.168.11.151 besu-sentry-2 ✅ Running P2P: 30303, Metrics: 9545 Sentry node 2
1502 192.168.11.152 besu-sentry-3 ✅ Running P2P: 30303, Metrics: 9545 Sentry node 3
1503 192.168.11.153 besu-sentry-4 ✅ Running P2P: 30303, Metrics: 9545 Sentry node 4
1504 192.168.11.154 besu-sentry-ali ✅ Running P2P: 30303, Metrics: 9545 Sentry node (Ali)
1505 192.168.11.213 besu-sentry-alltra-1 ✅ Running P2P: 30303, Metrics: 9545 Sentry (Alltra 1)
1506 192.168.11.214 besu-sentry-alltra-2 ✅ Running P2P: 30303, Metrics: 9545 Sentry (Alltra 2)
1507 192.168.11.244 besu-sentry-hybx-1 ✅ Running P2P: 30303, Metrics: 9545 Sentry (HYBX 1)
1508 192.168.11.245 besu-sentry-hybx-2 ✅ Running P2P: 30303, Metrics: 9545 Sentry (HYBX 2)
1509 192.168.11.219 besu-sentry-thirdweb-01 ✅ Running P2P: 30303, Metrics: 9545 Sentry (Thirdweb 1)
1510 192.168.11.220 besu-sentry-thirdweb-02 ✅ Running P2P: 30303, Metrics: 9545 Sentry (Thirdweb 2)

Note: 1505-1506 moved from .170/.171 to .213/.214 (2026-02-01) to free CCIP Ops interim range. Live SSH / cluster note (2026-04-24): 1500-1506 were found during the initial 3-host pass. Cluster-wide reconciliation then confirmed 1507, 1509, and 1510 on r630-03 (192.168.11.13), plus 1508 on r630-04 (192.168.11.14).

RPC Nodes - NEW VMID Structure (ChainID 138)

Migration Status: ✅ Complete (2026-01-18)

All RPC nodes have been migrated to a new VMID structure for better organization.

Core RPC Nodes

Live reconciliation note (2026-04-24):

  • Initial direct host pass confirmed 2101, 2103, 2201
  • Cluster-wide reconciliation then confirmed:
    • 2102 on r630-03
    • 2301 on r630-03
VMID IP Address Hostname Status Block Peers Endpoints Purpose
2101 192.168.11.211 besu-rpc-core-1 ✅ Running 1,145,367 7 Besu: 8545/8546, P2P: 30303, Metrics: 9545 Core RPC node
2102 192.168.11.212 besu-rpc-core-2 ✅ Running (live) (live) Besu: 8545/8546, P2P: 30303, Metrics: 9545 Core RPC node 2 (r630-03)
2103 192.168.11.217 besu-rpc-core-thirdweb ✅ Running Live SSH verified 2026-04-24 Live SSH verified 2026-04-24 Besu: 8545/8546, P2P: 30303, Metrics: 9545 Core Thirdweb admin RPC node
2201 192.168.11.221 besu-rpc-public-1 ✅ Running 1,145,367 7 Besu: 8545/8546, P2P: 30303, Metrics: 9545 Public RPC node (FIXED PERMANENT)
2301 192.168.11.232 besu-rpc-private-1 ✅ Running Cluster CT confirmed on r630-03 - Besu: 8545/8546, P2P: 30303, Metrics: 9545 Fireblocks-dedicated RPC on r630-03

Extra (non-canonical) Besu RPC — Justin / Jason variants

These LXCs are running and appear in scripts/verify/check-cluster-besu-inventory.sh as extra/non-canonical (parallel RPC paths). They are not in the minimal canonical Besu map; do not decommission without ops coordination.

VMID IP Address Hostname Node Endpoints (typical)
2104 192.168.11.222 besu-rpc-core-justin r630-03 Besu: 8545/8546, P2P: 30303, Metrics: 9545
2105 192.168.11.225 besu-rpc-core-jason r630-03 Besu: 8545/8546, P2P: 30303, Metrics: 9545
2202 192.168.11.223 besu-rpc-public-justin r630-02 Besu: 8545/8546, P2P: 30303, Metrics: 9545
2203 192.168.11.226 besu-rpc-public-jason r630-02 Besu: 8545/8546, P2P: 30303, Metrics: 9545
2309 192.168.11.224 besu-rpc-private-justin r630-03 Besu: 8545/8546, P2P: 30303, Metrics: 9545
2310 192.168.11.227 besu-rpc-private-jason r630-03 Besu: 8545/8546, P2P: 30303, Metrics: 9545

Named RPC Nodes (Ali/Luis/Putu)

VMID IP Address Hostname Status Block Peers Endpoints Purpose
2303 192.168.11.233 besu-rpc-ali-0x8a ✅ Running 1,145,367 7 Besu: 8545/8546, P2P: 30303, Metrics: 9545 Ali RPC (0x8a identity)
2304 192.168.11.234 besu-rpc-ali-0x1 ✅ Running Cluster CT confirmed on r630-03 - Besu: 8545/8546, P2P: 30303, Metrics: 9545 Ali RPC (0x1 identity) on r630-03
2305 192.168.11.235 besu-rpc-luis-0x8a ✅ Running 1,145,367 7 Besu: 8545/8546, P2P: 30303, Metrics: 9545 Luis RPC (0x8a identity)
2306 192.168.11.236 besu-rpc-luis-0x1 ✅ Running 1,145,367 7 Besu: 8545/8546, P2P: 30303, Metrics: 9545 Luis RPC (0x1 identity)
2307 192.168.11.237 besu-rpc-putu-0x8a ✅ Running 1,145,367 7 Besu: 8545/8546, P2P: 30303, Metrics: 9545 Putu RPC (0x8a identity)
2308 192.168.11.238 besu-rpc-putu-0x1 ✅ Running 1,145,367 7 Besu: 8545/8546, P2P: 30303, Metrics: 9545 Putu RPC (0x1 identity)

ThirdWeb RPC Nodes

VMID IP Address Hostname Status Block Peers Endpoints Purpose
2400 192.168.11.240 thirdweb-rpc-1 ✅ Running Cluster CT confirmed on r630-03 - Nginx: 443, Besu: 8545/8546, P2P: 30303, Metrics: 9545, Translator: 9645/9646 ThirdWeb RPC with translator (primary) on r630-03
2401 192.168.11.241 besu-rpc-thirdweb-0x8a-1 ✅ Running 1,149,992 2 Besu: 8545/8546, P2P: 30303, Metrics: 9545 ThirdWeb RPC instance 1
2402 192.168.11.242 besu-rpc-thirdweb-0x8a-2 ✅ Running Cluster CT confirmed on r630-03 - Besu: 8545/8546, P2P: 30303, Metrics: 9545 ThirdWeb RPC instance 2 on r630-03
2403 192.168.11.243 besu-rpc-thirdweb-0x8a-3 ✅ Running Cluster CT confirmed on r630-03 - Besu: 8545/8546, P2P: 30303 ThirdWeb RPC instance 3 on r630-03

Note: VMID 2400 is the primary ThirdWeb RPC with Nginx and RPC Translator. VMID 2403 metrics disabled due to port conflict, node is syncing.

Public Domain: rpc.public-0138.defi-oracle.io → Routes through NPMplus to VMID 2201 (8545 for HTTPS JSON-RPC, 8546 for WSS Upgrade). VMID 2400 remains the ThirdWeb RPC/translator service, but it is not the ChainList-facing public RPC route.

Additional Live Internal ALLTRA / HYBX RPC Nodes (SSH-verified 2026-04-24)

These are live Besu RPC containers and should not be confused with the older decommissioned migration rows below that used different IPs and hostnames.

VMID IP Address Hostname Status Endpoints Purpose
2500 192.168.11.172 besu-rpc-alltra-1 ✅ Running Besu: 8545/8546, P2P: 30303, Metrics: 9545 Internal ALLTRA RPC 1
2501 192.168.11.173 besu-rpc-alltra-2 ✅ Running Besu: 8545/8546, P2P: 30303, Metrics: 9545 Internal ALLTRA RPC 2
2502 192.168.11.174 besu-rpc-alltra-3 ✅ Running Besu: 8545/8546, P2P: 30303, Metrics: 9545 Internal ALLTRA RPC 3
2503 192.168.11.246 besu-rpc-hybx-1 ✅ Running Besu: 8545/8546, P2P: 30303, Metrics: 9545 Internal HYBX RPC 1
2504 192.168.11.247 besu-rpc-hybx-2 ✅ Running Besu: 8545/8546, P2P: 30303, Metrics: 9545 Internal HYBX RPC 2
2505 192.168.11.248 besu-rpc-hybx-3 ✅ Running Besu: 8545/8546, P2P: 30303, Metrics: 9545 Internal HYBX RPC 3

Destroyed Legacy Duplicate ALLTRA / HYBX RPC Containers (Not Canonical Fleet)

These were found live on r630-01 during the same SSH pass, but they do not exist in config/proxmox-operational-template.json. They were first retired, then permanently destroyed on 2026-04-24. Use the 2500-2505 rows above as the canonical intended fleet.

VMID IP Address Hostname Status Endpoints Purpose
2420 192.168.11.172 besu-rpc-alltra-1 🗑 Destroyed pct destroy --purge 1 completed Legacy duplicate of canonical VMID 2500
2430 192.168.11.173 besu-rpc-alltra-2 🗑 Destroyed pct destroy --purge 1 completed Legacy duplicate of canonical VMID 2501
2440 192.168.11.174 besu-rpc-alltra-3 🗑 Destroyed pct destroy --purge 1 completed Legacy duplicate of canonical VMID 2502
2460 192.168.11.246 besu-rpc-hybx-1 🗑 Destroyed pct destroy --purge 1 completed Legacy duplicate of canonical VMID 2503
2470 192.168.11.247 besu-rpc-hybx-2 🗑 Destroyed pct destroy --purge 1 completed Legacy duplicate of canonical VMID 2504
2480 192.168.11.248 besu-rpc-hybx-3 🗑 Destroyed pct destroy --purge 1 completed Legacy duplicate of canonical VMID 2505

OLD RPC Nodes (Decommissioned)

Status: Historical migration reference only. The rows below refer to the old .250-.255/.201-.204 plan, not the live .172-.174/.246-.248 ALLTRA/HYBX RPCs found during the 2026-04-24 SSH pass.

Historic VMIDs 2500–2505 (Besu RPC, destroyed): former assignments — 2500 @ .250, 2501 @ .251, 2502 @ .252, 2503 @ .253, 2504 @ .254, 2505 @ .201 — superseded by VMIDs 2101, 2201, 2301, 2303, 2304, 2305 respectively. Those numeric VMIDs were later reused for ALLTRA/HYBX internal RPC (same VMID number, different workload). Current IPs are in Additional Live Internal ALLTRA / HYBX RPC Nodes.

The following VMIDs have been permanently removed (no reuse on live cluster):

VMID Old IP Address Old Hostname Status Replaced By
2506 192.168.11.202 besu-rpc-luis-0x1 🗑️ Destroyed VMID 2306
2507 192.168.11.203 besu-rpc-putu-0x8a 🗑️ Destroyed VMID 2307
2508 192.168.11.204 besu-rpc-putu-0x1 🗑️ Destroyed VMID 2308

Public Domains (need updating to new IPs):

  • rpc-http-prv.d-bis.org → Should route to new RPC nodes
  • rpc-ws-prv.d-bis.org → Should route to new RPC nodes
  • rpc-http-pub.d-bis.org → Should route to new RPC nodes
  • rpc-ws-pub.d-bis.org → Should route to new RPC nodes
  • rpc.public-0138.defi-oracle.io → Should route to 2401-2403

Application Services

Order of Malta — DealFlow Command Center (prod Compose)

VMID IP Address Hostname Node Status Endpoints Purpose
10381 192.168.11.94 treasury-dealflow r630-03 ✅ Running HTTPS: 443 (nginx → frontend/backend), HTTP: 80→443; Grafana 3001, Prometheus 9090, MinIO 9000/9001 treasury_management_monorepo Docker Compose prod

Allocated: Sovereign Cloud band 10000–13999 (VMID 10381). Storage: thin2-r630-03 (~80 GiB root). App dir: /opt/treasury_management_monorepo, SSH user deploy.

CI/CD: Gitea .gitea/workflows/deploy.yml — secrets TREASURY_DEPLOY_HOST, TREASURY_DEPLOY_USER, TREASURY_DEPLOY_SSH_KEY, TREASURY_DEPLOY_PATH; runner must reach 192.168.11.94 on LAN.

Public edge (2026-05): dealflow.d-bis.org → Cloudflare A on d-bis.org (script: scripts/update-all-dns-to-public-ip.sh) → UDM/NPMplus 76.53.10.36:443 → NPMplus https://192.168.11.94:443 (forward_scheme https). Backend CORS_ORIGIN must list https://dealflow.d-bis.org for POST /api/auth/demo-login from the browser. TLS: NPM Let’s Encrypt ( request-npmplus-certificates.sh with CERT_DOMAINS_FILTER if needed).

Blockchain Explorer

VMID IP Address Hostname Status Endpoints Purpose
5000 192.168.11.140 blockscout-1 ✅ Running Web: 80, 443; API: 4000 Blockchain explorer

Public Domain: explorer.d-bis.org → Routes to VMID 5000:80 (nginx serves web UI, proxies /api/* to port 4000)

Firefly

VMID IP Address Hostname Status Endpoints Purpose
6200 192.168.11.35 firefly-1 ✅ Running Web: 80, 443, API: 5000 Firefly DLT platform
6201 192.168.11.57 firefly-ali-1 ✅ Running Web: 80, 443, API: 5000 Firefly (Ali instance)

Note: Firefly instances run on r630-02. VMID 6200 also on r630-02.

DBIS RTGS first-slice sidecars

VMID IP Address Hostname Status Endpoints Purpose
5802 192.168.11.89 rtgs-scsm-1 ✅ Running App: 8080, Redis: 6379 DBIS RTGS mifos-fineract-sidecar / SCSM
5803 192.168.11.90 rtgs-funds-1 ✅ Running App: 8080, Redis: 6379 DBIS RTGS server-funds-sidecar
5804 192.168.11.92 rtgs-xau-1 ✅ Running App: 8080, Redis: 6379 DBIS RTGS off-ledger-2-on-ledger-sidecar

Operational note (2026-03-28/29):

  • These three sidecars are deployed internally on r630-02 and return local actuator health.
  • They can reach the live Mifos / Fineract surface on VMID 5800 at the HTTP layer.
  • Canonical authenticated RTGS flow is still pending final Fineract tenant/auth freeze, so these should currently be treated as runtime deployed, functionally partial.

Hyperledger Fabric

VMID IP Address Hostname Status Endpoints Purpose
6000 192.168.11.113 fabric-1 ✅ Running Peer: 7051, Orderer: 7050 Hyperledger Fabric network

Hyperledger Indy

VMID IP Address Hostname Status Endpoints Purpose
6400 192.168.11.64 indy-1 ✅ Running Indy: 9701-9708 Hyperledger Indy network

DBIS Core Services

VMID IP Address Hostname Status Endpoints Purpose
10100 192.168.11.105 dbis-postgres-primary ✅ Running PostgreSQL: 5432 Primary database
10101 192.168.11.106 dbis-postgres-replica-1 ✅ Running PostgreSQL: 5432 Database replica
10120 192.168.11.125 dbis-redis ✅ Running Redis: 6379 Cache layer
10130 192.168.11.130 dbis-frontend ✅ Running Web: 80, 443 Frontend admin console
10150 192.168.11.155 dbis-api-primary ✅ Running API: 3000 Primary API server
10151 192.168.11.156 dbis-api-secondary ✅ Running API: 3000 Secondary API server

Public Domains:

  • dbis-admin.d-bis.org → Routes to VMID 10130:80
  • secure.d-bis.org → Routes to VMID 10130:80
  • dbis-api.d-bis.org → Routes to VMID 10150:3000
  • dbis-api-2.d-bis.org → Routes to VMID 10151:3000

Miracles In Motion (MIM4U)

VMID IP Address Hostname Status Endpoints Purpose
7810 192.168.11.37 mim-web-1 ✅ Running Web: 80, 443 MIM4U web frontend
7811 192.168.11.36 mim-api-1 ✅ Running Web: 80, 443, API: Various MIM4U service (web + API)

Public Domains (NPMplus config):

  • mim4u.org → Routes to http://192.168.11.37:80 (VMID 7810 mim-web-1)
  • www.mim4u.org → Routes to http://192.168.11.37:80 (VMID 7810; optional NPMplus redirect www → apex)
  • secure.mim4u.org → Routes to http://192.168.11.37:80 (VMID 7810)
  • training.mim4u.org → Routes to http://192.168.11.37:80 (VMID 7810)

Note: All MIM4U domains route to VMID 7810 (mim-web-1) at 192.168.11.37. nginx on 7810 proxies /api/ to VMID 7811 (192.168.11.36:3001).

Sankofa Phoenix Services

Status: ✅ DEPLOYED AND OPERATIONAL (2026-01-20)

Verified Deployed Services:

VMID IP Address Hostname Status Endpoints Purpose
7800 192.168.11.50 sankofa-api-1 ✅ Running GraphQL: 4000, Health: /health Phoenix API (Cloud Platform Portal)
7801 192.168.11.51 sankofa-portal-1 ✅ Running Web: 3000 Sankofa Portal (Company Website)
7802 192.168.11.52 sankofa-keycloak-1 ✅ Running Keycloak: 8080, Admin: /admin Identity and Access Management
7803 192.168.11.53 sankofa-postgres-1 ✅ Running PostgreSQL: 5432 Database Service
7804 192.168.11.54 (Gov Portals dev) ✅ Running Web: 80 Gov Portals — DBIS, ICCC, OMNL, XOM (*.xom-dev.phoenix.sankofa.nexus)
7805 192.168.11.72 sankofa-studio — API: 8000 Sankofa Studio (FusionAI Creator) — studio.sankofa.nexus (IP .72; .55 = VMID 10230 order-vault)
5010 192.168.11.91 tsunamiswap planned / documented target Web: 80 TsunamiSwap origin — landing page https://tsunamiswap.com, working app https://app.tsunamiswap.com via NPMplus/Cloudflare

Public Domains (NPMplus routing):

  • sankofa.nexus → Routes to http://192.168.11.51:3000 (Sankofa Portal/VMID 7801) ✅
  • www.sankofa.nexus → Same upstream as apex; NPM advanced_config issues 301 to https://sankofa.nexus (preserve path/query via $request_uri). ✅
  • phoenix.sankofa.nexus → Routes to http://192.168.11.50:4000 (Phoenix API/VMID 7800) ✅
  • www.phoenix.sankofa.nexus → Same upstream; 301 to https://phoenix.sankofa.nexus. ✅
  • tsunamiswap.com → Intended landing page route to http://192.168.11.91:80 (TsunamiSwap / VMID 5010)
  • app.tsunamiswap.com → Intended working application route to http://192.168.11.91:80 (TsunamiSwap / VMID 5010)
  • the-order.sankofa.nexus / www.the-order.sankofa.nexus → OSJ management portal (secure auth). App source: the_order at ~/projects/the_order. NPMplus default upstream: order-haproxy http://192.168.11.39:80 (VMID 10210), which proxies to Sankofa portal http://192.168.11.51:3000 (7801). Fallback: set THE_ORDER_UPSTREAM_IP / THE_ORDER_UPSTREAM_PORT to .51 / 3000 if HAProxy is offline. www.the-order.sankofa.nexus → 301 https://the-order.sankofa.nexus (same as www.sankofa / www.phoenix).
  • studio.sankofa.nexus → Routes to http://192.168.11.72:8000 (Sankofa Studio / VMID 7805)

Public verification evidence (2026-03-26): bash scripts/verify/verify-end-to-end-routing.sh --profile=public passed with Failed: 0; Sankofa root, Phoenix, Studio, and The Order returned 200. See verification_report.md.

Service Details:

  • Host: r630-01 (192.168.11.11)
  • Network: VLAN 11 (192.168.11.0/24)
  • Gateway: 192.168.11.1
  • All services verified and operational

Note: Sankofa services are deployed on VLAN 11 (192.168.11.x) as intended. All services are running and accessible.

The Order — microservices (mostly r630-01; portals r630-04)

VMID IP Address Hostname Status Endpoints Purpose
10030 192.168.11.40 order-identity ✅ Running API Identity
10040 192.168.11.41 order-intake ✅ Running API Intake
10050 192.168.11.49 order-finance ✅ Running API Finance
10060 192.168.11.42 order-dataroom ✅ Running Web: 80 Dataroom
10070 192.168.11.87 order-legal ✅ Running API Legal — use IP_ORDER_LEGAL (.87); not .54
10080 192.168.11.43 order-eresidency ✅ Running API eResidency
10090 192.168.11.180 order-portal-public ✅ Running Web Public portal
10091 192.168.11.181 order-portal-internal ✅ Running Web Internal portal
10092 192.168.11.182 order-mcp-legal ✅ Running API MCP legal
10200 192.168.11.46 order-prometheus ✅ Running 9090 Metrics (IP_ORDER_PROMETHEUS; not Order Redis)
10201 192.168.11.47 order-grafana ✅ Running 3000 Dashboards
10202 192.168.11.48 order-opensearch ✅ Running 9200 Search
10210 192.168.11.39 order-haproxy ✅ Running 80 (HAProxy → portal :3000) Edge for the-order.sankofa.nexus; HAProxy config via config/haproxy/order-haproxy-10210.cfg.template + scripts/deployment/provision-order-haproxy-10210.sh

Note: 10090–10092 are on r630-04 (not r630-01). MIM4U uses 7810/7811 on .37/.36 (r630-02) — do not conflate with Order portal IPs.

Gov portals vs Order: VMID 7804 alone uses 192.168.11.54 (IP_GOV_PORTALS_DEV). Order-legal must not use .54.

Phoenix Vault Cluster (8640-8642)

VMID IP Address Hostname Status Endpoints Purpose
8640 192.168.11.200 vault-phoenix-1 ✅ Running Vault: 8200 Phoenix Vault node 1
8641 192.168.11.215 vault-phoenix-2 ✅ Running Vault: 8200 Phoenix Vault node 2
8642 192.168.11.202 vault-phoenix-3 ✅ Running Vault: 8200 Phoenix Vault node 3

Note: 8641 moved from .201 to .215 (2026-02-01) to free CCIP Execute interim range. See IP_CONFLICTS_CCIP_RANGE_RESOLVED_20260201.md.

Phoenix Core Application Extensions

VMID IP Address Hostname Status Endpoints Purpose
8604 10.160.0.14 currencicombo-phoenix-1 ✅ Running Web: 3000, API: 8080 CurrenciCombo webapp + orchestrator

Operational note (2026-04-22):

  • Deployed on r630-01 through scripts/deployment/deploy-currencicombo-8604.sh.
  • Internal-only at present; no NPMplus / public hostname assigned yet.
  • Local PostgreSQL + Redis run inside the same CT for this first Phoenix deployment.

Other Services

VMID IP Address Hostname Status Endpoints Purpose Notes
5800 192.168.11.85 (Mifos) ✅ Running Web: 80 Mifos X + Fineract (OMNL) LXC on r630-02; mifos.d-bis.org; see MIFOS_R630_02_DEPLOYMENT.md
5801 192.168.11.58 dapp-smom — Web: 80 DApp (frontend-dapp) for Chain 138 bridge LXC; see DAPP_LXC_DEPLOYMENT.md; NPMplus/tunnel dapp.d-bis.org
10232 192.168.11.56 CT10232 ✅ Running Various Container service ✅ IP CONFLICT RESOLVED
10203 192.168.11.228 omdnl-org-web ✅ Running Web: 80 OMNL / org web (small CT) r630-01; renumbered from .222 (2026-05-11) to resolve duplicate with 2104
2421 192.168.11.229 mev-control-backend ✅ Running API / backend MEV control platform backend r630-04; renumbered from .223 (2026-05-11) to resolve duplicate with 2202

Note: 10234 is listed under NPMplus above (not stopped); older duplicate rows removed. 10203 / 2421 had briefly shared .222 / .223 with canonical Besu Justin RPC CTs — fixed by reassignment to .228 / .229.

Oracle & Monitoring

VMID IP Address Hostname Status Endpoints Purpose
3500 192.168.11.29 oracle-publisher-1 ✅ Running (verify on-chain) Oracle: Various r630-02 thin5. Reprovisioned 2026-03-28 via scripts/deployment/provision-oracle-publisher-lxc-3500.sh (systemd oracle-publisher). If updateAnswer txs revert, set PRIVATE_KEY in /opt/oracle-publisher/.env to an EOA authorized on the aggregator (may differ from deployer). Metrics: :8000/metrics.
3501 192.168.11.28 ccip-monitor-1 ✅ Running Monitor: Various CCIP monitoring; migrated 2026-03-28 to r630-02 thin5 (pvesh … /migrate --target-storage thin5).
5200 192.168.11.80 cacti-1 ✅ Running Web: 80, 443 Network monitoring (Cacti); host r630-02 (migrated 2026-02-15)

Machine Learning Nodes

Placement: LXCs 3000–3003 run on r630-01 (hostname field remains ml110 from template).

VMID IP Address Hostname Status Endpoints Purpose
3000 192.168.11.60 ml110 ✅ Running ML Services: Various ML node 1
3001 192.168.11.61 ml110 ✅ Running ML Services: Various ML node 2
3002 192.168.11.62 ml110 ✅ Running ML Services: Various ML node 3
3003 192.168.11.66 ml110 ✅ Running ML Services: Various ML node 4 (r630-01)

Supplementary cluster inventory (live 2026-05-11)

Guests present in r630-01:/opt/proxmox/reports/status/live_inventory.json at collection time but not listed in category tables above (for automation cross-checks). Canonical Besu / NPMplus rows are omitted here when already duplicated above.

VMID IP Address Hostname Node Notes
2410 192.168.11.218 info-defi-oracle-web r630-01
5201 192.168.11.177 cacti-alltra-1 r630-02
5202 192.168.11.251 cacti-hybx-1 r630-02
5700 192.168.11.59 dev-vm r630-04
5701 192.168.11.65 gitea-runner-1 r630-04
5702 192.168.11.82 ai-inf-1 r630-01
5705 192.168.11.86 ai-inf-2 r630-01
5751 192.168.11.69 op-stack-deployer-1 r630-02
5752 192.168.11.70 op-stack-ops-1 r630-02
6001 192.168.11.178 fabric-alltra-1 r630-02
6002 192.168.11.252 fabric-hybx-1 r630-02
6202 192.168.11.175 firefly-alltra-1 r630-02
6203 192.168.11.176 firefly-alltra-2 r630-02
6204 192.168.11.249 firefly-hybx-1 r630-02
6205 192.168.11.250 firefly-hybx-2 r630-02
6401 192.168.11.179 indy-alltra-1 r630-02
6402 192.168.11.253 indy-hybx-1 r630-02
6500 192.168.11.88 aries-1 r630-02
6600 192.168.11.93 caliper-1 r630-02
7806 192.168.11.63 sankofa-public-web r630-01
7807 — cc-phase1-lab r630-01 No static 192.168.11.x in net* (verify inside CT)
7808 — cc-phase1-k3s r630-01 No static 192.168.11.x in net* (verify inside CT)
7815 192.168.11.75 cc-phase1-lab r630-02 Second cc-phase1 lab CT
8604 10.160.0.14 currencicombo-phoenix-1 r630-01 Internal overlay; see Phoenix Extensions above
8811 192.168.11.111 sankofa-proxmox-mcp r630-04
8812 192.168.11.112 operator-services r630-04
10000 192.168.11.44 order-postgres-primary r630-01 Also referenced as ORDER_POSTGRES_PRIMARY
10001 192.168.11.45 order-postgres-replica r630-01
10020 192.168.11.38 order-redis r630-04
10230 192.168.11.55 order-vault r630-04
10900 192.168.11.115 mailcow-dbis r630-01

Port Reference

Standard Besu Ports

  • 8545: HTTP JSON-RPC
  • 8546: WebSocket JSON-RPC
  • 30303: P2P networking (TCP/UDP)
  • 9545: Prometheus metrics

Standard Application Ports

  • 80: HTTP
  • 443: HTTPS
  • 3000: Node.js API
  • 5432: PostgreSQL
  • 6379: Redis
  • 9000: Web3Signer
  • 8200: Vault

Network Architecture

Public Internet Access Flow

Internet
  ↓
Cloudflare (DNS + DDoS Protection)
  ↓
NPMplus (VMID 10233: 192.168.11.167:443)
  ↓
VM Nginx (443) → Backend Services

Internal RPC Access

Internal Network (192.168.11.0/24)
  ↓
Direct to RPC Nodes:
  - VMID 2101: 192.168.11.211:8545 (HTTP) / 8546 (WS) - Core RPC
  - VMID 2201: 192.168.11.221:8545 (HTTP) / 8546 (WS) - Public RPC
  - VMID 2303: 192.168.11.233:8545 (HTTP) / 8546 (WS) - Ali 0x8a
  - VMID 2304: 192.168.11.234:8545 (HTTP) / 8546 (WS) - Ali 0x1
  - VMID 2305: 192.168.11.235:8545 (HTTP) / 8546 (WS) - Luis 0x8a
  - VMID 2306: 192.168.11.236:8545 (HTTP) / 8546 (WS) - Luis 0x1
  - VMID 2307: 192.168.11.237:8545 (HTTP) / 8546 (WS) - Putu 0x8a
  - VMID 2308: 192.168.11.238:8545 (HTTP) / 8546 (WS) - Putu 0x1
  - VMID 2400: 192.168.11.240:8545 (HTTP) / 8546 (WS) - ThirdWeb Primary
  - VMID 2401: 192.168.11.241:8545 (HTTP) / 8546 (WS) - ThirdWeb 1
  - VMID 2402: 192.168.11.242:8545 (HTTP) / 8546 (WS) - ThirdWeb 2
  - VMID 2403: 192.168.11.243:8545 (HTTP) / 8546 (WS) - ThirdWeb 3

Known Issues & Notes

✅ IP Address Conflicts - RESOLVED

Status: ✅ RESOLVED - All conflicts fixed (2026-01-20)

  1. 192.168.11.50: ✅ RESOLVED

    • VMID 7800 (sankofa-api-1): 192.168.11.50 ✅ UNIQUE
    • VMID 10070 (order-legal): 192.168.11.87 (IP_ORDER_LEGAL) — moved off .54 2026-03-25 (ARP conflict with VMID 7804 gov-portals) ✅

  2. 192.168.11.51: ✅ RESOLVED

    • VMID 7801 (sankofa-portal-1): 192.168.11.51 ✅ UNIQUE
    • VMID 10230 (order-vault): Reassigned to 192.168.11.55 ✅

  3. 192.168.11.52: ✅ RESOLVED

    • VMID 7802 (sankofa-keycloak-1): 192.168.11.52 ✅ UNIQUE
    • VMID 10232 (CT10232): Reassigned to 192.168.11.56 ✅

  4. 192.168.11.55: ✅ IN USE — VMID 10230 (order-vault) only. Sankofa Studio (VMID 7805) uses 192.168.11.72 to avoid conflict.

Resolution: All IP conflicts resolved using scripts/resolve-ip-conflicts.sh

Verification: ✅ All IPs verified unique, all services operational

IP conflicts (canonical): reports/status/IP_CONFLICTS_RESOLUTION_COMPLETE.md; CCIP range move: reports/status/IP_CONFLICTS_CCIP_RANGE_RESOLVED_20260201.md. Script: scripts/resolve-ip-conflicts.sh (uses config/ip-addresses.conf).

Port Conflicts

  1. VMID 2400: Port conflict resolved ✅
    • Previous: Besu metrics (9545) conflicted with RPC Translator HTTP (9545)
    • Resolution: Translator moved to 9645/9646 (completed)
    • Current: Nginx routes to translator on 9645/9646

NPMplus Routing Issues

  1. rpc.public-0138.defi-oracle.io: Currently routes to wrong VMID
    • Stale historical target: https://192.168.11.252:443 (old VMID 2502 migration row, decommissioned)
    • Current intended target: https://192.168.11.240:443 (VMID 2400)
    • Fix: Update NPMplus proxy host configuration

Quick Access Commands

Test RPC Endpoints

# Public RPC (HTTP)
curl -X POST https://rpc-http-pub.d-bis.org \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}'

# Private RPC (HTTP) - requires JWT
curl -X POST https://rpc-http-prv.d-bis.org \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer <JWT_TOKEN>' \
  -d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}'

# ThirdWeb RPC
curl -X POST https://rpc.public-0138.defi-oracle.io \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}'

Check Container Status

# From Proxmox host
pct status <VMID>
qm status <VMID>

# Check specific service
pct exec <VMID> -- systemctl status <service-name>

Related Documentation

  • VMID IP List: reports/VMID_IP_ADDRESS_LIST.md
  • NPMplus Setup: docs/04-configuration/NPMPLUS_COMPLETE_SETUP_SUMMARY.md
  • Nginx Configurations: docs/04-configuration/NGINX_CONFIGURATIONS_VMIDS_2400-2508.md
  • RPC Translator: rpc-translator-138/VMID_ALLOCATION.md

NPMplus Endpoint Configuration Reference

This section lists all endpoints that should be configured in NPMplus, extracted from NPM (VMID 105) configuration files.

Complete NPMplus Domain Mapping

Domain Target Scheme Port WebSocket Notes
RPC Services
rpc.public-0138.defi-oracle.io 192.168.11.221 http 8545 ✅ Yes Public RPC (VMID 2201); WSS Upgrade internally routes to 8546
rpc-http-pub.d-bis.org 192.168.11.221 https 443 ✅ Yes Public RPC (VMID 2201)
rpc-ws-pub.d-bis.org 192.168.11.221 https 443 ✅ Yes Public WebSocket RPC (VMID 2201)
rpc-http-prv.d-bis.org 192.168.11.211 https 443 ✅ Yes Private RPC with JWT (VMID 2101)
rpc-ws-prv.d-bis.org 192.168.11.211 https 443 ✅ Yes Private WebSocket RPC with JWT (VMID 2101)
Explorer
explorer.d-bis.org 192.168.11.140 http 4000 ❌ No Blockchain Explorer (VMID 5000 - Direct Route)
DBIS Services
dbis-admin.d-bis.org 192.168.11.130 http 80 ❌ No DBIS Admin Frontend (VMID 10130)
dbis-api.d-bis.org 192.168.11.155 http 3000 ❌ No DBIS API Primary (VMID 10150)
dbis-api-2.d-bis.org 192.168.11.156 http 3000 ❌ No DBIS API Secondary (VMID 10151)
secure.d-bis.org 192.168.11.130 http 80 ❌ No DBIS Secure Portal (VMID 10130) - Path-based routing
MIM4U Services
mim4u.org 192.168.11.37 http 80 ❌ No MIM4U Main Site (VMID 7810 mim-web-1)
www.mim4u.org 192.168.11.37 http 80 ❌ No MIM4U (VMID 7810; optional redirect www → apex)
secure.mim4u.org 192.168.11.37 http 80 ❌ No MIM4U Secure Portal (VMID 7810)
training.mim4u.org 192.168.11.37 http 80 ❌ No MIM4U Training Portal (VMID 7810)
Sankofa Phoenix Services
sankofa.nexus 192.168.11.51 http 3000 ❌ No Sankofa Portal - Company Website (VMID 7801) ✅ Deployed
www.sankofa.nexus 192.168.11.51 http 3000 ❌ No Sankofa Portal (VMID 7801) ✅ Deployed
phoenix.sankofa.nexus 192.168.11.50 http 4000 ❌ No Phoenix API - Cloud Platform Portal (VMID 7800) ✅ Deployed
www.phoenix.sankofa.nexus 192.168.11.50 http 4000 ❌ No Phoenix API (VMID 7800) ✅ Deployed
the-order.sankofa.nexus, www.the-order.sankofa.nexus 192.168.11.39 (10210 HAProxy; default) or 192.168.11.51 (direct portal if env override) http 80 or 3000 ❌ No NPM → .39:80 by default; HAProxy → .51:3000
studio.sankofa.nexus 192.168.11.72 http 8000 ❌ No Sankofa Studio (FusionAI Creator) — VMID 7805

Path-Based Routing Notes

Some domains use path-based routing in NPM configs:

secure.d-bis.org:

  • /admin → http://192.168.11.130:80 (DBIS Frontend)
  • /api → http://192.168.11.155:3000 (DBIS API)
  • /graph → http://192.168.11.155:3000 (DBIS GraphQL)
  • / → http://192.168.11.130:80 (DBIS Frontend)

sankofa.nexus (per deploy script):

  • /api → http://10.160.0.10:4000 (Sankofa API)
  • / → http://10.160.0.11:3000 (Sankofa Portal)

Note: NPMplus may need custom location blocks or separate proxy hosts for path-based routing.

NPMplus routing (authoritative targets)

Use this document as the source of truth for domain → VMID:port. Only explorer.d-bis.org should point to Blockscout (VMID 5000, 192.168.11.140). All other domains must point to their correct VMID and port:

Domain Correct target (VMID, IP:port) Do NOT point to
explorer.d-bis.org 5000, 192.168.11.140:80 (web), :4000 (API) —
sankofa.nexus, www.sankofa.nexus 7801, 192.168.11.51:3000 192.168.11.140 (Blockscout)
phoenix.sankofa.nexus, www.phoenix.sankofa.nexus 7800, 192.168.11.50:4000 192.168.11.140 (Blockscout)
the-order.sankofa.nexus, www.the-order.sankofa.nexus 10210, 192.168.11.39:80 192.168.11.140 (Blockscout)
studio.sankofa.nexus 7805, 192.168.11.72:8000 —

If NPMplus proxy hosts for sankofa.nexus or phoenix.sankofa.nexus currently point to 192.168.11.140, update them to the correct IP:port above. See RPC_ENDPOINTS_MASTER.md and table "Sankofa Phoenix Services" in this document.

Note: All www.* subdomains redirect to their parent domains to reduce the number of proxy host configurations needed.

Last Updated: 2026-03-27
Maintained By: Infrastructure Team

RPC Node Quick Reference

Active RPC Endpoints (12/13 Running)

IP Address VMID Name Status
192.168.11.211 2101 besu-rpc-core-1 ✅ Running
192.168.11.221 2201 besu-rpc-public-1 ✅ Running
192.168.11.232 2301 besu-rpc-private-1 ⏸️ Stopped
192.168.11.233 2303 besu-rpc-ali-0x8a ✅ Running
192.168.11.234 2304 besu-rpc-ali-0x1 ✅ Running
192.168.11.235 2305 besu-rpc-luis-0x8a ✅ Running
192.168.11.236 2306 besu-rpc-luis-0x1 ✅ Running
192.168.11.237 2307 besu-rpc-putu-0x8a ✅ Running
192.168.11.238 2308 besu-rpc-putu-0x1 ✅ Running
192.168.11.240 2400 thirdweb-rpc-1 ✅ Running
192.168.11.241 2401 besu-rpc-thirdweb-0x8a-1 ✅ Running
192.168.11.242 2402 besu-rpc-thirdweb-0x8a-2 ✅ Running
192.168.11.243 2403 besu-rpc-thirdweb-0x8a-3 ✅ Running

Test All RPC Nodes

# Quick test all RPC nodes
for ip in 192.168.11.211 192.168.11.221 192.168.11.233 192.168.11.234 192.168.11.235 192.168.11.236 192.168.11.237 192.168.11.238 192.168.11.240 192.168.11.241 192.168.11.242 192.168.11.243; do
  curl -s -X POST -H "Content-Type: application/json" \
    --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
    http://$ip:8545 | grep -q "result" && echo "✓ $ip" || echo "✗ $ip"
done
Reference in New Issue View Git Blame Copy Permalink