From 91ba6f4f2fcffebc145fe2e6a30435617526314d Mon Sep 17 00:00:00 2001 From: defiQUG Date: Fri, 24 Apr 2026 11:45:04 -0700 Subject: [PATCH] docs: finalize besu host placement reconciliation --- docs/04-configuration/ALL_VMIDS_ENDPOINTS.md | 20 ++++++----- docs/04-configuration/RPC_ENDPOINTS_MASTER.md | 22 +++++++------ .../BESU_NODE_CONFIGURATION_MAP_20260424.md | 33 +++++++++++-------- 3 files changed, 42 insertions(+), 33 deletions(-) diff --git a/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md b/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md index 36fb7fa8..b8afdc86 100644 --- a/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md +++ b/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md @@ -87,7 +87,7 @@ | 1506 | 192.168.11.214 | besu-sentry-alltra-2 | ✅ Running | P2P: 30303, Metrics: 9545 | Sentry (Alltra 2) | **Note:** 1505-1506 moved from .170/.171 to .213/.214 (2026-02-01) to free CCIP Ops interim range. -**Live SSH note (2026-04-24):** `1500-1506` were found live on Proxmox. `1507` and `1508` are still present in the operational template / Besu allowlist, but were **not** found on any Proxmox host during the live pass. +**Live SSH / cluster note (2026-04-24):** `1500-1506` were found during the initial 3-host pass. Cluster-wide reconciliation then confirmed `1507` on `r630-03` (`192.168.11.13`) and `1508` on `r630-04` (`192.168.11.14`). --- @@ -99,23 +99,25 @@ All RPC nodes have been migrated to a new VMID structure for better organization ### Core RPC Nodes -**Live SSH note (2026-04-24):** -- Found live: `2101`, `2103`, `2201` -- Not found on any Proxmox host: `2102`, `2301` +**Live reconciliation note (2026-04-24):** +- Initial direct host pass confirmed `2101`, `2103`, `2201` +- Cluster-wide reconciliation then confirmed: + - `2102` on `r630-03` + - `2301` on `r630-03` | VMID | IP Address | Hostname | Status | Block | Peers | Endpoints | Purpose | |------|------------|----------|--------|-------|-------|-----------|---------| | 2101 | 192.168.11.211 | besu-rpc-core-1 | ✅ Running | 1,145,367 | 7 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Core RPC node | | 2103 | 192.168.11.217 | besu-rpc-core-thirdweb | ✅ Running | Live SSH verified 2026-04-24 | Live SSH verified 2026-04-24 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Core Thirdweb admin RPC node | | **2201** | **192.168.11.221** | besu-rpc-public-1 | ✅ Running | 1,145,367 | 7 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Public RPC node **(FIXED PERMANENT)** | -| 2301 | 192.168.11.232 | besu-rpc-private-1 | ⚠ Not found in live SSH pass | - | - | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Fireblocks-dedicated RPC, absent from 2026-04-24 live host inventory | +| 2301 | 192.168.11.232 | besu-rpc-private-1 | ✅ Running | Cluster CT confirmed on `r630-03` | - | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Fireblocks-dedicated RPC on `r630-03` | ### Named RPC Nodes (Ali/Luis/Putu) | VMID | IP Address | Hostname | Status | Block | Peers | Endpoints | Purpose | |------|------------|----------|--------|-------|-------|-----------|---------| | 2303 | 192.168.11.233 | besu-rpc-ali-0x8a | ✅ Running | 1,145,367 | 7 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Ali RPC (0x8a identity) | -| 2304 | 192.168.11.234 | besu-rpc-ali-0x1 | ✅ Running | 1,145,367 | 7 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Ali RPC (0x1 identity) | +| 2304 | 192.168.11.234 | besu-rpc-ali-0x1 | ✅ Running | Cluster CT confirmed on `r630-03` | - | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Ali RPC (0x1 identity) on `r630-03` | | 2305 | 192.168.11.235 | besu-rpc-luis-0x8a | ✅ Running | 1,145,367 | 7 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Luis RPC (0x8a identity) | | 2306 | 192.168.11.236 | besu-rpc-luis-0x1 | ✅ Running | 1,145,367 | 7 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Luis RPC (0x1 identity) | | 2307 | 192.168.11.237 | besu-rpc-putu-0x8a | ✅ Running | 1,145,367 | 7 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | Putu RPC (0x8a identity) | @@ -125,10 +127,10 @@ All RPC nodes have been migrated to a new VMID structure for better organization | VMID | IP Address | Hostname | Status | Block | Peers | Endpoints | Purpose | |------|------------|----------|--------|-------|-------|-----------|---------| -| 2400 | 192.168.11.240 | thirdweb-rpc-1 | ✅ Running | 1,149,992 | 2 | **Nginx: 443**, Besu: 8545/8546, P2P: 30303, Metrics: 9545, Translator: 9645/9646 | ThirdWeb RPC with translator (primary) | +| 2400 | 192.168.11.240 | thirdweb-rpc-1 | ✅ Running | Cluster CT confirmed on `r630-03` | - | **Nginx: 443**, Besu: 8545/8546, P2P: 30303, Metrics: 9545, Translator: 9645/9646 | ThirdWeb RPC with translator (primary) on `r630-03` | | 2401 | 192.168.11.241 | besu-rpc-thirdweb-0x8a-1 | ✅ Running | 1,149,992 | 2 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | ThirdWeb RPC instance 1 | -| 2402 | 192.168.11.242 | besu-rpc-thirdweb-0x8a-2 | ✅ Running | 1,149,992 | 2 | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | ThirdWeb RPC instance 2 | -| 2403 | 192.168.11.243 | besu-rpc-thirdweb-0x8a-3 | ✅ Running | 600,172 | 0 | Besu: 8545/8546, P2P: 30303 | ThirdWeb RPC instance 3 (syncing) | +| 2402 | 192.168.11.242 | besu-rpc-thirdweb-0x8a-2 | ✅ Running | Cluster CT confirmed on `r630-03` | - | Besu: 8545/8546, P2P: 30303, Metrics: 9545 | ThirdWeb RPC instance 2 on `r630-03` | +| 2403 | 192.168.11.243 | besu-rpc-thirdweb-0x8a-3 | ✅ Running | Cluster CT confirmed on `r630-03` | - | Besu: 8545/8546, P2P: 30303 | ThirdWeb RPC instance 3 on `r630-03` | **Note**: VMID 2400 is the primary ThirdWeb RPC with Nginx and RPC Translator. VMID 2403 metrics disabled due to port conflict, node is syncing. diff --git a/docs/04-configuration/RPC_ENDPOINTS_MASTER.md b/docs/04-configuration/RPC_ENDPOINTS_MASTER.md index a3492af0..fd40a40d 100644 --- a/docs/04-configuration/RPC_ENDPOINTS_MASTER.md +++ b/docs/04-configuration/RPC_ENDPOINTS_MASTER.md @@ -79,11 +79,13 @@ For **Ethereum mainnet and other public chains**, you can use: ## Active RPC Nodes -**Live SSH note (2026-04-24):** -- Verified live over Proxmox SSH: - - `2101`, `2103`, `2201`, `2303`, `2305`, `2306`, `2307`, `2308` -- Not found on any Proxmox host during the same live pass: - - `2102`, `2301`, `2304`, `2400`, `2402`, `2403` +**Live reconciliation note (2026-04-24):** +- Direct host pass plus cluster-wide reconciliation confirmed: + - `2102`, `2301`, `2304`, `2400`, `2402`, `2403` are on `r630-03` (`192.168.11.13`) + - `1507` is on `r630-03` + - `1508` is on `r630-04` (`192.168.11.14`) +- Confirmed live as Proxmox CTs: + - `2101`, `2102`, `2103`, `2201`, `2301`, `2303`, `2304`, `2305`, `2306`, `2307`, `2308`, `2400`, `2401`, `2402`, `2403` - The older `2500-2505` migration story is not the whole picture anymore: separate live ALLTRA/HYBX Besu RPC containers with those VMIDs are running on `r630-01` at `.172-.174` and `.246-.248`. - Live duplicate legacy Besu RPC containers were also found on `r630-01`: - `2420`, `2430`, `2440`, `2460`, `2470`, `2480` @@ -96,17 +98,17 @@ For **Ethereum mainnet and other public chains**, you can use: | 2101 | 192.168.11.211 | besu-rpc-core-1 | `http://192.168.11.211:8545` | `ws://192.168.11.211:8546` | ✅ Running | | 2103 | 192.168.11.217 | besu-rpc-core-thirdweb | `http://192.168.11.217:8545` | `ws://192.168.11.217:8546` | ✅ Running (Thirdweb admin / NPM `rpc.tw-core.d-bis.org`) | | **2201** | **192.168.11.221** | besu-rpc-public-1 | `http://192.168.11.221:8545` | `ws://192.168.11.221:8546` | ✅ Running | -| 2301 | 192.168.11.232 | besu-rpc-private-1 | `http://192.168.11.232:8545` | `ws://192.168.11.232:8546` | ⚠ Not found in 2026-04-24 live SSH pass | +| 2301 | 192.168.11.232 | besu-rpc-private-1 | `http://192.168.11.232:8545` | `ws://192.168.11.232:8546` | ✅ Running (`r630-03`) | | 2303 | 192.168.11.233 | besu-rpc-ali-0x8a | `http://192.168.11.233:8545` | `ws://192.168.11.233:8546` | ✅ Running | -| 2304 | 192.168.11.234 | besu-rpc-ali-0x1 | `http://192.168.11.234:8545` | `ws://192.168.11.234:8546` | ⚠ Not found in 2026-04-24 live SSH pass | +| 2304 | 192.168.11.234 | besu-rpc-ali-0x1 | `http://192.168.11.234:8545` | `ws://192.168.11.234:8546` | ✅ Running (`r630-03`) | | 2305 | 192.168.11.235 | besu-rpc-luis-0x8a | `http://192.168.11.235:8545` | `ws://192.168.11.235:8546` | ✅ Running | | 2306 | 192.168.11.236 | besu-rpc-luis-0x1 | `http://192.168.11.236:8545` | `ws://192.168.11.236:8546` | ✅ Running | | 2307 | 192.168.11.237 | besu-rpc-putu-0x8a | `http://192.168.11.237:8545` | `ws://192.168.11.237:8546` | ✅ Running | | 2308 | 192.168.11.238 | besu-rpc-putu-0x1 | `http://192.168.11.238:8545` | `ws://192.168.11.238:8546` | ✅ Running | -| 2400 | 192.168.11.240 | thirdweb-rpc-1 | `http://192.168.11.240:8545` | `ws://192.168.11.240:8546` | ⚠ Not found in 2026-04-24 live SSH pass | +| 2400 | 192.168.11.240 | thirdweb-rpc-1 | `http://192.168.11.240:8545` | `ws://192.168.11.240:8546` | ✅ Running (`r630-03`) | | 2401 | 192.168.11.241 | besu-rpc-thirdweb-0x8a-1 | `http://192.168.11.241:8545` | `ws://192.168.11.241:8546` | ✅ Running | -| 2402 | 192.168.11.242 | besu-rpc-thirdweb-0x8a-2 | `http://192.168.11.242:8545` | `ws://192.168.11.242:8546` | ⚠ Not found in 2026-04-24 live SSH pass | -| 2403 | 192.168.11.243 | besu-rpc-thirdweb-0x8a-3 | `http://192.168.11.243:8545` | `ws://192.168.11.243:8546` | ⚠ Not found in 2026-04-24 live SSH pass | +| 2402 | 192.168.11.242 | besu-rpc-thirdweb-0x8a-2 | `http://192.168.11.242:8545` | `ws://192.168.11.242:8546` | ✅ Running (`r630-03`) | +| 2403 | 192.168.11.243 | besu-rpc-thirdweb-0x8a-3 | `http://192.168.11.243:8545` | `ws://192.168.11.243:8546` | ✅ Running (`r630-03`) | ### Additional Live Internal RPC Nodes (SSH-verified 2026-04-24) diff --git a/docs/06-besu/BESU_NODE_CONFIGURATION_MAP_20260424.md b/docs/06-besu/BESU_NODE_CONFIGURATION_MAP_20260424.md index 03d07f5f..231cc77d 100644 --- a/docs/06-besu/BESU_NODE_CONFIGURATION_MAP_20260424.md +++ b/docs/06-besu/BESU_NODE_CONFIGURATION_MAP_20260424.md @@ -52,12 +52,17 @@ Fresh live SSH interrogation was run on `2026-04-24` against all three Proxmox h ### Intended Besu containers not found live in that pass -- `1503`, `1504`, `1505`, `1506`, `1507`, `1508` -- `2102` -- `2301`, `2304` -- `2400`, `2402`, `2403` - no Besu LXC containers were found on `ml110` +### Cluster reconciliation after the initial 3-host pass + +Cluster-wide `pvesh get /cluster/resources` resolved the remaining placements: + +- `r630-03` (`192.168.11.13`): + - `1507`, `2102`, `2301`, `2304`, `2400`, `2402`, `2403` +- `r630-04` (`192.168.11.14`): + - `1508` + ## Visual Map ```mermaid @@ -145,28 +150,28 @@ flowchart LR | 1504 | `192.168.11.154` | `besu-sentry-ali` | Yes | Yes | No generated TOML found | Mentioned as special-case in node-list README | | 1505 | `192.168.11.213` | `besu-sentry-alltra-1` | Yes | Yes | Yes | Generated TOML has explicit `p2p-host`, RPC enabled, `discovery-enabled=true` | | 1506 | `192.168.11.214` | `besu-sentry-alltra-2` | Yes | Yes | Yes | Same | -| 1507 | `192.168.11.244` | `besu-sentry-hybx-1` | Yes | Yes | Yes | Absent from `ALL_VMIDS_ENDPOINTS.md` | -| 1508 | `192.168.11.245` | `besu-sentry-hybx-2` | Yes | Yes | Yes | Absent from `ALL_VMIDS_ENDPOINTS.md` | +| 1507 | `192.168.11.244` | `besu-sentry-hybx-1` | Yes | Yes | Yes | Live CT confirmed on `r630-03` | +| 1508 | `192.168.11.245` | `besu-sentry-hybx-2` | Yes | Yes | Yes | Live CT confirmed on `r630-04` | ### RPC Nodes | VMID | IP | Hostname | In operational template | In allowlist | In `ALL_VMIDS` | In `RPC_ENDPOINTS_MASTER` | Notes | |---|---|---|---|---|---|---|---| | 2101 | `192.168.11.211` | `besu-rpc-core-1` | Yes | Yes | Yes | Yes | Core/admin RPC | -| 2102 | `192.168.11.212` | `besu-rpc-core-2` | Yes | Yes | No | No | Present in template and allowlist, missing from inventory docs | +| 2102 | `192.168.11.212` | `besu-rpc-core-2` | Yes | Yes | No | No | Live CT confirmed on `r630-03` | | 2103 | `192.168.11.217` | `besu-rpc-core-thirdweb` | Yes | Yes | No | Yes | Live SSH verified 2026-04-24; added to node allowlist during fix pass | | 2201 | `192.168.11.221` | `besu-rpc-public-1` | Yes | Yes | Yes | Yes | Public RPC | -| 2301 | `192.168.11.232` | `besu-rpc-private-1` | Yes | Yes | **Marked stopped** | **Marked running** | Cross-doc status conflict | +| 2301 | `192.168.11.232` | `besu-rpc-private-1` | Yes | Yes | **Running** | **Running** | Live CT confirmed on `r630-03` | | 2303 | `192.168.11.233` | `besu-rpc-ali-0x8a` | Yes | Yes | Yes | Yes | Named RPC | -| 2304 | `192.168.11.234` | `besu-rpc-ali-0x1` | Yes | Yes | Yes | Yes | Named RPC | +| 2304 | `192.168.11.234` | `besu-rpc-ali-0x1` | Yes | Yes | Yes | Yes | Live CT confirmed on `r630-03` | | 2305 | `192.168.11.235` | `besu-rpc-luis-0x8a` | Yes | Yes | Yes | Yes | Named RPC | | 2306 | `192.168.11.236` | `besu-rpc-luis-0x1` | Yes | Yes | Yes | Yes | Named RPC | | 2307 | `192.168.11.237` | `besu-rpc-putu-0x8a` | Yes | Yes | Yes | Yes | Named RPC | | 2308 | `192.168.11.238` | `besu-rpc-putu-0x1` | Yes | Yes | Yes | Yes | Named RPC | -| 2400 | `192.168.11.240` | `thirdweb-rpc-1` | Yes | Yes | Yes | Yes | Nginx + translator tier | +| 2400 | `192.168.11.240` | `thirdweb-rpc-1` | Yes | Yes | Yes | Yes | Live CT confirmed on `r630-03` | | 2401 | `192.168.11.241` | `besu-rpc-thirdweb-0x8a-1` | Yes | Yes | Yes | Yes | Thirdweb tier | -| 2402 | `192.168.11.242` | `besu-rpc-thirdweb-0x8a-2` | Yes | Yes | Yes | Yes | Thirdweb tier | -| 2403 | `192.168.11.243` | `besu-rpc-thirdweb-0x8a-3` | Yes | Yes | Yes | Yes | Thirdweb tier | +| 2402 | `192.168.11.242` | `besu-rpc-thirdweb-0x8a-2` | Yes | Yes | Yes | Yes | Live CT confirmed on `r630-03` | +| 2403 | `192.168.11.243` | `besu-rpc-thirdweb-0x8a-3` | Yes | Yes | Yes | Yes | Live CT confirmed on `r630-03` | | 2500 | `192.168.11.172` | `besu-rpc-alltra-1` | Yes | Yes | Yes | Yes | Canonical intended ALLTRA RPC 1 | | 2501 | `192.168.11.173` | `besu-rpc-alltra-2` | Yes | Yes | Yes | Yes | Canonical intended ALLTRA RPC 2 | | 2502 | `192.168.11.174` | `besu-rpc-alltra-3` | Yes | Yes | Yes | Yes | Canonical intended ALLTRA RPC 3 | @@ -267,12 +272,12 @@ Impact: `ALL_VMIDS_ENDPOINTS.md` says: - `2500-2505` are the intended internal ALLTRA/HYBX fleet -- `2301` is stopped +- `2301` is live on `r630-03` But: - live SSH found a second duplicate set `2420/2430/2440/2460/2470/2480` using the same role names and IPs, and a controlled shutdown pass then retired them - `RPC_ENDPOINTS_MASTER.md` historically mixed old migration rows with current fleet rows -- `2301` inventory language still varies by doc +- the initial 3-host pass missed `r630-03` and `r630-04`, which made live nodes appear unresolved until cluster-wide reconciliation Impact: - Operators cannot trust a single inventory doc during incident response.