d-bis/smom-dbis-138
4
0
Fork 0
Code Issues Pull Requests 3 Actions Packages Projects Releases Wiki Activity
Files
6817f53591c41cc9ee0fbf404fda7ebfe2a5bcb9
smom-dbis-138/terraform/phases/phase1/CONNECTIVITY_ISSUE_AND_SOLUTIONS.md
defiQUG 1fb7266469 Add Oracle Aggregator and CCIP Integration 
- Introduced Aggregator.sol for Chainlink-compatible oracle functionality, including round-based updates and access control.
- Added OracleWithCCIP.sol to extend Aggregator with CCIP cross-chain messaging capabilities.
- Created .gitmodules to include OpenZeppelin contracts as a submodule.
- Developed a comprehensive deployment guide in NEXT_STEPS_COMPLETE_GUIDE.md for Phase 2 and smart contract deployment.
- Implemented Vite configuration for the orchestration portal, supporting both Vue and React frameworks.
- Added server-side logic for the Multi-Cloud Orchestration Portal, including API endpoints for environment management and monitoring.
- Created scripts for resource import and usage validation across non-US regions.
- Added tests for CCIP error handling and integration to ensure robust functionality.
- Included various new files and directories for the orchestration portal and deployment scripts.
2025-12-12 14:57:48 -08:00

4.2 KiB
Raw Blame History

Connectivity Issue and Solutions

Problem Identified

No VNet Peering/Routing between:

  • Nginx Proxy VNet: az-p-wst-proxy-vnet (West Europe - 10.10.0.0/16)
  • Besu Node VNets: 5 US regions (10.1-10.5.0.0/16)

Impact

  • ❌ Nginx cannot reach backend Besu nodes (100% packet loss)
  • ❌ RPC requests timeout (Error 524)
  • ❌ Backend nodes use private IPs only (no public access)
  • ❌ Cross-region connectivity blocked

Current Architecture

West Europe (Nginx Proxy)
├── VNet: az-p-wst-proxy-vnet (10.10.0.0/16)
├── Subnet: 10.10.1.0/24
└── Nginx: 10.10.1.4 (public IP: 20.160.58.99)

US Regions (Besu Nodes)
├── Central US: 10.3.0.0/16 → 10.3.1.4
├── East US: 10.1.0.0/16 → 10.1.1.4
├── East US 2: 10.4.0.0/16 → 10.4.1.4
├── West US: 10.2.0.0/16 → 10.2.1.4
└── West US 2: 10.5.0.0/16 → 10.5.1.4

❌ No connectivity between regions

Solutions

Option 1: Cloudflare Tunnel on Backend VMs ⭐ Recommended

Deploy Cloudflare Tunnel on each Besu VM to expose RPC endpoints

Implementation

  1. Install cloudflared on each backend VM
  2. Create Cloudflare Tunnel for each VM
  3. Configure tunnel to expose port 8545
  4. Update Nginx backend configuration to use Cloudflare Tunnel endpoints

Pros

  • ✅ Quick to deploy (minutes per VM)
  • ✅ No Azure networking changes required
  • ✅ Already have Cloudflare Tunnel setup scripts
  • ✅ Works with existing Cloudflare account
  • ✅ Secure (encrypted tunnel)

Cons

  • ⚠️ Additional Cloudflare Tunnel instances (5 tunnels)
  • ⚠️ Cloudflare bandwidth usage

Steps

# For each backend VM:
1. SSH to VM
2. Run: setup-cloudflare-tunnel-backend.sh
3. Authenticate with Cloudflare
4. Create tunnel
5. Configure to expose port 8545
6. Update Nginx backend config with tunnel endpoint

Option 2: Azure VPN Gateway

Deploy VPN Gateways and establish site-to-site VPN connections

Implementation

  1. Deploy VPN Gateway in West Europe (Nginx region)
  2. Deploy VPN Gateways in each US region (Besu nodes)
  3. Establish site-to-site VPN connections
  4. Configure routing tables
  5. Update NSG rules for VPN traffic

Pros

  • ✅ Native Azure solution
  • ✅ Secure (IPSec VPN)
  • ✅ No external dependencies
  • ✅ Full network connectivity

Cons

  • ⚠️ Complex setup (multiple gateways)
  • ⚠️ Higher cost (VPN Gateway SKUs)
  • ⚠️ Longer deployment time
  • ⚠️ Requires VPN Gateway configuration

Estimated Cost

  • VPN Gateway Basic: ~$0.19/hour per gateway
  • 6 gateways (1 Nginx + 5 Besu): ~$1.14/hour = ~$820/month
  • Data transfer: Additional costs

Option 3: VNet Peering

Not Available - User confirmed no VNet peering/routing

Would require:

  • Peering between West Europe and each US region
  • Global VNet peering (if supported)
  • Route tables configuration

Recommendation

Option 1: Cloudflare Tunnel on Backend VMs

Why This Solution

  1. Fastest to implement - Scripts already exist
  2. No infrastructure changes - No Azure networking modifications
  3. Cost-effective - No additional Azure resources
  4. Already using Cloudflare - Nginx proxy already has tunnel
  5. Secure - Encrypted tunnel traffic

Implementation Plan

  1. Deploy Cloudflare Tunnel on each backend VM

    • Use existing script: setup-cloudflare-tunnel-backend.sh
    • Create tunnel per VM
    • Expose port 8545

  2. Update Nginx Backend Configuration

    • Replace private IPs with Cloudflare Tunnel endpoints
    • Or use Cloudflare Tunnel IP ranges

  3. Test Connectivity

    • Verify Nginx can reach backend nodes
    • Test RPC endpoint

Next Steps

  1. ✅ Documentation: Complete (this document)
  2. ⏳ Deploy Cloudflare Tunnels: On each backend VM
  3. ⏳ Update Nginx Config: Use tunnel endpoints
  4. ⏳ Test RPC Endpoint: Verify connectivity

Files Available

  • scripts/setup-cloudflare-tunnel-backend.sh - Backend VM tunnel setup
  • CLOUDFLARE_TUNNEL_SETUP.md - Detailed setup instructions
  • connectivity/README.md - Connectivity options documentation

Status: ⚠️ Connectivity Issue Identified, Solution Recommended

No VNet peering/routing exists. Cloudflare Tunnel on backend VMs is the recommended solution for Phase 1.

Reference in New Issue View Git Blame Copy Permalink