d-bis/proxmox
4
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
Files
9c37af10c0325987b78f774069e82d668fb970d0
proxmox/docs/05-network/RPC_PUBLIC_ENDPOINT_ROUTING.md
defiQUG 9c37af10c0 Complete optional next steps: fix references and consolidate duplicates 
- Fixed 104 broken references in 59 files
- Consolidated 40+ duplicate status files
- Archived duplicates to reports/archive/duplicates/
- Created scripts for reference fixing and consolidation
- Updated content inconsistency reports

All optional cleanup tasks complete.
2026-01-06 02:25:38 -08:00

8.7 KiB
Raw Blame History

Public RPC Endpoint Routing Architecture

Last Updated: 2025-01-27
Document Version: 1.0
Status: Active Documentation

Architecture Overview

The public RPC endpoints route through multiple layers:

Internet → Cloudflare (DNS/SSL) → Cloudflared Tunnel → Nginx → Besu RPC

Endpoint Routing

HTTP RPC Endpoint

URL: https://rpc-http-pub.d-bis.org

Routing Path:

  1. Cloudflare DNS/SSL: rpc-http-pub.d-bis.org resolves to Cloudflare IPs
  2. Cloudflare Edge: SSL termination, DDoS protection
  3. Cloudflared Tunnel: Encrypted tunnel from Cloudflare to internal network
  4. Nginx (VMID 2500): Receives request, proxies to Besu RPC
  5. Besu RPC: http://192.168.11.250:8545 (VMID 2500)

Configuration:

  • Should NOT require authentication (public endpoint)
  • Must accept requests without JWT tokens (for MetaMask compatibility)

WebSocket RPC Endpoint

URL: wss://rpc-ws-pub.d-bis.org

Routing Path:

  1. Cloudflare DNS/SSL: rpc-ws-pub.d-bis.org resolves to Cloudflare IPs
  2. Cloudflare Edge: SSL termination, WebSocket support
  3. Cloudflared Tunnel: Encrypted tunnel from Cloudflare to internal network
  4. Nginx (VMID 2500): Receives WebSocket upgrade, proxies to Besu RPC
  5. Besu RPC: ws://192.168.11.250:8546 (VMID 2500)

Configuration:

  • Should NOT require authentication (public endpoint)
  • Must accept WebSocket connections without JWT tokens

Components

1. Cloudflare DNS/SSL

  • DNS: rpc-http-pub.d-bis.org → CNAME to Cloudflared tunnel
  • SSL: Terminated at Cloudflare edge
  • DDoS Protection: Enabled (if proxied)

2. Cloudflared Tunnel

Location: VMID 102 (or wherever cloudflared is running)

Configuration: Routes traffic from Cloudflare to Nginx on VMID 2500

Example Config:

ingress:
  - hostname: rpc-http-pub.d-bis.org
    service: http://192.168.11.250:443  # Nginx on VMID 2500
  - hostname: rpc-ws-pub.d-bis.org
    service: http://192.168.11.250:443  # Nginx on VMID 2500

3. Nginx (VMID 2500)

IP: 192.168.11.250
Purpose: Reverse proxy to Besu RPC

Requirements:

  • MUST NOT require JWT authentication for public endpoints
  • Must proxy to 127.0.0.1:8545 (HTTP RPC)
  • Must proxy to 127.0.0.1:8546 (WebSocket RPC)
  • Must handle WebSocket upgrades correctly

4. Besu RPC (VMID 2500)

HTTP RPC: 127.0.0.1:8545 (internally) / 192.168.11.250:8545 (network)
WebSocket RPC: 127.0.0.1:8546 (internally) / 192.168.11.250:8546 (network)
Chain ID: 138 (0x8a in hex)

Nginx Configuration Requirements

Public HTTP RPC Endpoint

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name rpc-http-pub.d-bis.org;

    # SSL certificates
    ssl_certificate /etc/nginx/ssl/rpc-http-pub.crt;
    ssl_certificate_key /etc/nginx/ssl/rpc-http-pub.key;

    # Trust Cloudflare IPs for real IP
    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 188.114.96.0/20;
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 104.16.0.0/13;
    set_real_ip_from 104.24.0.0/14;
    set_real_ip_from 172.64.0.0/13;
    set_real_ip_from 131.0.72.0/22;
    real_ip_header CF-Connecting-IP;

    access_log /var/log/nginx/rpc-http-pub-access.log;
    error_log /var/log/nginx/rpc-http-pub-error.log;

    # Proxy to Besu RPC - NO AUTHENTICATION
    location / {
        proxy_pass http://127.0.0.1:8545;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # CORS headers (if needed)
        add_header Access-Control-Allow-Origin *;
        add_header Access-Control-Allow-Methods "GET, POST, OPTIONS";
        add_header Access-Control-Allow-Headers "Content-Type, Authorization";
        
        # NO JWT authentication here!
    }
}

Public WebSocket RPC Endpoint

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name rpc-ws-pub.d-bis.org;

    # SSL certificates
    ssl_certificate /etc/nginx/ssl/rpc-ws-pub.crt;
    ssl_certificate_key /etc/nginx/ssl/rpc-ws-pub.key;

    # Trust Cloudflare IPs for real IP
    set_real_ip_from 173.245.48.0/20;
    # ... (same Cloudflare IP ranges as above)
    real_ip_header CF-Connecting-IP;

    access_log /var/log/nginx/rpc-ws-pub-access.log;
    error_log /var/log/nginx/rpc-ws-pub-error.log;

    # Proxy to Besu WebSocket RPC - NO AUTHENTICATION
    location / {
        proxy_pass http://127.0.0.1:8546;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # WebSocket timeouts
        proxy_read_timeout 86400;
        proxy_send_timeout 86400;
        
        # NO JWT authentication here!
    }
}

Common Issues

Issue 1: "Could not fetch chain ID" Error in MetaMask

Symptom: MetaMask shows error when trying to connect to the network.

Root Cause: Nginx is requiring JWT authentication for the public endpoint.

Fix: Remove JWT authentication from the Nginx configuration for rpc-http-pub.d-bis.org.

Check:

ssh root@192.168.11.10 "pct exec 2500 -- nginx -T | grep -A 30 'rpc-http-pub'"

Look for:

  • auth_request directives (remove them)
  • Lua JWT validation scripts (remove them)

Issue 2: Cloudflared Tunnel Not Routing Correctly

Symptom: Requests don't reach Nginx.

Fix: Verify Cloudflared tunnel configuration is routing to 192.168.11.250:443.

Check:

# Check cloudflared config (adjust VMID if different)
ssh root@192.168.11.10 "pct exec 102 -- cat /etc/cloudflared/config.yml"

Issue 3: Nginx Not Listening on Port 443

Symptom: Connection refused errors.

Fix: Ensure Nginx is listening on port 443 and SSL certificates are configured.

Check:

ssh root@192.168.11.10 "pct exec 2500 -- ss -tuln | grep 443"
ssh root@192.168.11.10 "pct exec 2500 -- systemctl status nginx"

Testing

Test HTTP RPC Endpoint

curl -X POST https://rpc-http-pub.d-bis.org \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}'

Expected Response:

{"jsonrpc":"2.0","id":1,"result":"0x8a"}

Test WebSocket RPC Endpoint

wscat -c wss://rpc-ws-pub.d-bis.org

Then send:

{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}

Verification Checklist

  • Cloudflare DNS resolves rpc-http-pub.d-bis.org correctly
  • Cloudflared tunnel is running and routing to 192.168.11.250:443
  • Nginx on VMID 2500 is running and listening on port 443
  • Nginx configuration for rpc-http-pub.d-bis.org does NOT require JWT
  • Nginx proxies to 127.0.0.1:8545 correctly
  • Besu RPC on VMID 2500 is running and responding on port 8545
  • eth_chainId request returns 0x8a without authentication
  • MetaMask can connect to the network successfully

Related Documentation

Network Documents

Configuration Documents

Troubleshooting

Last Updated: 2025-01-27
Document Version: 1.0
Review Cycle: Quarterly

Last Updated: 2025-01-27

Reference in New Issue View Git Blame Copy Permalink