smom-dbis-138/terraform/phases/phase1/ALL_ISSUES_RESOLUTION.md

All Issues Resolution - Complete Fix

Issues Identified

1. SSH Keys Not Configured - Nginx proxy cannot SSH to backend VMs
2. Azure Run Command Failures - All attempts return "Bad Request" errors
3. Genesis File Not Deployed - VMs still have old 223-byte error XML

Root Cause Analysis

Issue 1: SSH Keys

• Root Cause: SSH keys were not copied to Nginx proxy
• Impact: Cannot use SSH to push files from Nginx proxy to backend VMs
• Workaround: Use Azure Run Command instead (bypasses SSH requirement)

Issue 2: Azure Run Command Failures

• Root Cause: Azure Run Command has limitations:
  • Fails on scripts with variables ($VAR, ${VAR})
  • Fails on complex conditionals (if [ ... ]; then)
  • Fails on multi-line scripts
  • Fails on long commands
• Working: Simple single-line commands work (echo "test", wc -c file)
• Solution: Use simplest possible commands, one at a time

Issue 3: Genesis File Deployment

• Root Cause: All deployment methods failed due to Azure Run Command limitations
• Solution: Use Nginx proxy as HTTP server + simple wget command

Solutions Implemented

Solution 1: Nginx Proxy as HTTP Server

1. ✅ Copy genesis file to Nginx proxy (/tmp/genesis-138.json)
2. ✅ Move to web directory (/var/www/genesis/)
3. ✅ Configure Nginx to serve on port 8080
4. ✅ Test accessibility from backend VMs

Solution 2: Simple Download Command

• Use absolute simplest command: wget http://10.10.1.4:8080/genesis-138.json -O /opt/besu/config/genesis.json
• No variables, no conditionals, no complex logic
• Run as separate commands for download, verify, and restart

Solution 3: Step-by-Step Deployment

1. Download genesis file (simple wget)
2. Verify file size (simple wc -c)
3. Set permissions (simple chmod)
4. Restart Besu (simple docker compose restart)

Deployment Process

Step 1: Setup Nginx HTTP Server

# On Nginx proxy
sudo mkdir -p /var/www/genesis
sudo cp /tmp/genesis-138.json /var/www/genesis/genesis-138.json
sudo chmod 644 /var/www/genesis/genesis-138.json
sudo chown www-data:www-data /var/www/genesis/genesis-138.json

# Configure Nginx
sudo tee /etc/nginx/sites-available/genesis > /dev/null << 'EOF'
server {
    listen 8080;
    server_name localhost;
    root /var/www/genesis;
    index genesis-138.json;
    
    location / {
        try_files $uri =404;
        add_header Content-Type application/json;
    }
}
EOF
sudo ln -sf /etc/nginx/sites-available/genesis /etc/nginx/sites-enabled/genesis
sudo nginx -t && sudo systemctl reload nginx

Step 2: Deploy to All VMs

# For each VM
az vm run-command invoke \
  --resource-group <RG> \
  --name <VM> \
  --command-id RunShellScript \
  --scripts "wget http://10.10.1.4:8080/genesis-138.json -O /opt/besu/config/genesis.json"

# Verify
az vm run-command invoke \
  --resource-group <RG> \
  --name <VM> \
  --command-id RunShellScript \
  --scripts "wc -c /opt/besu/config/genesis.json"

# Set permissions
az vm run-command invoke \
  --resource-group <RG> \
  --name <VM> \
  --command-id RunShellScript \
  --scripts "chmod 644 /opt/besu/config/genesis.json"

# Restart Besu
az vm run-command invoke \
  --resource-group <RG> \
  --name <VM> \
  --command-id RunShellScript \
  --scripts "cd /opt/besu && docker compose restart besu"

Status

• ✅ Nginx HTTP Server: Configured and serving genesis file on port 8080
• ✅ Genesis File: Available at http://10.10.1.4:8080/genesis-138.json
• ⏳ VM Deployment: In progress (using simple wget commands)
• ⏳ Besu Restart: Pending after genesis file deployment
• ⏳ RPC Testing: Pending after Besu restart

Next Steps

1. Complete genesis file deployment to all 5 VMs
2. Restart Besu containers
3. Wait 30-60 seconds for Besu to initialize
4. Test RPC endpoints
5. Verify Cloudflare endpoint

---

Last Updated: After implementing Nginx HTTP server solution