Add Oracle Aggregator and CCIP Integration

- Introduced Aggregator.sol for Chainlink-compatible oracle functionality, including round-based updates and access control.
- Added OracleWithCCIP.sol to extend Aggregator with CCIP cross-chain messaging capabilities.
- Created .gitmodules to include OpenZeppelin contracts as a submodule.
- Developed a comprehensive deployment guide in NEXT_STEPS_COMPLETE_GUIDE.md for Phase 2 and smart contract deployment.
- Implemented Vite configuration for the orchestration portal, supporting both Vue and React frameworks.
- Added server-side logic for the Multi-Cloud Orchestration Portal, including API endpoints for environment management and monitoring.
- Created scripts for resource import and usage validation across non-US regions.
- Added tests for CCIP error handling and integration to ensure robust functionality.
- Included various new files and directories for the orchestration portal and deployment scripts.
This commit is contained in:
defiQUG
2025-12-12 14:57:48 -08:00
parent a1466e4005
commit 1fb7266469
1720 changed files with 241279 additions and 16 deletions

View File

@@ -0,0 +1,146 @@
# VM Deployment Scripts
This directory contains scripts for deploying and managing Besu nodes on Azure Virtual Machines (VMs) or Virtual Machine Scale Sets (VMSS).
## Scripts
### Deployment Scripts
- **`deploy-vm-network.sh`** - Deploy Besu network on VMs using Terraform
- **`setup-vm.sh`** - Setup individual VM with Docker and Besu
- **`setup-cloud-init.sh`** - Generate cloud-init configuration
### Management Scripts
- **`monitor-vm.sh`** - Monitor all VMs and Besu containers
- **`update-vm-config.sh`** - Update Besu configuration on a VM
- **`get-vm-ips.sh`** - Get IP addresses of all VMs
- **`scale-vmss.sh`** - Scale VM Scale Set up or down
### Validation Scripts
- **`validate-vm-deployment.sh`** - Validate VM deployment
- **`health-check-vm.sh`** - Health check for VM-based Besu nodes
- **`run-all-checks.sh`** - Run all validation and health checks
### Backup/Restore Scripts
- **`backup-vm.sh`** - Backup Besu data from a VM
- **`restore-vm.sh`** - Restore Besu data to a VM
## Usage
### Deploy Network
```bash
./scripts/vm-deployment/deploy-vm-network.sh
```
### Setup Individual VM
```bash
./scripts/vm-deployment/setup-vm.sh validator 0
```
### Monitor VMs
```bash
./scripts/vm-deployment/monitor-vm.sh
```
### Validate Deployment
```bash
./scripts/vm-deployment/validate-vm-deployment.sh
```
### Health Check
```bash
./scripts/vm-deployment/health-check-vm.sh
```
### Get VM IPs
```bash
./scripts/vm-deployment/get-vm-ips.sh
```
### Scale VMSS
```bash
./scripts/vm-deployment/scale-vmss.sh besu-rpc-vmss 5
```
### Backup VM
```bash
./scripts/vm-deployment/backup-vm.sh <vm-ip>
```
### Restore VM
```bash
./scripts/vm-deployment/restore-vm.sh <vm-ip> <backup-file>
```
### Run All Checks
```bash
./scripts/vm-deployment/run-all-checks.sh
```
## Environment Variables
Most scripts support the following environment variables:
- `RESOURCE_GROUP` - Azure resource group name (default: `defi-oracle-mainnet-rg`)
- `CLUSTER_NAME` - Cluster name (default: `defi-oracle-aks`)
- `KEY_VAULT_NAME` - Key Vault name (default: `defi-oracle-kv`)
- `VALIDATOR_COUNT` - Number of validators (default: `2`)
- `SENTRY_COUNT` - Number of sentries (default: `2`)
- `RPC_COUNT` - Number of RPC nodes (default: `2`)
## Prerequisites
- Azure CLI installed and configured
- SSH key pair for VM access
- Terraform >= 1.0 (for deployment scripts)
- `jq` for JSON parsing
- `curl` for HTTP requests
## Examples
### Deploy Network
```bash
export RESOURCE_GROUP="defi-oracle-mainnet-rg"
export SSH_PUBLIC_KEY=$(cat ~/.ssh/id_rsa.pub)
./scripts/vm-deployment/deploy-vm-network.sh
```
### Monitor Specific VM
```bash
export VM_IP="20.123.45.67"
./scripts/vm-deployment/monitor-vm.sh
```
### Scale RPC VMSS
```bash
export RESOURCE_GROUP="defi-oracle-mainnet-rg"
./scripts/vm-deployment/scale-vmss.sh besu-rpc-vmss 5
```
## Troubleshooting
See [VM Deployment Troubleshooting Guide](../docs/VM_DEPLOYMENT_TROUBLESHOOTING.md) for common issues and solutions.
## Related Documentation
- [VM Deployment Guide](../docs/VM_DEPLOYMENT.md)
- [VM Deployment Quickstart](../docs/VM_DEPLOYMENT_QUICKSTART.md)
- [VM Deployment Checklist](../docs/VM_DEPLOYMENT_CHECKLIST.md)
- [Deployment Comparison](../docs/DEPLOYMENT_COMPARISON.md)

View File

@@ -0,0 +1,49 @@
#!/usr/bin/env bash
# Backup VM data script
# This script backs up Besu data from a VM
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
VM_IP="${1:-}"
BACKUP_DIR="${BACKUP_DIR:-/backup/vm}"
DATE=$(date +%Y%m%d-%H%M%S)
if [ -z "$VM_IP" ]; then
log_error "Error: VM IP not provided"
echo "Usage: $0 <vm-ip>"
exit 1
fi
log_success "Backing up VM data: $VM_IP"
# Create backup directory
mkdir -p "$BACKUP_DIR"
# Stop Besu container
log_warn "Stopping Besu container..."
ssh besuadmin@$VM_IP "docker compose -f /opt/besu/docker-compose.yml down"
# Backup data
log_warn "Backing up data..."
ssh besuadmin@$VM_IP "tar czf - /opt/besu/data /opt/besu/config" > "$BACKUP_DIR/vm-$VM_IP-$DATE.tar.gz"
if [ $? -eq 0 ]; then
log_success "✓ Backup created: vm-$VM_IP-$DATE.tar.gz"
else
log_error "✗ Backup failed"
exit 1
fi
# Restart Besu container
log_warn "Restarting Besu container..."
ssh besuadmin@$VM_IP "docker compose -f /opt/besu/docker-compose.yml up -d"
log_success "Backup completed!"

View File

@@ -0,0 +1,97 @@
#!/usr/bin/env bash
# Deploy Besu network on VMs/VMSS across multiple regions
# This script orchestrates VM deployment using Terraform
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
REGIONS=("eastus" "westus" "westeurope")
VALIDATOR_COUNT=2
SENTRY_COUNT=2
RPC_COUNT=2
USE_VMSS=false
log_success "Deploying Besu network on VMs..."
# Check prerequisites
if ! command -v terraform &> /dev/null; then
log_error "Error: Terraform not found. Please install Terraform."
exit 1
fi
if ! command -v az &> /dev/null; then
log_error "Error: Azure CLI not found. Please install Azure CLI."
exit 1
fi
# Check if logged in to Azure
if ! az account show &> /dev/null; then
log_error "Error: Not logged in to Azure. Please run 'az login'."
exit 1
fi
# Check for SSH public key
if [ -z "$SSH_PUBLIC_KEY" ]; then
if [ -f ~/.ssh/id_rsa.pub ]; then
SSH_PUBLIC_KEY=$(cat ~/.ssh/id_rsa.pub)
log_warn "Using SSH key from ~/.ssh/id_rsa.pub"
else
log_error "Error: SSH public key not found. Set SSH_PUBLIC_KEY environment variable or create ~/.ssh/id_rsa.pub"
exit 1
fi
fi
# Initialize Terraform
log_warn "Initializing Terraform..."
cd "$PROJECT_ROOT/terraform"
terraform init
# Create terraform.tfvars for VM deployment
cat > terraform.tfvars.vm <<EOF
vm_deployment_enabled = true
vm_regions = ${REGIONS[@]}
validator_vm_count = $VALIDATOR_COUNT
sentry_vm_count = $SENTRY_COUNT
rpc_vm_count = $RPC_COUNT
use_vmss = $USE_VMSS
ssh_public_key = "$SSH_PUBLIC_KEY"
vm_size_validator = "Standard_D4s_v3"
vm_size_sentry = "Standard_D4s_v3"
vm_size_rpc = "Standard_D8s_v3"
EOF
# Plan deployment
log_warn "Planning Terraform deployment..."
terraform plan -var-file=terraform.tfvars.vm -out=vm-deployment.tfplan
# Apply deployment
log_warn "Applying Terraform deployment..."
read -p "Do you want to proceed with VM deployment? (y/N) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
terraform apply vm-deployment.tfplan
log_success "✓ VM deployment completed"
else
log_warn "Deployment cancelled"
exit 0
fi
# Get VM information
log_warn "Getting VM information..."
terraform output -json > vm-deployment-output.json
log_success "VM deployment completed successfully!"
log_warn "VM information saved to terraform/vm-deployment-output.json"
# Display VM information
log_warn "VM Information:"
jq -r '.vm_names.value[]' terraform/vm-deployment-output.json 2>/dev/null || echo "VM information not available"
log_success "Deployment complete!"

View File

@@ -0,0 +1,36 @@
#!/usr/bin/env bash
# Get VM IP addresses
# This script retrieves and displays VM IP addresses
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
RESOURCE_GROUP="${RESOURCE_GROUP:-defi-oracle-mainnet-rg}"
log_success "Getting VM IP addresses..."
# Get all VMs
VMS=$(az vm list --resource-group "$RESOURCE_GROUP" --show-details --query "[].{Name:name, PrivateIP:privateIps, PublicIP:publicIps, PowerState:powerState}" -o json 2>/dev/null || echo "[]")
if [ "$VMS" == "[]" ]; then
log_warn "No VMs found in resource group: $RESOURCE_GROUP"
exit 0
fi
log_warn "=== Validator VMs ==="
echo "$VMS" | jq -r '.[] | select(.Name | contains("validator")) | "\(.Name):\n Private IP: \(.PrivateIP)\n Public IP: \(.PublicIP // "N/A")\n Status: \(.PowerState)\n"'
log_warn "=== Sentry VMs ==="
echo "$VMS" | jq -r '.[] | select(.Name | contains("sentry")) | "\(.Name):\n Private IP: \(.PrivateIP)\n Public IP: \(.PublicIP // "N/A")\n Status: \(.PowerState)\n"'
log_warn "=== RPC VMs ==="
echo "$VMS" | jq -r '.[] | select(.Name | contains("rpc")) | "\(.Name):\n Private IP: \(.PrivateIP)\n Public IP: \(.PublicIP // "N/A")\n Status: \(.PowerState)\n"'
log_success "VM IP addresses retrieved!"

View File

@@ -0,0 +1,86 @@
#!/usr/bin/env bash
# Health check script for VM-based Besu nodes
# This script checks the health of all Besu nodes on VMs
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
RESOURCE_GROUP="${RESOURCE_GROUP:-defi-oracle-mainnet-rg}"
TIMEOUT=5
log_success "Checking VM health..."
# Get all VMs
VMS=$(az vm list --resource-group "$RESOURCE_GROUP" --query "[].{Name:name, IP:publicIps}" -o json 2>/dev/null || echo "[]")
if [ "$VMS" == "[]" ]; then
log_error "Error: No VMs found"
exit 1
fi
# Check each VM
echo "$VMS" | jq -r '.[] | "\(.Name)|\(.IP)"' | while IFS='|' read -r VM_NAME VM_IP; do
if [ -z "$VM_IP" ] || [ "$VM_IP" == "None" ]; then
log_warn "$VM_NAME: No public IP (may be validator)"
continue
fi
# Check VM accessibility
if ping -c 1 -W $TIMEOUT "$VM_IP" &> /dev/null; then
log_success "$VM_NAME: VM is accessible"
else
log_error "$VM_NAME: VM is not accessible"
continue
fi
# Check Docker
if ssh -o ConnectTimeout=$TIMEOUT -o StrictHostKeyChecking=no besuadmin@$VM_IP "command -v docker &> /dev/null" 2>/dev/null; then
log_success " ✓ Docker is installed"
else
log_error " ✗ Docker is not installed"
continue
fi
# Check Besu container
if ssh -o ConnectTimeout=$TIMEOUT -o StrictHostKeyChecking=no besuadmin@$VM_IP "docker ps | grep -q besu" 2>/dev/null; then
log_success " ✓ Besu container is running"
# Get container status
CONTAINER_STATUS=$(ssh -o ConnectTimeout=$TIMEOUT -o StrictHostKeyChecking=no besuadmin@$VM_IP "docker ps --filter 'name=besu' --format '{{.Status}}'" 2>/dev/null || echo "unknown")
log_warn " Status: $CONTAINER_STATUS"
else
log_error " ✗ Besu container is not running"
continue
fi
# Check metrics endpoint
if ssh -o ConnectTimeout=$TIMEOUT -o StrictHostKeyChecking=no besuadmin@$VM_IP "curl -s http://localhost:9545/metrics | grep -q besu" 2>/dev/null; then
log_success " ✓ Metrics endpoint is accessible"
else
log_warn " ⚠ Metrics endpoint is not accessible"
fi
# Check RPC endpoint (if RPC node)
if echo "$VM_NAME" | grep -q "rpc"; then
RESPONSE=$(curl -s -X POST -H "Content-Type: application/json" \
--data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
http://$VM_IP:8545 2>/dev/null || echo "")
if echo "$RESPONSE" | jq -e '.result' > /dev/null 2>&1; then
BLOCK_NUMBER=$(echo "$RESPONSE" | jq -r '.result')
DECIMAL_BLOCK=$(printf "%d" $BLOCK_NUMBER 2>/dev/null || echo "unknown")
log_success " ✓ RPC endpoint responding (block: $DECIMAL_BLOCK)"
else
log_error " ✗ RPC endpoint not responding"
fi
fi
done
log_success "Health check complete!"

View File

@@ -0,0 +1,81 @@
#!/usr/bin/env bash
# Monitor Besu nodes on VMs
# This script monitors Besu nodes and displays status
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
VM_IPS="${VM_IPS:-}"
RESOURCE_GROUP="${RESOURCE_GROUP:-defi-oracle-mainnet-rg}"
log_success "Monitoring Besu nodes..."
# Get VM IPs from Azure if not provided
if [ -z "$VM_IPS" ]; then
log_warn "Getting VM IPs from Azure..."
VM_IPS=$(az vm list-ip-addresses --resource-group "$RESOURCE_GROUP" --query "[].virtualMachine.network.publicIpAddresses[0].ipAddress" -o tsv 2>/dev/null || echo "")
if [ -z "$VM_IPS" ]; then
log_error "Error: Could not get VM IPs from Azure"
echo "Please set VM_IPS environment variable or ensure Azure CLI is configured"
exit 1
fi
fi
# Monitor each VM
for VM_IP in $VM_IPS; do
if [ -z "$VM_IP" ] || [ "$VM_IP" == "None" ]; then
continue
fi
log_warn "Monitoring VM: $VM_IP"
# Check VM accessibility
if ! ping -c 1 -W 2 "$VM_IP" &> /dev/null; then
log_error "✗ VM $VM_IP is not accessible"
continue
fi
# Check Docker
if ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no besuadmin@$VM_IP "command -v docker &> /dev/null" 2>/dev/null; then
log_success "✓ Docker is installed"
else
log_error "✗ Docker is not installed"
continue
fi
# Check Besu container
if ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no besuadmin@$VM_IP "docker ps | grep -q besu" 2>/dev/null; then
log_success "✓ Besu container is running"
# Get container status
CONTAINER_STATUS=$(ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no besuadmin@$VM_IP "docker ps --filter 'name=besu' --format '{{.Names}} {{.Status}}'" 2>/dev/null || echo "")
log_warn " Container: $CONTAINER_STATUS"
else
log_error "✗ Besu container is not running"
continue
fi
# Check metrics endpoint
if ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no besuadmin@$VM_IP "curl -s http://localhost:9545/metrics | grep -q besu" 2>/dev/null; then
log_success "✓ Metrics endpoint is accessible"
else
log_warn "⚠ Metrics endpoint is not accessible"
fi
# Get block number (if RPC node)
if ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no besuadmin@$VM_IP "docker ps | grep -q rpc" 2>/dev/null; then
BLOCK_NUMBER=$(ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no besuadmin@$VM_IP "curl -s -X POST -H 'Content-Type: application/json' --data '{\"jsonrpc\":\"2.0\",\"method\":\"eth_blockNumber\",\"params\":[],\"id\":1}' http://localhost:8545 | jq -r '.result'" 2>/dev/null || echo "unknown")
log_warn " Block number: $BLOCK_NUMBER"
fi
done
log_success "Monitoring complete!"

View File

@@ -0,0 +1,57 @@
#!/usr/bin/env bash
# Restore VM data script
# This script restores Besu data to a VM
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
VM_IP="${1:-}"
BACKUP_FILE="${2:-}"
if [ -z "$VM_IP" ] || [ -z "$BACKUP_FILE" ]; then
log_error "Error: VM IP or backup file not provided"
echo "Usage: $0 <vm-ip> <backup-file>"
exit 1
fi
if [ ! -f "$BACKUP_FILE" ]; then
log_error "Error: Backup file not found: $BACKUP_FILE"
exit 1
fi
log_success "Restoring VM data: $VM_IP from $BACKUP_FILE"
log_error "WARNING: This will overwrite existing data. Continue? (y/N)"
read -r CONFIRM
if [ "$CONFIRM" != "y" ] && [ "$CONFIRM" != "Y" ]; then
echo "Restore cancelled"
exit 0
fi
# Stop Besu container
log_warn "Stopping Besu container..."
ssh besuadmin@$VM_IP "docker compose -f /opt/besu/docker-compose.yml down"
# Restore data
log_warn "Restoring data..."
cat "$BACKUP_FILE" | ssh besuadmin@$VM_IP "sudo tar xzf - -C /"
if [ $? -eq 0 ]; then
log_success "✓ Data restored"
else
log_error "✗ Restore failed"
exit 1
fi
# Restart Besu container
log_warn "Restarting Besu container..."
ssh besuadmin@$VM_IP "docker compose -f /opt/besu/docker-compose.yml up -d"
log_success "Restore completed!"

View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
# Run all VM deployment checks
# This script runs all validation and health checks
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
log_success "Running all VM deployment checks..."
# Run validation
log_warn "=== Running Validation ==="
"$SCRIPT_DIR/validate-vm-deployment.sh"
# Run health check
log_warn "=== Running Health Check ==="
"$SCRIPT_DIR/health-check-vm.sh"
# Get VM IPs
log_warn "=== VM IP Addresses ==="
"$SCRIPT_DIR/get-vm-ips.sh"
log_success "All checks complete!"

View File

@@ -0,0 +1,86 @@
#!/usr/bin/env bash
# Scale VM Scale Set
# This script scales a VM Scale Set up or down
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
RESOURCE_GROUP="${RESOURCE_GROUP:-defi-oracle-mainnet-rg}"
VMSS_NAME="${1:-}"
NEW_CAPACITY="${2:-}"
if [ -z "$VMSS_NAME" ] || [ -z "$NEW_CAPACITY" ]; then
log_error "Error: VMSS name and capacity required"
echo "Usage: $0 <vmss-name> <new-capacity>"
echo "Example:"
echo " $0 besu-rpc-vmss 5"
exit 1
fi
# Validate capacity
if ! [[ "$NEW_CAPACITY" =~ ^[0-9]+$ ]]; then
log_error "Error: Capacity must be a number"
exit 1
fi
if [ "$NEW_CAPACITY" -lt 1 ]; then
log_error "Error: Capacity must be at least 1"
exit 1
fi
log_success "Scaling VMSS: $VMSS_NAME to $NEW_CAPACITY instances"
# Get current capacity
CURRENT_CAPACITY=$(az vmss show --resource-group "$RESOURCE_GROUP" --name "$VMSS_NAME" --query "sku.capacity" -o tsv 2>/dev/null || echo "0")
if [ "$CURRENT_CAPACITY" == "0" ]; then
log_error "Error: VMSS not found: $VMSS_NAME"
exit 1
fi
log_warn "Current capacity: $CURRENT_CAPACITY"
log_warn "New capacity: $NEW_CAPACITY"
if [ "$CURRENT_CAPACITY" -eq "$NEW_CAPACITY" ]; then
log_warn "Capacity is already $NEW_CAPACITY"
exit 0
fi
# Confirm scaling
read -p "Do you want to scale from $CURRENT_CAPACITY to $NEW_CAPACITY? (y/N) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "Scaling cancelled"
exit 0
fi
# Scale VMSS
log_warn "Scaling VMSS..."
az vmss scale \
--resource-group "$RESOURCE_GROUP" \
--name "$VMSS_NAME" \
--new-capacity "$NEW_CAPACITY"
if [ $? -eq 0 ]; then
log_success "✓ VMSS scaled successfully"
# Wait for instances to be ready
log_warn "Waiting for instances to be ready..."
sleep 30
# Check instance status
INSTANCES=$(az vmss list-instances --resource-group "$RESOURCE_GROUP" --name "$VMSS_NAME" --query "[].{Name:name, ProvisioningState:provisioningState}" -o json 2>/dev/null || echo "[]")
echo "$INSTANCES" | jq -r '.[] | "\(.Name): \(.ProvisioningState)"'
else
log_error "✗ Failed to scale VMSS"
exit 1
fi
log_success "Scaling complete!"

View File

@@ -0,0 +1,72 @@
#!/usr/bin/env bash
# Setup cloud-init script for Besu nodes
# This script creates a cloud-init configuration file for VM deployment
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
NODE_TYPE="${1:-validator}"
NODE_INDEX="${2:-0}"
OUTPUT_FILE="${3:-/tmp/besu-cloud-init.yaml}"
log_success "Generating cloud-init configuration for $NODE_TYPE-$NODE_INDEX..."
# Create cloud-init file
cat > "$OUTPUT_FILE" <<'EOF'
#cloud-config
# Cloud-init configuration for Besu node setup
package_update: true
package_upgrade: true
packages:
- apt-transport-https
- ca-certificates
- curl
- gnupg
- lsb-release
- jq
- wget
- unzip
write_files:
- path: /opt/besu/setup.sh
content: |
#!/bin/bash
set -e
echo "Setting up Besu node..."
# Install Docker
if ! command -v docker &> /dev/null; then
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
systemctl enable docker
systemctl start docker
usermod -aG docker $USER
fi
# Create directories
mkdir -p /opt/besu/{data,config,keys,logs}
chown -R $USER:$USER /opt/besu
echo "Setup complete!"
permissions: '0755'
owner: root:root
runcmd:
- /opt/besu/setup.sh
- systemctl daemon-reload
final_message: "Besu node setup complete"
EOF
log_success "✓ Cloud-init configuration created: $OUTPUT_FILE"

253
scripts/vm-deployment/setup-vm.sh Executable file
View File

@@ -0,0 +1,253 @@
#!/usr/bin/env bash
# Setup script for Besu node on VM
# This script installs Docker, configures the node, and starts Besu
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
NODE_TYPE="${1:-validator}"
NODE_INDEX="${2:-0}"
CLUSTER_NAME="${CLUSTER_NAME:-defi-oracle-aks}"
KEY_VAULT_NAME="${KEY_VAULT_NAME:-defi-oracle-kv}"
GENESIS_FILE_PATH="${GENESIS_FILE_PATH:-/opt/besu/config/genesis.json}"
log_success "Setting up Besu node: $NODE_TYPE-$NODE_INDEX"
# Check if running as root
if [ "$EUID" -ne 0 ]; then
log_error "Please run as root or with sudo"
exit 1
fi
# Update system
log_warn "Updating system..."
apt-get update
apt-get upgrade -y
# Install dependencies
log_warn "Installing dependencies..."
apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg \
lsb-release \
jq \
wget \
unzip \
software-properties-common
# Install Docker
if ! command -v docker &> /dev/null; then
log_warn "Installing Docker..."
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
systemctl enable docker
systemctl start docker
usermod -aG docker $SUDO_USER
log_success "✓ Docker installed"
else
log_success "✓ Docker already installed"
fi
# Install Azure CLI
if ! command -v az &> /dev/null; then
log_warn "Installing Azure CLI..."
curl -sL https://aka.ms/InstallAzureCLIDeb | bash
log_success "✓ Azure CLI installed"
else
log_success "✓ Azure CLI already installed"
fi
# Create directories
log_warn "Creating directories..."
mkdir -p /opt/besu/{data,config,keys,logs}
chown -R $SUDO_USER:$SUDO_USER /opt/besu
log_success "✓ Directories created"
# Download genesis file
log_warn "Downloading genesis file..."
if [ -n "$GENESIS_FILE_URL" ]; then
wget -q -O "$GENESIS_FILE_PATH" "$GENESIS_FILE_URL"
log_success "✓ Genesis file downloaded"
else
log_warn "⚠ Genesis file URL not set, using local file"
if [ -f "$PROJECT_ROOT/config/genesis.json" ]; then
cp "$PROJECT_ROOT/config/genesis.json" "$GENESIS_FILE_PATH"
log_success "✓ Genesis file copied"
else
log_error "✗ Genesis file not found"
exit 1
fi
fi
# Download Besu configuration
log_warn "Downloading Besu configuration..."
CONFIG_FILE="/opt/besu/config/besu-config.toml"
case $NODE_TYPE in
validator)
if [ -f "$PROJECT_ROOT/config/validators/besu-config.toml" ]; then
cp "$PROJECT_ROOT/config/validators/besu-config.toml" "$CONFIG_FILE"
else
log_error "✗ Validator config file not found"
exit 1
fi
;;
sentry)
if [ -f "$PROJECT_ROOT/config/sentries/besu-config.toml" ]; then
cp "$PROJECT_ROOT/config/sentries/besu-config.toml" "$CONFIG_FILE"
else
log_error "✗ Sentry config file not found"
exit 1
fi
;;
rpc)
if [ -f "$PROJECT_ROOT/config/rpc/besu-config.toml" ]; then
cp "$PROJECT_ROOT/config/rpc/besu-config.toml" "$CONFIG_FILE"
else
log_error "✗ RPC config file not found"
exit 1
fi
;;
*)
log_error "✗ Invalid node type: $NODE_TYPE"
exit 1
;;
esac
log_success "✓ Configuration file copied"
# Download validator keys from Key Vault (if validator)
if [ "$NODE_TYPE" == "validator" ]; then
log_warn "Downloading validator keys from Key Vault..."
# This would use Azure Managed Identity to access Key Vault
# For now, we'll use a placeholder
log_warn "⚠ Key download not implemented (requires Key Vault access)"
fi
# Create Docker Compose file
log_warn "Creating Docker Compose file..."
cat > /opt/besu/docker-compose.yml <<EOF
version: '3.8'
services:
besu:
image: hyperledger/besu:23.10.0
container_name: besu-${NODE_TYPE}-${NODE_INDEX}
restart: unless-stopped
user: "$SUDO_USER"
volumes:
- /opt/besu/data:/data
- /opt/besu/config:/config
- /opt/besu/keys:/keys:ro
- /opt/besu/logs:/logs
ports:
- "8545:8545" # RPC
- "8546:8546" # WebSocket
- "9545:9545" # Metrics
EOF
if [ "$NODE_TYPE" == "validator" ] || [ "$NODE_TYPE" == "sentry" ]; then
cat >> /opt/besu/docker-compose.yml <<EOF
- "30303:30303" # P2P
- "30303:30303/udp" # P2P UDP
EOF
fi
cat >> /opt/besu/docker-compose.yml <<EOF
command:
- /opt/besu/bin/besu
- --config-file=/config/besu-config.toml
environment:
- BESU_OPTS=-Xmx4g -Xms4g
networks:
- besu-network
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
healthcheck:
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:9545/metrics"]
interval: 30s
timeout: 10s
retries: 3
start_period: 120s
networks:
besu-network:
driver: bridge
EOF
chown $SUDO_USER:$SUDO_USER /opt/besu/docker-compose.yml
log_success "✓ Docker Compose file created"
# Create systemd service
log_warn "Creating systemd service..."
cat > /etc/systemd/system/besu.service <<EOF
[Unit]
Description=Besu Node Service
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/opt/besu
ExecStart=/usr/bin/docker compose up -d
ExecStop=/usr/bin/docker compose down
Restart=on-failure
RestartSec=10
User=$SUDO_USER
Group=$SUDO_USER
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable besu.service
log_success "✓ Systemd service created"
# Start Besu service
log_warn "Starting Besu service..."
systemctl start besu.service
log_success "✓ Besu service started"
# Wait for Besu to be ready
log_warn "Waiting for Besu to be ready..."
sleep 30
# Check service status
if systemctl is-active --quiet besu.service; then
log_success "✓ Besu service is running"
else
log_error "✗ Besu service failed to start"
systemctl status besu.service
exit 1
fi
# Check container status
if docker ps | grep -q "besu-${NODE_TYPE}-${NODE_INDEX}"; then
log_success "✓ Besu container is running"
else
log_error "✗ Besu container is not running"
docker ps -a | grep besu
exit 1
fi
log_success "Setup complete!"
log_warn "Node type: $NODE_TYPE"
log_warn "Node index: $NODE_INDEX"
log_warn "Logs: /opt/besu/logs"
log_warn "Data: /opt/besu/data"

View File

@@ -0,0 +1,64 @@
#!/usr/bin/env bash
# Update Besu configuration on VM
# This script updates Besu configuration and restarts the service
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
VM_IP="${1:-}"
NODE_TYPE="${2:-validator}"
CONFIG_FILE="${3:-}"
if [ -z "$VM_IP" ]; then
log_error "Error: VM IP not provided"
echo "Usage: $0 <vm-ip> <node-type> [config-file]"
exit 1
fi
log_success "Updating Besu configuration on VM: $VM_IP"
# Copy configuration file
if [ -n "$CONFIG_FILE" ]; then
log_warn "Copying configuration file..."
scp "$CONFIG_FILE" besuadmin@$VM_IP:/tmp/besu-config.toml
ssh besuadmin@$VM_IP "sudo mv /tmp/besu-config.toml /opt/besu/config/besu-config.toml"
log_success "✓ Configuration file copied"
else
# Use default configuration
CONFIG_FILE="$PROJECT_ROOT/config/${NODE_TYPE}s/besu-config.toml"
if [ -f "$CONFIG_FILE" ]; then
log_warn "Copying default configuration file..."
scp "$CONFIG_FILE" besuadmin@$VM_IP:/tmp/besu-config.toml
ssh besuadmin@$VM_IP "sudo mv /tmp/besu-config.toml /opt/besu/config/besu-config.toml"
log_success "✓ Configuration file copied"
else
log_warn "⚠ Configuration file not found, skipping"
fi
fi
# Restart Besu service
log_warn "Restarting Besu service..."
ssh besuadmin@$VM_IP "sudo systemctl restart besu.service"
log_success "✓ Besu service restarted"
# Wait for service to be ready
log_warn "Waiting for Besu to be ready..."
sleep 30
# Check service status
if ssh besuadmin@$VM_IP "systemctl is-active --quiet besu.service"; then
log_success "✓ Besu service is running"
else
log_error "✗ Besu service failed to start"
ssh besuadmin@$VM_IP "systemctl status besu.service"
exit 1
fi
log_success "Configuration update complete!"

View File

@@ -0,0 +1,114 @@
#!/usr/bin/env bash
# Validate VM deployment
# This script validates that all VMs are properly deployed and configured
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Configuration
RESOURCE_GROUP="${RESOURCE_GROUP:-defi-oracle-mainnet-rg}"
VALIDATOR_COUNT="${VALIDATOR_COUNT:-2}"
SENTRY_COUNT="${SENTRY_COUNT:-2}"
RPC_COUNT="${RPC_COUNT:-2}"
log_success "Validating VM deployment..."
# Check prerequisites
if ! command -v az &> /dev/null; then
log_error "Error: Azure CLI not found"
exit 1
fi
if ! az account show &> /dev/null; then
log_error "Error: Not logged in to Azure"
exit 1
fi
# Get VM list
log_warn "Getting VM list..."
VMS=$(az vm list --resource-group "$RESOURCE_GROUP" --query "[].{Name:name, PowerState:powerState, IP:publicIps}" -o json 2>/dev/null || echo "[]")
if [ "$VMS" == "[]" ]; then
log_error "Error: No VMs found in resource group"
exit 1
fi
VALIDATOR_VMS=$(echo "$VMS" | jq -r '.[] | select(.Name | contains("validator")) | .Name')
SENTRY_VMS=$(echo "$VMS" | jq -r '.[] | select(.Name | contains("sentry")) | .Name')
RPC_VMS=$(echo "$VMS" | jq -r '.[] | select(.Name | contains("rpc")) | .Name')
# Validate validator VMs
log_warn "Validating validator VMs..."
VALIDATOR_COUNT_FOUND=$(echo "$VALIDATOR_VMS" | grep -c . || echo "0")
if [ "$VALIDATOR_COUNT_FOUND" -lt "$VALIDATOR_COUNT" ]; then
log_error "✗ Expected $VALIDATOR_COUNT validators, found $VALIDATOR_COUNT_FOUND"
else
log_success "✓ Found $VALIDATOR_COUNT_FOUND validator VMs"
fi
# Validate sentry VMs
log_warn "Validating sentry VMs..."
SENTRY_COUNT_FOUND=$(echo "$SENTRY_VMS" | grep -c . || echo "0")
if [ "$SENTRY_COUNT_FOUND" -lt "$SENTRY_COUNT" ]; then
log_error "✗ Expected $SENTRY_COUNT sentries, found $SENTRY_COUNT_FOUND"
else
log_success "✓ Found $SENTRY_COUNT_FOUND sentry VMs"
fi
# Validate RPC VMs
log_warn "Validating RPC VMs..."
RPC_COUNT_FOUND=$(echo "$RPC_VMS" | grep -c . || echo "0")
if [ "$RPC_COUNT_FOUND" -lt "$RPC_COUNT" ]; then
log_error "✗ Expected $RPC_COUNT RPC nodes, found $RPC_COUNT_FOUND"
else
log_success "✓ Found $RPC_COUNT_FOUND RPC VMs"
fi
# Check VM status
log_warn "Checking VM status..."
for VM in $VALIDATOR_VMS $SENTRY_VMS $RPC_VMS; do
STATUS=$(az vm show --resource-group "$RESOURCE_GROUP" --name "$VM" --show-details --query "powerState" -o tsv 2>/dev/null || echo "unknown")
if [ "$STATUS" == "VM running" ]; then
log_success "$VM is running"
else
log_error "$VM status: $STATUS"
fi
done
# Check Besu containers
log_warn "Checking Besu containers..."
for VM in $SENTRY_VMS $RPC_VMS; do
IP=$(az vm show --resource-group "$RESOURCE_GROUP" --name "$VM" --show-details --query "publicIps" -o tsv 2>/dev/null || echo "")
if [ -n "$IP" ] && [ "$IP" != "None" ]; then
if ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no besuadmin@$IP "docker ps | grep -q besu" 2>/dev/null; then
log_success "$VM: Besu container is running"
else
log_error "$VM: Besu container is not running"
fi
fi
done
# Check RPC endpoints
log_warn "Checking RPC endpoints..."
for VM in $RPC_VMS; do
IP=$(az vm show --resource-group "$RESOURCE_GROUP" --name "$VM" --show-details --query "publicIps" -o tsv 2>/dev/null || echo "")
if [ -n "$IP" ] && [ "$IP" != "None" ]; then
RESPONSE=$(curl -s -X POST -H "Content-Type: application/json" \
--data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
http://$IP:8545 2>/dev/null || echo "")
if echo "$RESPONSE" | jq -e '.result' > /dev/null 2>&1; then
BLOCK_NUMBER=$(echo "$RESPONSE" | jq -r '.result')
log_success "$VM: RPC endpoint responding (block: $BLOCK_NUMBER)"
else
log_error "$VM: RPC endpoint not responding"
fi
fi
done
log_success "Validation complete!"