# Deployment Configuration Audit Report ## Executive Summary This document provides a comprehensive audit of the deployment configuration for the DeFi Oracle Meta Mainnet (ChainID 138), identifying misconfigurations, gaps, and recommendations. **Audit Date**: $(date) **Chain ID**: 138 **Status**: ⚠️ **CONFIGURATION ISSUES FOUND** --- ## Critical Issues ### 1. ❌ Genesis File Missing Validators **Issue**: The `config/genesis.json` file has `extraData: "0x"` which means no validators are configured in the genesis block. **Impact**: - IBFT 2.0 requires validators to be specified in the genesis `extraData` field - Network cannot start without validators - Blocks cannot be produced **Location**: `config/genesis.json` line 35 **Current State**: ```json "extraData": "0x" ``` **Required**: Validator addresses must be encoded in `extraData` using IBFT 2.0 format: ``` extraData = RLP([32 bytes Vanity, [][20 bytes]Validators, 65 bytes Signature]) ``` **Fix**: Run `./scripts/generate-genesis.sh` to regenerate genesis with validator addresses from `keys/validators/`. --- ### 2. ❌ Terraform Node Counts Disabled **Issue**: In `terraform/terraform.tfvars`, sentries and RPC nodes are set to 0: ```hcl node_count = { system = 1 validators = 1 sentries = 0 # ❌ Disabled rpc = 0 # ❌ Disabled } ``` **Impact**: - No RPC endpoints will be available (explains why RPCs are not live) - No sentry nodes for P2P connectivity - Network cannot be accessed externally - Contracts cannot be deployed **Fix**: Update `terraform/terraform.tfvars`: ```hcl node_count = { system = 3 validators = 4 sentries = 3 rpc = 3 } ``` **Note**: Current configuration shows quota constraints (4 vCPUs remaining). Consider: 1. Requesting quota increase 2. Using smaller VM sizes 3. Staged deployment (deploy validators first, then sentries/RPC) --- ### 3. ⚠️ Kubernetes Version Mismatch **Issue**: `terraform/terraform.tfvars` specifies `kubernetes_version = "1.33"` which is likely invalid. **Impact**: - Terraform may fail during AKS cluster creation - AKS may not support version 1.33 **Current Supported Versions**: AKS typically supports versions up to 1.28-1.30 range. **Fix**: Update to a supported version: ```hcl kubernetes_version = "1.28" # or latest supported ``` **Verification**: Check supported versions: ```bash az aks get-versions --location westeurope --output table ``` --- ## Configuration Gaps ### 4. ⚠️ Missing Validator Addresses in Genesis **Issue**: Genesis file doesn't include validator addresses in `extraData`. **Required**: Validator public keys must be extracted from `keys/validators/` and encoded in genesis `extraData`. **Fix**: Ensure `scripts/generate-genesis.sh`: 1. Reads validator public keys from `keys/validators/*/key.pub` 2. Encodes them in IBFT 2.0 format 3. Updates `extraData` field --- ### 5. ⚠️ Static Nodes Configuration **Issue**: `config/static-nodes.json` may be empty or incomplete. **Impact**: Nodes may not be able to peer with each other. **Required**: Static nodes should include: - Validator enode addresses - Sentry enode addresses **Fix**: Ensure static-nodes.json is generated with all node enode addresses. --- ### 6. ⚠️ Terraform Backend Not Configured **Issue**: `terraform/main.tf` has backend configuration but it's commented/empty. **Impact**: - Terraform state may not be stored properly - State locking may not work - Team collaboration issues **Fix**: Configure Terraform backend: ```hcl backend "azurerm" { resource_group_name = "tfstate-rg" storage_account_name = "tfstate" container_name = "tfstate" key = "defi-oracle-mainnet.terraform.tfstate" } ``` --- ### 7. ⚠️ Missing Application Gateway Configuration **Issue**: Application Gateway configuration may be incomplete for RPC endpoints. **Required**: - Backend pool configuration for RPC nodes - HTTP settings - Listener configuration - Routing rules - WAF rules **Location**: Check `terraform/modules/networking/` for Application Gateway configuration. --- ### 8. ⚠️ Missing DNS Configuration **Issue**: DNS records for `rpc.d-bis.org` and `rpc2.d-bis.org` may not be configured. **Impact**: RPC endpoints won't be accessible via domain names. **Fix**: After Application Gateway deployment, configure Cloudflare DNS: ```bash ./scripts/deployment/cloudflare-dns.sh --zone-id $CLOUDFLARE_ZONE_ID --api-token $CLOUDFLARE_API_TOKEN --ip ``` --- ## Consistency Checks ### ✅ Chain ID Consistency **Status**: ✅ **CONSISTENT** All configurations use Chain ID 138: - `config/genesis.json`: ✅ 138 - `helm/besu-network/values.yaml`: ✅ 138 - `config/rpc/besu-config.toml`: ✅ network-id=138 - `config/validators/besu-config.toml`: ✅ network-id=138 - `config/sentries/besu-config.toml`: ✅ network-id=138 - `config/blockscout/config.json`: ✅ 138 --- ### ✅ IBFT 2.0 Configuration **Status**: ✅ **CONSISTENT** IBFT 2.0 parameters are consistent: - Block period: 2 seconds ✅ - Epoch length: 30,000 blocks ✅ - Request timeout: 10 seconds ✅ **Location**: `config/genesis.json` and all Besu config files. --- ### ⚠️ Resource Configuration Inconsistencies **Issue**: Resource requests/limits differ between Helm values and Terraform node sizes. **Helm values-validators.yaml**: - Requests: cpu: "4", memory: "8Gi" - Limits: cpu: "8", memory: "16Gi" **Helm values.yaml (base)**: - Requests: cpu: "2", memory: "4Gi" - Limits: cpu: "4", memory: "8Gi" **Terraform terraform.tfvars**: - VM Size: `Standard_D4s_v3` (4 vCPUs, 16 GiB RAM) **Analysis**: - Helm values-validators.yaml requests 4 CPUs but base values.yaml requests 2 CPUs - Terraform uses D4s_v3 (4 vCPUs) which matches values-validators.yaml - Base values.yaml may be overridden by values-validators.yaml (correct) **Recommendation**: Ensure values-validators.yaml is used when deploying validators. --- ### ⚠️ Storage Configuration **Status**: ⚠️ **INCONSISTENT** **Helm values-validators.yaml**: 512Gi **Helm values-rpc.yaml**: 500Gi **Helm values.yaml (base)**: 256Gi **k8s/base/validators/statefulset.yaml**: 512Gi ✅ **k8s/base/rpc/statefulset.yaml**: 256Gi ❌ (should be 500Gi per values-rpc.yaml) **Fix**: Update `k8s/base/rpc/statefulset.yaml` storage size to match Helm values. --- ## Blockchain Technology Configuration ### Besu Configuration #### Validators - ✅ Consensus: IBFT 2.0 - ✅ RPC: Disabled (correct for security) - ✅ P2P: Enabled on port 30303 - ✅ Sync Mode: FULL - ✅ Network ID: 138 - ✅ Metrics: Enabled on port 9545 #### Sentries - ✅ Consensus: IBFT 2.0 (read-only) - ✅ RPC: Enabled but internal only (127.0.0.1) - ✅ P2P: Enabled on port 30303 - ✅ Sync Mode: FULL - ✅ Network ID: 138 - ✅ Metrics: Enabled #### RPC Nodes - ✅ Consensus: IBFT 2.0 (read-only) - ✅ RPC: Enabled publicly (0.0.0.0) - ✅ P2P: Disabled (correct) - ✅ Sync Mode: SNAP (correct for RPC nodes) - ✅ Network ID: 138 - ✅ CORS: Enabled with wildcard (⚠️ should be restricted in production) - ✅ Host Allowlist: Wildcard (⚠️ should be restricted in production) **Security Concern**: RPC nodes have `corsOrigins: ["*"]` and `hostAllowlist: ["*"]`. For production, these should be restricted to specific domains. --- ### Network Architecture **Tiered Architecture**: ✅ **CORRECTLY CONFIGURED** 1. **Validators** (Private subnets) - ✅ No public IPs - ✅ RPC disabled - ✅ P2P to sentries only 2. **Sentries** (Public subnets) - ✅ Public P2P - ✅ Internal RPC only - ✅ Peer to validators and sentries 3. **RPC Nodes** (DMZ subnet) - ✅ No P2P - ✅ Public RPC - ✅ Behind Application Gateway --- ## Missing Configurations ### 1. Application Gateway Configuration **Status**: ⚠️ **MISSING OR INCOMPLETE** **Required**: - Backend pool with RPC node IPs - HTTP settings - Listener on port 443 (HTTPS) - Routing rules - WAF policy - SSL certificate configuration **Location**: Check `terraform/modules/networking/` for Application Gateway module. --- ### 2. Monitoring Configuration **Status**: ⚠️ **PARTIALLY CONFIGURED** **Found**: - ✅ Prometheus configuration referenced - ✅ Grafana optional - ✅ Metrics enabled on Besu nodes **Missing**: - ServiceMonitor CRD configuration - Alert rules - Alertmanager configuration --- ### 3. Key Management **Status**: ⚠️ **NEEDS VERIFICATION** **Found**: - ✅ Validator keys directory structure - ✅ Key generation scripts - ✅ Azure Key Vault module **Missing Verification**: - Keys stored in Azure Key Vault - Kubernetes secrets created from Key Vault - Key rotation procedures --- ### 4. Backup Configuration **Status**: ⚠️ **NOT CONFIGURED** **Missing**: - Backup storage account configuration - Backup schedule - Chaindata backup procedures - Key backup procedures --- ## Recommendations ### Immediate Actions (Before Deployment) 1. **Fix Genesis File** ```bash ./scripts/generate-genesis.sh ``` Verify `extraData` contains validator addresses. 2. **Update Terraform Node Counts** ```hcl node_count = { system = 3 validators = 4 sentries = 3 rpc = 3 } ``` 3. **Fix Kubernetes Version** ```hcl kubernetes_version = "1.28" # Check latest supported ``` 4. **Verify Validator Keys** ```bash ls -la keys/validators/*/key.pub ``` Ensure 4 validator public keys exist. ### Pre-Deployment Checklist - [ ] Genesis file has validators in extraData - [ ] Terraform node counts are correct - [ ] Kubernetes version is supported - [ ] Validator keys are generated - [ ] Static nodes are configured - [ ] Terraform backend is configured - [ ] Application Gateway is configured - [ ] DNS records are ready - [ ] Monitoring is configured - [ ] Backup procedures are defined ### Post-Deployment Verification - [ ] Validators are producing blocks - [ ] Sentries are peering correctly - [ ] RPC endpoints are accessible - [ ] Application Gateway is routing correctly - [ ] DNS is resolving - [ ] Monitoring is collecting metrics - [ ] Contracts can be deployed --- ## Configuration Files Summary ### ✅ Correctly Configured - Chain ID: 138 (consistent across all files) - IBFT 2.0 parameters (block period, epoch, timeout) - Network ID: 138 (consistent) - Besu image: hyperledger/besu:23.10.0 - Resource sizing (mostly consistent) - Storage classes: managed-premium - Namespace: besu-network ### ❌ Needs Fixing - Genesis extraData (missing validators) - Terraform node counts (sentries=0, rpc=0) - Kubernetes version (1.33 likely invalid) - RPC CORS/host allowlist (too permissive) - Storage size in k8s/rpc/statefulset.yaml (inconsistent) ### ⚠️ Needs Verification - Terraform backend configuration - Application Gateway configuration - DNS configuration - Key Vault integration - Monitoring setup - Backup procedures --- ## Next Steps 1. **Fix Critical Issues** (Genesis, Node Counts, K8s Version) 2. **Regenerate Genesis** with validator addresses 3. **Update Terraform Configuration** 4. **Verify All Configurations** 5. **Deploy Infrastructure** 6. **Deploy Kubernetes Resources** 7. **Deploy Contracts** 8. **Verify End-to-End** --- ## Support For questions or issues: - Review configuration files - Check deployment documentation - Verify prerequisites - Run validation scripts