Add Oracle Aggregator and CCIP Integration
- Introduced Aggregator.sol for Chainlink-compatible oracle functionality, including round-based updates and access control. - Added OracleWithCCIP.sol to extend Aggregator with CCIP cross-chain messaging capabilities. - Created .gitmodules to include OpenZeppelin contracts as a submodule. - Developed a comprehensive deployment guide in NEXT_STEPS_COMPLETE_GUIDE.md for Phase 2 and smart contract deployment. - Implemented Vite configuration for the orchestration portal, supporting both Vue and React frameworks. - Added server-side logic for the Multi-Cloud Orchestration Portal, including API endpoints for environment management and monitoring. - Created scripts for resource import and usage validation across non-US regions. - Added tests for CCIP error handling and integration to ensure robust functionality. - Included various new files and directories for the orchestration portal and deployment scripts.
This commit is contained in:
269
docs/configuration/AZURE_CLOUDFLARE_ENV_SETUP.md
Normal file
269
docs/configuration/AZURE_CLOUDFLARE_ENV_SETUP.md
Normal file
@@ -0,0 +1,269 @@
|
||||
# Azure and Cloudflare Environment Setup Guide
|
||||
|
||||
**Last Updated**: 2025-01-27
|
||||
**Status**: Active
|
||||
**Purpose**: Azure and Cloudflare environment variable configuration
|
||||
|
||||
This guide helps you populate your `.env` file with values from Azure and Cloudflare.
|
||||
|
||||
> **Related Documentation**:
|
||||
> - [Network Configuration Guide](NETWORK_CONFIGURATION_GUIDE.md) - For Besu network configuration
|
||||
> - [Contract Deployment Environment Setup](CONTRACT_DEPLOYMENT_ENV_SETUP.md) - For contract deployment environment variables
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Option 1: Interactive Script (Recommended)
|
||||
|
||||
Run the interactive script that will automatically populate your `.env` file:
|
||||
|
||||
```bash
|
||||
./scripts/deployment/populate-env.sh
|
||||
```
|
||||
|
||||
This script will:
|
||||
- ✅ Retrieve Azure Subscription ID and Tenant ID from Azure CLI
|
||||
- ✅ Get or create Terraform backend storage account
|
||||
- ✅ Prompt for Cloudflare Zone ID and API Token
|
||||
- ✅ Update your `.env` file automatically
|
||||
|
||||
### Option 2: View Values First
|
||||
|
||||
If you want to see what values will be populated before updating:
|
||||
|
||||
```bash
|
||||
# View all retrievable values
|
||||
./scripts/deployment/get-env-values.sh
|
||||
|
||||
# Or save to a file for review
|
||||
./scripts/deployment/get-env-values.sh > env-values.txt
|
||||
cat env-values.txt
|
||||
```
|
||||
|
||||
Then manually copy the values to your `.env` file.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before running the scripts:
|
||||
|
||||
1. **Azure CLI installed and logged in:**
|
||||
```bash
|
||||
az login
|
||||
az account show # Verify login
|
||||
```
|
||||
|
||||
2. **Cloudflare credentials ready:**
|
||||
- Zone ID: Found in Cloudflare Dashboard > Your Domain > Overview > Zone ID
|
||||
- API Token: Create in Cloudflare Dashboard > My Profile > API Tokens
|
||||
|
||||
## Examples
|
||||
|
||||
### Example: Complete .env File
|
||||
|
||||
```bash
|
||||
# Azure Configuration
|
||||
AZURE_SUBSCRIPTION_ID=12345678-1234-1234-1234-123456789012
|
||||
AZURE_TENANT_ID=87654321-4321-4321-4321-210987654321
|
||||
AZURE_RESOURCE_GROUP=az-p-wst-rg-comp-001
|
||||
AZURE_LOCATION=westeurope
|
||||
|
||||
# Terraform Backend
|
||||
TF_STATE_STORAGE_ACCOUNT=azpwesttfstate001
|
||||
TF_STATE_RESOURCE_GROUP=az-p-wst-rg-tfstate-001
|
||||
TF_STATE_CONTAINER=tfstate
|
||||
|
||||
# Cloudflare
|
||||
CLOUDFLARE_ZONE_ID=abc123def456ghi789
|
||||
CLOUDFLARE_API_TOKEN=your-api-token-here
|
||||
|
||||
# RPC Configuration
|
||||
RPC_URL=https://rpc.d-bis.org
|
||||
CHAIN_ID=138
|
||||
```
|
||||
|
||||
### Example: Retrieving Values
|
||||
|
||||
```bash
|
||||
# Get Azure subscription ID
|
||||
az account show --query id -o tsv
|
||||
|
||||
# Get Cloudflare zone ID
|
||||
# Found in Cloudflare Dashboard > Your Domain > Overview > Zone ID
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### Azure Configuration (Auto-retrieved)
|
||||
|
||||
These are automatically retrieved from Azure CLI:
|
||||
|
||||
- `AZURE_SUBSCRIPTION_ID` - Your Azure subscription ID
|
||||
- `AZURE_TENANT_ID` - Your Azure tenant ID
|
||||
- `AZURE_RESOURCE_GROUP` - Resource group name (default: `defi-oracle-mainnet-rg`)
|
||||
- `AZURE_LOCATION` - Azure region (default: `eastus`)
|
||||
|
||||
### Terraform Backend (Auto-created or retrieved)
|
||||
|
||||
- `ARM_RESOURCE_GROUP_NAME` - Resource group for Terraform state (default: `tfstate-rg`)
|
||||
- `ARM_STORAGE_ACCOUNT_NAME` - Storage account name (auto-generated or existing)
|
||||
- `ARM_CONTAINER_NAME` - Container name (default: `tfstate`)
|
||||
- `ARM_ACCESS_KEY` - Storage account access key
|
||||
|
||||
### Cloudflare Configuration (Manual input required)
|
||||
|
||||
- `CLOUDFLARE_ZONE_ID` - Your Cloudflare Zone ID
|
||||
- `CLOUDFLARE_API_TOKEN` - Your Cloudflare API Token
|
||||
|
||||
### Optional Variables
|
||||
|
||||
These can be set later:
|
||||
|
||||
- `AZURE_CLIENT_ID` - Service principal app ID (for CI/CD)
|
||||
- `AZURE_CLIENT_SECRET` - Service principal secret (for CI/CD)
|
||||
- `RPC_URL` - RPC endpoint URL (set after deployment)
|
||||
- `EXPLORER_URL` - Explorer URL (set after deployment)
|
||||
- `PRIVATE_KEY` - Private key for contract deployment
|
||||
|
||||
## Manual Setup
|
||||
|
||||
If you prefer to set values manually:
|
||||
|
||||
### 1. Get Azure Values
|
||||
|
||||
```bash
|
||||
# Subscription ID
|
||||
az account show --query id -o tsv
|
||||
|
||||
# Tenant ID
|
||||
az account show --query tenantId -o tsv
|
||||
|
||||
# Current location
|
||||
az account show --query location -o tsv
|
||||
```
|
||||
|
||||
### 2. Create Terraform Backend Storage
|
||||
|
||||
```bash
|
||||
# Set variables
|
||||
RESOURCE_GROUP="tfstate-rg"
|
||||
STORAGE_ACCOUNT="tfstate$(openssl rand -hex 4)"
|
||||
LOCATION="eastus"
|
||||
|
||||
# Create resource group
|
||||
az group create --name $RESOURCE_GROUP --location $LOCATION
|
||||
|
||||
# Create storage account
|
||||
az storage account create \
|
||||
--resource-group $RESOURCE_GROUP \
|
||||
--name $STORAGE_ACCOUNT \
|
||||
--sku Standard_LRS \
|
||||
--kind StorageV2 \
|
||||
--location $LOCATION
|
||||
|
||||
# Create container
|
||||
az storage container create \
|
||||
--name "tfstate" \
|
||||
--account-name $STORAGE_ACCOUNT
|
||||
|
||||
# Get access key
|
||||
az storage account keys list \
|
||||
--resource-group $RESOURCE_GROUP \
|
||||
--account-name $STORAGE_ACCOUNT \
|
||||
--query "[0].value" -o tsv
|
||||
```
|
||||
|
||||
### 3. Get Cloudflare Values
|
||||
|
||||
1. **Zone ID:**
|
||||
- Go to Cloudflare Dashboard
|
||||
- Select your domain
|
||||
- Overview page shows Zone ID
|
||||
|
||||
2. **API Token:**
|
||||
- Go to Cloudflare Dashboard > My Profile > API Tokens
|
||||
- Click "Create Token"
|
||||
- Use "Edit zone DNS" template or create custom token with:
|
||||
- Zone: DNS:Edit
|
||||
- Zone: Zone:Read
|
||||
|
||||
## Verify Setup
|
||||
|
||||
After populating your `.env` file, verify the values:
|
||||
|
||||
```bash
|
||||
# Check Azure authentication
|
||||
az account show
|
||||
|
||||
# Verify resource group exists (if already created)
|
||||
az group show --name defi-oracle-mainnet-rg
|
||||
|
||||
# Test Cloudflare API (if token is set)
|
||||
curl -X GET "https://api.cloudflare.com/client/v4/zones/${CLOUDFLARE_ZONE_ID}" \
|
||||
-H "Authorization: Bearer ${CLOUDFLARE_API_TOKEN}" \
|
||||
-H "Content-Type: application/json"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Azure CLI Not Found
|
||||
|
||||
```bash
|
||||
# Install Azure CLI
|
||||
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
|
||||
|
||||
# Or on macOS
|
||||
brew install azure-cli
|
||||
```
|
||||
|
||||
### Not Logged In to Azure
|
||||
|
||||
```bash
|
||||
# Login interactively
|
||||
az login
|
||||
|
||||
# Or use the helper script
|
||||
./scripts/deployment/azure-login.sh
|
||||
```
|
||||
|
||||
### Storage Account Creation Fails
|
||||
|
||||
- Check you have permissions to create resources
|
||||
- Verify subscription has available quota
|
||||
- Try a different storage account name (must be globally unique)
|
||||
|
||||
### Cloudflare API Token Issues
|
||||
|
||||
- Ensure token has correct permissions (Zone:Read, DNS:Edit)
|
||||
- Verify token hasn't expired
|
||||
- Check Zone ID is correct for your domain
|
||||
|
||||
## Next Steps
|
||||
|
||||
After setting up your `.env` file:
|
||||
|
||||
1. **Deploy Infrastructure:**
|
||||
```bash
|
||||
cd terraform
|
||||
terraform init
|
||||
terraform plan
|
||||
terraform apply
|
||||
```
|
||||
|
||||
2. **Configure DNS:**
|
||||
```bash
|
||||
./scripts/deployment/cloudflare-dns.sh \
|
||||
--zone-id $CLOUDFLARE_ZONE_ID \
|
||||
--api-token $CLOUDFLARE_API_TOKEN \
|
||||
--ip $(./scripts/deployment/get-app-gateway-ip.sh)
|
||||
```
|
||||
|
||||
3. **Deploy Kubernetes Resources:**
|
||||
```bash
|
||||
./scripts/deployment/deploy-all.sh
|
||||
```
|
||||
|
||||
## See Also
|
||||
|
||||
- [Deployment Guide](DEPLOYMENT.md)
|
||||
- [Terraform Backend Setup](TERRAFORM_BACKEND_SETUP.md)
|
||||
- [Next Steps List](NEXT_STEPS_LIST.md)
|
||||
|
||||
372
docs/configuration/AZURE_NAMING_CONVENTION_2CHAR.md
Normal file
372
docs/configuration/AZURE_NAMING_CONVENTION_2CHAR.md
Normal file
@@ -0,0 +1,372 @@
|
||||
# Azure Naming Convention (2-3 Character Region Codes)
|
||||
|
||||
**Last Updated**: 2025-01-27
|
||||
**Status**: Active - Legacy/Alternative Convention
|
||||
**Purpose**: Azure resource naming with 2-3 character region codes
|
||||
|
||||
## Overview
|
||||
|
||||
This document defines the standardized naming convention for all Azure resources in the DeFi Oracle Meta Mainnet (ChainID 138) deployment using **2-3 character region codes**. The convention ensures uniformity, clarity, and compliance with Azure naming restrictions.
|
||||
|
||||
> **Note**: This document uses 2-3 character region codes (e.g., `we`, `ne`, `fc`).
|
||||
> **Alternative**: See [Azure Naming Convention (3 Character Region Codes)](AZURE_NAMING_CONVENTION_3CHAR.md) for the 3-character standard.
|
||||
|
||||
> **Related Documentation**:
|
||||
> - [Azure Naming Convention (3 Character)](AZURE_NAMING_CONVENTION_3CHAR.md) - Standard 3-character region code convention
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Overview](#overview)
|
||||
- [Naming Pattern](#naming-pattern)
|
||||
- [Pattern Components](#pattern-components)
|
||||
- [Region Codes](#region-codes)
|
||||
- [Resource Type Abbreviations](#resource-type-abbreviations)
|
||||
- [Core Infrastructure](#core-infrastructure)
|
||||
- [Compute](#compute)
|
||||
- [Storage](#storage)
|
||||
- [Security & Identity](#security--identity)
|
||||
- [Monitoring & Logging](#monitoring--logging)
|
||||
- [Database](#database)
|
||||
- [Networking](#networking)
|
||||
- [Naming Examples](#naming-examples)
|
||||
- [Resource Groups](#resource-groups)
|
||||
- [AKS Resources](#aks-resources)
|
||||
- [Networking](#networking-1)
|
||||
- [Storage](#storage-1)
|
||||
- [Security](#security)
|
||||
- [Monitoring](#monitoring)
|
||||
- [Special Cases](#special-cases)
|
||||
- [Terraform Backend](#terraform-backend)
|
||||
- [Multi-Region](#multi-region)
|
||||
- [Node Pools](#node-pools)
|
||||
- [Azure Naming Restrictions](#azure-naming-restrictions)
|
||||
- [General Rules](#general-rules)
|
||||
- [Resource-Specific Limits](#resource-specific-limits)
|
||||
- [Implementation](#implementation)
|
||||
- [Terraform Variables](#terraform-variables)
|
||||
- [Naming Locals](#naming-locals)
|
||||
- [Validation](#validation)
|
||||
- [Naming Validation Script](#naming-validation-script)
|
||||
- [Manual Validation Checklist](#manual-validation-checklist)
|
||||
- [Migration Guide](#migration-guide)
|
||||
- [Updating Existing Resources](#updating-existing-resources)
|
||||
- [Example Migration](#example-migration)
|
||||
- [Best Practices](#best-practices)
|
||||
- [References](#references)
|
||||
|
||||
## Naming Pattern
|
||||
|
||||
```
|
||||
{cloud}-{env}-{region}-{resource}-{instance}
|
||||
```
|
||||
|
||||
### Pattern Components
|
||||
|
||||
| Component | Abbreviation | Length | Description | Examples |
|
||||
|-----------|-------------|--------|-------------|----------|
|
||||
| **Cloud** | `az` | 2 | Cloud provider identifier | `az` (Azure) |
|
||||
| **Environment** | `p`, `d`, `t`, `s` | 1 | Environment code | `p` (prod), `d` (dev), `t` (test), `s` (staging) |
|
||||
| **Region** | `we`, `ne`, `fc` | 2-3 | Azure region code | `we` (westeurope), `ne` (northeurope), `fc` (francecentral) |
|
||||
| **Resource** | `rg`, `aks`, `kv` | 2-6 | Resource type abbreviation | See resource abbreviations below |
|
||||
| **Instance** | `001`, `main`, `sys` | 2-4 | Instance identifier | Sequential number or descriptive name |
|
||||
|
||||
## Region Codes
|
||||
|
||||
| Full Name | Code | Full Name | Code |
|
||||
|-----------|------|-----------|------|
|
||||
| westeurope | `we` | northeurope | `ne` |
|
||||
| francecentral | `fc` | francesouth | `fs` |
|
||||
| germanywestcentral | `gw` | germanynorth | `gn` |
|
||||
| uksouth | `uk` | ukwest | `uw` |
|
||||
| swedencentral | `sc` | norwayeast | `ne` |
|
||||
| switzerlandnorth | `sn` | polandcentral | `pc` |
|
||||
| italynorth | `in` | spaincentral | `sp` |
|
||||
|
||||
## Resource Type Abbreviations
|
||||
|
||||
### Core Infrastructure
|
||||
|
||||
| Resource Type | Abbreviation | Example |
|
||||
|--------------|--------------|---------|
|
||||
| Resource Group | `rg` | `az-p-we-rg-net-001` |
|
||||
| Virtual Network | `vnet` | `az-p-we-vnet-main` |
|
||||
| Subnet | `snet` | `az-p-we-snet-aks` |
|
||||
| Network Security Group | `nsg` | `az-p-we-nsg-valid` |
|
||||
| Public IP | `pip` | `az-p-we-pip-gw` |
|
||||
| Load Balancer | `lb` | `az-p-we-lb-main` |
|
||||
| Application Gateway | `agw` | `az-p-we-agw-main` |
|
||||
|
||||
### Compute
|
||||
|
||||
| Resource Type | Abbreviation | Example |
|
||||
|--------------|--------------|---------|
|
||||
| AKS Cluster | `aks` | `az-p-we-aks-main` |
|
||||
| VM | `vm` | `az-p-we-vm-valid-001` |
|
||||
| VM Scale Set | `vmss` | `az-p-we-vmss-rpc` |
|
||||
| Container Instance | `aci` | `az-p-we-aci-oracle` |
|
||||
|
||||
### Storage
|
||||
|
||||
| Resource Type | Abbreviation | Example |
|
||||
|--------------|--------------|---------|
|
||||
| Storage Account | `st` | `az-p-we-st-backup-001` |
|
||||
| Storage Container | `cnt` | `az-p-we-cnt-chaindata` |
|
||||
| Disk | `disk` | `az-p-we-disk-valid-001` |
|
||||
| File Share | `share` | `az-p-we-share-config` |
|
||||
|
||||
### Security & Identity
|
||||
|
||||
| Resource Type | Abbreviation | Example |
|
||||
|--------------|--------------|---------|
|
||||
| Key Vault | `kv` | `az-p-we-kv-secrets-001` |
|
||||
| Managed Identity | `id` | `az-p-we-id-aks` |
|
||||
| Service Principal | `sp` | `az-p-we-sp-deploy` |
|
||||
|
||||
### Monitoring & Logging
|
||||
|
||||
| Resource Type | Abbreviation | Example |
|
||||
|--------------|--------------|---------|
|
||||
| Log Analytics Workspace | `law` | `az-p-we-law-main` |
|
||||
| Application Insights | `appi` | `az-p-we-appi-main` |
|
||||
| Action Group | `ag` | `az-p-we-ag-alerts` |
|
||||
| Alert Rule | `alert` | `az-p-we-alert-cpu` |
|
||||
|
||||
### Database
|
||||
|
||||
| Resource Type | Abbreviation | Example |
|
||||
|--------------|--------------|---------|
|
||||
| SQL Database | `sql` | `az-p-we-sql-blockscout` |
|
||||
| PostgreSQL | `psql` | `az-p-we-psql-main` |
|
||||
| Cosmos DB | `cosmos` | `az-p-we-cosmos-main` |
|
||||
|
||||
### Networking
|
||||
|
||||
| Resource Type | Abbreviation | Example |
|
||||
|--------------|--------------|---------|
|
||||
| Route Table | `rt` | `az-p-we-rt-main` |
|
||||
| NAT Gateway | `nat` | `az-p-we-nat-main` |
|
||||
| VPN Gateway | `vpngw` | `az-p-we-vpngw-main` |
|
||||
| ExpressRoute | `er` | `az-p-we-er-main` |
|
||||
|
||||
## Naming Examples
|
||||
|
||||
### Resource Groups
|
||||
|
||||
```
|
||||
az-p-we-rg-net-001 # Network resource group (prod, westeurope)
|
||||
az-p-we-rg-comp-001 # Compute resource group
|
||||
az-p-we-rg-stor-001 # Storage resource group
|
||||
az-p-we-rg-sec-001 # Security resource group
|
||||
az-d-we-rg-net-001 # Development environment
|
||||
```
|
||||
|
||||
### AKS Resources
|
||||
|
||||
```
|
||||
az-p-we-aks-main # Main AKS cluster
|
||||
az-p-we-aks-node-sys # System node pool
|
||||
az-p-we-aks-node-valid # Validator node pool
|
||||
az-p-we-aks-node-sent # Sentry node pool
|
||||
az-p-we-aks-node-rpc # RPC node pool
|
||||
```
|
||||
|
||||
### Networking
|
||||
|
||||
```
|
||||
az-p-we-vnet-main # Main virtual network
|
||||
az-p-we-snet-aks # AKS subnet
|
||||
az-p-we-snet-valid # Validator subnet
|
||||
az-p-we-snet-sent # Sentry subnet
|
||||
az-p-we-snet-rpc # RPC subnet
|
||||
az-p-we-snet-agw # Application Gateway subnet
|
||||
az-p-we-nsg-valid # Validator NSG
|
||||
az-p-we-pip-agw # Application Gateway public IP
|
||||
az-p-we-agw-main # Application Gateway
|
||||
```
|
||||
|
||||
### Storage
|
||||
|
||||
```
|
||||
az-p-we-st-backup-001 # Backup storage account
|
||||
az-p-we-st-shared-001 # Shared storage account
|
||||
az-p-we-st-tfstate-001 # Terraform state storage
|
||||
az-p-we-cnt-chaindata # Chaindata container
|
||||
az-p-we-cnt-config # Configuration container
|
||||
```
|
||||
|
||||
### Security
|
||||
|
||||
```
|
||||
az-p-we-kv-secrets-001 # Key Vault for secrets
|
||||
az-p-we-id-aks # AKS managed identity
|
||||
az-p-we-id-oracle # Oracle service identity
|
||||
```
|
||||
|
||||
### Monitoring
|
||||
|
||||
```
|
||||
az-p-we-law-main # Log Analytics workspace
|
||||
az-p-we-appi-main # Application Insights
|
||||
az-p-we-ag-alerts # Alert action group
|
||||
```
|
||||
|
||||
## Special Cases
|
||||
|
||||
### Terraform Backend
|
||||
|
||||
```
|
||||
az-p-we-st-tfstate-001 # Terraform state storage account
|
||||
az-p-we-rg-tfstate-001 # Terraform state resource group
|
||||
```
|
||||
|
||||
### Multi-Region
|
||||
|
||||
```
|
||||
az-p-we-aks-main # Primary region (westeurope)
|
||||
az-p-ne-aks-main # Secondary region (northeurope)
|
||||
az-p-fc-aks-main # Tertiary region (francecentral)
|
||||
```
|
||||
|
||||
### Node Pools
|
||||
|
||||
```
|
||||
az-p-we-aks-node-sys # System node pool
|
||||
az-p-we-aks-node-valid # Validator node pool (4 nodes)
|
||||
az-p-we-aks-node-sent # Sentry node pool (3 nodes)
|
||||
az-p-we-aks-node-rpc # RPC node pool (3 nodes)
|
||||
```
|
||||
|
||||
## Azure Naming Restrictions
|
||||
|
||||
### General Rules
|
||||
|
||||
- **Length**: Varies by resource (see table below)
|
||||
- **Characters**: Alphanumeric, hyphens, underscores (varies by resource)
|
||||
- **Case**: Lowercase only
|
||||
- **Uniqueness**: Must be globally unique for some resources (storage accounts, Key Vaults)
|
||||
|
||||
### Resource-Specific Limits
|
||||
|
||||
| Resource | Min | Max | Allowed Characters | Notes |
|
||||
|----------|-----|-----|-------------------|-------|
|
||||
| Resource Group | 1 | 90 | Alphanumeric, hyphens, underscores, periods, parentheses | |
|
||||
| Storage Account | 3 | 24 | Lowercase letters and numbers | Globally unique |
|
||||
| Key Vault | 3 | 24 | Alphanumeric and hyphens | Globally unique |
|
||||
| AKS Cluster | 1 | 63 | Alphanumeric, hyphens, underscores | |
|
||||
| Virtual Network | 2 | 64 | Alphanumeric, hyphens, underscores, periods | |
|
||||
| Subnet | 1 | 80 | Alphanumeric, hyphens, underscores, periods | |
|
||||
| NSG | 1 | 80 | Alphanumeric, hyphens, underscores, periods | |
|
||||
| Public IP | 1 | 80 | Alphanumeric, hyphens, underscores, periods | |
|
||||
|
||||
## Implementation
|
||||
|
||||
### Terraform Variables
|
||||
|
||||
```hcl
|
||||
# Naming convention variables
|
||||
variable "cloud_provider" {
|
||||
description = "Cloud provider code"
|
||||
type = string
|
||||
default = "az"
|
||||
}
|
||||
|
||||
variable "environment" {
|
||||
description = "Environment code (p=prod, d=dev, t=test, s=staging)"
|
||||
type = string
|
||||
default = "p"
|
||||
}
|
||||
|
||||
variable "region_code" {
|
||||
description = "Azure region code (we=westeurope, ne=northeurope)"
|
||||
type = string
|
||||
default = "we"
|
||||
}
|
||||
|
||||
variable "project_code" {
|
||||
description = "Project code"
|
||||
type = string
|
||||
default = "dbis" # DeFi Oracle Meta Mainnet
|
||||
}
|
||||
```
|
||||
|
||||
### Naming Locals
|
||||
|
||||
```hcl
|
||||
locals {
|
||||
# Region mapping
|
||||
region_codes = {
|
||||
westeurope = "we"
|
||||
northeurope = "ne"
|
||||
francecentral = "fc"
|
||||
uksouth = "uk"
|
||||
}
|
||||
|
||||
# Current region code
|
||||
region_code = local.region_codes[var.location]
|
||||
|
||||
# Naming prefix
|
||||
name_prefix = "${var.cloud_provider}-${var.environment}-${local.region_code}"
|
||||
|
||||
# Resource names
|
||||
rg_network = "${local.name_prefix}-rg-net-001"
|
||||
rg_compute = "${local.name_prefix}-rg-comp-001"
|
||||
rg_storage = "${local.name_prefix}-rg-stor-001"
|
||||
rg_security = "${local.name_prefix}-rg-sec-001"
|
||||
|
||||
aks_cluster = "${local.name_prefix}-aks-main"
|
||||
key_vault = "${local.name_prefix}-kv-secrets-001"
|
||||
vnet_main = "${local.name_prefix}-vnet-main"
|
||||
}
|
||||
```
|
||||
|
||||
## Validation
|
||||
|
||||
### Naming Validation Script
|
||||
|
||||
Use the provided script to validate naming:
|
||||
|
||||
```bash
|
||||
./scripts/azure/validate-naming.sh <resource-name>
|
||||
```
|
||||
|
||||
### Manual Validation Checklist
|
||||
|
||||
- [ ] Follows pattern: `{cloud}-{env}-{region}-{resource}-{instance}`
|
||||
- [ ] All segments are lowercase
|
||||
- [ ] Uses hyphens as separators
|
||||
- [ ] Within Azure length limits
|
||||
- [ ] Globally unique if required (storage, Key Vault)
|
||||
- [ ] No special characters except hyphens
|
||||
- [ ] Region code matches actual region
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### Updating Existing Resources
|
||||
|
||||
1. **Review current names**: Identify resources using old naming
|
||||
2. **Plan migration**: Create new names following convention
|
||||
3. **Update Terraform**: Modify resource names in code
|
||||
4. **Apply changes**: Use Terraform to rename (may require destroy/create)
|
||||
5. **Update references**: Update all scripts and documentation
|
||||
|
||||
### Example Migration
|
||||
|
||||
**Old Name**: `defi-oracle-mainnet-rg`
|
||||
**New Name**: `az-p-we-rg-comp-001`
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Consistency**: Always use the same pattern across all resources
|
||||
2. **Short Segments**: Keep abbreviations concise but clear
|
||||
3. **Environment Prefix**: Always include environment code
|
||||
4. **Region Code**: Include region for multi-region deployments
|
||||
5. **Sequential Numbers**: Use `001`, `002` for multiple instances
|
||||
6. **Documentation**: Document any custom abbreviations
|
||||
7. **Validation**: Validate names before deployment
|
||||
8. **Tags**: Use tags for additional metadata (don't rely on names alone)
|
||||
|
||||
## References
|
||||
|
||||
- [Azure Naming Conventions](https://docs.microsoft.com/azure/cloud-adoption-framework/ready/azure-best-practices/naming-and-tagging)
|
||||
- [Azure Resource Naming Rules](https://docs.microsoft.com/azure/azure-resource-manager/management/resource-name-rules)
|
||||
|
||||
230
docs/configuration/AZURE_NAMING_CONVENTION_3CHAR.md
Normal file
230
docs/configuration/AZURE_NAMING_CONVENTION_3CHAR.md
Normal file
@@ -0,0 +1,230 @@
|
||||
# Azure Resource Naming Conventions (3 Character Region Codes)
|
||||
|
||||
**Last Updated**: 2025-01-27
|
||||
**Status**: Active - Standard Convention
|
||||
**Purpose**: Azure resource naming with 3-character region codes (standard)
|
||||
|
||||
## Standard Naming Pattern
|
||||
|
||||
All Azure resources follow this standard naming convention using **exactly 3-character region codes**:
|
||||
|
||||
```
|
||||
{cloud_provider}-{env_code}-{region_code}-{resource_type}-{purpose}-{instance}
|
||||
```
|
||||
|
||||
> **Note**: This is the **standard convention** using exactly 3-character region codes (e.g., `wst`, `nor`, `frc`).
|
||||
> **Alternative**: See [Azure Naming Convention (2-3 Character)](AZURE_NAMING_CONVENTION_2CHAR.md) for 2-3 character codes.
|
||||
|
||||
> **Related Documentation**:
|
||||
> - [Azure Naming Convention (2-3 Character)](AZURE_NAMING_CONVENTION_2CHAR.md) - Alternative 2-3 character region code convention
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Standard Naming Pattern](#standard-naming-pattern)
|
||||
- [Components](#components)
|
||||
- [Examples](#examples)
|
||||
- [Region Codes (3-Character Standard)](#region-codes-3-character-standard)
|
||||
- [Resource Type Abbreviations](#resource-type-abbreviations)
|
||||
- [Purpose Abbreviations](#purpose-abbreviations)
|
||||
- [Naming Rules](#naming-rules)
|
||||
- [Legacy Naming (Deprecated)](#legacy-naming-deprecated)
|
||||
- [Migration Strategy](#migration-strategy)
|
||||
- [Scripts and Tools](#scripts-and-tools)
|
||||
- [Validation](#validation)
|
||||
- [Examples](#examples-1)
|
||||
- [Production Resources (West Europe)](#production-resources-west-europe)
|
||||
- [Dev Resources (West Europe)](#dev-resources-west-europe)
|
||||
- [Legacy Resources (Deprecated)](#legacy-resources-deprecated)
|
||||
|
||||
### Components
|
||||
|
||||
- **cloud_provider**: `az` (Azure)
|
||||
- **env_code**: `p` (production), `d` (dev), `t` (test), `s` (staging)
|
||||
- **region_code**: **3-letter region code** (e.g., `wst` for westeurope, `nor` for northeurope)
|
||||
- **resource_type**: Resource type abbreviation (e.g., `rg` for resource group, `kv` for key vault)
|
||||
- **purpose**: Purpose abbreviation (e.g., `sec` for security, `comp` for compute)
|
||||
- **instance**: Sequential number (e.g., `001`, `002`)
|
||||
|
||||
### Examples
|
||||
|
||||
#### Resource Groups
|
||||
- Compute: `az-p-{code}-rg-comp-001`
|
||||
- Network: `az-p-{code}-rg-net-001`
|
||||
- Storage: `az-p-{code}-rg-stor-001`
|
||||
- Security: `az-p-{code}-rg-sec-001`
|
||||
- Monitoring: `az-p-{code}-rg-mon-001`
|
||||
- Identity: `az-p-{code}-rg-id-001`
|
||||
|
||||
#### Key Vaults
|
||||
- Standard: `az-p-{code}-kv-secrets-001`
|
||||
- Legacy (deprecated): `azp{code}kvsecrets001` ❌
|
||||
|
||||
#### AKS Clusters
|
||||
- Standard: `az-p-{code}-aks-main`
|
||||
- Dev: `az-d-{code}-aks-dev-main`
|
||||
|
||||
#### Virtual Networks
|
||||
- Standard: `az-p-{code}-vnet-main`
|
||||
|
||||
#### Subnets
|
||||
- AKS: `az-p-{code}-snet-aks`
|
||||
- Validators: `az-p-{code}-snet-validators`
|
||||
- Sentries: `az-p-{code}-snet-sentries`
|
||||
- RPC: `az-p-{code}-snet-rpc`
|
||||
- Application Gateway: `az-p-{code}-snet-appgateway`
|
||||
|
||||
#### Network Security Groups
|
||||
- Validators: `az-p-{code}-nsg-validators`
|
||||
- Sentries: `az-p-{code}-nsg-sentries`
|
||||
- RPC: `az-p-{code}-nsg-rpc`
|
||||
|
||||
#### Application Gateways
|
||||
- Standard: `az-p-{code}-appgateway-main`
|
||||
|
||||
#### Public IPs
|
||||
- Application Gateway: `az-p-{code}-appgateway-ip`
|
||||
|
||||
#### Log Analytics Workspaces
|
||||
- Standard: `az-p-{code}-law-main`
|
||||
|
||||
## Region Codes (3-Character Standard)
|
||||
|
||||
**All region codes are now standardized to exactly 3 characters.**
|
||||
|
||||
| Region | Code | Full Name | Old Code |
|
||||
|--------|------|-----------|----------|
|
||||
| `northeurope` | `nor` | North Europe | `ne` |
|
||||
| `uksouth` | `uks` | UK South | `uks` |
|
||||
| `ukwest` | `ukw` | UK West | `ukw` |
|
||||
| `westeurope` | `wst` | West Europe | `we` |
|
||||
| `francecentral` | `frc` | France Central | `fc` |
|
||||
| `germanywestcentral` | `gwc` | Germany West Central | `gwc` |
|
||||
| `switzerlandnorth` | `swn` | Switzerland North | `sn` |
|
||||
| `switzerlandwest` | `swt` | Switzerland West | `sw` |
|
||||
| `swedencentral` | `swc` | Sweden Central | `swc` |
|
||||
| `norwayeast` | `noe` | Norway East | `noe` |
|
||||
| `polandcentral` | `pol` | Poland Central | `pc` |
|
||||
| `spaincentral` | `spa` | Spain Central | `sc` |
|
||||
| `italynorth` | `ita` | Italy North | `in` |
|
||||
| `southindia` | `sin` | South India | `si` |
|
||||
| `centralindia` | `cin` | Central India | `ci` |
|
||||
| `westindia` | `win` | West India | `wi` |
|
||||
| `belgiumcentral` | `bel` | Belgium Central | `bc` |
|
||||
| `eastasia` | `eas` | East Asia | `ea` |
|
||||
| `southeastasia` | `sea` | Southeast Asia | `sea` |
|
||||
| `japaneast` | `jpe` | Japan East | `je` |
|
||||
| `japanwest` | `jpw` | Japan West | `jw` |
|
||||
| `koreacentral` | `kor` | Korea Central | `kc` |
|
||||
| `koreasouth` | `kos` | Korea South | `ks` |
|
||||
| `australiaeast` | `aus` | Australia East | `ae` |
|
||||
| `australiasoutheast` | `ase` | Australia Southeast | `ase` |
|
||||
| `newzealandnorth` | `nzl` | New Zealand North | `nzn` |
|
||||
| `indonesiacentral` | `idn` | Indonesia Central | `ic` |
|
||||
| `malaysiawest` | `mys` | Malaysia West | `mw` |
|
||||
| `uaenorth` | `uae` | UAE North | `uan` |
|
||||
| `qatarcentral` | `qat` | Qatar Central | `qc` |
|
||||
| `israelcentral` | `ilc` | Israel Central | `ilc` |
|
||||
| `canadacentral` | `can` | Canada Central | `cc` |
|
||||
| `canadaeast` | `cae` | Canada East | `ce` |
|
||||
| `brazilsouth` | `bra` | Brazil South | `bs` |
|
||||
| `chilecentral` | `chl` | Chile Central | `chc` |
|
||||
| `mexicocentral` | `mex` | Mexico Central | `mc` |
|
||||
| `southafricanorth` | `zaf` | South Africa North | `san` |
|
||||
|
||||
## Resource Type Abbreviations
|
||||
|
||||
| Resource Type | Abbreviation | Example |
|
||||
|--------------|--------------|---------|
|
||||
| Resource Group | `rg` | `az-p-wst-rg-comp-001` |
|
||||
| Key Vault | `kv` | `az-p-wst-kv-secrets-001` |
|
||||
| AKS Cluster | `aks` | `az-p-wst-aks-main` |
|
||||
| Virtual Network | `vnet` | `az-p-wst-vnet-main` |
|
||||
| Subnet | `snet` | `az-p-wst-snet-aks` |
|
||||
| Network Security Group | `nsg` | `az-p-wst-nsg-validators` |
|
||||
| Application Gateway | `appgateway` | `az-p-wst-appgateway-main` |
|
||||
| Public IP | `pip` | `az-p-wst-appgateway-ip` |
|
||||
| Log Analytics Workspace | `law` | `az-p-wst-law-main` |
|
||||
| Storage Account | `sa` | `az-p-wst-sa-backups-001` |
|
||||
| Load Balancer | `lb` | `az-p-wst-lb-main` |
|
||||
|
||||
## Purpose Abbreviations
|
||||
|
||||
| Purpose | Abbreviation | Example |
|
||||
|---------|--------------|---------|
|
||||
| Compute | `comp` | `az-p-wst-rg-comp-001` |
|
||||
| Network | `net` | `az-p-wst-rg-net-001` |
|
||||
| Storage | `stor` | `az-p-wst-rg-stor-001` |
|
||||
| Security | `sec` | `az-p-wst-rg-sec-001` |
|
||||
| Monitoring | `mon` | `az-p-wst-rg-mon-001` |
|
||||
| Identity | `id` | `az-p-wst-rg-id-001` |
|
||||
| Secrets | `secrets` | `az-p-wst-kv-secrets-001` |
|
||||
| Main | `main` | `az-p-wst-aks-main` |
|
||||
| Backups | `backups` | `az-p-wst-sa-backups-001` |
|
||||
|
||||
## Naming Rules
|
||||
|
||||
1. **Use lowercase**: All names must be lowercase
|
||||
2. **Use dashes**: Separate components with dashes (`-`)
|
||||
3. **No underscores**: Do not use underscores in resource names
|
||||
4. **No spaces**: Do not use spaces in resource names
|
||||
5. **Keep it short**: Azure resource names have length limits
|
||||
6. **Be consistent**: Use the same pattern across all resources
|
||||
7. **Include region**: Always include region code for multi-region resources
|
||||
8. **3-character codes**: All region codes must be exactly 3 characters
|
||||
|
||||
## Legacy Naming (Deprecated)
|
||||
|
||||
Some older resources may use legacy naming:
|
||||
- Legacy Key Vault: `azp{code}kvsecrets001` ❌ (no dashes, old codes)
|
||||
- Legacy Resource Group: `azp{code}rgsec001` ❌ (no dashes, old codes)
|
||||
|
||||
**Note**: Azure resources cannot be renamed. Legacy resources should be:
|
||||
1. Migrated to new resources with standard naming (where possible)
|
||||
2. Documented as legacy
|
||||
3. Scripts should handle both patterns for backward compatibility
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
For resources that cannot be renamed (e.g., Key Vaults):
|
||||
1. Create new resources with standard naming (3-char codes)
|
||||
2. Migrate data/secrets from legacy resources
|
||||
3. Update scripts to prefer standard naming
|
||||
4. Deprecate legacy resources (but keep them for reference)
|
||||
|
||||
## Scripts and Tools
|
||||
|
||||
- **Check naming**: `scripts/azure/check-naming-conventions.sh`
|
||||
- **Fix naming**: `scripts/deployment/fix-resource-groups-and-keyvaults.sh`
|
||||
- **List resources**: `scripts/azure/list-all-resources.sh`
|
||||
|
||||
## Validation
|
||||
|
||||
All new resources should:
|
||||
1. Follow the standard naming pattern
|
||||
2. Use **exactly 3-character** region codes
|
||||
3. Use correct resource type abbreviations
|
||||
4. Use correct purpose abbreviations
|
||||
|
||||
Use the validation script to check:
|
||||
```bash
|
||||
./scripts/azure/check-naming-conventions.sh
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Production Resources (West Europe)
|
||||
- Resource Group (Compute): `az-p-wst-rg-comp-001`
|
||||
- Resource Group (Security): `az-p-wst-rg-sec-001`
|
||||
- Key Vault: `az-p-wst-kv-secrets-001`
|
||||
- AKS Cluster: `az-p-wst-aks-main`
|
||||
- Virtual Network: `az-p-wst-vnet-main`
|
||||
- Subnet (AKS): `az-p-wst-snet-aks`
|
||||
|
||||
### Dev Resources (West Europe)
|
||||
- Resource Group: `az-d-wst-rg-comp-001`
|
||||
- AKS Cluster: `az-d-wst-aks-dev-main`
|
||||
- Key Vault: `az-d-wst-kv-secrets-001`
|
||||
|
||||
### Legacy Resources (Deprecated)
|
||||
- Old Key Vault: `azpwekvsecrets001` (no dashes, old 2-char code)
|
||||
- Should migrate to: `az-p-wst-kv-secrets-001`
|
||||
247
docs/configuration/CONFIGURATION_FIXES_APPLIED.md
Normal file
247
docs/configuration/CONFIGURATION_FIXES_APPLIED.md
Normal file
@@ -0,0 +1,247 @@
|
||||
# Configuration Fixes Applied
|
||||
|
||||
## Date: $(date)
|
||||
|
||||
This document summarizes all configuration fixes that were automatically applied to resolve deployment configuration issues.
|
||||
|
||||
---
|
||||
|
||||
## ✅ Fixes Applied
|
||||
|
||||
### 1. Terraform Node Counts ✅ FIXED
|
||||
|
||||
**Issue**: Sentries and RPC nodes were set to 0, preventing RPC endpoints from being available.
|
||||
|
||||
**Fix Applied**:
|
||||
- Updated `terraform/terraform.tfvars`:
|
||||
- `system = 3` (was 1)
|
||||
- `validators = 4` (was 1)
|
||||
- `sentries = 3` (was 0) ✅
|
||||
- `rpc = 3` (was 0) ✅
|
||||
|
||||
**File**: `terraform/terraform.tfvars`
|
||||
|
||||
**Impact**: RPC endpoints will now be deployed, making the network accessible externally.
|
||||
|
||||
---
|
||||
|
||||
### 2. Kubernetes Version ✅ FIXED
|
||||
|
||||
**Issue**: Kubernetes version was set to `1.33` which may not be stable.
|
||||
|
||||
**Fix Applied**:
|
||||
- Updated `terraform/terraform.tfvars`:
|
||||
- Changed `kubernetes_version = "1.33"` to `kubernetes_version = "1.28"`
|
||||
|
||||
**File**: `terraform/terraform.tfvars`
|
||||
|
||||
**Note**: Version 1.28 is a stable LTS version. Verify with:
|
||||
```bash
|
||||
az aks get-versions --location westeurope --output table
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. RPC Storage Size Inconsistency ✅ FIXED
|
||||
|
||||
**Issue**: Storage size mismatch between k8s and Helm configurations.
|
||||
|
||||
**Fix Applied**:
|
||||
- Updated `k8s/base/rpc/statefulset.yaml`:
|
||||
- Changed storage from `256Gi` to `500Gi` to match Helm values
|
||||
|
||||
**File**: `k8s/base/rpc/statefulset.yaml`
|
||||
|
||||
**Impact**: Storage sizes are now consistent across all configurations.
|
||||
|
||||
---
|
||||
|
||||
### 4. Terraform Backend Configuration ✅ CREATED
|
||||
|
||||
**Issue**: `backend.tf` was missing, using only commented configuration in `main.tf`.
|
||||
|
||||
**Fix Applied**:
|
||||
- Created `terraform/backend.tf` from `terraform/backend.tf.example`
|
||||
|
||||
**File**: `terraform/backend.tf`
|
||||
|
||||
**Next Step**: Configure backend storage account details:
|
||||
```bash
|
||||
# Edit terraform/backend.tf and set:
|
||||
# - resource_group_name
|
||||
# - storage_account_name
|
||||
# - container_name
|
||||
# - key
|
||||
```
|
||||
|
||||
Or use environment variables:
|
||||
```bash
|
||||
export ARM_STORAGE_ACCOUNT_NAME="your-storage-account"
|
||||
export ARM_CONTAINER_NAME="tfstate"
|
||||
export ARM_RESOURCE_GROUP_NAME="your-rg"
|
||||
export ARM_ACCESS_KEY="your-access-key"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. RPC Security Configuration ⚠️ ANNOTATED
|
||||
|
||||
**Issue**: RPC CORS and host allowlist set to wildcard (`["*"]`), which is a security risk.
|
||||
|
||||
**Fix Applied**:
|
||||
- Added TODO comments in:
|
||||
- `config/rpc/besu-config.toml`
|
||||
- `k8s/base/rpc/statefulset.yaml`
|
||||
- `helm/besu-network/values-rpc.yaml`
|
||||
|
||||
**Files Updated**:
|
||||
- `config/rpc/besu-config.toml`
|
||||
- `k8s/base/rpc/statefulset.yaml`
|
||||
- `helm/besu-network/values-rpc.yaml`
|
||||
|
||||
**Status**: ⚠️ **REQUIRES MANUAL UPDATE AFTER DNS DEPLOYMENT**
|
||||
|
||||
**Action Required**: After DNS is configured, update CORS and host allowlist to:
|
||||
```toml
|
||||
rpc-http-cors-origins=["https://rpc.d-bis.org", "https://explorer.d-bis.org"]
|
||||
rpc-http-host-allowlist=["rpc.d-bis.org", "rpc2.d-bis.org"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6. Genesis Validator Configuration ⚠️ PARTIALLY FIXED
|
||||
|
||||
**Issue**: Genesis file has `extraData: "0x"` (no validators configured).
|
||||
|
||||
**Fix Applied**:
|
||||
- Created script: `scripts/deployment/generate-genesis-with-validators.sh`
|
||||
- Script generates validator keys if they don't exist
|
||||
- Script attempts to use Besu to generate proper genesis with extraData
|
||||
|
||||
**File**: `scripts/deployment/generate-genesis-with-validators.sh`
|
||||
|
||||
**Status**: ⚠️ **REQUIRES BESU CLI FOR COMPLETE FIX**
|
||||
|
||||
**Action Required**:
|
||||
1. Install Besu CLI (if not installed):
|
||||
```bash
|
||||
# See: https://besu.hyperledger.org/en/stable/HowTo/Get-Started/Installation-Options/
|
||||
```
|
||||
|
||||
2. Generate proper genesis:
|
||||
```bash
|
||||
./scripts/deployment/generate-genesis-with-validators.sh
|
||||
```
|
||||
|
||||
3. If Besu is not available, manually generate extraData:
|
||||
```bash
|
||||
# Extract validator addresses from keys
|
||||
# Use Besu's operator generate-blockchain-config
|
||||
besu operator generate-blockchain-config \
|
||||
--config-file=config/genesis-template.json \
|
||||
--to=keys/validators \
|
||||
--private-key-file-name=key.priv
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Summary of Changes
|
||||
|
||||
| Issue | Status | File(s) Modified |
|
||||
|-------|--------|-----------------|
|
||||
| Terraform node counts | ✅ Fixed | `terraform/terraform.tfvars` |
|
||||
| Kubernetes version | ✅ Fixed | `terraform/terraform.tfvars` |
|
||||
| RPC storage size | ✅ Fixed | `k8s/base/rpc/statefulset.yaml` |
|
||||
| Terraform backend | ✅ Created | `terraform/backend.tf` |
|
||||
| RPC security (CORS/host) | ⚠️ Annotated | `config/rpc/besu-config.toml`, `k8s/base/rpc/statefulset.yaml`, `helm/besu-network/values-rpc.yaml` |
|
||||
| Genesis validators | ⚠️ Script created | `scripts/deployment/generate-genesis-with-validators.sh` |
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Manual Actions Required
|
||||
|
||||
### 1. Configure Terraform Backend
|
||||
|
||||
Edit `terraform/backend.tf` and configure:
|
||||
- Storage account name
|
||||
- Container name
|
||||
- Resource group
|
||||
- Access key (or use Managed Identity)
|
||||
|
||||
### 2. Generate Genesis with Validators
|
||||
|
||||
Run the genesis generation script:
|
||||
```bash
|
||||
./scripts/deployment/generate-genesis-with-validators.sh
|
||||
```
|
||||
|
||||
If Besu is not installed, install it first:
|
||||
```bash
|
||||
# Ubuntu/Debian
|
||||
wget https://hyperledger.jfrog.io/hyperledger/besu-binaries/besu/23.10.0/besu-23.10.0.tar.gz
|
||||
tar -xzf besu-23.10.0.tar.gz
|
||||
export PATH=$PATH:$(pwd)/besu-23.10.0/bin
|
||||
```
|
||||
|
||||
### 3. Update RPC Security After DNS Deployment
|
||||
|
||||
After DNS records are configured, update:
|
||||
- `config/rpc/besu-config.toml`
|
||||
- `k8s/base/rpc/statefulset.yaml`
|
||||
- `helm/besu-network/values-rpc.yaml`
|
||||
|
||||
Replace wildcard CORS/host allowlist with specific domains.
|
||||
|
||||
### 4. Update All ConfigMaps with New Genesis
|
||||
|
||||
After generating proper genesis, update:
|
||||
- `k8s/base/validators/statefulset.yaml` (ConfigMap)
|
||||
- `k8s/base/sentries/statefulset.yaml` (ConfigMap)
|
||||
- `k8s/base/rpc/statefulset.yaml` (ConfigMap)
|
||||
|
||||
Or regenerate Helm ConfigMaps if using Helm deployment.
|
||||
|
||||
---
|
||||
|
||||
## ✅ Verification
|
||||
|
||||
Run validation script to verify all fixes:
|
||||
```bash
|
||||
./scripts/deployment/validate-deployment-config.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Notes
|
||||
|
||||
1. **Quota Constraints**: If Azure quota is limited, consider staged deployment:
|
||||
- Phase 1: Deploy validators only (4 nodes)
|
||||
- Phase 2: Deploy sentries (3 nodes)
|
||||
- Phase 3: Deploy RPC nodes (3 nodes)
|
||||
|
||||
2. **Genesis Generation**: Proper IBFT 2.0 extraData encoding requires Besu CLI. The script will attempt automatic generation, but manual steps may be needed.
|
||||
|
||||
3. **Security**: RPC security settings are currently permissive for initial deployment. **MUST** be restricted before production use.
|
||||
|
||||
4. **Backend Configuration**: Terraform backend is created but needs configuration. Use environment variables or edit `backend.tf` directly.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Next Steps
|
||||
|
||||
1. ✅ Review all changes
|
||||
2. ⚠️ Configure Terraform backend
|
||||
3. ⚠️ Generate genesis with validators (requires Besu)
|
||||
4. ⚠️ Update ConfigMaps with new genesis
|
||||
5. ⚠️ Deploy infrastructure
|
||||
6. ⚠️ Update RPC security settings after DNS deployment
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
- Review: `docs/DEPLOYMENT_CONFIGURATION_AUDIT.md`
|
||||
- Run: `./scripts/deployment/validate-deployment-config.sh`
|
||||
- Check: `docs/DEPLOYMENT_COMPLETE_GUIDE.md`
|
||||
|
||||
53
docs/configuration/CONFIGURATION_INDEX.md
Normal file
53
docs/configuration/CONFIGURATION_INDEX.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# Configuration Documentation Index
|
||||
|
||||
**Last Updated**: 2025-01-27
|
||||
**Status**: Active
|
||||
|
||||
This index helps you find the right configuration guide for your needs.
|
||||
|
||||
## Which Guide Should I Use?
|
||||
|
||||
### Network Configuration
|
||||
**Use**: [Network Configuration Guide](NETWORK_CONFIGURATION_GUIDE.md)
|
||||
**When**: You need to configure the Besu network, genesis file, or network topology
|
||||
**Covers**: Network configuration tool, genesis block, network settings
|
||||
|
||||
### Azure and Cloudflare Setup
|
||||
**Use**: [Azure/Cloudflare Environment Setup](AZURE_CLOUDFLARE_ENV_SETUP.md)
|
||||
**When**: You need to set up environment variables for Azure and Cloudflare integration
|
||||
**Covers**: Azure subscription, Cloudflare zone ID, API tokens, Terraform backend
|
||||
|
||||
### Contract Deployment Setup
|
||||
**Use**: [Contract Deployment Environment Setup](CONTRACT_DEPLOYMENT_ENV_SETUP.md)
|
||||
**When**: You need to configure environment variables for deploying contracts
|
||||
**Covers**: Private keys, CCIP configuration, WETH deployment, Oracle setup
|
||||
|
||||
### Azure Naming Conventions
|
||||
**Use**: [Azure Naming Convention (3 Character)](AZURE_NAMING_CONVENTION_3CHAR.md) - **Recommended**
|
||||
**When**: You need to name Azure resources using the standard 3-character region codes
|
||||
**Covers**: Standard naming pattern with 3-character region codes (e.g., `wst`, `nor`)
|
||||
|
||||
**Alternative**: [Azure Naming Convention (2-3 Character)](AZURE_NAMING_CONVENTION_2CHAR.md)
|
||||
**When**: You need to use 2-3 character region codes (legacy/alternative)
|
||||
**Covers**: Alternative naming pattern with 2-3 character region codes (e.g., `we`, `ne`)
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Guide | Purpose | When to Use |
|
||||
|-------|---------|-------------|
|
||||
| [Network Configuration Guide](NETWORK_CONFIGURATION_GUIDE.md) | Besu network setup | Configuring blockchain network |
|
||||
| [Azure/Cloudflare Environment Setup](AZURE_CLOUDFLARE_ENV_SETUP.md) | Azure/Cloudflare env vars | Setting up infrastructure |
|
||||
| [Contract Deployment Environment Setup](CONTRACT_DEPLOYMENT_ENV_SETUP.md) | Contract deployment env vars | Deploying smart contracts |
|
||||
| [Azure Naming (3 Char)](AZURE_NAMING_CONVENTION_3CHAR.md) | Azure naming (standard) | Naming Azure resources (standard) |
|
||||
| [Azure Naming (2-3 Char)](AZURE_NAMING_CONVENTION_2CHAR.md) | Azure naming (alternative) | Naming Azure resources (alternative) |
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Master Documentation Index](../MASTER_DOCUMENTATION_INDEX.md)
|
||||
- [Deployment Guide](../deployment/DEPLOYMENT.md)
|
||||
- [Architecture Documentation](../architecture/ARCHITECTURE.md)
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-01-27
|
||||
|
||||
242
docs/configuration/CONTRACT_DEPLOYMENT_ENV_SETUP.md
Normal file
242
docs/configuration/CONTRACT_DEPLOYMENT_ENV_SETUP.md
Normal file
@@ -0,0 +1,242 @@
|
||||
# Contract Deployment Environment Setup Guide
|
||||
|
||||
**Last Updated**: 2025-01-27
|
||||
**Status**: Active
|
||||
**Purpose**: Contract deployment and testing environment configuration
|
||||
|
||||
## Overview
|
||||
|
||||
This guide covers the environment variables and configuration required for deploying and testing the contracts.
|
||||
|
||||
> **Related Documentation**:
|
||||
> - [Network Configuration Guide](NETWORK_CONFIGURATION_GUIDE.md) - For Besu network configuration
|
||||
> - [Azure/Cloudflare Environment Setup](AZURE_CLOUDFLARE_ENV_SETUP.md) - For Azure and Cloudflare environment variables
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### Required Variables
|
||||
|
||||
#### Deployer Configuration
|
||||
- `PRIVATE_KEY` - Private key of the deployer account (without 0x prefix)
|
||||
|
||||
#### CCIP Configuration
|
||||
- `CCIP_ROUTER` - CCIP Router address on your chain
|
||||
- `CCIP_FEE_TOKEN` - LINK token address for paying CCIP fees
|
||||
|
||||
#### WETH Configuration (Optional)
|
||||
- `WETH9_ADDRESS` - WETH9 contract address (if not deploying new one)
|
||||
- `WETH10_ADDRESS` - WETH10 contract address (if not deploying new one)
|
||||
|
||||
#### Deployment Flags
|
||||
- `DEPLOY_WETH9` - Set to `true` to deploy WETH9
|
||||
- `DEPLOY_WETH10` - Set to `true` to deploy WETH10
|
||||
- `DEPLOY_BRIDGES` - Set to `true` to deploy CCIP bridges
|
||||
|
||||
#### Oracle Configuration (Optional)
|
||||
- `ORACLE_DESCRIPTION` - Oracle description (e.g., "ETH/USD Price Feed")
|
||||
- `ORACLE_HEARTBEAT` - Oracle heartbeat in seconds (default: 60)
|
||||
- `ORACLE_DEVIATION_THRESHOLD` - Oracle deviation threshold in basis points (default: 50)
|
||||
|
||||
#### MultiSig Configuration (Optional)
|
||||
- `MULTISIG_OWNER_1` - MultiSig owner address 1
|
||||
- `MULTISIG_OWNER_2` - MultiSig owner address 2
|
||||
- `MULTISIG_OWNER_3` - MultiSig owner address 3
|
||||
- `MULTISIG_REQUIRED` - Number of required signatures (must be <= number of owners)
|
||||
|
||||
### Optional Variables
|
||||
|
||||
#### RPC Configuration
|
||||
- `RPC_URL` - RPC URL for deployment (default: http://localhost:8545)
|
||||
- `CHAIN_ID` - Chain ID (default: 138)
|
||||
|
||||
#### Verification Configuration
|
||||
- `ETHERSCAN_API_KEY` - Etherscan API key for contract verification
|
||||
- `BLOCKSCOUT_API_KEY` - Blockscout API key for contract verification
|
||||
|
||||
## Examples
|
||||
|
||||
### Example: Complete .env File for Contract Deployment
|
||||
|
||||
```bash
|
||||
# Deployer Configuration
|
||||
PRIVATE_KEY=0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
|
||||
|
||||
# CCIP Configuration
|
||||
CCIP_ROUTER=0x1234567890123456789012345678901234567890
|
||||
CCIP_FEE_TOKEN=0x0987654321098765432109876543210987654321
|
||||
|
||||
# WETH Configuration (Optional - if not deploying new)
|
||||
WETH9_ADDRESS=0xabcdefabcdefabcdefabcdefabcdefabcdefabcd
|
||||
WETH10_ADDRESS=0xfedcbafedcbafedcbafedcbafedcbafedcba
|
||||
|
||||
# Deployment Flags
|
||||
DEPLOY_WETH9=true
|
||||
DEPLOY_WETH10=true
|
||||
DEPLOY_BRIDGES=true
|
||||
|
||||
# Oracle Configuration
|
||||
ORACLE_DESCRIPTION="ETH/USD Price Feed"
|
||||
ORACLE_HEARTBEAT=60
|
||||
ORACLE_DEVIATION_THRESHOLD=50
|
||||
|
||||
# MultiSig Configuration
|
||||
MULTISIG_OWNER_1=0x1111111111111111111111111111111111111111
|
||||
MULTISIG_OWNER_2=0x2222222222222222222222222222222222222222
|
||||
MULTISIG_OWNER_3=0x3333333333333333333333333333333333333333
|
||||
MULTISIG_REQUIRED=2
|
||||
|
||||
# RPC Configuration
|
||||
RPC_URL=http://localhost:8545
|
||||
CHAIN_ID=138
|
||||
|
||||
# Verification
|
||||
ETHERSCAN_API_KEY=your-etherscan-api-key
|
||||
BLOCKSCOUT_API_KEY=your-blockscout-api-key
|
||||
```
|
||||
|
||||
### Example: Minimal .env File
|
||||
|
||||
```bash
|
||||
# Minimum required for basic deployment
|
||||
PRIVATE_KEY=0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
|
||||
RPC_URL=http://localhost:8545
|
||||
CHAIN_ID=138
|
||||
```
|
||||
|
||||
## Setup Instructions
|
||||
|
||||
### 1. Create .env File
|
||||
|
||||
```bash
|
||||
# Copy example file
|
||||
cp .env.example .env
|
||||
|
||||
# Edit .env file with your values
|
||||
nano .env
|
||||
```
|
||||
|
||||
### 2. Configure Variables
|
||||
|
||||
Fill in the required variables in `.env`:
|
||||
|
||||
```bash
|
||||
# Deployer private key (required)
|
||||
PRIVATE_KEY=your_private_key_here
|
||||
|
||||
# CCIP Router address (required)
|
||||
CCIP_ROUTER=0x...
|
||||
|
||||
# LINK token address (required)
|
||||
CCIP_FEE_TOKEN=0x...
|
||||
|
||||
# Deployment flags (optional)
|
||||
DEPLOY_WETH9=true
|
||||
DEPLOY_WETH10=true
|
||||
DEPLOY_BRIDGES=true
|
||||
```
|
||||
|
||||
### 3. Verify Configuration
|
||||
|
||||
```bash
|
||||
# Check if variables are set
|
||||
source .env
|
||||
echo $PRIVATE_KEY
|
||||
echo $CCIP_ROUTER
|
||||
echo $CCIP_FEE_TOKEN
|
||||
```
|
||||
|
||||
### 4. Test Configuration
|
||||
|
||||
```bash
|
||||
# Test deployment script (dry run)
|
||||
forge script script/DeployWETH.s.sol:DeployWETH --rpc-url $RPC_URL -vvvv
|
||||
```
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
### 1. Private Key Management
|
||||
- **Never commit .env to version control**
|
||||
- Use environment variables in production
|
||||
- Use hardware wallets for production deployments
|
||||
- Rotate private keys regularly
|
||||
- Store sensitive credentials in Azure Key Vault or similar
|
||||
|
||||
### 2. Environment Variables
|
||||
- Use separate .env files for different environments (dev, staging, production)
|
||||
- Never hardcode credentials in code
|
||||
- Use secure key management services
|
||||
- Rotate credentials regularly
|
||||
|
||||
### 3. Access Control
|
||||
- Limit access to .env files
|
||||
- Use least privilege principle
|
||||
- Monitor access to sensitive credentials
|
||||
- Use multi-factor authentication
|
||||
|
||||
## Deployment Scripts
|
||||
|
||||
### Scripts Requiring PRIVATE_KEY
|
||||
- `Deploy.s.sol` - Main deployment script
|
||||
- `DeployWETH.s.sol` - WETH deployment
|
||||
- `DeployWETH10.s.sol` - WETH10 deployment
|
||||
- `DeployCCIPWETH9Bridge.s.sol` - CCIPWETH9Bridge deployment
|
||||
- `DeployCCIPWETH10Bridge.s.sol` - CCIPWETH10Bridge deployment
|
||||
- `DeployWETHWithCCIP.s.sol` - Combined WETH + CCIP deployment
|
||||
- `DeployOracle.s.sol` - Oracle deployment
|
||||
- `DeployMulticall.s.sol` - Multicall deployment
|
||||
- `DeployMultiSig.s.sol` - MultiSig deployment
|
||||
|
||||
### Scripts Requiring Additional Variables
|
||||
- `DeployCCIPWETH9Bridge.s.sol` - Requires CCIP_ROUTER, WETH9_ADDRESS, CCIP_FEE_TOKEN
|
||||
- `DeployCCIPWETH10Bridge.s.sol` - Requires CCIP_ROUTER, WETH10_ADDRESS, CCIP_FEE_TOKEN
|
||||
- `DeployWETHWithCCIP.s.sol` - Requires CCIP_ROUTER, CCIP_FEE_TOKEN, DEPLOY_WETH9, DEPLOY_WETH10, DEPLOY_BRIDGES
|
||||
- `DeployMultiSig.s.sol` - Requires MULTISIG_OWNER_1, MULTISIG_OWNER_2, MULTISIG_OWNER_3, MULTISIG_REQUIRED
|
||||
|
||||
## Testing
|
||||
|
||||
### Test Configuration
|
||||
Tests don't require environment variables - they use mock contracts and test fixtures.
|
||||
|
||||
### Running Tests
|
||||
```bash
|
||||
# Run all tests
|
||||
forge test
|
||||
|
||||
# Run specific test
|
||||
forge test --match-test testSendCrossChain
|
||||
|
||||
# Run with verbose output
|
||||
forge test -vvvv
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Missing Environment Variables**
|
||||
- Error: `Error: Missing environment variable: PRIVATE_KEY`
|
||||
- Solution: Create .env file and set PRIVATE_KEY
|
||||
|
||||
2. **Invalid Private Key**
|
||||
- Error: `Error: Invalid private key format`
|
||||
- Solution: Ensure private key is hex format without 0x prefix
|
||||
|
||||
3. **Invalid Address**
|
||||
- Error: `Error: Invalid address format`
|
||||
- Solution: Ensure addresses are valid Ethereum addresses
|
||||
|
||||
4. **Missing CCIP Router**
|
||||
- Error: `Error: CCIP_ROUTER not set`
|
||||
- Solution: Set CCIP_ROUTER in .env file
|
||||
|
||||
5. **Missing Fee Token**
|
||||
- Error: `Error: CCIP_FEE_TOKEN not set`
|
||||
- Solution: Set CCIP_FEE_TOKEN in .env file
|
||||
|
||||
## References
|
||||
|
||||
- [Foundry Documentation](https://book.getfoundry.sh/)
|
||||
- [Chainlink CCIP Documentation](https://docs.chain.link/ccip)
|
||||
- [WETH Deployment Guide](docs/WETH_CCIP_DEPLOYMENT.md)
|
||||
- [Azure/Cloudflare Environment Setup](AZURE_CLOUDFLARE_ENV_SETUP.md)
|
||||
|
||||
302
docs/configuration/NETWORK_CONFIGURATION_GUIDE.md
Normal file
302
docs/configuration/NETWORK_CONFIGURATION_GUIDE.md
Normal file
@@ -0,0 +1,302 @@
|
||||
# Network Configuration Guide
|
||||
|
||||
**Last Updated**: 2025-01-27
|
||||
**Status**: Active
|
||||
**Purpose**: Network and Besu configuration setup
|
||||
|
||||
## Overview
|
||||
|
||||
This guide describes how to use the interactive configuration tool to set up all necessary configuration files for the Besu network.
|
||||
|
||||
> **Related Documentation**:
|
||||
> - [Azure/Cloudflare Environment Setup](AZURE_CLOUDFLARE_ENV_SETUP.md) - For Azure and Cloudflare environment variables
|
||||
> - [Contract Deployment Environment Setup](CONTRACT_DEPLOYMENT_ENV_SETUP.md) - For contract deployment environment variables
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Overview](#overview)
|
||||
- [Quick Start](#quick-start)
|
||||
- [Basic Configuration](#basic-configuration)
|
||||
- [Advanced Configuration](#advanced-configuration)
|
||||
- [Configuration Steps](#configuration-steps)
|
||||
- [1. Genesis Block Configuration](#1-genesis-block-configuration)
|
||||
- [2. Network Configuration](#2-network-configuration)
|
||||
- [3. Besu Node Configuration](#3-besu-node-configuration)
|
||||
- [4. Deployment Configuration](#4-deployment-configuration)
|
||||
- [5. Advanced Configuration](#5-advanced-configuration-advanced-tool-only)
|
||||
- [Generated Files](#generated-files)
|
||||
- [Core Configuration](#core-configuration)
|
||||
- [Infrastructure Configuration](#infrastructure-configuration)
|
||||
- [Advanced Configuration](#advanced-configuration-advanced-tool-only-1)
|
||||
- [Documentation](#documentation)
|
||||
- [Backup](#backup)
|
||||
- [Validation](#validation)
|
||||
- [Next Steps](#next-steps)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [Configuration Tool Not Running](#configuration-tool-not-running)
|
||||
- [Invalid Configuration Values](#invalid-configuration-values)
|
||||
- [Missing Configuration Files](#missing-configuration-files)
|
||||
- [Backup Restoration](#backup-restoration)
|
||||
- [Best Practices](#best-practices)
|
||||
- [Examples](#examples)
|
||||
- [Basic Configuration](#basic-configuration-1)
|
||||
- [Advanced Configuration](#advanced-configuration-1)
|
||||
- [Custom Configuration](#custom-configuration)
|
||||
- [References](#references)
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Basic Configuration
|
||||
|
||||
Run the basic configuration tool:
|
||||
|
||||
```bash
|
||||
./scripts/configure-network.sh
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Backup existing configuration files
|
||||
2. Prompt for all necessary configuration values
|
||||
3. Generate all configuration files
|
||||
4. Create a configuration summary
|
||||
|
||||
### Advanced Configuration
|
||||
|
||||
For advanced options (security, monitoring, backups, etc.):
|
||||
|
||||
```bash
|
||||
./scripts/configure-network-advanced.sh
|
||||
```
|
||||
|
||||
## Configuration Steps
|
||||
|
||||
### 1. Genesis Block Configuration
|
||||
|
||||
The tool will prompt for:
|
||||
- **Chain ID**: Unique identifier for your network (default: 138)
|
||||
- **Block Period**: Time between blocks in seconds (default: 2)
|
||||
- **Epoch Length**: Number of blocks in an epoch (default: 30000)
|
||||
- **Request Timeout**: Timeout for consensus requests (default: 10 seconds)
|
||||
- **Gas Limit**: Maximum gas per block (default: 0x1c9c380)
|
||||
- **Validators**: Validator addresses (will be updated after key generation)
|
||||
- **Pre-allocated Accounts**: Accounts with initial balances
|
||||
|
||||
### 2. Network Configuration
|
||||
|
||||
The tool will prompt for:
|
||||
- **Cluster Name**: Name of your Kubernetes cluster
|
||||
- **Resource Group**: Azure resource group name
|
||||
- **Location**: Azure region
|
||||
- **VNet Address Space**: CIDR notation for VNet
|
||||
- **Subnets**: CIDR notation for each subnet (validators, sentries, RPC, AKS)
|
||||
- **Node Counts**: Number of validators, sentries, and RPC nodes
|
||||
- **VM Sizes**: VM sizes for each node type
|
||||
- **Ports**: P2P, RPC HTTP, RPC WebSocket, and metrics ports
|
||||
|
||||
### 3. Besu Node Configuration
|
||||
|
||||
The tool will prompt for:
|
||||
- **Sync Mode**: FULL, FAST, or SNAP
|
||||
- **RPC Configuration**: Enable/disable RPC, host, port
|
||||
- **WebSocket Configuration**: Enable/disable WebSocket, host, port
|
||||
- **CORS Origins**: Allowed CORS origins (for RPC nodes)
|
||||
- **Host Allowlist**: Allowed hosts (for RPC nodes)
|
||||
- **JVM Options**: JVM memory and options for each node type
|
||||
|
||||
### 4. Deployment Configuration
|
||||
|
||||
The tool will prompt for:
|
||||
- **Deployment Type**: AKS, VM, or both
|
||||
- **VM Configuration**: VM Scale Sets, regions, SSH keys
|
||||
- **Key Vault**: Key Vault name and settings
|
||||
- **Monitoring**: Enable/disable monitoring components
|
||||
- **Blockscout**: Enable/disable Blockscout explorer
|
||||
|
||||
### 5. Advanced Configuration (Advanced Tool Only)
|
||||
|
||||
The advanced tool also prompts for:
|
||||
- **Security**: Network Policies, RBAC, Pod Security, WAF
|
||||
- **Key Management**: Azure Key Vault, key rotation
|
||||
- **Access Control**: IP whitelisting, admin IPs
|
||||
- **Monitoring**: Prometheus, Grafana, Loki, Alertmanager
|
||||
- **Backups**: Backup frequency, retention, storage
|
||||
- **Oracle**: Oracle publisher configuration
|
||||
|
||||
## Generated Files
|
||||
|
||||
The configuration tool generates the following files:
|
||||
|
||||
### Core Configuration
|
||||
|
||||
- `config/genesis.json` - Genesis block configuration
|
||||
- `config/validators/besu-config.toml` - Validator node configuration
|
||||
- `config/sentries/besu-config.toml` - Sentry node configuration
|
||||
- `config/rpc/besu-config.toml` - RPC node configuration
|
||||
- `config/permissions-nodes.toml` - Node permissions
|
||||
- `config/permissions-accounts.toml` - Account permissions
|
||||
- `config/static-nodes.json` - Static nodes configuration
|
||||
|
||||
### Infrastructure Configuration
|
||||
|
||||
- `terraform/terraform.tfvars` - Terraform variables
|
||||
- `helm/besu-network/values.yaml` - Helm values
|
||||
|
||||
### Advanced Configuration (Advanced Tool Only)
|
||||
|
||||
- `k8s/network-policies/default-deny.yaml` - Network Policies
|
||||
- `monitoring/prometheus/prometheus.yml` - Prometheus configuration
|
||||
- `scripts/backup/backup-config.json` - Backup configuration
|
||||
|
||||
### Documentation
|
||||
|
||||
- `CONFIG_SUMMARY.md` - Configuration summary
|
||||
|
||||
## Backup
|
||||
|
||||
Before generating new configuration files, the tool automatically backs up existing files to `.config-backup/` directory.
|
||||
|
||||
To restore from backup:
|
||||
|
||||
```bash
|
||||
cp -r .config-backup/* .
|
||||
```
|
||||
|
||||
## Validation
|
||||
|
||||
The configuration tool validates:
|
||||
|
||||
- **Chain ID**: Must be between 1 and 2147483647
|
||||
- **IP Addresses**: Must be valid IPv4 addresses
|
||||
- **CIDR Notation**: Must be valid CIDR notation
|
||||
- **Ports**: Must be between 1 and 65535
|
||||
- **Hex Values**: Must be valid hexadecimal values
|
||||
|
||||
## Next Steps
|
||||
|
||||
After running the configuration tool:
|
||||
|
||||
1. **Generate Validator Keys**:
|
||||
```bash
|
||||
./scripts/key-management/generate-validator-keys.sh <count>
|
||||
```
|
||||
|
||||
2. **Generate Proper Genesis**:
|
||||
```bash
|
||||
./scripts/generate-genesis-proper.sh
|
||||
```
|
||||
This will update `genesis.json` with proper `extraData` using validator keys.
|
||||
|
||||
3. **Review Configuration**:
|
||||
- Review `CONFIG_SUMMARY.md`
|
||||
- Review generated configuration files
|
||||
- Update `permissions-nodes.toml` with node enodes after deployment
|
||||
- Update `static-nodes.json` with node enodes after deployment
|
||||
|
||||
4. **Deploy Infrastructure**:
|
||||
```bash
|
||||
cd terraform
|
||||
terraform init
|
||||
terraform plan
|
||||
terraform apply
|
||||
```
|
||||
|
||||
5. **Deploy Kubernetes Resources**:
|
||||
```bash
|
||||
kubectl apply -k k8s/base
|
||||
helm install besu-validators ./helm/besu-network -f helm/besu-network/values.yaml -n besu-network
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Configuration Tool Not Running
|
||||
|
||||
**Issue**: Python script not executing
|
||||
|
||||
**Solution**:
|
||||
- Ensure Python 3.8+ is installed: `python3 --version`
|
||||
- Make script executable: `chmod +x scripts/configure-network.sh`
|
||||
- Run directly: `python3 scripts/configure-network.py`
|
||||
|
||||
### Invalid Configuration Values
|
||||
|
||||
**Issue**: Configuration tool rejects valid values
|
||||
|
||||
**Solution**:
|
||||
- Check validation rules in the tool
|
||||
- Ensure values are in correct format (hex, decimal, CIDR, etc.)
|
||||
- Review error messages for specific requirements
|
||||
|
||||
### Missing Configuration Files
|
||||
|
||||
**Issue**: Some configuration files are not generated
|
||||
|
||||
**Solution**:
|
||||
- Check if advanced options are enabled (use advanced tool)
|
||||
- Review configuration summary for missing files
|
||||
- Manually create missing files if needed
|
||||
|
||||
### Backup Restoration
|
||||
|
||||
**Issue**: Need to restore from backup
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
# Restore all files
|
||||
cp -r .config-backup/* .
|
||||
|
||||
# Restore specific file
|
||||
cp .config-backup/config/genesis.json config/genesis.json
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Review Before Deploying**: Always review generated configuration files before deploying
|
||||
2. **Backup First**: The tool automatically backs up, but you can also manually backup
|
||||
3. **Validate Configuration**: Use validation scripts to verify configuration
|
||||
4. **Update After Deployment**: Update permissions and static nodes after deployment
|
||||
5. **Document Changes**: Document any manual changes to configuration files
|
||||
|
||||
## Examples
|
||||
|
||||
### Basic Configuration
|
||||
|
||||
```bash
|
||||
./scripts/configure-network.sh
|
||||
```
|
||||
|
||||
Follow the prompts to configure:
|
||||
- Chain ID: 138
|
||||
- Block period: 2 seconds
|
||||
- 4 validators, 3 sentries, 3 RPC nodes
|
||||
- AKS deployment
|
||||
- Basic monitoring
|
||||
|
||||
### Advanced Configuration
|
||||
|
||||
```bash
|
||||
./scripts/configure-network-advanced.sh
|
||||
```
|
||||
|
||||
Follow the prompts to configure:
|
||||
- All basic options
|
||||
- Security policies
|
||||
- Advanced monitoring
|
||||
- Backup configuration
|
||||
- Oracle publisher
|
||||
|
||||
### Custom Configuration
|
||||
|
||||
```bash
|
||||
# Edit configuration directly
|
||||
vi config/genesis.json
|
||||
vi terraform/terraform.tfvars
|
||||
vi helm/besu-network/values.yaml
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- [Besu Configuration](https://besu.hyperledger.org/stable/Reference/CLI/CLI-Syntax/)
|
||||
- [Terraform Documentation](https://www.terraform.io/docs)
|
||||
- [Helm Documentation](https://helm.sh/docs/)
|
||||
- [Kubernetes Documentation](https://kubernetes.io/docs/)
|
||||
|
||||
236
docs/configuration/TERRAFORM_BACKEND_SETUP.md
Normal file
236
docs/configuration/TERRAFORM_BACKEND_SETUP.md
Normal file
@@ -0,0 +1,236 @@
|
||||
# Terraform Backend Setup Guide
|
||||
|
||||
## Overview
|
||||
|
||||
This guide explains how to configure Terraform backend for state management using Azure Storage.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Azure CLI installed and configured
|
||||
- Terraform >= 1.0
|
||||
- Azure subscription with appropriate permissions
|
||||
- Storage account (or create one)
|
||||
|
||||
## Step 1: Create Storage Account
|
||||
|
||||
### Option 1: Using Azure CLI
|
||||
|
||||
```bash
|
||||
# Set variables
|
||||
RESOURCE_GROUP="tfstate-rg"
|
||||
STORAGE_ACCOUNT="tfstate$(openssl rand -hex 4)"
|
||||
CONTAINER_NAME="tfstate"
|
||||
LOCATION="eastus"
|
||||
|
||||
# Create resource group
|
||||
az group create --name $RESOURCE_GROUP --location $LOCATION
|
||||
|
||||
# Create storage account
|
||||
az storage account create \
|
||||
--resource-group $RESOURCE_GROUP \
|
||||
--name $STORAGE_ACCOUNT \
|
||||
--sku Standard_LRS \
|
||||
--kind StorageV2 \
|
||||
--location $LOCATION
|
||||
|
||||
# Create container
|
||||
az storage container create \
|
||||
--name $CONTAINER_NAME \
|
||||
--account-name $STORAGE_ACCOUNT
|
||||
|
||||
# Get access key
|
||||
ACCESS_KEY=$(az storage account keys list \
|
||||
--resource-group $RESOURCE_GROUP \
|
||||
--account-name $STORAGE_ACCOUNT \
|
||||
--query "[0].value" -o tsv)
|
||||
|
||||
echo "Storage Account: $STORAGE_ACCOUNT"
|
||||
echo "Container: $CONTAINER_NAME"
|
||||
echo "Access Key: $ACCESS_KEY"
|
||||
```
|
||||
|
||||
### Option 2: Using Terraform
|
||||
|
||||
```bash
|
||||
cd terraform/backend-setup
|
||||
terraform init
|
||||
terraform apply
|
||||
```
|
||||
|
||||
## Step 2: Configure Backend
|
||||
|
||||
### Option 1: Environment Variables
|
||||
|
||||
```bash
|
||||
export ARM_STORAGE_ACCOUNT_NAME="tfstate<your-random>"
|
||||
export ARM_CONTAINER_NAME="tfstate"
|
||||
export ARM_RESOURCE_GROUP_NAME="tfstate-rg"
|
||||
export ARM_ACCESS_KEY="<your-access-key>"
|
||||
```
|
||||
|
||||
### Option 2: Backend Config File
|
||||
|
||||
Create `terraform/backend.hcl`:
|
||||
|
||||
```hcl
|
||||
resource_group_name = "tfstate-rg"
|
||||
storage_account_name = "tfstate<random>"
|
||||
container_name = "tfstate"
|
||||
key = "defi-oracle-mainnet.terraform.tfstate"
|
||||
```
|
||||
|
||||
Then initialize:
|
||||
|
||||
```bash
|
||||
terraform init -backend-config=backend.hcl
|
||||
```
|
||||
|
||||
### Option 3: Command Line
|
||||
|
||||
```bash
|
||||
terraform init \
|
||||
-backend-config="resource_group_name=tfstate-rg" \
|
||||
-backend-config="storage_account_name=tfstate<random>" \
|
||||
-backend-config="container_name=tfstate" \
|
||||
-backend-config="key=defi-oracle-mainnet.terraform.tfstate" \
|
||||
-backend-config="access_key=<your-access-key>"
|
||||
```
|
||||
|
||||
## Step 3: Update terraform/main.tf
|
||||
|
||||
Uncomment the backend configuration in `terraform/main.tf`:
|
||||
|
||||
```hcl
|
||||
terraform {
|
||||
backend "azurerm" {
|
||||
resource_group_name = "tfstate-rg"
|
||||
storage_account_name = "tfstate<random>"
|
||||
container_name = "tfstate"
|
||||
key = "defi-oracle-mainnet.terraform.tfstate"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Step 4: Initialize Terraform
|
||||
|
||||
```bash
|
||||
cd terraform
|
||||
terraform init
|
||||
```
|
||||
|
||||
## Step 5: Verify Backend
|
||||
|
||||
```bash
|
||||
# Check backend configuration
|
||||
terraform show -backend-config
|
||||
|
||||
# List state
|
||||
terraform state list
|
||||
```
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
### 1. Use Azure Key Vault
|
||||
|
||||
Store access keys in Azure Key Vault:
|
||||
|
||||
```bash
|
||||
# Store access key in Key Vault
|
||||
az keyvault secret set \
|
||||
--vault-name <key-vault-name> \
|
||||
--name "terraform-backend-key" \
|
||||
--value $ACCESS_KEY
|
||||
```
|
||||
|
||||
Reference in Terraform:
|
||||
|
||||
```hcl
|
||||
data "azurerm_key_vault_secret" "backend_key" {
|
||||
name = "terraform-backend-key"
|
||||
key_vault_id = azurerm_key_vault.main.id
|
||||
}
|
||||
|
||||
terraform {
|
||||
backend "azurerm" {
|
||||
# Use Key Vault for access key
|
||||
access_key = data.azurerm_key_vault_secret.backend_key.value
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Enable Storage Account Security
|
||||
|
||||
- Enable soft delete
|
||||
- Enable versioning
|
||||
- Restrict network access
|
||||
- Enable encryption at rest
|
||||
- Use managed identity when possible
|
||||
|
||||
### 3. State Locking
|
||||
|
||||
Terraform automatically uses blob leasing for state locking. Ensure:
|
||||
|
||||
- Storage account supports blob leasing
|
||||
- Network access allows Terraform to access storage
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Backend Initialization Failed
|
||||
|
||||
**Error**: "Failed to get existing workspaces"
|
||||
|
||||
**Solution**:
|
||||
1. Verify storage account exists
|
||||
2. Check access key is correct
|
||||
3. Verify container exists
|
||||
4. Check network access rules
|
||||
|
||||
### State Lock Error
|
||||
|
||||
**Error**: "Error acquiring the state lock"
|
||||
|
||||
**Solution**:
|
||||
1. Check if another Terraform process is running
|
||||
2. Manually release lock if process crashed:
|
||||
```bash
|
||||
az storage blob lease break \
|
||||
--account-name $STORAGE_ACCOUNT \
|
||||
--container-name $CONTAINER_NAME \
|
||||
--blob-name "defi-oracle-mainnet.terraform.tfstate"
|
||||
```
|
||||
|
||||
### Access Denied
|
||||
|
||||
**Error**: "Access denied to storage account"
|
||||
|
||||
**Solution**:
|
||||
1. Verify access key is correct
|
||||
2. Check storage account firewall rules
|
||||
3. Verify network access
|
||||
4. Check RBAC permissions
|
||||
|
||||
## Migration
|
||||
|
||||
### Migrate from Local to Remote Backend
|
||||
|
||||
```bash
|
||||
# Initialize with backend
|
||||
terraform init -migrate-state
|
||||
|
||||
# Verify migration
|
||||
terraform state list
|
||||
```
|
||||
|
||||
### Migrate Between Backends
|
||||
|
||||
```bash
|
||||
# Update backend configuration
|
||||
terraform init -migrate-state -backend-config=backend.hcl
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- [Terraform Azure Backend](https://www.terraform.io/docs/language/settings/backends/azurerm.html)
|
||||
- [Azure Storage Documentation](https://docs.microsoft.com/azure/storage/)
|
||||
- [Terraform State Management](https://www.terraform.io/docs/language/state/index.html)
|
||||
|
||||
Reference in New Issue
Block a user