1fb7266469
- Introduced Aggregator.sol for Chainlink-compatible oracle functionality, including round-based updates and access control. - Added OracleWithCCIP.sol to extend Aggregator with CCIP cross-chain messaging capabilities. - Created .gitmodules to include OpenZeppelin contracts as a submodule. - Developed a comprehensive deployment guide in NEXT_STEPS_COMPLETE_GUIDE.md for Phase 2 and smart contract deployment. - Implemented Vite configuration for the orchestration portal, supporting both Vue and React frameworks. - Added server-side logic for the Multi-Cloud Orchestration Portal, including API endpoints for environment management and monitoring. - Created scripts for resource import and usage validation across non-US regions. - Added tests for CCIP error handling and integration to ensure robust functionality. - Included various new files and directories for the orchestration portal and deployment scripts.
186 lines
5.6 KiB
Markdown
186 lines
5.6 KiB
Markdown
# Parallel Deployment Strategy
|
||
|
||
## Overview
|
||
|
||
This deployment strategy optimizes for **full parallel execution** across all 24 regions, maximizing deployment speed while excluding West Europe from validator deployment (admin use only).
|
||
|
||
## Key Optimizations
|
||
|
||
### 1. Parallel Terraform Execution
|
||
- **Terraform Parallelism**: Set to 50 (allows concurrent resource creation)
|
||
- **Region Independence**: Each region deploys independently
|
||
- **No Dependencies**: Regions don't depend on each other
|
||
|
||
### 2. Parallel Kubernetes Configuration
|
||
- **xargs -P 24**: Configure all 24 regions simultaneously
|
||
- **Independent Operations**: Each region configured separately
|
||
- **No Blocking**: One region's failure doesn't block others
|
||
|
||
### 3. Parallel Besu Deployment
|
||
- **Concurrent Deployments**: Deploy validators to all regions at once
|
||
- **Independent State**: Each region's Besu network is independent
|
||
- **Fast Startup**: All validators start simultaneously
|
||
|
||
## Region Strategy
|
||
|
||
### Deployment Regions (24)
|
||
All regions have 10+ vCPUs available:
|
||
1. North Europe
|
||
2. UK South
|
||
3. UK West
|
||
4. France Central
|
||
5. Germany West Central
|
||
6. Switzerland North
|
||
7. Italy North
|
||
8. Norway East
|
||
9. Poland Central
|
||
10. Spain Central
|
||
11. Sweden Central
|
||
12. Belgium Central
|
||
13. Australia East
|
||
14. Australia Southeast
|
||
15. East Asia
|
||
16. Southeast Asia
|
||
17. Central India
|
||
18. South India
|
||
19. West India (replacement for West Europe)
|
||
20. Japan East
|
||
21. Japan West
|
||
22. Korea Central
|
||
23. Korea South
|
||
24. Canada Central
|
||
|
||
**Total**: 24 regions × 10 vCPUs = 240 vCPUs ✅
|
||
|
||
### Administrative Region (West Europe)
|
||
- **Purpose**: Monitoring, management, control plane only
|
||
- **No Validators**: West Europe does not host validators
|
||
- **No Sentries**: West Europe does not host sentries
|
||
- **Optional System Nodes**: For admin tools only
|
||
- **Quota**: Not counted in 240 vCPU deployment
|
||
|
||
## Deployment Phases (Parallel)
|
||
|
||
### Phase 1: Infrastructure (Parallel)
|
||
```bash
|
||
cd terraform/well-architected/cloud-sovereignty
|
||
terraform init
|
||
terraform plan -out=tfplan-parallel -parallelism=50
|
||
terraform apply -parallelism=50 tfplan-parallel
|
||
```
|
||
- **Time**: 30-60 minutes (vs 2-4 hours sequential)
|
||
- **Speed**: 4-8x faster than sequential
|
||
|
||
### Phase 2: Kubernetes Configuration (Parallel)
|
||
```bash
|
||
./scripts/deployment/configure-kubernetes-parallel.sh
|
||
```
|
||
- **Time**: 5-10 minutes (vs 1-2 hours sequential)
|
||
- **Speed**: 12-24x faster than sequential
|
||
|
||
### Phase 3: Besu Network (Parallel)
|
||
```bash
|
||
./scripts/deployment/deploy-besu-parallel.sh
|
||
```
|
||
- **Time**: 10-15 minutes (vs 2-3 hours sequential)
|
||
- **Speed**: 12-18x faster than sequential
|
||
|
||
### Phase 4: Contracts (Can be parallel)
|
||
- Deploy Mainnet contracts
|
||
- Deploy Chain-138 contracts
|
||
- Can run in parallel if using different networks
|
||
|
||
### Phase 5: CCIP Integration (Sequential)
|
||
- Configure routers (sequential due to dependencies)
|
||
- Fund contracts (sequential for safety)
|
||
|
||
## Performance Comparison
|
||
|
||
| Phase | Sequential | Parallel | Speedup |
|
||
|-------|-----------|----------|---------|
|
||
| Infrastructure | 2-4 hours | 30-60 min | 4-8x |
|
||
| Kubernetes | 1-2 hours | 5-10 min | 12-24x |
|
||
| Besu Network | 2-3 hours | 10-15 min | 12-18x |
|
||
| **Total** | **5-9 hours** | **45-85 min** | **6-12x** |
|
||
|
||
## Parallel Execution Scripts
|
||
|
||
### Master Script
|
||
```bash
|
||
./scripts/deployment/deploy-parallel.sh
|
||
```
|
||
- Deploys all infrastructure in parallel
|
||
- Uses Terraform parallelism=50
|
||
|
||
### Kubernetes Configuration
|
||
```bash
|
||
./scripts/deployment/configure-kubernetes-parallel.sh
|
||
```
|
||
- Configures all 24 regions simultaneously
|
||
- Uses xargs -P 24
|
||
|
||
### Besu Deployment
|
||
```bash
|
||
./scripts/deployment/deploy-besu-parallel.sh
|
||
```
|
||
- Deploys Besu to all regions simultaneously
|
||
- Uses xargs -P 24
|
||
|
||
### Verification
|
||
```bash
|
||
./scripts/deployment/verify-all-clusters-parallel.sh
|
||
```
|
||
- Verifies all clusters in parallel
|
||
- Uses xargs -P 24
|
||
|
||
## Benefits
|
||
|
||
1. **Speed**: 6-12x faster deployment
|
||
2. **Efficiency**: All regions deploy simultaneously
|
||
3. **Scalability**: Easy to add more regions
|
||
4. **Resilience**: One region's failure doesn't block others
|
||
5. **Resource Utilization**: Maximum use of available resources
|
||
|
||
## Considerations
|
||
|
||
### Resource Limits
|
||
- **Azure API Rate Limits**: May need to throttle if hitting limits
|
||
- **Terraform State**: Concurrent writes to state file (Terraform handles this)
|
||
- **Network Bandwidth**: High bandwidth usage during parallel deployment
|
||
|
||
### Error Handling
|
||
- **Partial Failures**: Some regions may fail, others succeed
|
||
- **Retry Logic**: Failed regions can be retried independently
|
||
- **Verification**: Always verify all regions after deployment
|
||
|
||
### Monitoring
|
||
- **Progress Tracking**: Monitor deployment progress per region
|
||
- **Error Detection**: Detect and report failures quickly
|
||
- **Resource Usage**: Monitor quota and API usage
|
||
|
||
## West Europe Administrative Setup
|
||
|
||
### Resources in West Europe
|
||
- **Monitoring Stack**: Prometheus, Grafana (centralized)
|
||
- **Management Tools**: Control plane, admin dashboards
|
||
- **Log Aggregation**: Centralized logging from all regions
|
||
- **Backup Storage**: Centralized backup storage
|
||
|
||
### Configuration
|
||
```bash
|
||
# Deploy admin resources to West Europe only
|
||
cd terraform/well-architected/cloud-sovereignty
|
||
terraform apply -target=module.region_deployment["westeurope"] \
|
||
-var="deploy_aks_clusters=false" \
|
||
-var="validator_count=0" \
|
||
-var="sentry_count=0"
|
||
```
|
||
|
||
## Next Steps
|
||
|
||
1. ✅ Configuration updated (24 regions, excluding West Europe)
|
||
2. ⏳ Run parallel deployment: `./scripts/deployment/deploy-parallel.sh`
|
||
3. ⏳ Verify all clusters: `./scripts/deployment/verify-all-clusters-parallel.sh`
|
||
4. ⏳ Configure Kubernetes: `./scripts/deployment/configure-kubernetes-parallel.sh`
|
||
5. ⏳ Deploy Besu: `./scripts/deployment/deploy-besu-parallel.sh`
|