# Maximum Parallel Deployment Strategy

## Overview

This strategy maximizes parallel execution across **ALL** deployment phases, achieving the fastest possible deployment time.

## Parallelism Settings

### Terraform
- **Parallelism**: 128 (maximum)
- **Default**: 10
- **Speedup**: 12.8x faster than default

### Kubernetes Configuration
- **Parallelism**: 24 (one per region)
- **Method**: xargs -P 24
- **Speedup**: 24x faster than sequential

### Besu Deployment
- **Parallelism**: 48 (one per validator)
- **Method**: xargs -P 48
- **Speedup**: 48x faster than sequential

### Contract Deployment
- **Parallelism**: 2 (Mainnet + Chain-138)
- **Method**: Background jobs
- **Speedup**: 2x faster than sequential

### Monitoring
- **Parallelism**: 24 (one per region)
- **Method**: xargs -P 24
- **Speedup**: 24x faster than sequential

## Deployment Modes

### Mode 1: Maximum Parallel (Recommended)
```bash
./scripts/deployment/deploy-max-parallel.sh
```
- **Terraform**: parallelism=128
- **Phases**: Sequential but each phase is maximally parallel
- **Time**: 30-60 minutes
- **Risk**: Medium (may hit API limits)

### Mode 2: Ultra Parallel (Fastest)
```bash
./scripts/deployment/deploy-ultra-parallel.sh
```
- **All Phases**: Run simultaneously
- **Terraform**: parallelism=128
- **Time**: 20-40 minutes
- **Risk**: High (may hit API limits, requires careful monitoring)

## Performance Comparison

| Mode | Time | Speedup | Risk |
|------|------|---------|------|
| Sequential | 5-9 hours | 1x | Low |
| Standard Parallel | 45-85 min | 6-12x | Low |
| Maximum Parallel | 30-60 min | 10-18x | Medium |
| Ultra Parallel | 20-40 min | 15-27x | High |

## Azure API Rate Limits

### Considerations
- **Subscription Limits**: May have per-subscription rate limits
- **Resource Provider Limits**: Each provider (Compute, Network, etc.) has limits
- **Regional Limits**: Each region has independent limits

### Mitigation
- **Retry Logic**: Automatic retries with exponential backoff
- **Throttling**: Terraform handles throttling automatically
- **Monitoring**: Monitor API usage during deployment

## Deployment Scripts

### Maximum Parallel
- `deploy-max-parallel.sh` - Maximum parallelism per phase
- `configure-kubernetes-max-parallel.sh` - Maximum K8s parallelism
- `deploy-besu-max-parallel.sh` - Maximum Besu parallelism
- `verify-all-max-parallel.sh` - Maximum verification parallelism

### Ultra Parallel
- `deploy-ultra-parallel.sh` - All phases simultaneously

## Best Practices

### 1. Start with Maximum Parallel
- Use `deploy-max-parallel.sh` first
- Monitor for API rate limit errors
- Adjust parallelism if needed

### 2. Monitor Progress
- Watch Terraform output
- Monitor Azure Portal for resource creation
- Check logs for errors

### 3. Handle Failures
- Some regions may fail (retry independently)
- Verify successful regions
- Retry failed regions separately

### 4. Gradual Rollout (Optional)
- Deploy 8 regions first (test)
- Then deploy remaining 16 regions
- Reduces risk of hitting limits

## Recommended Approach

### Phase 1: Test with Maximum Parallel
```bash
./scripts/deployment/deploy-max-parallel.sh
```

### Phase 2: If Successful, Use Ultra Parallel
```bash
./scripts/deployment/deploy-ultra-parallel.sh
```

### Phase 3: Monitor and Adjust
- Monitor API usage
- Adjust parallelism if hitting limits
- Retry failed regions

## Expected Results

### Maximum Parallel
- **Infrastructure**: 20-30 minutes
- **Kubernetes**: 3-5 minutes
- **Besu**: 5-10 minutes
- **Total**: 30-60 minutes

### Ultra Parallel
- **All Phases**: 20-40 minutes
- **Fastest Possible**: All operations concurrent

## Troubleshooting

### API Rate Limits
- **Symptom**: "Too Many Requests" errors
- **Solution**: Reduce parallelism or add delays
- **Command**: `terraform apply -parallelism=64` (half)

### Partial Failures
- **Symptom**: Some regions succeed, others fail
- **Solution**: Retry failed regions independently
- **Command**: `terraform apply -target=module.region_deployment["region"]`

### Timeout Issues
- **Symptom**: Operations timing out
- **Solution**: Increase timeout or reduce parallelism
- **Note**: Some operations (AKS cluster creation) take 10-15 minutes