docs/TERRAFORM_MODULES_CONSOLIDATION.md

Terraform Modules Consolidation Plan

Last Updated: 2025-01-27 Purpose: Plan for consolidating and standardizing Terraform modules across projects

---

Overview

Multiple projects contain Terraform modules that can be consolidated into shared, reusable modules. This document identifies consolidation opportunities and provides a plan for implementation.

---

Current Terraform Module Inventory

Project: smom-dbis-138

Location: smom-dbis-138/terraform/modules/

Modules:

• networking - Virtual networks, subnets, NSGs
• kubernetes - AKS cluster, node pools
• storage - Storage accounts, containers
• secrets - Key Vault
• resource-groups - Resource group management
• keyvault-enhanced - Enhanced Key Vault with RBAC
• budget - Consumption budgets
• monitoring - Monitoring and observability
• backup - Backup configurations
• nginx-proxy - Nginx proxy configuration
• networking-vm - VM networking
• application-gateway - Application Gateway configuration

Multi-Cloud Modules:

• modules/azure/ - Azure-specific modules
• modules/aws/ - AWS-specific modules
• modules/gcp/ - GCP-specific modules
• modules/onprem-hci/ - On-premises HCI modules
• modules/azure-arc/ - Azure Arc integration
• modules/service-mesh/ - Service mesh configuration
• modules/observability/ - Observability stack

Project: the_order

Location: the_order/infra/terraform/modules/

Modules:

• regional-landing-zone/ - Regional landing zone
• well-architected/ - Well-Architected Framework modules

Project: loc_az_hci

Location: loc_az_hci/terraform/

Modules:

• Proxmox integration modules
• Azure Arc modules
• Kubernetes modules

Project: Sankofa

Location: Sankofa/cloudflare/terraform/

Modules:

• Cloudflare DNS configuration
• Cloudflare Tunnel configuration

---

Consolidation Opportunities

High Priority Modules (Used Across Multiple Projects)

1. Networking Module

Current Locations:

• smom-dbis-138/terraform/modules/networking
• Used for: Virtual networks, subnets, NSGs

Consolidation:

• Create shared module: infrastructure/terraform/modules/azure/networking
• Standardize interface
• Support multiple projects

2. Kubernetes Module

Current Locations:

• smom-dbis-138/terraform/modules/kubernetes
• loc_az_hci/terraform/ (K3s configuration)

Consolidation:

• Create shared module: infrastructure/terraform/modules/azure/kubernetes
• Support AKS and K3s
• Standardize configuration

3. Key Vault Module

Current Locations:

• smom-dbis-138/terraform/modules/keyvault-enhanced
• the_order/infra/terraform/ (if present)

Consolidation:

• Create shared module: infrastructure/terraform/modules/azure/keyvault
• Enhanced version with RBAC
• Support both access policies and RBAC

4. Storage Module

Current Locations:

• smom-dbis-138/terraform/modules/storage

Consolidation:

• Create shared module: infrastructure/terraform/modules/azure/storage
• Standardize storage account configuration
• Support multiple storage types

5. Monitoring Module

Current Locations:

• smom-dbis-138/terraform/modules/monitoring
• loc_az_hci/ (monitoring configuration)

Consolidation:

• Create shared module: infrastructure/terraform/modules/azure/monitoring
• Unified monitoring stack
• Support Prometheus, Grafana, Application Insights

---

Proposed Shared Module Structure

infrastructure/
├── terraform/
│   ├── modules/
│   │   ├── azure/
│   │   │   ├── networking/
│   │   │   ├── kubernetes/
│   │   │   ├── storage/
│   │   │   ├── keyvault/
│   │   │   ├── monitoring/
│   │   │   ├── database/
│   │   │   └── compute/
│   │   ├── multi-cloud/
│   │   │   ├── azure/
│   │   │   ├── aws/
│   │   │   ├── gcp/
│   │   │   └── onprem-hci/
│   │   └── shared/
│   │       ├── resource-groups/
│   │       ├── tags/
│   │       └── naming/
│   ├── environments/
│   │   ├── dev/
│   │   ├── staging/
│   │   └── prod/
│   └── README.md

---

Module Standardization

Standard Module Structure

module-name/
├── main.tf              # Main module resources
├── variables.tf         # Input variables
├── outputs.tf           # Output values
├── versions.tf          # Version constraints
├── README.md            # Module documentation
└── examples/            # Usage examples
    └── basic/
        └── main.tf

Standard Variables

• environment - Environment name (dev/staging/prod)
• location - Azure region
• project_name - Project identifier
• tags - Resource tags
• resource_group_name - Resource group name

Standard Outputs

• Resource IDs
• Resource names
• Connection strings (when applicable)
• Configuration values

---

Migration Strategy

Phase 1: Identify and Document (Week 1-2)

• Inventory all Terraform modules ✅
• Document module interfaces
• Identify common patterns
• Document dependencies

Phase 2: Create Shared Module Structure (Week 3-4)

• Create infrastructure/terraform/modules/ structure
• Create shared module templates
• Document module standards
• Create module registry

Phase 3: Consolidate High-Priority Modules (Week 5-8)

• Networking module
• Kubernetes module
• Key Vault module
• Storage module
• Monitoring module

Phase 4: Migrate Projects (Week 9-12)

• Update smom-dbis-138 to use shared modules
• Update the_order to use shared modules
• Update loc_az_hci to use shared modules
• Update Sankofa to use shared modules (if applicable)

Phase 5: Documentation and Testing (Week 13-14)

• Complete module documentation
• Create usage examples
• Test module compatibility
• Update project documentation

---

Module Registry

Azure Modules

networking

• Purpose: Virtual networks, subnets, NSGs, Application Gateway
• Used By: smom-dbis-138, the_order
• Status: To be consolidated

kubernetes

• Purpose: AKS cluster, node pools, networking
• Used By: smom-dbis-138, loc_az_hci
• Status: To be consolidated

keyvault

• Purpose: Azure Key Vault with RBAC
• Used By: smom-dbis-138, the_order
• Status: To be consolidated

storage

• Purpose: Storage accounts, containers, file shares
• Used By: smom-dbis-138
• Status: To be consolidated

monitoring

• Purpose: Log Analytics, Application Insights, monitoring
• Used By: smom-dbis-138, loc_az_hci
• Status: To be consolidated

Multi-Cloud Modules

azure

• Purpose: Azure-specific resources
• Used By: smom-dbis-138
• Status: Existing, to be enhanced

aws

• Purpose: AWS-specific resources
• Used By: smom-dbis-138
• Status: Existing

gcp

• Purpose: GCP-specific resources
• Used By: smom-dbis-138
• Status: Existing

onprem-hci

• Purpose: On-premises HCI infrastructure
• Used By: smom-dbis-138
• Status: Existing

---

Best Practices

Module Design

1. Single Responsibility: Each module should have one clear purpose
2. Composable: Modules should work together
3. Configurable: Use variables for flexibility
4. Documented: Clear README and examples
5. Tested: Test modules in isolation

Versioning

• Use semantic versioning
• Tag module releases
• Document breaking changes
• Maintain changelog

Testing

• Test modules in isolation
• Use Terratest for automated testing
• Validate module outputs
• Test error scenarios

---

Usage Examples

Using Shared Networking Module

module "networking" {
  source = "../../infrastructure/terraform/modules/azure/networking"

  environment     = var.environment
  location        = var.location
  project_name    = "dbis-core"
  resource_group_name = azurerm_resource_group.main.name

  vnet_address_space = ["10.0.0.0/16"]
  subnets = {
    app = {
      address_prefixes = ["10.0.1.0/24"]
      service_endpoints = ["Microsoft.Storage"]
    }
    db = {
      address_prefixes = ["10.0.2.0/24"]
      service_endpoints = ["Microsoft.Sql"]
    }
  }

  tags = var.tags
}

---

Next Steps

1. Create Infrastructure Directory Structure

   • Set up infrastructure/terraform/modules/
   • Create module templates
   • Document standards

2. Prioritize Module Consolidation

   • Start with networking module
   • Consolidate Kubernetes module
   • Standardize Key Vault module

3. Migration Planning

   • Plan migration for each project
   • Test compatibility
   • Update documentation

---

Last Updated: 2025-01-27 Status: Planning Phase