scripts/SCRIPTS_MODULARIZATION_PLAN.md

Scripts Directory Modularization Plan

Date: 2025-01-27 Purpose: Evaluate and implement a modular approach for the scripts directory Status: Analysis Complete - Ready for Implementation

---

📊 Current State Analysis

Main Scripts Directory (/home/intlc/projects/scripts/)

• Total Scripts: 30 scripts
• Structure: Flat directory (no organization)
• Categories Identified: 5 main categories

Script Categories

1. Migration Scripts (7 scripts)

• migrate-readme.sh
• migrate-terraform.sh
• migrate-to-api-gateway.sh
• migrate-to-k8s.sh
• migrate-to-monitoring.sh
• migrate-to-shared-packages.sh
• migrate-dbis-project.sh

2. Metrics Scripts (8 scripts)

• collect-code-metrics.sh
• collect-deployment-metrics.sh
• collect-developer-metrics.sh
• collect-infrastructure-metrics.sh
• collect-operational-metrics.sh
• collect-service-metrics.sh
• track-all-metrics.sh
• track-success-metrics.sh
• update-metrics.sh
• generate-metrics-report.sh

3. DBIS Scripts (4 scripts)

• automate-dbis-migration.sh
• migrate-all-dbis-projects.sh
• test-dbis-migration.sh

4. Infrastructure Scripts (2 scripts)

• setup-shared-infrastructure.sh
• setup.sh

5. Utility Scripts (8 scripts)

• analyze-costs.sh
• optimize-builds.sh
• deps-analyze.sh
• deps-audit.sh
• build-all.sh
• test-all.sh
• verify-all.sh
• cleanup.sh

---

🏗️ Proposed Modular Structure

Option 1: Category-Based (Recommended)

scripts/
├── README.md                    # Main documentation
├── lib/                         # Shared libraries
│   ├── common/                 # Common utilities
│   │   ├── colors.sh           # Color definitions
│   │   ├── logging.sh          # Logging functions
│   │   ├── utils.sh            # Utility functions
│   │   ├── validation.sh       # Input validation
│   │   └── error-handling.sh   # Error handling
│   ├── config/                 # Configuration
│   │   └── env.sh              # Environment loading
│   └── init.sh                 # Initialize all libraries
├── migration/                   # Migration scripts
│   ├── migrate-readme.sh
│   ├── migrate-terraform.sh
│   ├── migrate-to-api-gateway.sh
│   ├── migrate-to-k8s.sh
│   ├── migrate-to-monitoring.sh
│   └── migrate-to-shared-packages.sh
├── metrics/                     # Metrics scripts
│   ├── collect/
│   │   ├── collect-code-metrics.sh
│   │   ├── collect-deployment-metrics.sh
│   │   ├── collect-developer-metrics.sh
│   │   ├── collect-infrastructure-metrics.sh
│   │   ├── collect-operational-metrics.sh
│   │   └── collect-service-metrics.sh
│   ├── track-all-metrics.sh
│   ├── track-success-metrics.sh
│   ├── update-metrics.sh
│   └── generate-metrics-report.sh
├── dbis/                        # DBIS-specific scripts
│   ├── automate-dbis-migration.sh
│   ├── migrate-all-dbis-projects.sh
│   ├── migrate-dbis-project.sh
│   └── test-dbis-migration.sh
├── infrastructure/              # Infrastructure scripts
│   ├── setup-shared-infrastructure.sh
│   └── setup.sh
└── utils/                       # Utility scripts
    ├── analyze-costs.sh
    ├── optimize-builds.sh
    ├── deps-analyze.sh
    ├── deps-audit.sh
    ├── build-all.sh
    ├── test-all.sh
    ├── verify-all.sh
    └── cleanup.sh

Option 2: Function-Based

scripts/
├── README.md
├── lib/                         # Shared libraries
├── deploy/                      # Deployment scripts
├── migrate/                     # Migration scripts
├── monitor/                     # Monitoring/metrics scripts
├── test/                        # Testing scripts
├── analyze/                     # Analysis scripts
└── setup/                       # Setup scripts

Option 3: Hybrid (Recommended for Future)

scripts/
├── README.md
├── lib/                         # Shared libraries
│   ├── common/
│   ├── config/
│   └── init.sh
├── migration/                   # Migration scripts
│   ├── terraform/
│   ├── kubernetes/
│   ├── api-gateway/
│   ├── monitoring/
│   └── shared-packages/
├── metrics/                     # Metrics scripts
│   ├── collect/
│   ├── track/
│   └── report/
├── dbis/                        # DBIS-specific
├── infrastructure/              # Infrastructure
└── utils/                       # Utilities
    ├── analysis/
    ├── optimization/
    └── maintenance/

---

✅ Recommended Approach: Option 1 (Category-Based)

Benefits

1. Clear Organization: Easy to find scripts by purpose
2. Scalable: Can add subdirectories as needed
3. Maintainable: Related scripts grouped together
4. Consistent: Matches patterns from other projects

Implementation Steps

1. Create Directory Structure

   mkdir -p scripts/{lib/{common,config},migration,metrics/{collect},dbis,infrastructure,utils}
   

2. Create Shared Libraries

   • Extract common functions to lib/common/
   • Create initialization script lib/init.sh

3. Move Scripts to Categories

   • Move scripts to appropriate directories
   • Update shebang paths if needed

4. Create Wrapper Scripts (Optional)

   • Create symlinks or wrapper scripts in root for backward compatibility

5. Update Documentation

   • Update README.md with new structure
   • Document library usage

---

📚 Shared Library Design

Common Utilities (lib/common/)

colors.sh

# Color definitions for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

logging.sh

# Logging functions
log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }

utils.sh

# Common utility functions
check_command() { command -v "$1" &> /dev/null; }
require_command() { check_command "$1" || { log_error "$1 not found"; exit 1; }; }

validation.sh

# Input validation functions
validate_project_name() { ... }
validate_environment() { ... }

error-handling.sh

# Error handling
set_error_handler() { ... }
cleanup_on_error() { ... }

Configuration (lib/config/)

env.sh

# Environment loading
load_env() { ... }
require_env() { ... }

Initialization (lib/init.sh)

# Initialize all libraries
source "$(dirname "$0")/common/colors.sh"
source "$(dirname "$0")/common/logging.sh"
# ... etc

---

🔄 Migration Strategy

Phase 1: Create Structure (Non-Breaking)

1. Create new directory structure
2. Create shared libraries
3. Keep existing scripts in place

Phase 2: Refactor Scripts (Gradual)

1. Update scripts to use shared libraries
2. Move scripts to new locations
3. Create wrapper scripts for backward compatibility

Phase 3: Cleanup

1. Remove wrapper scripts after migration period
2. Update all documentation
3. Update CI/CD workflows

---

📋 Implementation Checklist

Structure Creation

• Create directory structure
• Create lib/common/ directory
• Create lib/config/ directory
• Create category directories

Shared Libraries

• Create lib/common/colors.sh
• Create lib/common/logging.sh
• Create lib/common/utils.sh
• Create lib/common/validation.sh
• Create lib/common/error-handling.sh
• Create lib/config/env.sh
• Create lib/init.sh

Script Migration

• Move migration scripts
• Move metrics scripts
• Move DBIS scripts
• Move infrastructure scripts
• Move utility scripts

Documentation

• Create scripts/README.md
• Document library usage
• Update main README.md
• Create migration guide

Testing

• Test all scripts in new locations
• Verify backward compatibility
• Update CI/CD workflows

---

🎯 Best Practices from Other Projects

From loc_az_hci/scripts/

• ✅ Category-based organization
• ✅ Library structure (lib/)
• ✅ Helper functions in separate files

From smom-dbis-138/scripts/

• ✅ Common library initialization
• ✅ Configurable logging levels
• ✅ Error handling patterns

From the_order/scripts/

• ✅ Clear category separation
• ✅ README documentation
• ✅ Usage examples

---

📊 Comparison with Other Projects

Project	Structure	Libraries	Organization
loc_az_hci/scripts/	Category-based	✅ Yes	Excellent
smom-dbis-138/scripts/	Category-based	✅ Yes	Excellent
the_order/scripts/	Category-based	❌ No	Good
metaverseDubai/scripts/	Category-based	❌ No	Good
Current (scripts/)	Flat	❌ No	Needs Improvement

---

🚀 Next Steps

1. Review and Approve this plan
2. Create Structure - Set up directories
3. Create Libraries - Extract common functions
4. Migrate Scripts - Move to new locations
5. Update Documentation - Document new structure
6. Test - Verify all scripts work
7. Deploy - Update CI/CD and workflows

---

Status: Ready for Implementation Recommended: Option 1 (Category-Based with Libraries) Priority: High (Improves maintainability and scalability)