d-bis/scripts
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
scripts/NEXT_STEPS_DETAILED.md
defiQUG 5d47b3a5d9 Initial commit: add .gitignore and README
2026-02-09 21:51:52 -08:00

16 KiB
Raw Permalink Blame History

Detailed Next Steps - Scripts Modularization

Date: 2025-01-27 Status: Structure Complete - Ready for Implementation Priority: High

📋 Complete Implementation Checklist

Phase 1: Script Migration (Move Files)

1.1 Migration Scripts → migration/

  • Move migrate-readme.sh

    mv scripts/migrate-readme.sh scripts/migration/migrate-readme.sh

  • Move migrate-terraform.sh

    mv scripts/migrate-terraform.sh scripts/migration/migrate-terraform.sh

  • Move migrate-to-api-gateway.sh

    mv scripts/migrate-to-api-gateway.sh scripts/migration/migrate-to-api-gateway.sh

  • Move migrate-to-k8s.sh

    mv scripts/migrate-to-k8s.sh scripts/migration/migrate-to-k8s.sh

  • Move migrate-to-monitoring.sh

    mv scripts/migrate-to-monitoring.sh scripts/migration/migrate-to-monitoring.sh

  • Move migrate-to-shared-packages.sh

    mv scripts/migrate-to-shared-packages.sh scripts/migration/migrate-to-shared-packages.sh

1.2 Metrics Scripts → metrics/ and metrics/collect/

  • Move collection scripts to metrics/collect/

    mv scripts/collect-code-metrics.sh scripts/metrics/collect/collect-code-metrics.sh
mv scripts/collect-deployment-metrics.sh scripts/metrics/collect/collect-deployment-metrics.sh
mv scripts/collect-developer-metrics.sh scripts/metrics/collect/collect-developer-metrics.sh
mv scripts/collect-infrastructure-metrics.sh scripts/metrics/collect/collect-infrastructure-metrics.sh
mv scripts/collect-operational-metrics.sh scripts/metrics/collect/collect-operational-metrics.sh
mv scripts/collect-service-metrics.sh scripts/metrics/collect/collect-service-metrics.sh

  • Move tracking scripts to metrics/

    mv scripts/track-all-metrics.sh scripts/metrics/track-all-metrics.sh
mv scripts/track-success-metrics.sh scripts/metrics/track-success-metrics.sh
mv scripts/update-metrics.sh scripts/metrics/update-metrics.sh
mv scripts/generate-metrics-report.sh scripts/metrics/generate-metrics-report.sh

1.3 DBIS Scripts → dbis/

  • Move DBIS scripts 
    mv scripts/automate-dbis-migration.sh scripts/dbis/automate-dbis-migration.sh
mv scripts/migrate-all-dbis-projects.sh scripts/dbis/migrate-all-dbis-projects.sh
mv scripts/migrate-dbis-project.sh scripts/dbis/migrate-dbis-project.sh
mv scripts/test-dbis-migration.sh scripts/dbis/test-dbis-migration.sh

1.4 Infrastructure Scripts → infrastructure/

  • Move infrastructure scripts 
    mv scripts/setup-shared-infrastructure.sh scripts/infrastructure/setup-shared-infrastructure.sh
mv scripts/setup.sh scripts/infrastructure/setup.sh

1.5 Utility Scripts → utils/

  • Move utility scripts 
    mv scripts/analyze-costs.sh scripts/utils/analyze-costs.sh
mv scripts/optimize-builds.sh scripts/utils/optimize-builds.sh
mv scripts/deps-analyze.sh scripts/utils/deps-analyze.sh
mv scripts/deps-audit.sh scripts/utils/deps-audit.sh
mv scripts/build-all.sh scripts/utils/build-all.sh
mv scripts/test-all.sh scripts/utils/test-all.sh
mv scripts/verify-all.sh scripts/utils/verify-all.sh
mv scripts/cleanup.sh scripts/utils/cleanup.sh

Phase 2: Update Script Paths and Add Libraries

2.1 Update Migration Scripts

For each script in migration/, add library initialization at the top:

  • Update migration/migrate-readme.sh

    # Add after shebang:
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"

  • Update migration/migrate-terraform.sh

    • Add library initialization
    • Replace echo with log_info, log_error, etc.
    • Add input validation using validate_project_name

  • Update migration/migrate-to-api-gateway.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add validation for project name and URL

  • Update migration/migrate-to-k8s.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add validate_project_name for input
    • Use require_command kubectl

  • Update migration/migrate-to-monitoring.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add validation for project name and environment

  • Update migration/migrate-to-shared-packages.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add input validation

2.2 Update Metrics Scripts

  • Update all scripts in metrics/collect/

    • Add library initialization
    • Replace echo with logging functions
    • Standardize output format

  • Update metrics/track-all-metrics.sh

    • Add library initialization
    • Replace echo with logging functions
    • Use require_command jq if needed

  • Update metrics/track-success-metrics.sh

    • Add library initialization
    • Replace echo with logging functions

  • Update metrics/update-metrics.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add validation for metric category parameter

  • Update metrics/generate-metrics-report.sh

    • Add library initialization
    • Replace echo with logging functions
    • Use ensure_dir for output directories

2.3 Update DBIS Scripts

  • Update dbis/automate-dbis-migration.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add validation for project name

  • Update dbis/migrate-all-dbis-projects.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add error handling with set_error_trap

  • Update dbis/migrate-dbis-project.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add validate_project_name

  • Update dbis/test-dbis-migration.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add validation

2.4 Update Infrastructure Scripts

  • Update infrastructure/setup-shared-infrastructure.sh

    • Add library initialization
    • Replace echo with logging functions
    • Use require_command for required tools
    • Add set_error_trap for error handling

  • Update infrastructure/setup.sh

    • Add library initialization
    • Replace check_tool function with require_command
    • Replace echo with logging functions
    • Use log_success and log_failure for status

2.5 Update Utility Scripts

  • Update utils/analyze-costs.sh

    • Add library initialization
    • Replace echo with logging functions

  • Update utils/optimize-builds.sh

    • Add library initialization
    • Replace echo with logging functions

  • Update utils/deps-analyze.sh

    • Add library initialization
    • Replace echo with logging functions
    • Use require_command for required tools

  • Update utils/deps-audit.sh

    • Add library initialization
    • Replace echo with logging functions
    • Use require_command for required tools

  • Update utils/build-all.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add error handling

  • Update utils/test-all.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add error handling

  • Update utils/verify-all.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add error handling

  • Update utils/cleanup.sh

    • Add library initialization
    • Replace echo with logging functions
    • Add confirmation prompt using confirm

Phase 3: Create Backward Compatibility

3.1 Create Wrapper Scripts (Optional - for backward compatibility)

Create wrapper scripts in root scripts/ directory that call the new locations:

  • Create scripts/migrate-to-k8s.sh wrapper

    #!/bin/bash
# Wrapper for backward compatibility
# This script has moved to scripts/migration/migrate-to-k8s.sh
exec "$(dirname "$0")/migration/migrate-to-k8s.sh" "$@"

  • Create wrapper for each migrated script

    • migrate-readme.sh → migration/migrate-readme.sh
    • migrate-terraform.sh → migration/migrate-terraform.sh
    • migrate-to-api-gateway.sh → migration/migrate-to-api-gateway.sh
    • migrate-to-monitoring.sh → migration/migrate-to-monitoring.sh
    • migrate-to-shared-packages.sh → migration/migrate-to-shared-packages.sh
    • All metrics scripts
    • All DBIS scripts
    • All infrastructure scripts
    • All utility scripts

Alternative: Update all references instead of creating wrappers (recommended)

Phase 4: Update Script References

4.1 Update Documentation

  • Update main README.md

    • Update script paths in examples
    • Update quick start guide
    • Update script references

  • Update QUICK_START_GUIDE.md

    • Update all script paths
    • Update examples

  • Update INTEGRATION_TASKS_LIST.md

    • Update script references if any

  • Update all migration guides in docs/

    • Search for script references
    • Update paths

4.2 Update CI/CD Workflows

  • Update .github/workflows/ci-pilot-template.yml

    • Update script paths if referenced

  • Update .github/workflows/publish-shared-packages.yml

    • Update script paths if referenced

  • Update .github/workflows/infrastructure-deploy.yml

    • Update script paths if referenced

  • Update .github/workflows/metrics-collection.yml

    • Update script paths for metrics scripts

4.3 Update Other Scripts

  • Search for script references in other scripts

    grep -r "scripts/migrate-to-k8s.sh" . --exclude-dir=node_modules
grep -r "scripts/collect-" . --exclude-dir=node_modules
grep -r "scripts/track-" . --exclude-dir=node_modules

  • Update found references

    • Update paths to new locations
    • Test after updates

Phase 5: Testing

5.1 Test Library Loading

  • Test lib/init.sh

    source scripts/lib/init.sh
log_info "Test message"
log_error "Test error"

  • Test each library individually

    • Test colors.sh
    • Test logging.sh
    • Test utils.sh
    • Test validation.sh
    • Test error-handling.sh
    • Test env.sh

5.2 Test Migration Scripts

  • Test migration/migrate-to-k8s.sh

    ./scripts/migration/migrate-to-k8s.sh test-project

  • Test migration/migrate-to-api-gateway.sh

    ./scripts/migration/migrate-to-api-gateway.sh test-service http://test-service:8080

  • Test all migration scripts with help flags

    • Verify help text displays correctly
    • Verify error messages are clear

5.3 Test Metrics Scripts

  • Test metrics/update-metrics.sh

    ./scripts/metrics/update-metrics.sh all

  • Test metrics/generate-metrics-report.sh

    ./scripts/metrics/generate-metrics-report.sh

  • Test each collection script

    • Verify they create output files
    • Verify output format is correct

5.4 Test DBIS Scripts

  • Test dbis/test-dbis-migration.sh

    ./scripts/dbis/test-dbis-migration.sh dbis_monorepo

  • Test all DBIS scripts with help flags

5.5 Test Infrastructure Scripts

  • Test infrastructure/setup.sh

    ./scripts/infrastructure/setup.sh

  • Test infrastructure/setup-shared-infrastructure.sh

    • Verify it checks for required tools
    • Verify error handling works

5.6 Test Utility Scripts

  • Test utils/build-all.sh
  • Test utils/test-all.sh
  • Test utils/verify-all.sh
  • Test utils/cleanup.sh (with confirmation)

5.7 Integration Testing

  • Test script calling other scripts

    • Verify paths work correctly
    • Verify error propagation

  • Test from project root

    • All scripts should work from project root
    • Paths should resolve correctly

Phase 6: Documentation Updates

6.1 Update Scripts README

  • Add usage examples for each category

    • Migration examples
    • Metrics examples
    • DBIS examples
    • Infrastructure examples
    • Utility examples

  • Add library usage examples

    • Show how to use each library
    • Show common patterns

  • Add troubleshooting section

    • Common issues
    • Solutions

6.2 Create Migration Guide

  • Create docs/SCRIPTS_MIGRATION_GUIDE.md
    • Document the migration process
    • Document breaking changes
    • Document new paths
    • Document library usage

6.3 Update Main Documentation

  • Update README.md (project root)

    • Update script references
    • Update quick start examples

  • Update QUICK_START_GUIDE.md

    • Update all script paths
    • Update examples

  • Update PROJECT_INDEX.md

    • Update script counts
    • Update structure description

Phase 7: Quality Assurance

7.1 Code Quality

  • Lint all scripts

    find scripts -name "*.sh" -exec shellcheck {} \;

  • Fix any linting errors

    • Address shellcheck warnings
    • Fix syntax errors

  • Verify shebang lines

    • All scripts should have #!/bin/bash or #!/usr/bin/env bash
    • Make all scripts executable: chmod +x scripts/**/*.sh

7.2 Consistency Check

  • Verify all scripts use libraries

    • Check for scripts still using echo instead of logging
    • Check for scripts not using validation
    • Check for scripts not using error handling

  • Verify consistent patterns

    • All scripts should initialize libraries the same way
    • All scripts should have help text
    • All scripts should validate inputs

7.3 Performance Check

  • Test script execution time
    • Compare before/after library loading
    • Verify no significant performance impact

Phase 8: Final Verification

8.1 Complete Verification Checklist

  • All scripts moved to correct directories
  • All scripts updated to use libraries
  • All scripts tested and working
  • All documentation updated
  • All references updated
  • No broken links or paths
  • All scripts executable
  • No linting errors
  • Backward compatibility verified (if using wrappers)

8.2 Create Verification Script

  • Create scripts/verify-structure.sh 
    #!/bin/bash
# Verify scripts directory structure

# Check all directories exist
# Check all scripts are in correct locations
# Check all scripts are executable
# Check all scripts can load libraries

8.3 Run Final Tests

  • Run all scripts with --help or no args

    • Verify help text displays
    • Verify no errors

  • Run integration tests

    • Test complete workflows
    • Verify end-to-end functionality

📊 Progress Tracking

Current Status

  • ✅ Directory structure: Complete
  • ✅ Shared libraries: Complete
  • ✅ Documentation (structure): Complete
  • ⏳ Script migration: 0/30 (0%)
  • ⏳ Script updates: 0/30 (0%)
  • ⏳ Testing: 0%
  • ⏳ Documentation updates: 0%

Estimated Time

  • Phase 1 (Migration): 1-2 hours
  • Phase 2 (Updates): 4-6 hours
  • Phase 3 (Compatibility): 1 hour (optional)
  • Phase 4 (References): 2-3 hours
  • Phase 5 (Testing): 3-4 hours
  • Phase 6 (Documentation): 2-3 hours
  • Phase 7 (QA): 1-2 hours
  • Phase 8 (Verification): 1 hour

Total Estimated Time: 15-22 hours

🎯 Priority Order

High Priority (Do First)

  1. Phase 1: Script Migration
  2. Phase 2: Update Scripts (at least library initialization)
  3. Phase 5: Basic Testing

Medium Priority

  1. Phase 4: Update References
  2. Phase 6: Documentation Updates

Low Priority (Can be done later)

  1. Phase 3: Backward Compatibility (if needed)
  2. Phase 7: Quality Assurance
  3. Phase 8: Final Verification

🚀 Quick Start Commands

Move All Scripts at Once

cd /home/intlc/projects/scripts

# Migration scripts
mv migrate-readme.sh migrate-terraform.sh migrate-to-*.sh migration/

# Metrics scripts
mv collect-*.sh metrics/collect/
mv track-*.sh update-metrics.sh generate-metrics-report.sh metrics/

# DBIS scripts
mv *dbis*.sh dbis/

# Infrastructure scripts
mv setup*.sh infrastructure/

# Utility scripts
mv analyze-costs.sh optimize-builds.sh deps-*.sh build-all.sh test-all.sh verify-all.sh cleanup.sh utils/

Update All Scripts to Use Libraries

# Add to top of each script (after shebang):
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../lib/init.sh"

Make All Scripts Executable

find scripts -name "*.sh" -exec chmod +x {} \;

Status: Ready for Implementation Next Action: Begin Phase 1 - Script Migration

Reference in New Issue View Git Blame Copy Permalink