Apply Composer changes: comprehensive API updates, migrations, middleware, and infrastructure improvements

- Add comprehensive database migrations (001-024) for schema evolution - Enhance API schema with expanded type definitions and resolvers - Add new middleware: audit logging, rate limiting, MFA enforcement, security, tenant auth - Implement new services: AI optimization, billing, blockchain, compliance, marketplace - Add adapter layer for cloud integrations (Cloudflare, Kubernetes, Proxmox, storage) - Update Crossplane provider with enhanced VM management capabilities - Add comprehensive test suite for API endpoints and services - Update frontend components with improved GraphQL subscriptions and real-time updates - Enhance security configurations and headers (CSP, CORS, etc.) - Update documentation and configuration files - Add new CI/CD workflows and validation scripts - Implement design system improvements and UI enhancements
2025-12-12 18:01:35 -08:00
parent e01131efaf
commit 9daf1fd378
968 changed files with 160890 additions and 1092 deletions
--- a/infrastructure/inventory/README.md
+++ b/infrastructure/inventory/README.md
@@ -0,0 +1,222 @@
+# Infrastructure Inventory
+
+Centralized inventory and discovery system for all infrastructure components in Sankofa Phoenix.
+
+## Overview
+
+The infrastructure inventory system provides:
+- Auto-discovery of infrastructure components
+- Centralized inventory database
+- Asset tracking and lifecycle management
+- Configuration drift detection
+- Change history and audit trails
+
+## Components
+
+### Discovery (`discovery/`)
+
+Auto-discovery scripts for:
+- Proxmox clusters and nodes
+- Network devices (switches, routers)
+- Omada controllers and access points
+- Storage systems
+- Other infrastructure components
+
+### Database (`database/`)
+
+Inventory database schema and management:
+- PostgreSQL schema for inventory
+- Migration scripts
+- Query utilities
+- Backup/restore procedures
+
+## Discovery
+
+### Auto-Discovery
+
+```bash
+# Discover all infrastructure
+./discovery/discover-all.sh --site us-east-1
+
+# Discover Proxmox infrastructure
+./discovery/discover-proxmox.sh --site us-east-1
+
+# Discover network infrastructure
+./discovery/discover-network.sh --site us-east-1
+
+# Discover Omada infrastructure
+./discovery/discover-omada.sh --controller omada.sankofa.nexus
+```
+
+### Scheduled Discovery
+
+Discovery can be scheduled via cron or Kubernetes CronJob:
+
+```yaml
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: infrastructure-discovery
+spec:
+  schedule: "0 */6 * * *"  # Every 6 hours
+  jobTemplate:
+    spec:
+      template:
+        spec:
+          containers:
+          - name: discovery
+            image: infrastructure-discovery:latest
+            command: ["./discovery/discover-all.sh"]
+```
+
+## Database Schema
+
+### Tables
+
+- **sites**: Physical sites/locations
+- **nodes**: Compute nodes (Proxmox, Kubernetes)
+- **vms**: Virtual machines
+- **network_devices**: Switches, routers, access points
+- **storage_pools**: Storage systems
+- **networks**: Network segments and VLANs
+- **inventory_history**: Change history
+
+### Schema Location
+
+See `database/schema.sql` for complete database schema.
+
+## Usage
+
+### Query Inventory
+
+```bash
+# List all sites
+./database/query.sh "SELECT * FROM sites"
+
+# List nodes for a site
+./database/query.sh "SELECT * FROM nodes WHERE site_id = 'us-east-1'"
+
+# Get VM inventory
+./database/query.sh "SELECT * FROM vms WHERE site_id = 'us-east-1'"
+```
+
+### Update Inventory
+
+```bash
+# Update node information
+./database/update-node.sh \
+  --node pve1 \
+  --site us-east-1 \
+  --status online \
+  --cpu 32 \
+  --memory 128GB
+```
+
+### Configuration Drift Detection
+
+```bash
+# Detect configuration drift
+./discovery/detect-drift.sh --site us-east-1
+
+# Compare with expected configuration
+./discovery/compare-config.sh \
+  --site us-east-1 \
+  --expected expected-config.yaml
+```
+
+## Integration
+
+### API Integration
+
+The inventory system provides a REST API for integration:
+
+```bash
+# Get site inventory
+curl https://api.sankofa.nexus/inventory/sites/us-east-1
+
+# Get node details
+curl https://api.sankofa.nexus/inventory/nodes/pve1
+
+# Update inventory
+curl -X POST https://api.sankofa.nexus/inventory/nodes \
+  -H "Content-Type: application/json" \
+  -d '{"name": "pve1", "site": "us-east-1", ...}'
+```
+
+### Portal Integration
+
+The inventory is accessible via the Portal UI:
+- Infrastructure explorer
+- Asset management
+- Configuration comparison
+- Change history
+
+## Configuration
+
+### Discovery Configuration
+
+```yaml
+discovery:
+  sites:
+    - id: us-east-1
+      proxmox:
+        endpoints:
+          - https://pve1.sankofa.nexus:8006
+          - https://pve2.sankofa.nexus:8006
+      network:
+        snmp_community: public
+        devices:
+          - 10.1.0.1  # switch-01
+          - 10.1.0.254  # router-01
+      omada:
+        controller: omada.sankofa.nexus
+        site_id: us-east-1
+```
+
+### Database Configuration
+
+```yaml
+database:
+  host: postgres.inventory.svc.cluster.local
+  port: 5432
+  database: infrastructure
+  username: inventory
+  password: ${DB_PASSWORD}
+  ssl_mode: require
+```
+
+## Backup and Recovery
+
+### Backup Inventory
+
+```bash
+# Backup inventory database
+./database/backup.sh --output inventory-backup-$(date +%Y%m%d).sql
+```
+
+### Restore Inventory
+
+```bash
+# Restore inventory database
+./database/restore.sh --backup inventory-backup-20240101.sql
+```
+
+## Reporting
+
+### Generate Reports
+
+```bash
+# Generate inventory report
+./database/report.sh --site us-east-1 --format html
+
+# Generate asset report
+./database/asset-report.sh --format csv
+```
+
+## Related Documentation
+
+- [Proxmox Management](../proxmox/README.md)
+- [Omada Management](../omada/README.md)
+- [Network Management](../network/README.md)
+- [Infrastructure Management](../README.md)
+
--- a/infrastructure/inventory/database/schema.sql
+++ b/infrastructure/inventory/database/schema.sql
@@ -0,0 +1,133 @@
+-- Infrastructure Inventory Database Schema
+-- PostgreSQL schema for tracking infrastructure components
+
+-- Sites table
+CREATE TABLE IF NOT EXISTS sites (
+    id VARCHAR(50) PRIMARY KEY,
+    name VARCHAR(255) NOT NULL,
+    location VARCHAR(255),
+    timezone VARCHAR(50) DEFAULT 'UTC',
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Nodes table (Proxmox, Kubernetes, etc.)
+CREATE TABLE IF NOT EXISTS nodes (
+    id VARCHAR(50) PRIMARY KEY,
+    site_id VARCHAR(50) REFERENCES sites(id) ON DELETE CASCADE,
+    name VARCHAR(255) NOT NULL,
+    type VARCHAR(50) NOT NULL, -- 'proxmox', 'kubernetes', etc.
+    ip_address INET,
+    status VARCHAR(20) DEFAULT 'unknown', -- 'online', 'offline', 'maintenance'
+    cpu_cores INTEGER,
+    memory_gb INTEGER,
+    storage_gb INTEGER,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Virtual machines table
+CREATE TABLE IF NOT EXISTS vms (
+    id VARCHAR(50) PRIMARY KEY,
+    node_id VARCHAR(50) REFERENCES nodes(id) ON DELETE CASCADE,
+    site_id VARCHAR(50) REFERENCES sites(id) ON DELETE CASCADE,
+    name VARCHAR(255) NOT NULL,
+    vmid INTEGER,
+    status VARCHAR(20) DEFAULT 'unknown',
+    cpu_cores INTEGER,
+    memory_gb INTEGER,
+    disk_gb INTEGER,
+    ip_address INET,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Network devices table
+CREATE TABLE IF NOT EXISTS network_devices (
+    id VARCHAR(50) PRIMARY KEY,
+    site_id VARCHAR(50) REFERENCES sites(id) ON DELETE CASCADE,
+    name VARCHAR(255) NOT NULL,
+    type VARCHAR(50) NOT NULL, -- 'switch', 'router', 'access_point', 'gateway'
+    model VARCHAR(255),
+    ip_address INET,
+    mac_address MACADDR,
+    status VARCHAR(20) DEFAULT 'unknown',
+    firmware_version VARCHAR(50),
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Storage pools table
+CREATE TABLE IF NOT EXISTS storage_pools (
+    id VARCHAR(50) PRIMARY KEY,
+    site_id VARCHAR(50) REFERENCES sites(id) ON DELETE CASCADE,
+    name VARCHAR(255) NOT NULL,
+    type VARCHAR(50) NOT NULL, -- 'local', 'ceph', 'nfs', etc.
+    total_gb BIGINT,
+    used_gb BIGINT,
+    available_gb BIGINT,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Networks/VLANs table
+CREATE TABLE IF NOT EXISTS networks (
+    id VARCHAR(50) PRIMARY KEY,
+    site_id VARCHAR(50) REFERENCES sites(id) ON DELETE CASCADE,
+    name VARCHAR(255) NOT NULL,
+    vlan_id INTEGER,
+    subnet CIDR,
+    gateway INET,
+    description TEXT,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Inventory history table (for change tracking)
+CREATE TABLE IF NOT EXISTS inventory_history (
+    id SERIAL PRIMARY KEY,
+    table_name VARCHAR(50) NOT NULL,
+    record_id VARCHAR(50) NOT NULL,
+    action VARCHAR(20) NOT NULL, -- 'create', 'update', 'delete'
+    changes JSONB,
+    changed_by VARCHAR(255),
+    changed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Indexes
+CREATE INDEX IF NOT EXISTS idx_nodes_site_id ON nodes(site_id);
+CREATE INDEX IF NOT EXISTS idx_vms_node_id ON vms(node_id);
+CREATE INDEX IF NOT EXISTS idx_vms_site_id ON vms(site_id);
+CREATE INDEX IF NOT EXISTS idx_network_devices_site_id ON network_devices(site_id);
+CREATE INDEX IF NOT EXISTS idx_storage_pools_site_id ON storage_pools(site_id);
+CREATE INDEX IF NOT EXISTS idx_networks_site_id ON networks(site_id);
+CREATE INDEX IF NOT EXISTS idx_inventory_history_record ON inventory_history(table_name, record_id);
+
+-- Function to update updated_at timestamp
+CREATE OR REPLACE FUNCTION update_updated_at_column()
+RETURNS TRIGGER AS $$
+BEGIN
+    NEW.updated_at = CURRENT_TIMESTAMP;
+    RETURN NEW;
+END;
+$$ language 'plpgsql';
+
+-- Triggers for updated_at
+CREATE TRIGGER update_sites_updated_at BEFORE UPDATE ON sites
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
+CREATE TRIGGER update_nodes_updated_at BEFORE UPDATE ON nodes
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
+CREATE TRIGGER update_vms_updated_at BEFORE UPDATE ON vms
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
+CREATE TRIGGER update_network_devices_updated_at BEFORE UPDATE ON network_devices
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
+CREATE TRIGGER update_storage_pools_updated_at BEFORE UPDATE ON storage_pools
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
+CREATE TRIGGER update_networks_updated_at BEFORE UPDATE ON networks
+    FOR EACH ROW EXECUTE FUNCTION update_updated_at_column();
+
--- a/infrastructure/inventory/discovery/discover-all.sh
+++ b/infrastructure/inventory/discovery/discover-all.sh
@@ -0,0 +1,97 @@
+#!/bin/bash
+set -euo pipefail
+
+# Infrastructure Discovery Script
+# Discovers all infrastructure components for a site
+
+SITE="${SITE:-}"
+OUTPUT_DIR="${OUTPUT_DIR:-/tmp/infrastructure-inventory}"
+
+log() {
+    echo "[$(date +'%Y-%m-%d %H:%M:%S')] $*" >&2
+}
+
+error() {
+    log "ERROR: $*"
+    exit 1
+}
+
+check_prerequisites() {
+    if [ -z "${SITE}" ]; then
+        error "SITE environment variable is required"
+    fi
+    
+    mkdir -p "${OUTPUT_DIR}"
+}
+
+discover_proxmox() {
+    log "Discovering Proxmox infrastructure..."
+    
+    # Check if discovery script exists
+    if [ -f "../../proxmox/scripts/discover-cluster.sh" ]; then
+        ../../proxmox/scripts/discover-cluster.sh --site "${SITE}" > "${OUTPUT_DIR}/proxmox-${SITE}.json" 2>&1 || log "  ⚠️  Proxmox discovery failed"
+    else
+        log "  ⚠️  Proxmox discovery script not found"
+    fi
+}
+
+discover_omada() {
+    log "Discovering Omada infrastructure..."
+    
+    if [ -f "../../omada/scripts/discover-aps.sh" ]; then
+        ../../omada/scripts/discover-aps.sh --site "${SITE}" > "${OUTPUT_DIR}/omada-${SITE}.json" 2>&1 || log "  ⚠️  Omada discovery failed"
+    else
+        log "  ⚠️  Omada discovery script not found"
+    fi
+}
+
+discover_network() {
+    log "Discovering network infrastructure..."
+    
+    # Network discovery would use SNMP or other protocols
+    log "  ⚠️  Network discovery not yet implemented"
+}
+
+generate_inventory() {
+    log "Generating inventory report..."
+    
+    REPORT_FILE="${OUTPUT_DIR}/inventory-${SITE}-$(date +%Y%m%d-%H%M%S).json"
+    
+    cat > "${REPORT_FILE}" <<EOF
+{
+  "site": "${SITE}",
+  "discovery_date": "$(date -Iseconds)",
+  "components": {
+    "proxmox": {
+      "file": "proxmox-${SITE}.json",
+      "status": "$([ -f "${OUTPUT_DIR}/proxmox-${SITE}.json" ] && echo "discovered" || echo "failed")"
+    },
+    "omada": {
+      "file": "omada-${SITE}.json",
+      "status": "$([ -f "${OUTPUT_DIR}/omada-${SITE}.json" ] && echo "discovered" || echo "failed")"
+    },
+    "network": {
+      "status": "not_implemented"
+    }
+  }
+}
+EOF
+    
+    log "Inventory report generated: ${REPORT_FILE}"
+    cat "${REPORT_FILE}"
+}
+
+main() {
+    log "Starting infrastructure discovery for site: ${SITE}"
+    
+    check_prerequisites
+    discover_proxmox
+    discover_omada
+    discover_network
+    generate_inventory
+    
+    log "Discovery completed! Results in: ${OUTPUT_DIR}"
+}
+
+main "$@"
+