# Architecture Overview ## System Architecture This document describes the complete architecture of the Proxmox VE → Azure Arc → Hybrid Cloud Stack implementation. ## High-Level Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ Azure Portal │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Azure Arc │ │ Azure Policy │ │ Azure Monitor │ │ │ │ Servers │ │ │ │ │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Arc K8s │ │ GitOps │ │ Defender │ │ │ │ │ │ (Flux) │ │ for Cloud │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └─────────────────────────────────────────────────────────────────┘ │ │ Azure Arc Connection │ ┌─────────────────────────────────────────────────────────────────┐ │ On-Premises Infrastructure │ │ │ │ ┌──────────────────────────────────────────────────────────┐ │ │ │ Proxmox VE Cluster (2 Nodes) │ │ │ │ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ PVE Node 1 │◄────────────►│ PVE Node 2 │ │ │ │ │ │ │ Cluster │ │ │ │ │ │ │ Azure Arc │ Network │ Azure Arc │ │ │ │ │ │ Agent │ │ Agent │ │ │ │ │ └──────────────┘ └──────────────┘ │ │ │ │ │ │ │ │ │ │ └───────────┬───────────────┘ │ │ │ │ │ │ │ │ │ ┌──────▼──────┐ │ │ │ │ │ NFS Storage │ │ │ │ │ │ (Shared) │ │ │ │ │ └─────────────┘ │ │ │ └──────────────────────────────────────────────────────────┘ │ │ │ │ ┌──────────────────────────────────────────────────────────┐ │ │ │ Proxmox VMs │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ K3s VM │ │ Git Server │ │ Other VMs │ │ │ │ │ │ │ │ (Gitea/ │ │ │ │ │ │ │ │ Azure Arc │ │ GitLab) │ │ Azure Arc │ │ │ │ │ │ K8s │ │ │ │ Agents │ │ │ │ │ │ Resource │ │ │ │ │ │ │ │ │ │ Bridge │ │ │ │ │ │ │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ │ └──────────────────────────────────────────────────────────┘ │ │ │ │ ┌──────────────────────────────────────────────────────────┐ │ │ │ Kubernetes Cluster (K3s) │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ Ingress │ │ Cert- │ │ GitOps │ │ │ │ │ │ Controller │ │ Manager │ │ (Flux) │ │ │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ │ │ │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ Besu │ │ Firefly │ │ Chainlink │ │ │ │ │ │ (Ethereum) │ │ (Middleware)│ │ CCIP │ │ │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ │ │ │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ Blockscout │ │ Cacti │ │ NGINX │ │ │ │ │ │ (Explorer) │ │ (Monitoring) │ │ Proxy │ │ │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ │ └──────────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────┘ ``` ## Component Details ### 1. Proxmox VE Cluster **Purpose**: Hypervisor layer providing virtualization and high availability **Components**: - 2 Proxmox nodes in cluster configuration - Shared NFS storage for VM data - Linux bridge networking (vmbr0) - Corosync for cluster communication **Features**: - High availability (HA) for VMs - Live migration between nodes - Centralized management via web UI - Azure Arc integration for portal visibility ### 2. Azure Arc Integration **Purpose**: Extend Azure management capabilities to on-premises infrastructure **Components**: - **Azure Connected Machine Agent**: Installed on Proxmox hosts and VMs - **Azure Arc Kubernetes**: K3s cluster onboarded to Azure Arc - **Resource Bridge**: Custom Kubernetes-based bridge for VM lifecycle control - **GitOps Extension**: Flux-based GitOps for declarative deployments **Capabilities**: - VM visibility in Azure Portal - Azure Policy enforcement - Azure Update Manager - Defender for Cloud - Azure Monitor integration - GitOps-based deployments ### 3. Kubernetes (K3s) **Purpose**: Container orchestration platform for HC Stack services **Components**: - K3s lightweight Kubernetes distribution - NGINX Ingress Controller - Cert-Manager for TLS certificates - Flux GitOps operator **Namespaces**: - `hc-stack`: Core infrastructure - `blockchain`: Blockchain services (Besu, Firefly, Chainlink) - `monitoring`: Monitoring tools (Cacti) - `ingress-nginx`: Ingress controller - `cert-manager`: Certificate management ### 4. Hybrid Cloud Stack Services #### Hyperledger Besu - Ethereum client for blockchain operations - RPC endpoints (HTTP/WebSocket) - P2P networking - Metrics and monitoring #### Hyperledger Firefly - Blockchain middleware and API layer - Multi-party system support - Token and asset management - Event streaming #### Chainlink CCIP - Cross-chain interoperability protocol - Oracle services - Secure cross-chain messaging #### Blockscout - Blockchain explorer - Transaction and block visualization - Contract verification - Analytics dashboard #### Cacti - Network monitoring and graphing - Performance metrics - Alerting capabilities #### NGINX Proxy - Reverse proxy for all services - Load balancing - SSL termination ### 5. Private Git/DevOps **Options**: - **Gitea**: Lightweight Git server (recommended for small deployments) - **GitLab CE**: Full-featured DevOps platform - **Azure DevOps**: Self-hosted agents for Azure DevOps pipelines **Purpose**: - Version control for infrastructure and applications - CI/CD pipeline execution - GitOps repository for Kubernetes deployments ## Data Flow 1. **Infrastructure Management**: - Terraform → Proxmox API → VM Creation - Azure Arc Agent → Azure Portal → Visibility & Management 2. **Application Deployment**: - Git Repository → Flux GitOps → Kubernetes API → Pod Deployment - Azure Arc GitOps → Flux → Kubernetes → Application Updates 3. **Monitoring & Observability**: - Services → Metrics → Azure Monitor / Cacti - Logs → Azure Log Analytics / Local Storage ## Security Architecture - **Network Isolation**: Separate networks for management, storage, and application traffic - **Azure Arc Security**: Managed identities and RBAC - **Kubernetes Security**: RBAC, network policies, pod security policies - **TLS/SSL**: Cert-Manager for automatic certificate management - **Secrets Management**: Kubernetes secrets (consider Azure Key Vault integration) ## High Availability - **Proxmox Cluster**: 2-node cluster with shared storage - **VM HA**: Automatic failover for VMs - **Kubernetes**: Multiple replicas for stateless services - **Storage**: NFS shared storage for persistent data - **Load Balancing**: NGINX Ingress for service distribution ## Scalability - **Horizontal Scaling**: Add more Proxmox nodes to cluster - **Kubernetes Scaling**: Add worker nodes to K3s cluster - **Application Scaling**: Kubernetes HPA for automatic scaling - **Storage Scaling**: Expand NFS storage as needed ## Integration Points 1. **Azure Portal**: Full visibility and management 2. **Git Repository**: Source of truth for infrastructure and applications 3. **Kubernetes API**: Application deployment and management 4. **Proxmox API**: VM lifecycle management 5. **Monitoring Systems**: Metrics and alerting