Sankofa/docs/vm/MONITORING_SUMMARY.md

VM Deployment Monitoring Summary

Date: 2025-12-13
Status: 🟡 MONITORING IN PROGRESS - AUTHENTICATION ISSUE DETECTED

---

Current Status

Provider

• Status: Running (1/1 Ready)
• Image: crossplane-provider-proxmox:latest
• Token Auth: Detected in logs ("Using token authentication")
• Issue: Still getting "invalid PVE ticket" errors

VMs

• Total: 30
• Deployed in K8s: 30
• Created on Proxmox: 0 (blocked by authentication errors)

---

Issue Identified

Authentication Errors

Symptom: "401 permission denied - invalid PVE ticket"
Frequency: All node health checks failing
Impact: VM creation blocked - cannot proceed past health check

Root Cause Analysis

1. Token authentication is being detected in logs ✅
2. Token format appears correct: tokenid=token ✅
3. Cookie header implementation is correct ✅
4. But API calls still fail with "invalid PVE ticket" ❌

Possible Causes

1. Token might be expired or invalid
2. Token permissions might be insufficient
3. Token format might need adjustment
4. Provider might not be using latest code (unlikely - pod restarted)

---

Monitoring Commands

Real-Time Monitoring

# Watch all VMs
kubectl get proxmoxvm -A -w

# Continuous monitor
./scripts/continuous-monitor.sh 30

# View provider logs
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox -f

Check Authentication Status

# Count authentication errors
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox --since=5m | grep -i "invalid PVE ticket" | wc -l

# Check token authentication usage
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox --since=5m | grep "Using token authentication" | wc -l

---

Next Steps

1. ⚠️ URGENT: Fix token authentication issue

   • Verify token is valid and not expired
   • Check token permissions
   • Test token manually with curl
   • Verify token format is correct

2. Monitor VM creation once authentication is fixed

3. Verify VMs are being created on Proxmox

4. Post-deployment verification

---

Last Updated: 2025-12-13
Status: 🟡 MONITORING - AUTHENTICATION ISSUE BLOCKING DEPLOYMENT