Sankofa/docs/proxmox/AUTHENTICATION_ERROR_FIX.md

Authentication Error Fix - "invalid PVE ticket"

Date: 2025-12-13
Status: 🔧 FIXING

---

Issue

Symptom

• Error: "401 permission denied - invalid PVE ticket"
• Frequency: Intermittent to continuous
• Impact: BLOCKS ALL VM CREATION
• Affected: Both ML110-01 and R630-01 nodes

Error Details

GET /nodes/r630-01/status failed: 401 permission denied - invalid PVE ticket
node r630-01 is not reachable or unhealthy

---

Root Cause Analysis

Previous Fix Applied

• Changed from req.AddCookie() to req.Header.Set("Cookie", ...)
• Reason: AddCookie() automatically URL-encodes values, breaking token format

Current Status

• Code fix is in place ✅
• Provider rebuilt and restarted ✅
• But errors still occurring ⚠️

Possible Causes

1. Provider not using latest code - Pod may be using old image
2. Token format issue - Token may not be in correct format
3. Token validity - Token may be expired or invalid
4. Cookie header format - May need different format

---

Investigation Steps

1. Verify Token Validity

• Test token manually with curl
• Verify token works with both endpoints
• Check token format matches Proxmox expectations

2. Verify Code Fix

• Check if Cookie header fix is in code
• Verify provider is using latest image
• Ensure provider pod restarted with new image

3. Test Authentication

• Test token with exact format from controller
• Verify Cookie header is set correctly
• Check HTTP request format

---

Fix Applied

Code Changes

• File: crossplane-provider-proxmox/pkg/proxmox/http_client.go
• Change: req.AddCookie() → req.Header.Set("Cookie", fmt.Sprintf("PVEAuthCookie=%s", c.token))

Provider Rebuild

• Rebuilt provider image
• Loaded into kind cluster
• Restarted provider pod

---

Verification

After Fix

• Check authentication errors (should be 0)
• Verify node health checks pass
• Monitor VM creation progress

---

Next Steps

1. ✅ Verify token is valid
2. ✅ Test token manually
3. ✅ Rebuild provider with latest code
4. ✅ Restart provider pod
5. ⏳ Monitor for resolution

---

Last Updated: 2025-12-13
Status: 🔧 FIXING - VERIFYING RESULTS