Apply Composer changes: comprehensive API updates, migrations, middleware, and infrastructure improvements
- Add comprehensive database migrations (001-024) for schema evolution - Enhance API schema with expanded type definitions and resolvers - Add new middleware: audit logging, rate limiting, MFA enforcement, security, tenant auth - Implement new services: AI optimization, billing, blockchain, compliance, marketplace - Add adapter layer for cloud integrations (Cloudflare, Kubernetes, Proxmox, storage) - Update Crossplane provider with enhanced VM management capabilities - Add comprehensive test suite for API endpoints and services - Update frontend components with improved GraphQL subscriptions and real-time updates - Enhance security configurations and headers (CSP, CORS, etc.) - Update documentation and configuration files - Add new CI/CD workflows and validation scripts - Implement design system improvements and UI enhancements
This commit is contained in:
134
docs/FORCE_UNLOCK_INSTRUCTIONS.md
Normal file
134
docs/FORCE_UNLOCK_INSTRUCTIONS.md
Normal file
@@ -0,0 +1,134 @@
|
||||
# Force Unlock VM Instructions
|
||||
|
||||
**Date**: 2025-12-09
|
||||
**Issue**: `qm unlock 100` is timing out
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
The `qm unlock` command is timing out, which indicates:
|
||||
- A stuck process is holding the lock
|
||||
- The lock file is corrupted or in an invalid state
|
||||
- Another operation is blocking the unlock
|
||||
|
||||
---
|
||||
|
||||
## Solution: Force Unlock
|
||||
|
||||
### Option 1: Use the Script (Recommended)
|
||||
|
||||
**On Proxmox Node (root@ml110-01)**:
|
||||
|
||||
```bash
|
||||
# Copy the script to the Proxmox node
|
||||
# Or run commands manually (see Option 2)
|
||||
|
||||
# Run the script
|
||||
bash force-unlock-vm-proxmox.sh 100
|
||||
```
|
||||
|
||||
### Option 2: Manual Commands
|
||||
|
||||
**On Proxmox Node (root@ml110-01)**:
|
||||
|
||||
```bash
|
||||
# 1. Check for stuck processes
|
||||
ps aux | grep -E 'qm|qemu' | grep 100
|
||||
|
||||
# 2. Check lock file
|
||||
ls -la /var/lock/qemu-server/lock-100.conf
|
||||
cat /var/lock/qemu-server/lock-100.conf 2>/dev/null
|
||||
|
||||
# 3. Kill stuck processes (if found)
|
||||
pkill -9 -f 'qm.*100'
|
||||
pkill -9 -f 'qemu.*100'
|
||||
|
||||
# 4. Wait a moment
|
||||
sleep 2
|
||||
|
||||
# 5. Force remove lock file
|
||||
rm -f /var/lock/qemu-server/lock-100.conf
|
||||
|
||||
# 6. Verify lock is gone
|
||||
ls -la /var/lock/qemu-server/lock-100.conf
|
||||
# Should show: No such file or directory
|
||||
|
||||
# 7. Check VM status
|
||||
qm status 100
|
||||
|
||||
# 8. Try unlock again (should work now)
|
||||
qm unlock 100
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## If Lock Persists
|
||||
|
||||
### Check for Other Issues
|
||||
|
||||
```bash
|
||||
# Check if VM is in a transitional state
|
||||
qm status 100
|
||||
|
||||
# Check VM configuration
|
||||
qm config 100
|
||||
|
||||
# Check for other locks
|
||||
ls -la /var/lock/qemu-server/lock-*.conf
|
||||
|
||||
# Check system resources
|
||||
df -h
|
||||
free -h
|
||||
```
|
||||
|
||||
### Nuclear Option: Restart Proxmox Services
|
||||
|
||||
**⚠️ WARNING: This will affect all VMs on the node**
|
||||
|
||||
```bash
|
||||
# Only if absolutely necessary
|
||||
systemctl restart pve-cluster
|
||||
systemctl restart pvedaemon
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## After Successful Unlock
|
||||
|
||||
1. **Monitor VM Status**:
|
||||
```bash
|
||||
qm status 100
|
||||
```
|
||||
|
||||
2. **Check Provider Logs** (from Kubernetes):
|
||||
```bash
|
||||
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox --tail=50 -f
|
||||
```
|
||||
|
||||
3. **Watch VM Resource**:
|
||||
```bash
|
||||
kubectl get proxmoxvm basic-vm-001 -w
|
||||
```
|
||||
|
||||
4. **Expected Outcome**:
|
||||
- Provider will retry within 1 minute
|
||||
- VM configuration will complete
|
||||
- VM will boot successfully
|
||||
|
||||
---
|
||||
|
||||
## Prevention
|
||||
|
||||
To prevent this issue in the future:
|
||||
|
||||
1. **Ensure proper VM shutdown** before operations
|
||||
2. **Wait for operations to complete** before starting new ones
|
||||
3. **Monitor for stuck processes** regularly
|
||||
4. **Implement lock timeout handling** in provider code (already added)
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-12-09
|
||||
**Status**: ⚠️ **MANUAL FORCE UNLOCK REQUIRED**
|
||||
|
||||
Reference in New Issue
Block a user