144 lines
2.7 KiB
Markdown
144 lines
2.7 KiB
Markdown
# Ceph OSD Creation Issues - Summary
|
|
|
|
**Date**: 2025-12-13
|
|
**Status**: 🔍 **ISSUES IDENTIFIED - NEEDS DIAGNOSIS**
|
|
|
|
---
|
|
|
|
## Issues Identified
|
|
|
|
### 1. OSD Creation Hanging/Timeout
|
|
|
|
**Symptom**: `ceph-volume lvm create` hangs at "osd new" step
|
|
|
|
**Root Causes** (likely):
|
|
- ❌ **Bootstrap keyring missing** - Cannot authenticate with cluster
|
|
- ❌ **Ceph cluster not accessible** - RADOS timeout errors
|
|
- ❌ **Monitor connectivity issues** - Cannot reach Ceph monitors
|
|
|
|
### 2. Bootstrap Keyring Issue
|
|
|
|
**Error**:
|
|
```
|
|
unable to find a keyring on /etc/pve/priv/ceph.client.bootstrap-osd.keyring
|
|
RADOS timed out (error connecting to the cluster)
|
|
```
|
|
|
|
**Possible Locations**:
|
|
- `/var/lib/ceph/bootstrap-osd/ceph.keyring` (standard)
|
|
- `/etc/pve/priv/ceph.client.bootstrap-osd.keyring` (Proxmox)
|
|
|
|
**Status**: Unknown - needs verification
|
|
|
|
### 3. Ceph Cluster Connectivity
|
|
|
|
**Issue**: Commands timeout when trying to connect to cluster
|
|
|
|
**Possible Causes**:
|
|
- Monitors not running
|
|
- Network connectivity issues
|
|
- Firewall blocking ports
|
|
- Cluster configuration issues
|
|
|
|
---
|
|
|
|
## Diagnostic Steps
|
|
|
|
Run these commands on R630-01 to diagnose:
|
|
|
|
### Quick Check
|
|
|
|
```bash
|
|
# 1. Check Ceph health (does it respond?)
|
|
ceph health
|
|
|
|
# 2. Check monitors
|
|
ceph mon stat
|
|
|
|
# 3. Check bootstrap keyring
|
|
ls -la /var/lib/ceph/bootstrap-osd/ceph.keyring
|
|
ls -la /etc/pve/priv/ceph.client.bootstrap-osd.keyring
|
|
|
|
# 4. Check if pveceph available
|
|
which pveceph
|
|
```
|
|
|
|
### Full Diagnostic
|
|
|
|
```bash
|
|
# Run comprehensive diagnostic
|
|
bash /tmp/diagnose-ceph-issues.sh
|
|
```
|
|
|
|
---
|
|
|
|
## Likely Solutions
|
|
|
|
### Solution 1: Use pveceph (If Proxmox-Managed)
|
|
|
|
If `pveceph` is available:
|
|
|
|
```bash
|
|
for drive in sdc sdd sde sdf sdg sdh; do
|
|
pveceph create /dev/$drive
|
|
done
|
|
```
|
|
|
|
### Solution 2: Fix Bootstrap Keyring
|
|
|
|
If cluster is accessible:
|
|
|
|
```bash
|
|
# Get bootstrap keyring
|
|
mkdir -p /var/lib/ceph/bootstrap-osd
|
|
ceph auth get client.bootstrap-osd -o /var/lib/ceph/bootstrap-osd/ceph.keyring
|
|
|
|
# Then create OSDs
|
|
for drive in sdc sdd sde sdf sdg sdh; do
|
|
ceph-volume lvm create --data /dev/$drive
|
|
done
|
|
```
|
|
|
|
### Solution 3: Fix Cluster Connectivity
|
|
|
|
If cluster is not accessible:
|
|
|
|
1. Check monitor services
|
|
2. Check network connectivity
|
|
3. Check firewall rules
|
|
4. Fix connectivity issues
|
|
5. Then retry OSD creation
|
|
|
|
---
|
|
|
|
## Current State
|
|
|
|
- ✅ **6x 250GB drives**: Wiped and ready
|
|
- ❌ **OSD creation**: Hanging due to authentication/connectivity
|
|
- 🔍 **Root cause**: Needs diagnosis
|
|
|
|
---
|
|
|
|
## Next Action
|
|
|
|
**Run diagnostic script** to understand the issue:
|
|
|
|
```bash
|
|
# On R630-01
|
|
bash /tmp/diagnose-ceph-issues.sh
|
|
```
|
|
|
|
This will show:
|
|
- Ceph cluster status
|
|
- Bootstrap keyring locations
|
|
- Monitor connectivity
|
|
- Service status
|
|
- Network connectivity
|
|
|
|
Then we can apply the appropriate fix.
|
|
|
|
---
|
|
|
|
**Last Updated**: 2025-12-13
|
|
|