proxmox/scripts/DEPLOYMENT_README_R630-01.md

# Sankofa & Phoenix Deployment Guide for r630-01

**Target Server:** r630-01 (192.168.11.11)  
**Deployment Date:** $(date +%Y-%m-%d)  
**Status:** Ready for Deployment

---

## Overview

This guide provides step-by-step instructions for deploying Sankofa and Phoenix control plane services to r630-01 Proxmox node.

### Architecture

```
r630-01 (192.168.11.11)
├── VMID 7803: PostgreSQL (10.160.0.13)
├── VMID 7802: Keycloak (10.160.0.12)
├── VMID 7800: Sankofa API (10.160.0.10)
└── VMID 7801: Sankofa Portal (10.160.0.11)
```

### Network Configuration

- **VLAN:** 160
- **Subnet:** 10.160.0.0/22
- **Gateway:** 10.160.0.1
- **Storage:** thin1 (208GB available)

---

## Prerequisites

1. **SSH Access to r630-01**
   ```bash
   ssh root@192.168.11.11
   ```

2. **Sankofa Project Available**
   - Location: `/home/intlc/projects/Sankofa`
   - Must contain `api/` and `portal/` directories

3. **Proxmox Storage**
   - Verify `thin1` storage is available
   - Check available space: `pvesm status`

4. **Network Configuration**
   - Verify VLAN 160 is configured
   - Verify gateway (10.160.0.1) is accessible

---

## Deployment Steps

### Step 1: Prepare Configuration

1. Copy environment template:
   ```bash
   cd /home/intlc/projects/proxmox/scripts
   cp env.r630-01.example .env.r630-01
   ```

2. Edit `.env.r630-01` and update:
   - Database passwords
   - Keycloak admin password
   - Client secrets
   - JWT secrets
   - Any other production values

### Step 2: Deploy Containers

Deploy all LXC containers:

```bash
cd /home/intlc/projects/proxmox/scripts
./deploy-sankofa-r630-01.sh
```

This will create:
- PostgreSQL container (VMID 7803)
- Keycloak container (VMID 7802)
- API container (VMID 7800)
- Portal container (VMID 7801)

### Step 3: Setup PostgreSQL

Configure PostgreSQL database:

```bash
./setup-postgresql-r630-01.sh
```

This will:
- Install PostgreSQL 16
- Create `sankofa` database
- Create `sankofa` user
- Configure network access
- Enable required extensions

**Note:** The script will generate a random password. Update `.env.r630-01` with the actual password.

### Step 4: Setup Keycloak

Configure Keycloak identity service:

```bash
./setup-keycloak-r630-01.sh
```

This will:
- Install Java 21
- Download and install Keycloak 24.0.0
- Create Keycloak database
- Configure PostgreSQL connection
- Create admin user
- Create API and Portal clients

**Note:** The script will generate random passwords and secrets. Update `.env.r630-01` with the actual values.

### Step 5: Deploy API

Deploy Sankofa API service:

```bash
./deploy-api-r630-01.sh
```

This will:
- Install Node.js 18
- Install pnpm
- Copy API project files
- Install dependencies
- Configure environment
- Run database migrations
- Build API
- Create systemd service
- Start API service

### Step 6: Run Database Migrations

If migrations weren't run during API deployment:

```bash
./run-migrations-r630-01.sh
```

### Step 7: Deploy Portal

Deploy Sankofa Portal:

```bash
./deploy-portal-r630-01.sh
```

This will:
- Install Node.js 18
- Install pnpm
- Copy Portal project files
- Install dependencies
- Configure environment
- Build Portal (Next.js)
- Create systemd service
- Start Portal service

---

## Verification

### Check Container Status

```bash
ssh root@192.168.11.11 "pct list | grep -E '780[0-3]'"
```

### Check Service Status

**PostgreSQL:**
```bash
ssh root@192.168.11.11 "pct exec 7803 -- systemctl status postgresql"
```

**Keycloak:**
```bash
ssh root@192.168.11.11 "pct exec 7802 -- systemctl status keycloak"
curl http://10.160.0.12:8080/health/ready
```

**API:**
```bash
ssh root@192.168.11.11 "pct exec 7800 -- systemctl status sankofa-api"
curl http://10.160.0.10:4000/health
```

**Portal:**
```bash
ssh root@192.168.11.11 "pct exec 7801 -- systemctl status sankofa-portal"
curl http://10.160.0.11:3000
```

### Test GraphQL Endpoint

```bash
curl -X POST http://10.160.0.10:4000/graphql \
  -H "Content-Type: application/json" \
  -d '{"query": "{ __typename }"}'
```

---

## Service URLs

| Service | URL | Description |
|---------|-----|-------------|
| PostgreSQL | `10.160.0.13:5432` | Database |
| Keycloak | `http://10.160.0.12:8080` | Identity Provider |
| Keycloak Admin | `http://10.160.0.12:8080/admin` | Admin Console |
| API | `http://10.160.0.10:4000` | GraphQL API |
| API GraphQL | `http://10.160.0.10:4000/graphql` | GraphQL Endpoint |
| API Health | `http://10.160.0.10:4000/health` | Health Check |
| Portal | `http://10.160.0.11:3000` | Web Portal |

---

## Troubleshooting

### Container Won't Start

```bash
# Check container status
ssh root@192.168.11.11 "pct status 7800"

# Check container logs
ssh root@192.168.11.11 "pct exec 7800 -- journalctl -n 50"
```

### Database Connection Issues

```bash
# Test database connection from API container
ssh root@192.168.11.11 "pct exec 7800 -- bash -c 'PGPASSWORD=your_password psql -h 10.160.0.13 -U sankofa -d sankofa -c \"SELECT 1;\"'"
```

### Keycloak Not Starting

```bash
# Check Keycloak logs
ssh root@192.168.11.11 "pct exec 7802 -- journalctl -u keycloak -n 100"

# Check Keycloak process
ssh root@192.168.11.11 "pct exec 7802 -- ps aux | grep keycloak"
```

### API Service Issues

```bash
# Check API logs
ssh root@192.168.11.11 "pct exec 7800 -- journalctl -u sankofa-api -n 100"

# Restart API service
ssh root@192.168.11.11 "pct exec 7800 -- systemctl restart sankofa-api"
```

### Portal Build Failures

```bash
# Check build logs
ssh root@192.168.11.11 "pct exec 7801 -- journalctl -u sankofa-portal -n 100"

# Rebuild Portal
ssh root@192.168.11.11 "pct exec 7801 -- bash -c 'cd /opt/sankofa-portal && pnpm build'"
```

---

## Post-Deployment Tasks

1. **Update Environment Variables**
   - Update `.env.r630-01` with actual passwords and secrets
   - Update service configurations if needed

2. **Configure Firewall Rules**
   - Allow access to service ports
   - Configure VLAN 160 routing if needed

3. **Set Up Cloudflare Tunnels**
   - Configure tunnels for external access
   - Set up DNS records

4. **Configure Monitoring**
   - Set up Prometheus exporters
   - Configure Grafana dashboards
   - Set up alerting

5. **Backup Configuration**
   - Document all passwords and secrets
   - Create backup procedures
   - Test restore procedures

---

## Maintenance

### Update Services

**Update API:**
```bash
ssh root@192.168.11.11 "pct exec 7800 -- bash -c 'cd /opt/sankofa-api && git pull && pnpm install && pnpm build && systemctl restart sankofa-api'"
```

**Update Portal:**
```bash
ssh root@192.168.11.11 "pct exec 7801 -- bash -c 'cd /opt/sankofa-portal && git pull && pnpm install && pnpm build && systemctl restart sankofa-portal'"
```

### Backup Database

```bash
ssh root@192.168.11.11 "pct exec 7803 -- bash -c 'PGPASSWORD=your_password pg_dump -h localhost -U sankofa sankofa > /tmp/sankofa_backup_$(date +%Y%m%d).sql'"
```

### View Logs

**API Logs:**
```bash
ssh root@192.168.11.11 "pct exec 7800 -- journalctl -u sankofa-api -f"
```

**Portal Logs:**
```bash
ssh root@192.168.11.11 "pct exec 7801 -- journalctl -u sankofa-portal -f"
```

---

## Support

For issues or questions:
1. Check logs using troubleshooting commands above
2. Review deployment scripts for configuration
3. Verify network connectivity between containers
4. Check Proxmox storage and resource availability

---

**Last Updated:** $(date +%Y-%m-%d)