loc_az_hci/infrastructure/proxmox/README.md

# Proxmox Network Configuration

This directory contains scripts for configuring Proxmox VE networking on ML110 and R630 servers.

## Network Configuration Overview

Both Proxmox servers use a two-NIC setup:

- **NIC 1** → `vmbr0` (LAN Bridge)
  - Connected to 192.168.1.0/24 network
  - DHCP client for management network
  - Route metric: 200

- **NIC 2** → `vmbr1` (WAN Bridge)
  - Connected directly to Spectrum cable modem
  - DHCP client for public IP address
  - Route metric: 100 (preferred for default route)

## Scripts

### `validate-network-setup.sh`

Validation script that checks system readiness before configuration. Run this first to ensure all prerequisites are met.

**Usage:**
```bash
sudo ./validate-network-setup.sh
```

**Options:**
- `--show-network` - Display current network configuration

**Checks:**
- Root access
- Proxmox VE installation
- Physical interface detection (needs at least 2)
- Existing bridge configuration
- DHCP client availability
- Network management tools
- Network service status

### `configure-proxmox-networking.sh`

Main entry point script that detects the server type (ML110/R630) and calls the network configuration script.

**Usage:**
```bash
sudo ./configure-proxmox-networking.sh
```

**Dry Run (preview changes without applying):**
```bash
sudo DRY_RUN=true ./configure-proxmox-networking.sh
```

### `network-config.sh`

Core network configuration script that:
- Auto-detects physical network interfaces
- Configures two bridges (vmbr0 and vmbr1) with DHCP
- Sets proper routing priorities
- Backs up existing configuration
- Validates the setup

**Usage:**
```bash
sudo ./network-config.sh
```

**Options:**
- `DRY_RUN=true` - Preview configuration without applying changes
- `NODE_HOSTNAME=<hostname>` - Set custom hostname (defaults to current hostname)
- `NIC1_OVERRIDE=<interface>` - Manually specify NIC 1 (LAN) interface name
- `NIC2_OVERRIDE=<interface>` - Manually specify NIC 2 (WAN) interface name

**Example:**
```bash
# Preview configuration
sudo DRY_RUN=true ./network-config.sh

# Apply configuration
sudo ./network-config.sh

# Apply with custom hostname
sudo NODE_HOSTNAME=pve-ml110 ./network-config.sh
```

## Prerequisites

Run the validation script first to check prerequisites:

```bash
sudo ./validate-network-setup.sh
```

Required:
1. **Root access** - Scripts must be run as root
2. **Two physical network interfaces** - Script will auto-detect available NICs
3. **Proxmox VE installed** - Scripts are designed for Proxmox hosts
4. **DHCP servers available** - Both interfaces require DHCP:
   - LAN interface needs DHCP on 192.168.1.0/24 network
   - WAN interface needs DHCP from Spectrum cable modem

## Interface Detection

The script automatically detects physical network interfaces by:
- Scanning `/sys/class/net/` for physical devices
- Excluding virtual interfaces, bridges, bonds, and VLANs
- Detecting interface speeds using `ethtool` or `/sys/class/net/*/speed`
- **Prioritizing 1 Gbps interfaces** for vmbr0 and vmbr1
- Showing all detected interfaces with their speeds

**Automatic Selection:**
- If 2+ 1 Gbps interfaces are found, they are selected automatically
- Otherwise, falls back to first two physical interfaces
- Interface speeds are displayed during detection

**Manual Override:**
If automatic detection selects wrong interfaces, you can override:
```bash
NIC1_OVERRIDE=nic2 NIC2_OVERRIDE=nic3 ./network-config.sh
```

**Note:** Speed detection requires the interface to have a link or be queryable via ethtool. Interfaces without link may show "unknown" speed.

## Configuration Files

The script generates `/etc/network/interfaces` with the following structure:

```
# Loopback
auto lo
iface lo inet loopback

# NIC 1 (LAN)
auto <nic1>
iface <nic1> inet manual

# vmbr0 (LAN Bridge)
auto vmbr0
iface vmbr0 inet dhcp
    bridge-ports <nic1>
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware no
    metric 200

# NIC 2 (WAN)
auto <nic2>
iface <nic2> inet manual

# vmbr1 (WAN Bridge)
auto vmbr1
iface vmbr1 inet dhcp
    bridge-ports <nic2>
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware no
    metric 100
```

## Safety Features

1. **Automatic Backup** - Creates timestamped backup of `/etc/network/interfaces`
2. **Dry Run Mode** - Preview changes before applying
3. **Validation** - Checks interface availability before configuration
4. **Rollback** - Backup files can be restored if needed

## Verification

After running the script, it will:
- Verify both bridges are up
- Check IP address assignment via DHCP
- Display routing table
- Show current network status

**Manual Verification:**
```bash
# Check bridges
ip addr show vmbr0
ip addr show vmbr1

# Check routing
ip route show

# Check interfaces
ip link show
```

## Troubleshooting

### Interface Detection Issues

If the script can't find 2 physical interfaces:
```bash
# List all interfaces
ls -la /sys/class/net/

# Check physical interfaces
for iface in /sys/class/net/*; do
    echo "$(basename $iface): $(readlink -f $iface)"
done
```

### DHCP Not Working

If DHCP doesn't assign IP addresses:
1. Check cable connections
2. Verify DHCP servers are available
3. Check DHCP client logs: `journalctl -u networking`
4. Manually test: `dhclient -v vmbr0` or `dhclient -v vmbr1`

### Restore Backup

If you need to restore the previous configuration:
```bash
# List backups
ls -la /etc/network/interfaces.backup.*

# Restore (replace with actual backup filename)
sudo cp /etc/network/interfaces.backup.YYYYMMDD_HHMMSS /etc/network/interfaces
sudo systemctl restart networking
```

### Default Route Issues

If the default route goes through the wrong interface:
- WAN should have metric 100 (preferred)
- LAN should have metric 200
- Check: `ip route show default`

## Related Files

- `config/hardware/nic-mapping.yaml` - Hardware NIC configuration
- `infrastructure/network/ip-schema-config.yaml` - IP address schema
- `diagrams/network-topology.mmd` - Network topology diagram

## Notes

- **VLAN Configuration**: The VLAN scripts (`configure-proxmox-vlans.sh`) are kept for reference but are not used in the current physical setup
- **Static IPs**: The scripts use DHCP. If you need static IPs, you'll need to modify the configuration manually or extend the scripts
- **Multiple Interfaces**: If servers have more than 2 NICs, additional interfaces will be ignored (first two are used)

## Migration from VLAN-Based Setup

If migrating from a VLAN-based configuration:
1. Backup current configuration
2. Review current `/etc/network/interfaces`
3. Run with `DRY_RUN=true` to preview changes
4. Apply new configuration
5. Verify connectivity on both networks