loc_az_hci/docs/deployment/azure-arc-onboarding.md

# Azure Arc Onboarding Guide

## Overview

This document describes the Azure Arc onboarding process for all Linux hosts and VMs in the Azure Stack HCI environment, enabling Azure governance, monitoring, and management.

## Architecture

### Azure Arc Architecture

```
┌─────────────────────────────────────────────────────────┐
│                    Azure Portal                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐ │
│  │ Azure Arc    │  │ Azure Policy │  │ Azure Monitor │ │
│  │ Servers      │  │              │  │              │ │
│  └──────────────┘  └──────────────┘  └──────────────┘ │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐ │
│  │ Defender     │  │ Update       │  │ GitOps       │ │
│  │ for Cloud    │  │ Management   │  │ (Flux)       │ │
│  └──────────────┘  └──────────────┘  └──────────────┘ │
└─────────────────────────────────────────────────────────┘
                              │
                              │ HTTPS (443) Outbound
                              │
┌─────────────────────────────────────────────────────────┐
│              On-Premises Infrastructure                   │
│                                                           │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐ │
│  │ Router       │  │  Proxmox     │  │  Ubuntu      │ │
│  │ Server       │  │  ML110/R630  │  │  Service VMs │ │
│  │              │  │              │  │              │ │
│  │ Arc Agent    │  │  Arc Agent   │  │  Arc Agent   │ │
│  └──────────────┘  └──────────────┘  └──────────────┘ │
└─────────────────────────────────────────────────────────┘
```

## Prerequisites

### Azure Requirements

- Azure subscription with Contributor role
- Resource group created (or will be created)
- Azure CLI installed and authenticated
- Service principal or managed identity (optional)

### Network Requirements

- Outbound HTTPS (443) connectivity to Azure
- Proxy support if needed (see Proxy Configuration section)
- DNS resolution for Azure endpoints

### Target Systems

- Linux hosts (Proxmox VE, Ubuntu)
- Windows Server (optional, for management VM)
- Ubuntu VMs (service VMs)

### Environment Configuration

Before starting, ensure your `.env` file is configured with Azure credentials:

```bash
# Copy template if not already done
cp .env.example .env

# Edit .env and set:
# - AZURE_SUBSCRIPTION_ID
# - AZURE_TENANT_ID
# - AZURE_CLIENT_ID (optional, for service principal)
# - AZURE_CLIENT_SECRET (optional, for service principal)
# - AZURE_RESOURCE_GROUP
# - AZURE_LOCATION
```

## Installation

### Step 1: Prepare Azure Environment

```bash
# Load environment variables from .env (if using .env file)
export $(cat .env | grep -v '^#' | xargs)

# Set variables (use from .env or set manually)
export SUBSCRIPTION_ID="${AZURE_SUBSCRIPTION_ID:-your-subscription-id}"
export RESOURCE_GROUP="${AZURE_RESOURCE_GROUP:-HC-Stack}"
export LOCATION="${AZURE_LOCATION:-eastus}"
export TENANT_ID="${AZURE_TENANT_ID:-$(az account show --query tenantId -o tsv)}"

# Login to Azure
az login

# Set subscription
az account set --subscription $SUBSCRIPTION_ID

# Create resource group (if not exists)
az group create \
  --name $RESOURCE_GROUP \
  --location $LOCATION
```

### Step 2: Install Arc Agent on Linux

#### Ubuntu/Debian

```bash
# Download installation script
curl -s https://aka.ms/azcmagent -o /tmp/install_linux_azcmagent.sh

# Run installation
bash /tmp/install_linux_azcmagent.sh

# Verify installation
azcmagent version
```

#### Proxmox VE (Debian-based)

```bash
# Same as Ubuntu/Debian
curl -s https://aka.ms/azcmagent -o /tmp/install_linux_azcmagent.sh
bash /tmp/install_linux_azcmagent.sh
azcmagent version
```

### Step 3: Onboard to Azure Arc

#### Using Service Principal

```bash
# Load environment variables from .env
export $(cat .env | grep -v '^#' | xargs)

# Use service principal from .env or create new one
if [ -z "$AZURE_CLIENT_ID" ] || [ -z "$AZURE_CLIENT_SECRET" ]; then
  # Create service principal (if not exists)
  az ad sp create-for-rbac \
    --name "ArcOnboarding" \
    --role "Azure Connected Machine Onboarding" \
    --scopes "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP"
  
  # Note: AppId, Password, Tenant - add these to .env file
else
  export SUBSCRIPTION_ID="${AZURE_SUBSCRIPTION_ID}"
  export RESOURCE_GROUP="${AZURE_RESOURCE_GROUP:-HC-Stack}"
  export LOCATION="${AZURE_LOCATION:-eastus}"
  export TENANT_ID="${AZURE_TENANT_ID}"
fi

# Onboard machine
azcmagent connect \
  --service-principal-id "${AZURE_CLIENT_ID:-<app-id>}" \
  --service-principal-secret "${AZURE_CLIENT_SECRET:-<password>}" \
  --tenant-id "$TENANT_ID" \
  --subscription-id "$SUBSCRIPTION_ID" \
  --resource-group "$RESOURCE_GROUP" \
  --location "$LOCATION" \
  --tags "Environment=Production,Role=Router"
```

#### Using Interactive Login

```bash
# Load environment variables from .env
export $(cat .env | grep -v '^#' | xargs)

export SUBSCRIPTION_ID="${AZURE_SUBSCRIPTION_ID}"
export RESOURCE_GROUP="${AZURE_RESOURCE_GROUP:-HC-Stack}"
export LOCATION="${AZURE_LOCATION:-eastus}"

# Onboard machine (will prompt for login)
azcmagent connect \
  --subscription-id "$SUBSCRIPTION_ID" \
  --resource-group "$RESOURCE_GROUP" \
  --location "$LOCATION" \
  --tags "Environment=Production,Role=Router"
```

### Step 4: Verify Onboarding

```bash
# Check agent status
azcmagent show

# Verify in Azure Portal
az connectedmachine list \
  --resource-group $RESOURCE_GROUP \
  --output table
```

## Proxy Configuration

### If Outbound Proxy Required

#### Configure Proxy for Arc Agent

```bash
# Set proxy environment variables
export https_proxy="http://proxy.example.com:8080"
export http_proxy="http://proxy.example.com:8080"
export no_proxy="localhost,127.0.0.1,.local"

# Configure Arc agent proxy
azcmagent config set proxy.url "http://proxy.example.com:8080"
azcmagent config set proxy.bypass "localhost,127.0.0.1,.local"

# Restart agent
azcmagent restart
```

#### Proxy Authentication

```bash
# If proxy requires authentication
azcmagent config set proxy.url "http://user:password@proxy.example.com:8080"
azcmagent restart
```

## Governance Configuration

### Azure Policy

#### Enable Policy for Arc Servers

```bash
# Assign built-in policy: "Enable Azure Monitor for VMs"
az policy assignment create \
  --name "EnableAzureMonitorForVMs" \
  --display-name "Enable Azure Monitor for VMs" \
  --scope "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP" \
  --policy "/providers/Microsoft.Authorization/policyDefinitions/0ef5aac7-c064-427a-b87b-d47b3ddcaf73"
```

#### Custom Policy Example

```json
{
  "if": {
    "allOf": [
      {
        "field": "type",
        "equals": "Microsoft.HybridCompute/machines"
      },
      {
        "field": "Microsoft.HybridCompute/machines/osName",
        "notEquals": "Ubuntu"
      }
    ]
  },
  "then": {
    "effect": "audit"
  }
}
```

### Azure Monitor

#### Enable Log Analytics

```bash
# Create Log Analytics workspace
az monitor log-analytics workspace create \
  --resource-group $RESOURCE_GROUP \
  --workspace-name "hci-logs-$LOCATION"

# Enable VM insights
az monitor log-analytics solution create \
  --resource-group $RESOURCE_GROUP \
  --name "VMInsights" \
  --workspace "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.OperationalInsights/workspaces/hci-logs-$LOCATION" \
  --plan-publisher "Microsoft" \
  --plan-product "OMSGallery/VMInsights"
```

#### Configure Data Collection

```bash
# Enable data collection rule
az monitor data-collection rule create \
  --resource-group $RESOURCE_GROUP \
  --name "hci-dcr" \
  --location "$LOCATION" \
  --log-analytics "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.OperationalInsights/workspaces/hci-logs-$LOCATION"
```

### Azure Defender

#### Enable Defender for Servers

```bash
# Enable Defender for Cloud
az security pricing create \
  --name "VirtualMachines" \
  --tier "Standard" \
  --resource-group $RESOURCE_GROUP
```

#### Onboard Arc Servers to Defender

```bash
# Install Defender extension (via Azure Portal or CLI)
az connectedmachine extension create \
  --machine-name "<machine-name>" \
  --resource-group $RESOURCE_GROUP \
  --name "WindowsDefenderATP" \
  --publisher "Microsoft.AzureDefender" \
  --type "MDE.Linux"
```

### Update Management

#### Enable Update Management

```bash
# Enable Update Management via Azure Automation
# This is typically done through Azure Portal:
# 1. Create Automation Account
# 2. Enable Update Management solution
# 3. Add Arc servers to Update Management
```

## Tagging Strategy

### Recommended Tags

```bash
# Tag machines during onboarding
azcmagent connect \
  --subscription-id "$SUBSCRIPTION_ID" \
  --resource-group "$RESOURCE_GROUP" \
  --location "$LOCATION" \
  --tags "Environment=Production,Role=Router,Project=AzureStackHCI,ManagedBy=Arc"
```

### Update Tags

```bash
# Update tags after onboarding
az connectedmachine update \
  --name "<machine-name>" \
  --resource-group $RESOURCE_GROUP \
  --tags "Environment=Production,Role=Router,Updated=2024-01-01"
```

## Verification

### Check Agent Status

```bash
# On each machine
azcmagent show

# Expected output:
# Agent Status: Connected
# Azure Resource ID: /subscriptions/.../resourceGroups/.../providers/Microsoft.HybridCompute/machines/...
```

### Verify in Azure Portal

1. Navigate to Azure Portal > Azure Arc > Servers
2. Verify all machines listed
3. Check machine status (Connected)
4. Review machine details and tags

### Test Policy Enforcement

```bash
# Check policy compliance
az policy state list \
  --resource "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP" \
  --output table
```

## Troubleshooting

### Agent Not Connecting

**Problem:** Agent shows as disconnected
- **Solution:**
  - Check network connectivity (HTTPS 443)
  - Verify proxy configuration if needed
  - Check agent logs: `azcmagent logs`
  - Verify Azure credentials

### Proxy Issues

**Problem:** Agent can't connect through proxy
- **Solution:**
  - Verify proxy URL and credentials
  - Check proxy bypass list
  - Test proxy connectivity manually
  - Review agent logs

### Policy Not Applying

**Problem:** Azure Policy not enforcing
- **Solution:**
  - Verify policy assignment scope
  - Check policy evaluation status
  - Verify machine tags match policy conditions
  - Review policy compliance reports

### Monitoring Not Working

**Problem:** Azure Monitor not collecting data
- **Solution:**
  - Verify Log Analytics workspace configuration
  - Check data collection rules
  - Verify agent extension installed
  - Review Log Analytics workspace logs

## Best Practices

1. **Use Service Principals:**
   - Create dedicated service principal for Arc onboarding
   - Use least privilege permissions
   - Rotate credentials regularly

2. **Tagging:**
   - Use consistent tagging strategy
   - Include environment, role, project tags
   - Enable tag-based policy enforcement

3. **Monitoring:**
   - Enable Azure Monitor for all Arc servers
   - Configure alert rules
   - Set up log retention policies

4. **Security:**
   - Enable Azure Defender for all servers
   - Configure security policies
   - Review security recommendations regularly

5. **Updates:**
   - Enable Update Management
   - Schedule regular maintenance windows
   - Test updates in dev environment first

## Related Documentation

- [Complete Architecture](complete-architecture.md) - Full architecture overview
- [Bring-Up Checklist](bring-up-checklist.md) - Installation guide
- [Microsoft Azure Arc Documentation](https://docs.microsoft.com/azure/azure-arc/)