# Unreal Engine Avatar Setup

This directory contains the Unreal Engine project for the Virtual Banker avatar.

## Prerequisites

- Unreal Engine 5.3+ (or 5.4+ recommended)
- PixelStreaming plugin enabled
- Digital human character asset (Ready Player Me, MetaHuman, or custom)

## Setup Instructions

### 1. Create Unreal Project

1. Open Unreal Engine Editor
2. Create new project:
   - Template: Blank
   - Blueprint or C++: Blueprint (or C++ if custom code needed)
   - Target Platform: Desktop
   - Quality: Maximum
   - Raytracing: Enabled (optional, for better quality)

### 2. Enable PixelStreaming

1. Edit → Plugins
2. Search for "Pixel Streaming"
3. Enable the plugin
4. Restart Unreal Editor

### 3. Import Digital Human

1. Import your digital human character:
   - Ready Player Me: Use their Unreal plugin
   - MetaHuman: Use MetaHuman Creator
   - Custom: Import FBX/glTF with blendshapes

2. Set up blendshapes for visemes:
   - Import viseme blendshapes (aa, ee, oh, ou, mbp, etc.)
   - Map to animation system

### 4. Configure PixelStreaming

1. Edit → Project Settings → Plugins → Pixel Streaming
2. Configure:
   - Streamer Port: 8888
   - WebRTC Port Range: 8888-8897
   - Enable WebRTC

### 5. Set Up Animation Blueprint

1. Create Animation Blueprint for avatar
2. Set up state machine:
   - Idle
   - Speaking (viseme-driven)
   - Gesturing
   - Expressions

3. Connect viseme blendshapes to animation graph

### 6. Create Control Blueprint

1. Create Blueprint Actor for avatar control
2. Add functions:
   - SetVisemes(VisemeData)
   - SetExpression(Valence, Arousal)
   - SetGesture(GestureType)
   - SetGaze(Target)

### 7. Build and Package

1. Package project for Linux (for server deployment):
   - File → Package Project → Linux
   - Or use command line:
     ```
     UnrealEditor-Cmd.exe -run=UnrealVersionSelector -project="path/to/project.uproject" -game -cook -package -build
     ```

## Deployment

The packaged Unreal project should be deployed to a GPU-enabled server with:
- NVIDIA GPU (RTX 3090+ recommended)
- CUDA drivers
- Sufficient VRAM (8GB+ per instance)

## Integration

The renderer service (`avatar/renderer/service.go`) controls Unreal instances via:
- Process management (start/stop instances)
- WebSocket communication (animation parameters)
- PixelStreaming WebRTC streams

## Notes

- Each active session requires one Unreal instance
- GPU resources should be allocated per instance
- Consider using Unreal's multi-instance support for scaling