Add full monorepo: virtual-banker, backend, frontend, docs, scripts, deployment
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
97
avatar/unreal/README.md
Normal file
97
avatar/unreal/README.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# Unreal Engine Avatar Setup
|
||||
|
||||
This directory contains the Unreal Engine project for the Virtual Banker avatar.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Unreal Engine 5.3+ (or 5.4+ recommended)
|
||||
- PixelStreaming plugin enabled
|
||||
- Digital human character asset (Ready Player Me, MetaHuman, or custom)
|
||||
|
||||
## Setup Instructions
|
||||
|
||||
### 1. Create Unreal Project
|
||||
|
||||
1. Open Unreal Engine Editor
|
||||
2. Create new project:
|
||||
- Template: Blank
|
||||
- Blueprint or C++: Blueprint (or C++ if custom code needed)
|
||||
- Target Platform: Desktop
|
||||
- Quality: Maximum
|
||||
- Raytracing: Enabled (optional, for better quality)
|
||||
|
||||
### 2. Enable PixelStreaming
|
||||
|
||||
1. Edit → Plugins
|
||||
2. Search for "Pixel Streaming"
|
||||
3. Enable the plugin
|
||||
4. Restart Unreal Editor
|
||||
|
||||
### 3. Import Digital Human
|
||||
|
||||
1. Import your digital human character:
|
||||
- Ready Player Me: Use their Unreal plugin
|
||||
- MetaHuman: Use MetaHuman Creator
|
||||
- Custom: Import FBX/glTF with blendshapes
|
||||
|
||||
2. Set up blendshapes for visemes:
|
||||
- Import viseme blendshapes (aa, ee, oh, ou, mbp, etc.)
|
||||
- Map to animation system
|
||||
|
||||
### 4. Configure PixelStreaming
|
||||
|
||||
1. Edit → Project Settings → Plugins → Pixel Streaming
|
||||
2. Configure:
|
||||
- Streamer Port: 8888
|
||||
- WebRTC Port Range: 8888-8897
|
||||
- Enable WebRTC
|
||||
|
||||
### 5. Set Up Animation Blueprint
|
||||
|
||||
1. Create Animation Blueprint for avatar
|
||||
2. Set up state machine:
|
||||
- Idle
|
||||
- Speaking (viseme-driven)
|
||||
- Gesturing
|
||||
- Expressions
|
||||
|
||||
3. Connect viseme blendshapes to animation graph
|
||||
|
||||
### 6. Create Control Blueprint
|
||||
|
||||
1. Create Blueprint Actor for avatar control
|
||||
2. Add functions:
|
||||
- SetVisemes(VisemeData)
|
||||
- SetExpression(Valence, Arousal)
|
||||
- SetGesture(GestureType)
|
||||
- SetGaze(Target)
|
||||
|
||||
### 7. Build and Package
|
||||
|
||||
1. Package project for Linux (for server deployment):
|
||||
- File → Package Project → Linux
|
||||
- Or use command line:
|
||||
```
|
||||
UnrealEditor-Cmd.exe -run=UnrealVersionSelector -project="path/to/project.uproject" -game -cook -package -build
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
The packaged Unreal project should be deployed to a GPU-enabled server with:
|
||||
- NVIDIA GPU (RTX 3090+ recommended)
|
||||
- CUDA drivers
|
||||
- Sufficient VRAM (8GB+ per instance)
|
||||
|
||||
## Integration
|
||||
|
||||
The renderer service (`avatar/renderer/service.go`) controls Unreal instances via:
|
||||
- Process management (start/stop instances)
|
||||
- WebSocket communication (animation parameters)
|
||||
- PixelStreaming WebRTC streams
|
||||
|
||||
## Notes
|
||||
|
||||
- Each active session requires one Unreal instance
|
||||
- GPU resources should be allocated per instance
|
||||
- Consider using Unreal's multi-instance support for scaling
|
||||
|
||||
Reference in New Issue
Block a user