359 lines
7.6 KiB
Markdown
359 lines
7.6 KiB
Markdown
# MetaHuman Pipeline - Dubai Metaverse
|
|
|
|
## Overview
|
|
|
|
This document describes the MetaHuman creation and integration pipeline for the Dubai Metaverse project.
|
|
|
|
## MetaHuman Framework
|
|
|
|
### What are MetaHumans?
|
|
|
|
MetaHumans are Unreal Engine's high-fidelity digital humans created using the MetaHuman Creator. They provide photorealistic human characters with advanced facial animation capabilities.
|
|
|
|
### MetaHuman Creator
|
|
|
|
**Tool**: MetaHuman Creator (web-based or plugin)
|
|
|
|
**Features**:
|
|
- High-quality human models
|
|
- Extensive customization
|
|
- Realistic facial features
|
|
- Hair and clothing options
|
|
|
|
---
|
|
|
|
## MetaHuman Creation
|
|
|
|
### Step 1: Access MetaHuman Creator
|
|
|
|
1. **Epic Games Account**: Sign in to Epic Games account
|
|
2. **MetaHuman Creator**: Access via web or Unreal plugin
|
|
3. **Create New MetaHuman**: Start new MetaHuman creation
|
|
|
|
### Step 2: Base Character
|
|
|
|
1. **Preset Selection**: Choose base preset
|
|
2. **Gender**: Select gender
|
|
3. **Age**: Adjust age
|
|
4. **Ethnicity**: Select ethnicity (Dubai multicultural mix)
|
|
|
|
### Step 3: Facial Features
|
|
|
|
1. **Face Shape**: Adjust face shape
|
|
2. **Eyes**: Customize eyes
|
|
3. **Nose**: Customize nose
|
|
4. **Mouth**: Customize mouth
|
|
5. **Ears**: Customize ears
|
|
6. **Skin**: Adjust skin tone and features
|
|
|
|
### Step 4: Hair and Appearance
|
|
|
|
1. **Hair**: Select hair style and color
|
|
2. **Eyebrows**: Customize eyebrows
|
|
3. **Facial Hair**: Add facial hair (if applicable)
|
|
4. **Makeup**: Add makeup (if applicable)
|
|
|
|
### Step 5: Clothing
|
|
|
|
1. **Outfit**: Select clothing style
|
|
2. **Colors**: Adjust clothing colors
|
|
3. **Accessories**: Add accessories (optional)
|
|
|
|
### Step 6: Export
|
|
|
|
1. **Review**: Review MetaHuman
|
|
2. **Export**: Export to Unreal Engine
|
|
3. **Save**: Save MetaHuman asset
|
|
|
|
---
|
|
|
|
## Import to Unreal Engine
|
|
|
|
### Import Process
|
|
|
|
1. **MetaHuman Plugin**: Ensure MetaHuman plugin is enabled
|
|
2. **Import**: Import MetaHuman from MetaHuman Creator
|
|
3. **Validation**: Verify import is successful
|
|
4. **Placement**: Place in level
|
|
|
|
### Asset Organization
|
|
|
|
```
|
|
Content/Assets/Characters/MetaHumans/
|
|
├── MH_Dubai_01/
|
|
│ ├── SK_Character_Male_01
|
|
│ ├── [Animation assets]
|
|
│ └── [Material assets]
|
|
├── MH_Dubai_02/
|
|
└── ...
|
|
```
|
|
|
|
---
|
|
|
|
## Animation Setup
|
|
|
|
### Animation Blueprints
|
|
|
|
#### ABP_NPC_Walk
|
|
|
|
**Walking animation blueprint**
|
|
|
|
- **Locomotion**: Walking animations
|
|
- **Blend Spaces**: Speed-based blending
|
|
- **Transitions**: Smooth transitions
|
|
|
|
#### ABP_NPC_Sit
|
|
|
|
**Sitting animation blueprint**
|
|
|
|
- **Idle Sitting**: Sitting idle animations
|
|
- **Sitting Variations**: Different sitting poses
|
|
|
|
#### ABP_NPC_Phone
|
|
|
|
**Phone usage animation blueprint**
|
|
|
|
- **Phone Usage**: Using phone animations
|
|
- **Variations**: Different phone poses
|
|
|
|
#### ABP_NPC_Gestures
|
|
|
|
**Gesture animation blueprint**
|
|
|
|
- **Gestures**: Hand gestures, talking
|
|
- **Variations**: Different gesture types
|
|
|
|
### Animation Assets
|
|
|
|
- **AM_Walking**: Walking animation
|
|
- **AM_Sitting**: Sitting animation
|
|
- **AM_PhoneUsage**: Phone usage animation
|
|
- **AM_Gestures**: Gesture animations
|
|
|
|
---
|
|
|
|
## Behavior Trees
|
|
|
|
### BT_NPC_Wander
|
|
|
|
**Wandering behavior for NPCs**
|
|
|
|
#### Behavior
|
|
|
|
1. **Random Movement**: Move to random locations
|
|
2. **Wait**: Wait at locations
|
|
3. **Repeat**: Repeat wandering pattern
|
|
|
|
#### Implementation
|
|
|
|
- **Blackboard**: Store target location
|
|
- **Tasks**: Move to location, wait
|
|
- **Decorators**: Conditions for behavior
|
|
|
|
### BT_NPC_Social
|
|
|
|
**Social behavior for NPCs**
|
|
|
|
#### Behavior
|
|
|
|
1. **Group Formation**: Form groups with other NPCs
|
|
2. **Conversation**: Simulate conversation
|
|
3. **Gestures**: Use gestures during conversation
|
|
|
|
#### Implementation
|
|
|
|
- **Blackboard**: Store group members, conversation state
|
|
- **Tasks**: Group formation, conversation
|
|
- **Decorators**: Social conditions
|
|
|
|
### BT_NPC_Phone
|
|
|
|
**Phone usage behavior**
|
|
|
|
#### Behavior
|
|
|
|
1. **Phone Usage**: Use phone at intervals
|
|
2. **Walking with Phone**: Walk while using phone
|
|
3. **Stop and Use**: Stop to use phone
|
|
|
|
#### Implementation
|
|
|
|
- **Blackboard**: Store phone usage state
|
|
- **Tasks**: Phone usage, movement
|
|
- **Decorators**: Phone usage conditions
|
|
|
|
---
|
|
|
|
## NPC Controller
|
|
|
|
### BP_NPCController
|
|
|
|
**AI controller for NPCs**
|
|
|
|
### Components
|
|
|
|
1. **Behavior Tree**: Reference to behavior tree
|
|
2. **Blackboard**: NPC state and data
|
|
3. **Perception**: AI perception (sight, hearing)
|
|
4. **Movement**: AI movement component
|
|
|
|
### Settings
|
|
|
|
- **Behavior Tree**: Assign behavior tree
|
|
- **Perception**: Configure perception settings
|
|
- **Movement**: Configure movement settings
|
|
|
|
---
|
|
|
|
## ChatGPT Integration (Optional)
|
|
|
|
### BP_DialogueSystem
|
|
|
|
**ChatGPT-powered dialogue system**
|
|
|
|
### Implementation
|
|
|
|
1. **API Integration**: Integrate ChatGPT API
|
|
2. **Dialogue Generation**: Generate dynamic dialogue
|
|
3. **NPC Responses**: NPC responds to player
|
|
4. **Context Awareness**: Context-aware dialogue
|
|
|
|
### Setup
|
|
|
|
1. **API Key**: Store API key securely (not in repository)
|
|
2. **API Client**: Create API client
|
|
3. **Dialogue Handler**: Handle dialogue requests
|
|
4. **Response Processing**: Process API responses
|
|
|
|
### Usage
|
|
|
|
1. **Player Interaction**: Player interacts with NPC
|
|
2. **Dialogue Request**: Request dialogue from ChatGPT
|
|
3. **NPC Response**: NPC speaks generated dialogue
|
|
4. **Continuation**: Continue conversation
|
|
|
|
---
|
|
|
|
## MetaHuman Animator
|
|
|
|
### Facial Capture
|
|
|
|
**Purpose**: Capture facial animation for MetaHumans
|
|
|
|
### Setup
|
|
|
|
1. **MetaHuman Animator Plugin**: Enable plugin
|
|
2. **Camera Setup**: Set up camera for capture
|
|
3. **Calibration**: Calibrate facial capture
|
|
4. **Recording**: Record facial animation
|
|
|
|
### Usage
|
|
|
|
1. **Record**: Record facial performance
|
|
2. **Process**: Process facial capture
|
|
3. **Apply**: Apply to MetaHuman
|
|
4. **Refine**: Refine animation
|
|
|
|
---
|
|
|
|
## NPC Placement
|
|
|
|
### Placement Strategy
|
|
|
|
1. **Density**: Moderate NPC density
|
|
2. **Distribution**: Distribute across district
|
|
3. **Groups**: Place NPCs in social groups
|
|
4. **Activities**: Place NPCs in activity areas
|
|
|
|
### Areas
|
|
|
|
- **Marina Promenade**: Pedestrians walking
|
|
- **Cafes**: NPCs sitting, socializing
|
|
- **Roads**: NPCs walking along roads
|
|
- **Parks**: NPCs in parks, green spaces
|
|
|
|
---
|
|
|
|
## Performance Optimization
|
|
|
|
### NPC Optimization
|
|
|
|
1. **LODs**: Use LODs for distant NPCs
|
|
2. **Culling**: Cull NPCs outside view
|
|
3. **Instancing**: Use instanced meshes where possible
|
|
4. **Animation**: Optimize animation complexity
|
|
|
|
### Behavior Optimization
|
|
|
|
1. **Update Frequency**: Optimize behavior tree updates
|
|
2. **Perception**: Optimize perception checks
|
|
3. **Pathfinding**: Optimize pathfinding
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
### NPC Testing
|
|
|
|
1. **Behavior**: Test NPC behavior
|
|
2. **Animation**: Test animations
|
|
3. **Performance**: Test performance with NPCs
|
|
4. **Interactions**: Test NPC interactions
|
|
|
|
### Dialogue Testing
|
|
|
|
1. **ChatGPT Integration**: Test API integration
|
|
2. **Dialogue Quality**: Test dialogue quality
|
|
3. **Response Time**: Test response time
|
|
4. **Context**: Test context awareness
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
**Issue**: MetaHuman not importing
|
|
- **Solution**: Check MetaHuman plugin is enabled
|
|
- **Solution**: Verify MetaHuman Creator connection
|
|
|
|
**Issue**: Animations not working
|
|
- **Solution**: Check animation blueprint setup
|
|
- **Solution**: Verify animation assets are assigned
|
|
|
|
**Issue**: Behavior tree not executing
|
|
- **Solution**: Check behavior tree assignment
|
|
- **Solution**: Verify blackboard setup
|
|
|
|
**Issue**: Performance issues
|
|
- **Solution**: Reduce NPC count
|
|
- **Solution**: Use LODs
|
|
- **Solution**: Optimize behavior trees
|
|
|
|
---
|
|
|
|
## Documentation
|
|
|
|
### MetaHuman Catalog
|
|
|
|
Document each MetaHuman:
|
|
- Name
|
|
- Appearance
|
|
- Clothing
|
|
- Behavior
|
|
- Usage
|
|
|
|
### Behavior Documentation
|
|
|
|
Document behavior trees:
|
|
- Behavior description
|
|
- Blackboard variables
|
|
- Tasks and decorators
|
|
- Usage
|
|
|
|
---
|
|
|
|
**Version**: 1.0
|
|
**Last Updated**: [Current Date]
|
|
|