# MetaHuman Pipeline - Dubai Metaverse ## Overview This document describes the MetaHuman creation and integration pipeline for the Dubai Metaverse project. ## MetaHuman Framework ### What are MetaHumans? MetaHumans are Unreal Engine's high-fidelity digital humans created using the MetaHuman Creator. They provide photorealistic human characters with advanced facial animation capabilities. ### MetaHuman Creator **Tool**: MetaHuman Creator (web-based or plugin) **Features**: - High-quality human models - Extensive customization - Realistic facial features - Hair and clothing options --- ## MetaHuman Creation ### Step 1: Access MetaHuman Creator 1. **Epic Games Account**: Sign in to Epic Games account 2. **MetaHuman Creator**: Access via web or Unreal plugin 3. **Create New MetaHuman**: Start new MetaHuman creation ### Step 2: Base Character 1. **Preset Selection**: Choose base preset 2. **Gender**: Select gender 3. **Age**: Adjust age 4. **Ethnicity**: Select ethnicity (Dubai multicultural mix) ### Step 3: Facial Features 1. **Face Shape**: Adjust face shape 2. **Eyes**: Customize eyes 3. **Nose**: Customize nose 4. **Mouth**: Customize mouth 5. **Ears**: Customize ears 6. **Skin**: Adjust skin tone and features ### Step 4: Hair and Appearance 1. **Hair**: Select hair style and color 2. **Eyebrows**: Customize eyebrows 3. **Facial Hair**: Add facial hair (if applicable) 4. **Makeup**: Add makeup (if applicable) ### Step 5: Clothing 1. **Outfit**: Select clothing style 2. **Colors**: Adjust clothing colors 3. **Accessories**: Add accessories (optional) ### Step 6: Export 1. **Review**: Review MetaHuman 2. **Export**: Export to Unreal Engine 3. **Save**: Save MetaHuman asset --- ## Import to Unreal Engine ### Import Process 1. **MetaHuman Plugin**: Ensure MetaHuman plugin is enabled 2. **Import**: Import MetaHuman from MetaHuman Creator 3. **Validation**: Verify import is successful 4. **Placement**: Place in level ### Asset Organization ``` Content/Assets/Characters/MetaHumans/ ├── MH_Dubai_01/ │ ├── SK_Character_Male_01 │ ├── [Animation assets] │ └── [Material assets] ├── MH_Dubai_02/ └── ... ``` --- ## Animation Setup ### Animation Blueprints #### ABP_NPC_Walk **Walking animation blueprint** - **Locomotion**: Walking animations - **Blend Spaces**: Speed-based blending - **Transitions**: Smooth transitions #### ABP_NPC_Sit **Sitting animation blueprint** - **Idle Sitting**: Sitting idle animations - **Sitting Variations**: Different sitting poses #### ABP_NPC_Phone **Phone usage animation blueprint** - **Phone Usage**: Using phone animations - **Variations**: Different phone poses #### ABP_NPC_Gestures **Gesture animation blueprint** - **Gestures**: Hand gestures, talking - **Variations**: Different gesture types ### Animation Assets - **AM_Walking**: Walking animation - **AM_Sitting**: Sitting animation - **AM_PhoneUsage**: Phone usage animation - **AM_Gestures**: Gesture animations --- ## Behavior Trees ### BT_NPC_Wander **Wandering behavior for NPCs** #### Behavior 1. **Random Movement**: Move to random locations 2. **Wait**: Wait at locations 3. **Repeat**: Repeat wandering pattern #### Implementation - **Blackboard**: Store target location - **Tasks**: Move to location, wait - **Decorators**: Conditions for behavior ### BT_NPC_Social **Social behavior for NPCs** #### Behavior 1. **Group Formation**: Form groups with other NPCs 2. **Conversation**: Simulate conversation 3. **Gestures**: Use gestures during conversation #### Implementation - **Blackboard**: Store group members, conversation state - **Tasks**: Group formation, conversation - **Decorators**: Social conditions ### BT_NPC_Phone **Phone usage behavior** #### Behavior 1. **Phone Usage**: Use phone at intervals 2. **Walking with Phone**: Walk while using phone 3. **Stop and Use**: Stop to use phone #### Implementation - **Blackboard**: Store phone usage state - **Tasks**: Phone usage, movement - **Decorators**: Phone usage conditions --- ## NPC Controller ### BP_NPCController **AI controller for NPCs** ### Components 1. **Behavior Tree**: Reference to behavior tree 2. **Blackboard**: NPC state and data 3. **Perception**: AI perception (sight, hearing) 4. **Movement**: AI movement component ### Settings - **Behavior Tree**: Assign behavior tree - **Perception**: Configure perception settings - **Movement**: Configure movement settings --- ## ChatGPT Integration (Optional) ### BP_DialogueSystem **ChatGPT-powered dialogue system** ### Implementation 1. **API Integration**: Integrate ChatGPT API 2. **Dialogue Generation**: Generate dynamic dialogue 3. **NPC Responses**: NPC responds to player 4. **Context Awareness**: Context-aware dialogue ### Setup 1. **API Key**: Store API key securely (not in repository) 2. **API Client**: Create API client 3. **Dialogue Handler**: Handle dialogue requests 4. **Response Processing**: Process API responses ### Usage 1. **Player Interaction**: Player interacts with NPC 2. **Dialogue Request**: Request dialogue from ChatGPT 3. **NPC Response**: NPC speaks generated dialogue 4. **Continuation**: Continue conversation --- ## MetaHuman Animator ### Facial Capture **Purpose**: Capture facial animation for MetaHumans ### Setup 1. **MetaHuman Animator Plugin**: Enable plugin 2. **Camera Setup**: Set up camera for capture 3. **Calibration**: Calibrate facial capture 4. **Recording**: Record facial animation ### Usage 1. **Record**: Record facial performance 2. **Process**: Process facial capture 3. **Apply**: Apply to MetaHuman 4. **Refine**: Refine animation --- ## NPC Placement ### Placement Strategy 1. **Density**: Moderate NPC density 2. **Distribution**: Distribute across district 3. **Groups**: Place NPCs in social groups 4. **Activities**: Place NPCs in activity areas ### Areas - **Marina Promenade**: Pedestrians walking - **Cafes**: NPCs sitting, socializing - **Roads**: NPCs walking along roads - **Parks**: NPCs in parks, green spaces --- ## Performance Optimization ### NPC Optimization 1. **LODs**: Use LODs for distant NPCs 2. **Culling**: Cull NPCs outside view 3. **Instancing**: Use instanced meshes where possible 4. **Animation**: Optimize animation complexity ### Behavior Optimization 1. **Update Frequency**: Optimize behavior tree updates 2. **Perception**: Optimize perception checks 3. **Pathfinding**: Optimize pathfinding --- ## Testing ### NPC Testing 1. **Behavior**: Test NPC behavior 2. **Animation**: Test animations 3. **Performance**: Test performance with NPCs 4. **Interactions**: Test NPC interactions ### Dialogue Testing 1. **ChatGPT Integration**: Test API integration 2. **Dialogue Quality**: Test dialogue quality 3. **Response Time**: Test response time 4. **Context**: Test context awareness --- ## Troubleshooting ### Common Issues **Issue**: MetaHuman not importing - **Solution**: Check MetaHuman plugin is enabled - **Solution**: Verify MetaHuman Creator connection **Issue**: Animations not working - **Solution**: Check animation blueprint setup - **Solution**: Verify animation assets are assigned **Issue**: Behavior tree not executing - **Solution**: Check behavior tree assignment - **Solution**: Verify blackboard setup **Issue**: Performance issues - **Solution**: Reduce NPC count - **Solution**: Use LODs - **Solution**: Optimize behavior trees --- ## Documentation ### MetaHuman Catalog Document each MetaHuman: - Name - Appearance - Clothing - Behavior - Usage ### Behavior Documentation Document behavior trees: - Behavior description - Blackboard variables - Tasks and decorators - Usage --- **Version**: 1.0 **Last Updated**: [Current Date]