Initial commit: Complete project foundation with all documentation, scripts, and project structure

This commit is contained in:
Dubai Metaverse Team
2025-11-20 15:13:53 -08:00
commit 0804197325
96 changed files with 16912 additions and 0 deletions

107
scripts/README.md Normal file
View File

@@ -0,0 +1,107 @@
# Scripts Directory - Dubai Metaverse
This directory contains automation scripts for the Dubai Metaverse project.
## Setup Scripts
### setup_project.sh
Initializes the project structure, Git LFS, and validates prerequisites.
**Usage**:
```bash
./scripts/setup_project.sh
```
### setup_ue5_project.sh
Validates and helps configure the Unreal Engine 5 project structure.
**Usage**:
```bash
./scripts/setup_ue5_project.sh
```
## Validation Scripts
### validate_assets.sh
Validates asset naming conventions and structure.
**Usage**:
```bash
./scripts/validate_assets.sh
```
### generate_docs.sh
Validates documentation completeness.
**Usage**:
```bash
./scripts/generate_docs.sh
```
## Data Import Scripts
### import_osm_data.py
Imports OpenStreetMap data for Dubai Marina.
**Usage**:
```bash
python3 scripts/import_osm_data.py --output data/processed/dubai_marina_buildings.geojson
```
**Requirements**:
- overpy
- geojson
### gis_to_unreal.py
Converts GIS elevation data to Unreal Engine terrain format.
**Usage**:
```bash
python3 scripts/gis_to_unreal.py data/elevation/dubai_marina_dem.tif --output data/processed/terrain_heightmap.raw
```
**Requirements**:
- rasterio
- numpy
## Phase-Specific Scripts
### Phase 2 (Weeks 3-5)
- `pcg_validation.py` - Validates PCG graph outputs
- `texture_validation.py` - Validates texture resolution and format
### Phase 3 (Week 6)
- `lighting_validation.py` - Validates lighting setup and performance
### Phase 4 (Week 10)
- `performance_audit.py` - Audits performance metrics
- `render_cinematic.py` - Automates Movie Render Queue rendering
### Phase 5 (Weeks 11-12)
- `automated_tests.py` - Runs automated test suite
- `package_build.sh` - Packages final build
## Optional Scripts
- `setup_pixel_streaming.sh` - Sets up Pixel Streaming (optional)
- `npc_dialogue_api.py` - ChatGPT API integration (optional)
## Installation
Install Python dependencies:
```bash
pip install -r requirements.txt
```
Make scripts executable:
```bash
chmod +x scripts/*.sh scripts/*.py
```
## Notes
- Some scripts are placeholders for future phases
- Python scripts require Python 3.8+
- Shell scripts require bash
- All scripts should be run from project root directory

12
scripts/automated_tests.py Executable file
View File

@@ -0,0 +1,12 @@
#!/usr/bin/env python3
"""
Dubai Metaverse - Automated Test Suite
Runs automated tests for gameplay systems
"""
# Placeholder script - to be implemented during Phase 5, Week 11-12
# This script will run automated tests for gameplay systems
print("Automated Test Suite")
print("To be implemented during Phase 5, Week 11-12")

60
scripts/generate_docs.sh Executable file
View File

@@ -0,0 +1,60 @@
#!/bin/bash
# Dubai Metaverse - Documentation Generation Script
# Generates documentation from templates and validates completeness
set -e # Exit on error
echo "=========================================="
echo "Dubai Metaverse - Documentation Generator"
echo "=========================================="
echo ""
# Check for required documentation files
REQUIRED_DOCS=(
"README.md"
"PROJECT_CHARTER.md"
"TECHNICAL_BRIEF.md"
"ART_BIBLE.md"
"DISTRICT_SELECTION.md"
"PROJECT_PLAN.md"
"PIPELINE.md"
"NAMING_CONVENTIONS.md"
"UE5_SETUP.md"
"PROJECT_SETTINGS.md"
"PLUGINS.md"
"VERSION_CONTROL.md"
"MILESTONES.md"
)
MISSING_DOCS=()
echo "Checking required documentation..."
echo ""
for doc in "${REQUIRED_DOCS[@]}"; do
if [ -f "$doc" ]; then
echo "$doc"
else
echo "⚠ Missing: $doc"
MISSING_DOCS+=("$doc")
fi
done
echo ""
if [ ${#MISSING_DOCS[@]} -eq 0 ]; then
echo "✓ All required documentation files present"
else
echo "⚠ Missing documentation files:"
for doc in "${MISSING_DOCS[@]}"; do
echo " - $doc"
done
fi
echo ""
echo "=========================================="
echo "Documentation Check Complete"
echo "=========================================="
echo ""

158
scripts/gis_to_unreal.py Executable file
View File

@@ -0,0 +1,158 @@
#!/usr/bin/env python3
"""
Dubai Metaverse - GIS to Unreal Terrain Conversion Script
Converts elevation data (DEM/GeoTIFF) to Unreal Engine terrain format
"""
import os
import sys
import argparse
from typing import Tuple, Optional
try:
import rasterio
import numpy as np
except ImportError:
print("Error: Required packages not installed.")
print("Install with: pip install rasterio numpy")
sys.exit(1)
def load_elevation_data(dem_file: str) -> Tuple[np.ndarray, dict]:
"""
Load elevation data from GeoTIFF file.
Returns:
elevation_data: NumPy array of elevation values
metadata: Dictionary with geospatial metadata
"""
try:
with rasterio.open(dem_file) as src:
elevation_data = src.read(1) # Read first band
metadata = {
'width': src.width,
'height': src.height,
'crs': src.crs,
'transform': src.transform,
'bounds': src.bounds
}
return elevation_data, metadata
except Exception as e:
print(f"Error loading elevation data: {e}")
sys.exit(1)
def normalize_elevation(elevation_data: np.ndarray, min_elev: float, max_elev: float) -> np.ndarray:
"""
Normalize elevation data to 0-1 range for Unreal Engine.
Unreal Engine uses 0-1 normalized height values.
"""
# Clip to min/max range
elevation_data = np.clip(elevation_data, min_elev, max_elev)
# Normalize to 0-1
normalized = (elevation_data - min_elev) / (max_elev - min_elev)
return normalized
def export_heightmap(normalized_data: np.ndarray, output_file: str, format: str = 'raw'):
"""
Export normalized elevation data as heightmap.
Formats:
- 'raw': Raw binary format (16-bit)
- 'png': PNG format (16-bit grayscale)
"""
# Convert to 16-bit integer (0-65535)
heightmap = (normalized_data * 65535).astype(np.uint16)
if format == 'raw':
heightmap.tofile(output_file)
print(f"✓ Exported heightmap to {output_file} (RAW format)")
elif format == 'png':
try:
from PIL import Image
# Convert to 16-bit PNG
img = Image.fromarray(heightmap, mode='I;16')
img.save(output_file)
print(f"✓ Exported heightmap to {output_file} (PNG format)")
except ImportError:
print("Warning: PIL not installed, falling back to RAW format")
heightmap.tofile(output_file)
print(f"✓ Exported heightmap to {output_file} (RAW format)")
else:
print(f"Error: Unknown format '{format}'")
sys.exit(1)
def main():
parser = argparse.ArgumentParser(description='Convert GIS elevation data to Unreal terrain')
parser.add_argument('input', help='Input DEM/GeoTIFF file')
parser.add_argument('--output', '-o', default='data/processed/terrain_heightmap.raw',
help='Output heightmap file')
parser.add_argument('--format', choices=['raw', 'png'], default='raw',
help='Output format (raw or png)')
parser.add_argument('--min-elev', type=float, help='Minimum elevation (meters)')
parser.add_argument('--max-elev', type=float, help='Maximum elevation (meters)')
args = parser.parse_args()
if not os.path.exists(args.input):
print(f"Error: Input file not found: {args.input}")
sys.exit(1)
# Create output directory if it doesn't exist
os.makedirs(os.path.dirname(args.output), exist_ok=True)
print("==========================================")
print("Dubai Metaverse - GIS to Unreal Terrain")
print("==========================================")
print("")
print(f"Loading elevation data: {args.input}")
elevation_data, metadata = load_elevation_data(args.input)
print(f"Data dimensions: {metadata['width']} x {metadata['height']}")
print(f"Bounds: {metadata['bounds']}")
print("")
# Determine elevation range
if args.min_elev is None:
min_elev = float(np.nanmin(elevation_data))
else:
min_elev = args.min_elev
if args.max_elev is None:
max_elev = float(np.nanmax(elevation_data))
else:
max_elev = args.max_elev
print(f"Elevation range: {min_elev:.2f}m to {max_elev:.2f}m")
print("")
print("Normalizing elevation data...")
normalized = normalize_elevation(elevation_data, min_elev, max_elev)
print("Exporting heightmap...")
export_heightmap(normalized, args.output, args.format)
print("")
print("==========================================")
print("Conversion Complete")
print("==========================================")
print("")
print("Next steps:")
print("1. Import heightmap to Unreal Engine")
print("2. Create landscape from heightmap")
print("3. Adjust landscape material and settings")
print("")
print(f"Heightmap file: {args.output}")
print(f"Dimensions: {metadata['width']} x {metadata['height']}")
print("")
if __name__ == '__main__':
main()

134
scripts/import_osm_data.py Executable file
View File

@@ -0,0 +1,134 @@
#!/usr/bin/env python3
"""
Dubai Metaverse - OpenStreetMap Data Import Script
Imports OpenStreetMap data for Dubai Marina and converts to Unreal-compatible format
"""
import os
import sys
import json
import argparse
from typing import List, Dict, Tuple
try:
import overpy
import geojson
except ImportError:
print("Error: Required packages not installed.")
print("Install with: pip install overpy geojson")
sys.exit(1)
def get_marina_buildings(api: overpy.Overpass) -> List[Dict]:
"""
Query OpenStreetMap for buildings in Dubai Marina area.
Note: Coordinates should be adjusted based on actual Dubai Marina location.
"""
# Dubai Marina approximate bounding box
# These coordinates should be verified and adjusted
query = """
[out:json][timeout:25];
(
way["building"](25.0750,55.1350,25.0850,55.1450);
);
out geom;
"""
try:
result = api.query(query)
buildings = []
for way in result.ways:
building = {
'id': way.id,
'nodes': [(node.lat, node.lon) for node in way.nodes],
'tags': way.tags
}
buildings.append(building)
return buildings
except Exception as e:
print(f"Error querying OpenStreetMap: {e}")
return []
def buildings_to_geojson(buildings: List[Dict], output_file: str):
"""Convert buildings to GeoJSON format."""
features = []
for building in buildings:
# Create polygon from nodes
coordinates = [[node[1], node[0]] for node in building['nodes']] # GeoJSON uses [lon, lat]
coordinates.append(coordinates[0]) # Close polygon
feature = {
'type': 'Feature',
'geometry': {
'type': 'Polygon',
'coordinates': [coordinates]
},
'properties': {
'id': building['id'],
'tags': building['tags']
}
}
features.append(feature)
geojson_data = {
'type': 'FeatureCollection',
'features': features
}
with open(output_file, 'w') as f:
json.dump(geojson_data, f, indent=2)
print(f"✓ Exported {len(features)} buildings to {output_file}")
def main():
parser = argparse.ArgumentParser(description='Import OpenStreetMap data for Dubai Marina')
parser.add_argument('--output', '-o', default='data/processed/dubai_marina_buildings.geojson',
help='Output GeoJSON file path')
parser.add_argument('--area', help='Custom bounding box (lat1,lon1,lat2,lon2)')
args = parser.parse_args()
# Create output directory if it doesn't exist
os.makedirs(os.path.dirname(args.output), exist_ok=True)
print("==========================================")
print("Dubai Metaverse - OSM Data Import")
print("==========================================")
print("")
print("Connecting to OpenStreetMap API...")
api = overpy.Overpass()
print("Querying Dubai Marina buildings...")
buildings = get_marina_buildings(api)
if not buildings:
print("⚠ No buildings found. Check coordinates and query.")
return
print(f"Found {len(buildings)} buildings")
print("")
print("Converting to GeoJSON...")
buildings_to_geojson(buildings, args.output)
print("")
print("==========================================")
print("Import Complete")
print "=========================================="
print("")
print("Next steps:")
print("1. Review GeoJSON file in GIS software (QGIS)")
print("2. Import to Unreal Engine using GIS import tools")
print("3. Generate building meshes from footprints")
print("")
if __name__ == '__main__':
main()

12
scripts/lighting_validation.py Executable file
View File

@@ -0,0 +1,12 @@
#!/usr/bin/env python3
"""
Dubai Metaverse - Lighting Validation Script
Validates lighting setup and performance
"""
# Placeholder script - to be implemented during Phase 3, Week 6
# This script will validate lighting configuration and performance
print("Lighting Validation Script")
print("To be implemented during Phase 3, Week 6")

12
scripts/npc_dialogue_api.py Executable file
View File

@@ -0,0 +1,12 @@
#!/usr/bin/env python3
"""
Dubai Metaverse - NPC Dialogue API Integration
Integrates ChatGPT API for dynamic NPC dialogue
"""
# Placeholder script - to be implemented during Phase 3, Week 8 (optional)
# This script will integrate ChatGPT API for NPC dialogue
print("NPC Dialogue API Integration")
print("To be implemented during Phase 3, Week 8 (optional)")

11
scripts/package_build.sh Executable file
View File

@@ -0,0 +1,11 @@
#!/bin/bash
# Dubai Metaverse - Build Packaging Script
# Packages the final build for distribution
# Placeholder script - to be implemented during Phase 5, Week 11-12
# This script will package the final build
echo "Build Packaging Script"
echo "To be implemented during Phase 5, Week 11-12"

12
scripts/pcg_validation.py Executable file
View File

@@ -0,0 +1,12 @@
#!/usr/bin/env python3
"""
Dubai Metaverse - PCG Validation Script
Validates Procedural Content Generation graphs and outputs
"""
# Placeholder script - to be implemented during Phase 2, Week 3
# This script will validate PCG graph outputs and placement rules
print("PCG Validation Script")
print("To be implemented during Phase 2, Week 3")

12
scripts/performance_audit.py Executable file
View File

@@ -0,0 +1,12 @@
#!/usr/bin/env python3
"""
Dubai Metaverse - Performance Audit Script
Audits performance metrics and identifies bottlenecks
"""
# Placeholder script - to be implemented during Phase 4, Week 10
# This script will audit performance and generate optimization reports
print("Performance Audit Script")
print("To be implemented during Phase 4, Week 10")

12
scripts/render_cinematic.py Executable file
View File

@@ -0,0 +1,12 @@
#!/usr/bin/env python3
"""
Dubai Metaverse - Cinematic Render Script
Automates Movie Render Queue rendering
"""
# Placeholder script - to be implemented during Phase 4, Week 10
# This script will automate Movie Render Queue rendering
print("Cinematic Render Script")
print("To be implemented during Phase 4, Week 10")

View File

@@ -0,0 +1,11 @@
#!/bin/bash
# Dubai Metaverse - Pixel Streaming Setup Script
# Sets up Pixel Streaming for cloud-based access
# Placeholder script - to be implemented during Phase 5 (optional)
# This script will configure Pixel Streaming
echo "Pixel Streaming Setup Script"
echo "To be implemented during Phase 5 (optional)"

87
scripts/setup_project.sh Executable file
View File

@@ -0,0 +1,87 @@
#!/bin/bash
# Dubai Metaverse - Project Setup Script
# This script sets up the initial project structure and configuration
set -e # Exit on error
echo "=========================================="
echo "Dubai Metaverse - Project Setup"
echo "=========================================="
echo ""
# Check if Git is installed
if ! command -v git &> /dev/null; then
echo "Error: Git is not installed. Please install Git first."
exit 1
fi
# Check if Git LFS is installed
if ! command -v git-lfs &> /dev/null; then
echo "Warning: Git LFS is not installed. Large file tracking will not work."
echo "Please install Git LFS: https://git-lfs.github.com"
read -p "Continue anyway? (y/n) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
exit 1
fi
else
echo "✓ Git LFS is installed"
# Initialize Git LFS
git lfs install
echo "✓ Git LFS initialized"
fi
# Initialize Git repository if not already initialized
if [ ! -d ".git" ]; then
echo "Initializing Git repository..."
git init
echo "✓ Git repository initialized"
else
echo "✓ Git repository already initialized"
fi
# Create directory structure
echo ""
echo "Creating directory structure..."
mkdir -p TASKS
mkdir -p docs
mkdir -p scripts
mkdir -p PROGRESS_REPORTS
mkdir -p houdini
mkdir -p data/{osm,elevation,references,processed}
echo "✓ Directory structure created"
# Set script permissions
echo ""
echo "Setting script permissions..."
chmod +x scripts/*.sh 2>/dev/null || true
echo "✓ Script permissions set"
# Check Python installation (for Python scripts)
if command -v python3 &> /dev/null; then
echo "✓ Python 3 is installed"
PYTHON_VERSION=$(python3 --version)
echo " Version: $PYTHON_VERSION"
else
echo "Warning: Python 3 is not installed. Some scripts may not work."
fi
# Summary
echo ""
echo "=========================================="
echo "Setup Complete!"
echo "=========================================="
echo ""
echo "Next steps:"
echo "1. Review documentation in root directory"
echo "2. Follow UE5_SETUP.md to install Unreal Engine 5.4"
echo "3. Create Unreal Engine project"
echo "4. Follow PROJECT_SETTINGS.md to configure engine"
echo "5. Install plugins (see PLUGINS.md)"
echo "6. Begin Phase 1, Week 2: Geospatial acquisition"
echo ""
echo "For more information, see README.md"
echo ""

94
scripts/setup_ue5_project.sh Executable file
View File

@@ -0,0 +1,94 @@
#!/bin/bash
# Dubai Metaverse - UE5 Project Setup Script
# This script provides instructions and validation for UE5 project setup
set -e # Exit on error
echo "=========================================="
echo "Dubai Metaverse - UE5 Project Setup"
echo "=========================================="
echo ""
# Check if UE5 project file exists
if [ -f "DubaiMetaverse.uproject" ]; then
echo "✓ UE5 project file found: DubaiMetaverse.uproject"
else
echo "⚠ UE5 project file not found"
echo ""
echo "Please create the UE5 project manually:"
echo "1. Open Epic Games Launcher"
echo "2. Launch Unreal Engine 5.4"
echo "3. Create new project: 'DubaiMetaverse'"
echo "4. Template: Blank (or Third Person)"
echo "5. Blueprint (C++ can be added later)"
echo "6. Target Platform: Desktop"
echo "7. Quality Preset: Maximum"
echo "8. Starter Content: No"
echo "9. Save project in this directory"
echo ""
read -p "Press Enter when project is created..."
fi
# Check for Content directory
if [ -d "Content" ]; then
echo "✓ Content directory found"
# Check for expected folder structure
echo ""
echo "Checking folder structure..."
DIRS=("Content/Maps" "Content/Assets" "Content/Blueprints" "Content/PCG" "Content/Cinematics" "Content/Audio")
MISSING_DIRS=()
for dir in "${DIRS[@]}"; do
if [ ! -d "$dir" ]; then
MISSING_DIRS+=("$dir")
fi
done
if [ ${#MISSING_DIRS[@]} -eq 0 ]; then
echo "✓ All expected directories exist"
else
echo "⚠ Missing directories:"
for dir in "${MISSING_DIRS[@]}"; do
echo " - $dir"
done
echo ""
read -p "Create missing directories? (y/n) " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
for dir in "${MISSING_DIRS[@]}"; do
mkdir -p "$dir"
echo "✓ Created: $dir"
done
fi
fi
else
echo "⚠ Content directory not found"
echo "This script should be run from the UE5 project root directory"
exit 1
fi
# Check for Config directory
if [ -d "Config" ]; then
echo "✓ Config directory found"
else
echo "⚠ Config directory not found"
echo "This may be normal for a new project"
fi
# Summary
echo ""
echo "=========================================="
echo "UE5 Project Setup Check Complete"
echo "=========================================="
echo ""
echo "Next steps:"
echo "1. Open project in Unreal Editor"
echo "2. Configure project settings (see PROJECT_SETTINGS.md)"
echo "3. Install required plugins (see PLUGINS.md)"
echo "4. Set up World Partition"
echo "5. Begin asset import"
echo ""

12
scripts/texture_validation.py Executable file
View File

@@ -0,0 +1,12 @@
#!/usr/bin/env python3
"""
Dubai Metaverse - Texture Validation Script
Validates texture resolution, format, and naming conventions
"""
# Placeholder script - to be implemented during Phase 2, Week 4-5
# This script will validate texture files for resolution, format, and naming
print("Texture Validation Script")
print("To be implemented during Phase 2, Week 4-5")

87
scripts/validate_assets.sh Executable file
View File

@@ -0,0 +1,87 @@
#!/bin/bash
# Dubai Metaverse - Asset Validation Script
# Validates asset naming conventions and structure
set -e # Exit on error
echo "=========================================="
echo "Dubai Metaverse - Asset Validation"
echo "=========================================="
echo ""
# Check if Content directory exists
if [ ! -d "Content" ]; then
echo "Error: Content directory not found"
echo "Run this script from the UE5 project root directory"
exit 1
fi
ERRORS=0
WARNINGS=0
# Function to check naming convention
check_naming() {
local file="$1"
local basename=$(basename "$file")
# Check for spaces
if [[ "$basename" == *" "* ]]; then
echo " ❌ ERROR: Contains spaces: $basename"
((ERRORS++))
return 1
fi
# Check for special characters (except underscores and hyphens)
if [[ "$basename" =~ [^a-zA-Z0-9_\-\.] ]]; then
echo " ❌ ERROR: Contains invalid characters: $basename"
((ERRORS++))
return 1
fi
# Check prefix based on extension
local ext="${basename##*.}"
case "$ext" in
uasset)
if [[ ! "$basename" =~ ^(SM_|SK_|M_|MI_|BP_|ABP_|BT_|P_|A_|SQ_|CineCamera_|DA_|WBP_|PCG_|D_|PP_) ]]; then
echo " ⚠ WARNING: Missing prefix: $basename"
((WARNINGS++))
fi
;;
*)
# Other file types - basic validation only
;;
esac
return 0
}
# Scan Content directory for assets
echo "Scanning Content directory..."
echo ""
find Content -type f -name "*.uasset" -o -name "*.umap" | while read -r file; do
check_naming "$file"
done
# Summary
echo ""
echo "=========================================="
echo "Validation Complete"
echo "=========================================="
echo ""
echo "Errors: $ERRORS"
echo "Warnings: $WARNINGS"
echo ""
if [ $ERRORS -eq 0 ] && [ $WARNINGS -eq 0 ]; then
echo "✓ All assets passed validation"
exit 0
elif [ $ERRORS -eq 0 ]; then
echo "⚠ Some warnings found, but no errors"
exit 0
else
echo "❌ Validation failed. Please fix errors before committing."
exit 1
fi