Initial commit: Complete project foundation with all documentation, scripts, and project structure
This commit is contained in:
Dubai Metaverse Team
292
DATA_SOURCES.md
Normal file
292
DATA_SOURCES.md
Normal file
@@ -0,0 +1,292 @@
|
||||
# Data Sources - Dubai Metaverse
|
||||
|
||||
## Overview
|
||||
|
||||
This document lists all data sources for geospatial data, elevation data, and reference materials needed for the Dubai Metaverse project.
|
||||
|
||||
## Geospatial Data Sources
|
||||
|
||||
### OpenStreetMap (OSM)
|
||||
|
||||
**URL**: [openstreetmap.org](https://www.openstreetmap.org)
|
||||
|
||||
**Usage**: Building footprints, road networks, points of interest
|
||||
|
||||
**Data Format**:
|
||||
- OSM XML (.osm)
|
||||
- GeoJSON
|
||||
- Shapefile (via conversion)
|
||||
|
||||
**Extraction Methods**:
|
||||
1. **Overpass API**: Query specific areas
|
||||
2. **OSM Export**: Download area via export tool
|
||||
3. **Python Scripts**: Use `osmnx` or `overpy` libraries
|
||||
|
||||
**Target Area**: Dubai Marina district
|
||||
- Coordinates: [To be determined]
|
||||
- Bounding box: [To be determined]
|
||||
|
||||
**Script**: `scripts/import_osm_data.py`
|
||||
|
||||
---
|
||||
|
||||
### GIS Data Sources
|
||||
|
||||
#### Dubai Municipality GIS
|
||||
|
||||
**Source**: Dubai Municipality (if available)
|
||||
|
||||
**Usage**: Official building data, cadastral information
|
||||
|
||||
**Access**: May require permission/access
|
||||
|
||||
**Alternative**: Use OpenStreetMap if official data unavailable
|
||||
|
||||
---
|
||||
|
||||
#### USGS EarthExplorer
|
||||
|
||||
**URL**: [earthexplorer.usgs.gov](https://earthexplorer.usgs.gov)
|
||||
|
||||
**Usage**: Satellite imagery, elevation data
|
||||
|
||||
**Data Types**:
|
||||
- Landsat imagery
|
||||
- ASTER GDEM elevation data
|
||||
- SRTM elevation data
|
||||
|
||||
**Access**: Free registration required
|
||||
|
||||
---
|
||||
|
||||
### Elevation Data
|
||||
|
||||
#### ASTER GDEM
|
||||
|
||||
**Source**: USGS EarthExplorer
|
||||
|
||||
**Resolution**: 30m
|
||||
|
||||
**Usage**: Terrain generation
|
||||
|
||||
**Format**: GeoTIFF
|
||||
|
||||
**Script**: `scripts/gis_to_unreal.py`
|
||||
|
||||
---
|
||||
|
||||
#### SRTM
|
||||
|
||||
**Source**: USGS EarthExplorer
|
||||
|
||||
**Resolution**: 30m (SRTM-30) or 90m (SRTM-90)
|
||||
|
||||
**Usage**: Terrain generation (alternative to ASTER)
|
||||
|
||||
**Format**: GeoTIFF
|
||||
|
||||
---
|
||||
|
||||
#### OpenDEM
|
||||
|
||||
**URL**: [opentopography.org](https://opentopography.org)
|
||||
|
||||
**Usage**: High-resolution elevation data
|
||||
|
||||
**Access**: Free registration required
|
||||
|
||||
---
|
||||
|
||||
## Reference Materials
|
||||
|
||||
### Photography
|
||||
|
||||
#### On-Site Photography (If Available)
|
||||
|
||||
**Requirements**:
|
||||
- High-resolution camera (DSLR or mirrorless)
|
||||
- Multiple angles per building
|
||||
- Consistent lighting
|
||||
- Overlapping photos (70-80% overlap)
|
||||
|
||||
**Target Buildings**:
|
||||
- Cayan Tower (hero landmark)
|
||||
- Marina Towers
|
||||
- Primary buildings (if accessible)
|
||||
|
||||
**Usage**: Photogrammetry, texture reference
|
||||
|
||||
---
|
||||
|
||||
#### Stock Photography
|
||||
|
||||
**Sources**:
|
||||
- Unsplash
|
||||
- Pexels
|
||||
- Shutterstock (if budget allows)
|
||||
- Getty Images (if budget allows)
|
||||
|
||||
**Usage**: Reference images, texture sources
|
||||
|
||||
---
|
||||
|
||||
#### Google Earth / Street View
|
||||
|
||||
**URL**: [earth.google.com](https://earth.google.com)
|
||||
|
||||
**Usage**:
|
||||
- Aerial views
|
||||
- Street-level reference
|
||||
- Building context
|
||||
- Material reference
|
||||
|
||||
**Limitations**:
|
||||
- Image quality may vary
|
||||
- Copyright considerations
|
||||
- Not suitable for direct texture use
|
||||
|
||||
---
|
||||
|
||||
### Architectural Plans
|
||||
|
||||
#### Public Records
|
||||
|
||||
**Source**: Dubai Municipality (if available)
|
||||
|
||||
**Usage**: Accurate building dimensions, layouts
|
||||
|
||||
**Access**: May require permission
|
||||
|
||||
---
|
||||
|
||||
#### Online Resources
|
||||
|
||||
**Sources**:
|
||||
- Building websites
|
||||
- Architectural firm websites
|
||||
- Real estate listings
|
||||
|
||||
**Usage**: Building information, dimensions
|
||||
|
||||
---
|
||||
|
||||
## Data Processing Tools
|
||||
|
||||
### Python Libraries
|
||||
|
||||
- **osmnx**: OpenStreetMap data extraction
|
||||
- **overpy**: Overpass API queries
|
||||
- **rasterio**: GeoTIFF processing
|
||||
- **gdal**: GIS data processing
|
||||
- **shapely**: Geometric operations
|
||||
|
||||
### GIS Software
|
||||
|
||||
- **QGIS**: Free, open-source GIS
|
||||
- **ArcGIS**: Professional GIS (if available)
|
||||
- **Google Earth Pro**: Visualization and export
|
||||
|
||||
### Photogrammetry Tools
|
||||
|
||||
- **RealityCapture**: Professional photogrammetry
|
||||
- **Luma AI**: Simplified photogrammetry
|
||||
- **Agisoft Metashape**: Alternative photogrammetry
|
||||
|
||||
---
|
||||
|
||||
## Data Acquisition Workflow
|
||||
|
||||
### Step 1: OpenStreetMap Data
|
||||
|
||||
1. Identify target area (Dubai Marina)
|
||||
2. Extract building footprints via Overpass API
|
||||
3. Export to GeoJSON or Shapefile
|
||||
4. Process and clean data
|
||||
5. Import to Unreal (via script)
|
||||
|
||||
**Script**: `scripts/import_osm_data.py`
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Elevation Data
|
||||
|
||||
1. Identify area coordinates
|
||||
2. Download ASTER GDEM or SRTM data
|
||||
3. Process elevation data
|
||||
4. Convert to Unreal terrain format
|
||||
5. Import to Unreal
|
||||
|
||||
**Script**: `scripts/gis_to_unreal.py`
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Reference Materials
|
||||
|
||||
1. Collect photography (on-site or stock)
|
||||
2. Organize by building/area
|
||||
3. Use for modeling and texturing reference
|
||||
4. Store in reference folder
|
||||
|
||||
---
|
||||
|
||||
## Data Organization
|
||||
|
||||
### Folder Structure
|
||||
|
||||
```
|
||||
data/
|
||||
├── osm/
|
||||
│ └── dubai_marina.osm
|
||||
├── elevation/
|
||||
│ └── dubai_marina_dem.tif
|
||||
├── references/
|
||||
│ ├── cayan_tower/
|
||||
│ ├── marina_towers/
|
||||
│ └── buildings/
|
||||
└── processed/
|
||||
└── [processed data files]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Validation
|
||||
|
||||
### Checklist
|
||||
|
||||
- [ ] OSM data covers target area
|
||||
- [ ] Building footprints are accurate
|
||||
- [ ] Elevation data matches area
|
||||
- [ ] Reference materials are sufficient
|
||||
- [ ] Data formats are compatible
|
||||
- [ ] Data is properly organized
|
||||
|
||||
---
|
||||
|
||||
## Legal Considerations
|
||||
|
||||
### Copyright
|
||||
|
||||
- **OpenStreetMap**: Open Database License (ODbL)
|
||||
- **Photography**: Respect copyright, use licensed images
|
||||
- **Architectural Plans**: May require permission
|
||||
|
||||
### Attribution
|
||||
|
||||
- **OpenStreetMap**: Attribution required
|
||||
- **Other Sources**: Follow license requirements
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Extract OpenStreetMap data for Dubai Marina
|
||||
2. Download elevation data
|
||||
3. Collect reference photography
|
||||
4. Process and import data to Unreal
|
||||
5. Validate data accuracy
|
||||
|
||||
---
|
||||
|
||||
**Version**: 1.0
|
||||
**Last Updated**: [Current Date]
|
||||
|
||||
Reference in New Issue
Block a user