Enhance README.md with badges, installation instructions, and community support details. Improved project structure and added quick contribution guidelines.

2025-08-06 04:09:56 +00:00
parent c8fb525651
commit 9f9f217175
47 changed files with 14593 additions and 36 deletions
--- a/docs/API_REFERENCE.md
+++ b/docs/API_REFERENCE.md
@@ -0,0 +1,541 @@
+# NowYouSeeMe API Reference
+
+## Overview
+
+The NowYouSeeMe holodeck environment provides comprehensive APIs for multi-device connectivity, Azure integration, and real-time 6DOF pose estimation. This document describes the complete API surface for all components.
+
+## Table of Contents
+
+1. [Device Management API](#device-management-api)
+2. [Azure Integration API](#azure-integration-api)
+3. [SLAM Processing API](#slam-processing-api)
+4. [UI/UX API](#uiux-api)
+5. [Configuration API](#configuration-api)
+6. [Network Protocols](#network-protocols)
+
+## Device Management API
+
+### DeviceManager Class
+
+The `DeviceManager` class handles discovery, connection, and management of multiple device types simultaneously.
+
+#### Constructor
+```cpp
+DeviceManager(int discoveryPort = 8080, int maxConnections = 100)
+```
+
+#### Device Types
+- `WIFI_CARD` - WiFi network interfaces
+- `ACCESS_POINT` - WiFi access points
+- `CELL_PHONE` - Mobile devices
+- `HOTSPOT` - Mobile hotspots
+- `BLUETOOTH` - Bluetooth devices
+- `ZIGBEE` - ZigBee devices
+- `LORA` - LoRa devices
+- `CUSTOM` - Custom network interfaces
+
+#### Device Status
+- `DISCONNECTED` - Device not connected
+- `CONNECTING` - Connection in progress
+- `CONNECTED` - Successfully connected
+- `STREAMING` - Actively streaming data
+- `ERROR` - Connection error
+
+#### Methods
+
+##### Discovery Management
+```cpp
+bool startDiscovery()
+void stopDiscovery()
+```
+
+##### Device Connection
+```cpp
+bool connectToDevice(const std::string& deviceId)
+bool disconnectFromDevice(const std::string& deviceId)
+void disconnectAll()
+```
+
+##### Data Transmission
+```cpp
+bool sendData(const std::string& deviceId, const std::vector<uint8_t>& data)
+std::vector<uint8_t> receiveData(const std::string& deviceId, int timeout_ms = 1000)
+```
+
+##### Device Information
+```cpp
+std::vector<DeviceInfo> getConnectedDevices()
+std::vector<DeviceInfo> getAllDevices()
+DeviceInfo getDeviceInfo(const std::string& deviceId)
+```
+
+##### Callbacks
+```cpp
+void setDeviceCallback(std::function<void(const DeviceInfo&)> callback)
+void setDataCallback(std::function<void(const std::string&, const std::vector<uint8_t>&)> callback)
+```
+
+##### Statistics
+```cpp
+DeviceStats getStats()
+```
+
+#### DeviceInfo Structure
+```cpp
+struct DeviceInfo {
+    std::string id;
+    std::string name;
+    DeviceType type;
+    DeviceStatus status;
+    std::string mac_address;
+    std::string ip_address;
+    int port;
+    double signal_strength;
+    std::map<std::string, std::string> capabilities;
+    std::chrono::steady_clock::time_point last_seen;
+    bool is_active;
+};
+```
+
+#### DeviceStats Structure
+```cpp
+struct DeviceStats {
+    int total_devices;
+    int connected_devices;
+    int active_streams;
+    double average_signal_strength;
+    std::chrono::steady_clock::time_point last_update;
+};
+```
+
+## Azure Integration API
+
+### AzureIntegration Class
+
+The `AzureIntegration` class provides GPU computing and AI Foundry resources through Azure services.
+
+#### Constructor
+```cpp
+AzureIntegration(const AzureConfig& config)
+```
+
+#### AzureConfig Structure
+```cpp
+struct AzureConfig {
+    std::string subscription_id;
+    std::string resource_group;
+    std::string location;
+    std::string storage_account;
+    std::string container_name;
+    std::string compute_cluster;
+    std::string ai_workspace;
+    std::string tenant_id;
+    std::string client_id;
+    std::string client_secret;
+};
+```
+
+#### Methods
+
+##### Authentication
+```cpp
+bool authenticate()
+```
+
+##### GPU Resource Management
+```cpp
+bool provisionGPUResource(const std::string& vmName, const std::string& gpuType, 
+                         int gpuCount, int memoryGB)
+bool deprovisionGPUResource(const std::string& vmName)
+std::vector<GPUResource> getAvailableGPUResources()
+```
+
+##### AI Foundry Integration
+```cpp
+bool setupAIWorkspace(const std::string& workspaceName)
+bool deployModel(const std::string& workspaceName, const std::string& modelName,
+                const std::vector<uint8_t>& modelData)
+```
+
+##### Compute Job Management
+```cpp
+std::string submitComputeJob(const std::string& jobType, const std::string& resourceId,
+                            const std::vector<uint8_t>& inputData,
+                            std::function<void(const std::vector<uint8_t>&)> callback)
+bool cancelJob(const std::string& jobId)
+std::vector<ComputeJob> getActiveJobs()
+```
+
+##### Monitoring
+```cpp
+bool startMonitoring()
+void stopMonitoring()
+```
+
+##### Callbacks
+```cpp
+void setJobCompletionCallback(std::function<void(const std::string&, const std::vector<uint8_t>&)> callback)
+void setResourceStatusCallback(std::function<void(const std::string&, const std::string&)> callback)
+```
+
+##### Statistics
+```cpp
+AzureStats getStats()
+```
+
+#### GPUResource Structure
+```cpp
+struct GPUResource {
+    std::string vm_id;
+    std::string vm_name;
+    std::string gpu_type;
+    int gpu_count;
+    int memory_gb;
+    std::string status;
+    std::chrono::steady_clock::time_point last_used;
+    bool is_available;
+};
+```
+
+#### AIFoundryResource Structure
+```cpp
+struct AIFoundryResource {
+    std::string workspace_id;
+    std::string workspace_name;
+    std::string compute_target;
+    std::string model_registry;
+    std::vector<std::string> available_models;
+    std::string status;
+    int active_jobs;
+    int max_jobs;
+};
+```
+
+#### ComputeJob Structure
+```cpp
+struct ComputeJob {
+    std::string job_id;
+    std::string job_type;
+    std::string resource_id;
+    std::string status;
+    std::chrono::steady_clock::time_point start_time;
+    std::chrono::steady_clock::time_point end_time;
+    double progress;
+    std::vector<uint8_t> input_data;
+    std::vector<uint8_t> output_data;
+    std::function<void(const std::vector<uint8_t>&)> callback;
+};
+```
+
+## SLAM Processing API
+
+### SLAMProcessor Class
+
+The `SLAMProcessor` class handles real-time SLAM processing with RF-vision fusion.
+
+#### Methods
+```cpp
+void start()
+void stop()
+void setParameters(const SLAMParameters& params)
+SLAMResult getCurrentPose()
+std::vector<MapPoint> getMapPoints()
+```
+
+#### SLAMParameters Structure
+```cpp
+struct SLAMParameters {
+    double map_scale;
+    int update_rate;
+    bool enable_rf_fusion;
+    bool enable_vision_slam;
+    bool enable_nerf_rendering;
+};
+```
+
+#### SLAMResult Structure
+```cpp
+struct SLAMResult {
+    Eigen::Matrix4d pose;
+    double confidence;
+    std::chrono::steady_clock::time_point timestamp;
+    std::vector<double> uncertainties;
+};
+```
+
+## UI/UX API
+
+### HolodeckUI Class
+
+The `HolodeckUI` class provides a comprehensive PyQt6-based user interface.
+
+#### Constructor
+```python
+HolodeckUI()
+```
+
+#### Methods
+
+##### Device Management
+```python
+def start_device_discovery(self)
+def stop_device_discovery(self)
+def connect_to_device(self)
+def disconnect_from_device(self)
+```
+
+##### SLAM Processing
+```python
+def start_slam_processing(self)
+def stop_slam_processing(self)
+```
+
+##### Azure Integration
+```python
+def connect_to_azure(self)
+def disconnect_from_azure(self)
+def provision_gpu_resource(self)
+def deprovision_gpu_resource(self)
+def setup_ai_workspace(self)
+def deploy_model(self)
+```
+
+##### View Management
+```python
+def set_view(self, view_type: str)
+def update_camera_position(self)
+```
+
+##### Configuration
+```python
+def open_configuration(self)
+def save_configuration(self)
+def calibrate_camera(self)
+```
+
+#### View Types
+- `"3D"` - 3D OpenGL visualization
+- `"2D"` - 2D map view
+- `"NeRF"` - Neural Radiance Fields rendering
+
+### HolodeckGLWidget Class
+
+OpenGL widget for 3D visualization.
+
+#### Methods
+```python
+def set_view_type(self, view_type: str)
+def set_camera_position(self, x: float, y: float, z: float)
+def update(self)
+```
+
+## Configuration API
+
+### Configuration File Format
+
+The system uses JSON configuration files for all settings.
+
+#### Main Configuration
+```json
+{
+  "holodeck": {
+    "enabled": true,
+    "view_type": "3D",
+    "update_rate": 30
+  },
+  "device_management": {
+    "discovery_port": 8080,
+    "max_connections": 100,
+    "discovery_interval": 5,
+    "connection_timeout": 30
+  },
+  "azure_integration": {
+    "subscription_id": "your-subscription-id",
+    "resource_group": "nowyouseeme-rg",
+    "location": "eastus",
+    "tenant_id": "your-tenant-id",
+    "client_id": "your-client-id",
+    "client_secret": "your-client-secret"
+  },
+  "slam_processing": {
+    "map_scale": 1.0,
+    "update_rate": 30,
+    "enable_rf_fusion": true,
+    "enable_vision_slam": true,
+    "enable_nerf_rendering": true
+  },
+  "ui_settings": {
+    "window_width": 1600,
+    "window_height": 1200,
+    "theme": "dark",
+    "auto_save": true
+  }
+}
+```
+
+## Network Protocols
+
+### Device Communication Protocol
+
+#### Connection Establishment
+1. Device discovery via UDP broadcast on port 8080
+2. TCP connection establishment on device-specific port
+3. Handshake protocol for capability negotiation
+4. Data streaming with protocol headers
+
+#### Message Format
+```
+[Header: 8 bytes][Payload: variable length]
+Header: [MessageType: 2 bytes][Length: 4 bytes][Flags: 2 bytes]
+```
+
+#### Message Types
+- `0x0001` - Device Info
+- `0x0002` - Data Stream
+- `0x0003` - Control Command
+- `0x0004` - Status Update
+- `0x0005` - Error Report
+
+### Azure REST API Integration
+
+#### Authentication
+- OAuth 2.0 client credentials flow
+- Bearer token authentication
+- Automatic token refresh
+
+#### GPU Resource Management
+- Azure Compute REST API
+- VM provisioning with GPU extensions
+- Real-time status monitoring
+
+#### AI Foundry Integration
+- Azure Machine Learning REST API
+- Model deployment and management
+- Workspace and compute target management
+
+## Error Handling
+
+### Error Codes
+- `0x0001` - Device not found
+- `0x0002` - Connection failed
+- `0x0003` - Authentication failed
+- `0x0004` - Resource unavailable
+- `0x0005` - Invalid parameters
+- `0x0006` - Timeout
+- `0x0007` - Network error
+- `0x0008` - Azure API error
+
+### Exception Handling
+```cpp
+try {
+    device_manager.connectToDevice(device_id);
+} catch (const DeviceException& e) {
+    spdlog::error("Device error: {}", e.what());
+} catch (const NetworkException& e) {
+    spdlog::error("Network error: {}", e.what());
+} catch (const AzureException& e) {
+    spdlog::error("Azure error: {}", e.what());
+}
+```
+
+## Performance Considerations
+
+### Multi-threading
+- Device discovery runs in separate thread
+- Data reception uses thread pool
+- UI updates on main thread
+- Azure operations use async/await pattern
+
+### Memory Management
+- Device connections use smart pointers
+- Data buffers are pre-allocated
+- GPU memory is managed by Azure
+- UI elements are properly disposed
+
+### Network Optimization
+- Connection pooling for multiple devices
+- Data compression for large payloads
+- Heartbeat mechanism for connection monitoring
+- Automatic reconnection on failure
+
+## Security
+
+### Authentication
+- Azure AD integration for cloud resources
+- Device-specific authentication tokens
+- Encrypted communication channels
+- Certificate-based device verification
+
+### Data Protection
+- End-to-end encryption for sensitive data
+- Secure storage of configuration files
+- Access control for device management
+- Audit logging for all operations
+
+## Examples
+
+### Basic Device Management
+```cpp
+DeviceManager manager(8080, 100);
+manager.setDeviceCallback([](const DeviceInfo& device) {
+    std::cout << "Found device: " << device.name << std::endl;
+});
+
+manager.startDiscovery();
+manager.connectToDevice("device-001");
+```
+
+### Azure GPU Provisioning
+```cpp
+AzureConfig config;
+config.subscription_id = "your-subscription-id";
+// ... set other config parameters
+
+AzureIntegration azure(config);
+azure.authenticate();
+azure.provisionGPUResource("gpu-vm-001", "V100", 1, 32);
+```
+
+### UI Integration
+```python
+ui = HolodeckUI()
+ui.start_device_discovery()
+ui.connect_to_azure()
+ui.start_slam_processing()
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Device Discovery Fails**
+   - Check network permissions
+   - Verify firewall settings
+   - Ensure discovery port is available
+
+2. **Azure Authentication Fails**
+   - Verify credentials in configuration
+   - Check Azure AD permissions
+   - Ensure subscription is active
+
+3. **SLAM Processing Errors**
+   - Check camera calibration
+   - Verify sensor data quality
+   - Monitor system resources
+
+4. **UI Performance Issues**
+   - Reduce update rate
+   - Disable unnecessary features
+   - Monitor GPU usage
+
+### Debug Mode
+Enable debug logging by setting environment variable:
+```bash
+export NOWYOUSEE_DEBUG=1
+```
+
+### Log Files
+- Device logs: `/var/log/nowyouseeme/device.log`
+- Azure logs: `/var/log/nowyouseeme/azure.log`
+- SLAM logs: `/var/log/nowyouseeme/slam.log`
+- UI logs: `/var/log/nowyouseeme/ui.log`