Add full monorepo: virtual-banker, backend, frontend, docs, scripts, deployment

Co-authored-by: Cursor <cursoragent@cursor.com>

docs/specs/mempool/realtime-events.md

@@ -0,0 +1,267 @@
+# Real-time Event System Specification
+
+## Overview
+
+This document specifies the event bus architecture for real-time event distribution across the explorer platform services.
+
+## Architecture
+
+**Technology**: Kafka or RabbitMQ
+
+**Recommendation**: Kafka for high throughput, RabbitMQ for simpler setup
+
+## Event Bus Architecture
+
+```mermaid
+flowchart LR
+    Producers[Event Producers]
+    Bus[Event Bus<br/>Kafka/RabbitMQ]
+    Consumers[Event Consumers]
+    
+    Producers --> Bus
+    Bus --> Consumers
+```
+
+## Event Types
+
+### Block Events
+
+**Event**: `block.new`
+
+**Payload**:
+```json
+{
+  "chain_id": 138,
+  "block_number": 12345,
+  "hash": "0x...",
+  "timestamp": "2024-01-01T00:00:00Z",
+  "transaction_count": 100
+}
+```
+
+**Producers**: Indexer
+**Consumers**: API Gateway (cache invalidation), WebSocket service, Analytics
+
+### Transaction Events
+
+**Event**: `transaction.confirmed`
+
+**Payload**:
+```json
+{
+  "chain_id": 138,
+  "hash": "0x...",
+  "block_number": 12345,
+  "from_address": "0x...",
+  "to_address": "0x...",
+  "status": "success"
+}
+```
+
+**Producers**: Indexer, Mempool service
+**Consumers**: WebSocket service, Search indexer, Analytics
+
+### Mempool Events
+
+**Event**: `transaction.pending`
+
+**Payload**: Transaction data
+
+**Producers**: Mempool service
+**Consumers**: WebSocket service, Fee oracle
+
+### Token Transfer Events
+
+**Event**: `token.transfer`
+
+**Payload**:
+```json
+{
+  "chain_id": 138,
+  "token_address": "0x...",
+  "from_address": "0x...",
+  "to_address": "0x...",
+  "amount": "1000000000000000000",
+  "transaction_hash": "0x..."
+}
+```
+
+**Producers**: Indexer
+**Consumers**: Token balance updater, Analytics, Notifications
+
+### Contract Verification Events
+
+**Event**: `contract.verified`
+
+**Payload**:
+```json
+{
+  "chain_id": 138,
+  "address": "0x...",
+  "verification_status": "verified",
+  "verified_at": "2024-01-01T00:00:00Z"
+}
+```
+
+**Producers**: Verification service
+**Consumers**: Search indexer, Cache invalidation
+
+## Event Schema
+
+### Standard Event Format
+
+```json
+{
+  "event_type": "block.new",
+  "event_id": "uuid",
+  "timestamp": "2024-01-01T00:00:00Z",
+  "chain_id": 138,
+  "payload": { ... },
+  "metadata": {
+    "producer": "indexer",
+    "version": "1.0"
+  }
+}
+```
+
+## Topic/Queue Structure
+
+### Kafka Topics
+
+**Naming**: `{event_type}.{chain_id}` (e.g., `block.new.138`)
+
+**Topics**:
+- `block.new.{chain_id}`
+- `transaction.confirmed.{chain_id}`
+- `transaction.pending.{chain_id}`
+- `token.transfer.{chain_id}`
+- `contract.verified.{chain_id}`
+
+**Partitioning**: By chain_id (all events for chain in same partition)
+
+### RabbitMQ Exchanges
+
+**Exchange Type**: Topic exchange
+
+**Exchange Name**: `explorer.events`
+
+**Routing Keys**: `{event_type}.{chain_id}` (e.g., `block.new.138`)
+
+## Consumer Groups and Partitioning
+
+### Consumer Groups
+
+**Purpose**: Enable parallel processing and load balancing
+
+**Groups**:
+- `websocket-service`: WebSocket real-time updates
+- `search-indexer`: Search index updates
+- `analytics`: Analytics aggregation
+- `cache-invalidation`: Cache invalidation
+
+### Partitioning Strategy
+
+**Kafka**:
+- Partition by chain_id
+- Same chain_id → same partition (maintains ordering)
+
+**RabbitMQ**:
+- Use consistent hash exchange for partitioning
+- Partition by chain_id
+
+## Delivery Guarantees
+
+### At-Least-Once Delivery
+
+**Default**: At-least-once delivery (events may be delivered multiple times)
+
+**Handling**:
+- Idempotent consumers
+- Deduplication by event_id
+- Track processed events
+
+### Exactly-Once Delivery
+
+**Use Case**: Critical financial events (banking layer)
+
+**Implementation**:
+- Kafka exactly-once semantics (if using Kafka)
+- Idempotent consumers with deduplication
+- Transactional processing
+
+## Backpressure Handling
+
+### Strategy
+
+**Flow Control**:
+- Consumer lag monitoring
+- Slow consumer detection
+- Automatic scaling
+
+**Handling Slow Consumers**:
+1. Scale consumers horizontally
+2. Increase consumer resources
+3. Alert on persistent lag
+
+### Lag Monitoring
+
+**Metrics**:
+- Consumer lag per topic
+- Processing rate
+- Consumer health
+
+**Alerts**: Lag > 1000 events or > 5 minutes
+
+## Event Ordering
+
+### Ordering Guarantees
+
+**Within Partition**: Events processed in order
+**Across Partitions**: No ordering guarantee
+
+**Use Cases Requiring Ordering**:
+- Block events (must process in order)
+- Transaction events per address (maintain order)
+
+**Implementation**: Use same partition for related events
+
+## Error Handling
+
+### Dead Letter Queue
+
+**Purpose**: Store failed events for retry or investigation
+
+**Strategy**:
+- Retry failed events (exponential backoff)
+- Move to DLQ after max retries
+- Alert on DLQ events
+
+### Retry Strategy
+
+**Configuration**:
+- Max retries: 3
+- Backoff: Exponential (1s, 2s, 4s)
+- Dead letter after max retries
+
+## Monitoring
+
+### Metrics
+
+- Event production rate
+- Event consumption rate
+- Consumer lag
+- Error rate
+- Dead letter queue size
+
+### Dashboards
+
+- Event throughput per topic
+- Consumer lag per consumer group
+- Error rates
+- System health
+
+## References
+
+- Mempool Service: See `mempool-service.md`
+- WebSocket API: See `../api/websocket-api.md`
+