Add full monorepo: virtual-banker, backend, frontend, docs, scripts, deployment

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-10 11:32:49 -08:00
parent aafcd913c2
commit 88bc76da91
815 changed files with 125522 additions and 264 deletions
--- a/docs/specs/multichain/multichain-indexing.md
+++ b/docs/specs/multichain/multichain-indexing.md
@@ -0,0 +1,179 @@
+# Multi-Chain Indexing Strategy
+
+## Overview
+
+This document specifies the strategy for indexing multiple blockchain networks with shared schema and per-chain workers.
+
+## Architecture
+
+```mermaid
+flowchart TB
+    subgraph Chains[Blockchain Networks]
+        C1[Chain 138]
+        C2[Chain 1]
+        C3[Chain 137]
+    end
+    
+    subgraph Workers[Indexer Workers]
+        W1[Worker: Chain 138]
+        W2[Worker: Chain 1]
+        W3[Worker: Chain 137]
+    end
+    
+    subgraph DB[(Shared Database)]
+        PG[(PostgreSQL<br/>chain_id partitioned)]
+        ES[(Elasticsearch<br/>per-chain indices)]
+    end
+    
+    C1 --> W1
+    C2 --> W2
+    C3 --> W3
+    
+    W1 --> PG
+    W2 --> PG
+    W3 --> PG
+    
+    W1 --> ES
+    W2 --> ES
+    W3 --> ES
+```
+
+## Per-Chain Indexer Workers
+
+### Worker Architecture
+
+**Design**: One indexer worker process per chain
+
+**Isolation**:
+- Separate processes/containers per chain
+- Independent scaling per chain
+- Chain-specific configuration
+
+**Shared Components**:
+- Database connection pool
+- Message queue consumers
+- Common indexing logic
+
+### Worker Configuration
+
+```yaml
+workers:
+  - chain_id: 138
+    name: "indexer-chain-138"
+    rpc_url: "http://192.168.11.221:8545"  # Internal IP for direct connection
+    adapter: "evm"
+    enabled: true
+    
+  - chain_id: 1
+    name: "indexer-chain-1"
+    rpc_url: "https://eth-mainnet.alchemyapi.io/..."
+    adapter: "evm"
+    enabled: true
+```
+
+### Worker Scaling
+
+**Horizontal Scaling**:
+- Multiple workers per chain for high-throughput chains
+- Partition blocks across workers (by block number range)
+- Coordinate via database locks or message queue
+
+**Scaling Strategy**:
+- Scale workers based on chain activity
+- Monitor worker lag per chain
+- Auto-scale based on metrics
+
+## Shared Schema with Chain ID Partitioning
+
+### Database Partitioning
+
+**Strategy**: Partition all tables by `chain_id`
+
+**Benefits**:
+- Efficient queries (partition pruning)
+- Independent maintenance per chain
+- Easy data isolation
+
+**Implementation**: See `../database/postgres-schema.md`
+
+### Index Strategy
+
+**Global Indexes**: Across all chains for unified queries
+**Chain-Specific Indexes**: Within partitions for chain-specific queries
+
+## Cross-Chain Data Consistency
+
+### Consistency Model
+
+**Per-Chain Consistency**: Strong consistency within each chain
+**Cross-Chain Consistency**: Eventual consistency (independent chains)
+
+### Transaction Ordering
+
+**Within Chain**: Maintain strict block/transaction ordering
+**Across Chains**: No ordering guarantees (independent chains)
+
+## Worker Coordination
+
+### Block Processing Coordination
+
+**Strategy**: Use database locks or message queue for coordination
+
+**Approach 1: Database Locks**
+- Acquire lock before processing block
+- Release lock after completion
+- Prevent duplicate processing
+
+**Approach 2: Message Queue**
+- Single consumer per chain partition
+- Queue ensures ordering
+- Automatic retry on failure
+
+### Checkpointing
+
+**Per-Chain Checkpoints**:
+- Track last processed block per chain
+- Stored in database
+- Enable recovery and resume
+
+## Performance Optimization
+
+### Chain-Specific Optimizations
+
+**High-Activity Chains**:
+- More workers
+- Larger batch sizes
+- Optimized processing
+
+**Low-Activity Chains**:
+- Fewer workers
+- Longer polling intervals
+- Resource-efficient processing
+
+### Resource Allocation
+
+**CPU/Memory**: Allocate based on chain activity
+**Database Connections**: Pool per chain
+**Network Bandwidth**: Prioritize high-activity chains
+
+## Monitoring
+
+### Per-Chain Metrics
+
+- Block lag (current block - last indexed)
+- Processing rate (blocks/minute)
+- Error rate
+- Worker health
+
+### Cross-Chain Metrics
+
+- Total chains indexed
+- Total blocks indexed across all chains
+- System-wide throughput
+
+## References
+
+- Chain Adapter Interface: See `chain-adapter-interface.md`
+- Database Schema: See `../database/postgres-schema.md`
+- Indexer Architecture: See `../indexing/indexer-architecture.md`
+