Initial commit: MEV taxonomy, production pipeline, mermaid architecture, scaling notes

Made-with: Cursor

docs/PRODUCTION_PIPELINE.md

@@ -0,0 +1,114 @@
+# Production pipeline (reference)
+
+**Last Updated:** 2026-04-13  
+**Document Version:** 1.0  
+**Status:** Reference
+
+This is a **systems-level** description of how serious searchers and builders often structure work: **predict post-trade state, simulate, bid for ordering, settle**. Details differ by chain, relay, and builder market.
+
+---
+
+## Core loop (conceptual)
+
+1. **Ingest** chain head state, pool reserves, and (where available) **pending** transactions or private flow.  
+2. **Model** candidate state transitions (e.g. apply pending tx to a local view of state).  
+3. **Search** for profitable routes or reactions (cycles, backruns, liquidations).  
+4. **Simulate** deterministically in an EVM-equivalent environment with **gas** and fee models.  
+5. **Decide** under risk limits (inventory, gas caps, failure modes).  
+6. **Package** signed transactions into **bundles** or other relay payloads.  
+7. **Submit** to relays, builders, or public mempool; track inclusion and PnL.  
+8. **Observe** failures, latency, and counterparties; feed back into strategy.
+
+The system is not simply “find arb then send tx”; it is **compete for ordering** against others running similar loops.
+
+---
+
+## Layer breakdown
+
+### Data layer
+
+**Inputs:** RPC (archive when historical replay matters), mempool or builder streams, pool factories, logs, optional CEX/oracle feeds.
+
+**Outputs:** Normalized pool graph edges, reserves, fees, and **candidate triggers** (new block, new pending tx, new pool).
+
+**Typical stack (illustrative):** Self-hosted or low-latency RPC, websocket or fiber mempool feeds, indexers, Redis or in-memory hot state, PostgreSQL for metadata and analytics.
+
+---
+
+### Simulation engine
+
+**Role:** Given state `S` and a candidate action, compute post-state, balances, and **net** profit after gas and protocol fees.
+
+**Requirements:** Deterministic EVM behavior, correct token transfer semantics, gas metering, and **parallel** evaluation of many candidates.
+
+This layer is often the **main technical moat** after raw data access.
+
+---
+
+### Strategy engine
+
+**Role:** Choose **which** opportunities to pursue, size, and gas or builder tip policy under constraints (slippage, inventory, max loss, cooldowns).
+
+**Includes:** Graph search / path enumeration, scoring, and integration with **capital** limits.
+
+---
+
+### Execution layer
+
+**Role:** Turn a simulated opportunity into an **included** on-chain outcome.
+
+**Mechanisms (chain-dependent):** Bundles via relays (e.g. Flashbots-style flows), direct builder relationships, private orderflow, or public mempool with aggressive priority fees.
+
+**Win rate** is dominated by this layer plus simulation quality, not only “alpha” detection.
+
+---
+
+### Smart contract layer
+
+**Role:** Atomic multi-hop swaps, flash loans, callbacks, and **revert-safe** accounting so failed attempts do not destroy inventory.
+
+**Requirements:** Gas discipline, minimal external calls, clear failure modes.
+
+---
+
+### Capital layer
+
+**Role:** Inventory across tokens and chains, treasury movements, and **rebalancing** after runs.
+
+**Techniques:** Pre-funded inventory, internal netting across strategies, flash liquidity (bounded by depth and fees).
+
+---
+
+### Latency and placement
+
+**Role:** Reduce time from **signal** to **submission**, and improve placement in the **auction** for blockspace.
+
+**Knobs:** Geography, peering, NIC/kernel tuning, colocation (where allowed), direct builder APIs.
+
+---
+
+## Mapping to `MEV_Bot` implementation specs
+
+The **`MEV_Bot`** submodule (see parent **proxmox** repo) names concrete services in `specs/SERVICE_ARCHITECTURE_AND_MESSAGE_CONTRACTS.md`. Rough alignment:
+
+| This reference layer | `MEV_Bot` service (MVP spec) |
+|----------------------|----------------------------|
+| Data: pools, reserves, blocks | `PoolIndexerService`, `StateIngestionService` |
+| Liquidity graph | `LiquidityGraphService` |
+| Opportunity search | `OpportunitySearchService` |
+| Simulation | `SimulationService` |
+| Strategy / risk (partially in sim + ops) | Config + scoring in `SEARCH_AND_SIMULATION_SPEC` / ops docs |
+| Bundle build | `BundleBuilderService` |
+| Execution / relay | `ExecutionGateway` |
+| Settlement and PnL | `SettlementAnalyticsService` |
+| Mempool-driven rescoring (optional) | `MempoolWatcherService` (noted as optional in spec) |
+
+Use **`MEV_Bot/specs/README.md`** for the authoritative reading order and scope boundaries.
+
+---
+
+## Related
+
+- [ARCHITECTURE.md](ARCHITECTURE.md) — Mermaid diagrams of the same pipeline  
+- [OPPORTUNITY_TAXONOMY.md](OPPORTUNITY_TAXONOMY.md) — what kinds of opportunities exist  
+- [SCALING_AND_REALITY.md](SCALING_AND_REALITY.md) — competition and scaling