chore: sync submodule state (parent ref update)

Made-with: Cursor
This commit is contained in:
defiQUG
2026-03-02 12:14:09 -08:00
parent 50ab378da9
commit 5efe36b1e0
1100 changed files with 155024 additions and 8674 deletions

View File

@@ -0,0 +1,154 @@
# CMC and CoinGecko Reporting — Full API for All Tokens and Liquidity
This document describes the Token Aggregation Service reporting API used to expose **all coins, tokens, liquidity, and other reportable data** in formats suitable for **CoinMarketCap (CMC)** and **CoinGecko** listing submissions and sync.
## Endpoints
Base path: `/api/v1/report`
| Endpoint | Description | Cache |
|----------|-------------|--------|
| `GET /report/all` | Unified report: all tokens, pools, liquidity, volume, summary, and cross-chain data | 2 min |
| `GET /report/coingecko` | Token list and market data in CoinGecko-friendly format (includes crossChain) | 2 min |
| `GET /report/cmc` | Token list and DEX pairs in CoinMarketCap-friendly format (includes crossChain) | 2 min |
| `GET /report/cross-chain` | Cross-chain pools, bridge volume by lane, atomic swaps (138↔1, 138↔651940, etc.) | 2 min |
| `GET /report/token-list` | Flat canonical token list (all chains or `?chainId=138`) | 5 min |
| `GET /report/canonical` | Raw canonical token spec (symbol, name, type, decimals, addresses) | 10 min |
## Query Parameters
- **`chainId`** (optional): For `/report/all` and `/report/token-list`, limit to one chain (e.g. `138`, `651940`). For `/report/coingecko` and `/report/cmc`, set the chain for the response (default `138`).
## 1. Full Report — `GET /api/v1/report/all`
Returns all canonical tokens with DB-backed market data and pools, plus per-chain pool list and summary (total liquidity, volume, counts).
**Example:**
```bash
curl "http://localhost:3000/api/v1/report/all"
curl "http://localhost:3000/api/v1/report/all?chainId=138"
```
**Response shape:**
- `generatedAt`: ISO timestamp
- `chains`: list of chain IDs included
- `tokens`: `{ [chainId]: TokenReport[] }` — each token has `address`, `symbol`, `name`, `type`, `decimals`, `market` (priceUsd, volume24h, volume7d, volume30d, marketCapUsd, liquidityUsd), `pools` (poolAddress, dex, tvl, volume24h), `fromDb`
- `pools`: `{ [chainId]: LiquidityPool[] }`
- `summary`: `totalLiquidityUsdByChain`, `totalVolume24hUsdByChain`, `tokenCountByChain`, `poolCountByChain`
Use this for dashboards, internal analytics, and as the source of truth for “all reportable data.”
## 2. CoinGecko Format — `GET /api/v1/report/coingecko`
Returns tokens in a structure aligned with CoinGeckos API and listing expectations: contract address, symbol, name, decimals, optional market_data (current_price, total_volume, market_cap, liquidity_usd), and liquidity_pools (pool_address, dex_id, tvl_usd, volume_24h_usd).
**Example:**
```bash
curl "http://localhost:3000/api/v1/report/coingecko?chainId=138"
```
**Use for:**
- Preparing CoinGecko listing submissions (contract address, symbol, name, decimals, platform).
- Feeding an internal or external job that syncs to CoinGecko (e.g. after they support custom chains).
- One-shot exports for manual submission.
**Note:** CoinGecko may not list ChainID 138 or 651940 until they add the platform. The response still provides a single, consistent format for all our tokens and liquidity.
## 3. CoinMarketCap Format — `GET /api/v1/report/cmc`
Returns tokens with contract address, symbol, name, decimals, volume_24h, market_cap, liquidity_usd, and `pairs` (pair_address, dex_id, base, quote, liquidity_usd, volume_24h_usd).
**Example:**
```bash
curl "http://localhost:3000/api/v1/report/cmc?chainId=138"
```
**Use for:**
- CMC DEX/listing submission or sync when the chain is supported.
- Internal reporting that mirrors CMCs expected fields.
## 4. Token List — `GET /api/v1/report/token-list`
Flat list of all canonical tokens (per chain or all supported chains): `chainId`, `address`, `symbol`, `name`, `decimals`, `type` (base, w, asset, debt).
**Example:**
```bash
curl "http://localhost:3000/api/v1/report/token-list"
curl "http://localhost:3000/api/v1/report/token-list?chainId=651940"
```
**Use for:**
- DEX/UIs that need a single token list.
- Indexer or cron jobs that ensure every canonical token is registered in the DB.
## 5. Canonical Spec — `GET /api/v1/report/canonical`
Raw canonical token definitions (no DB merge): symbol, name, type, decimals, currencyCode, addresses by chainId. No market or pool data.
**Use for:** Tooling and config that need the authoritative list of tokens and their chains/addresses.
## Canonical Token Set and Addresses
The report uses the **canonical token list** in `src/config/canonical-tokens.ts`. It includes:
- **Base (GRU-M1):** cUSDC, cUSDT, cEURC, cEURT, cGBPC, cGBPT, cAUDC, cJPYC, cCHFC, cCADC, cXAUC, cXAUT
- **W-tokens (ISO-4217):** USDW, EURW, GBPW, AUDW, JPYW, CHFW, CADW
- **Asset (ac*):** acUSDC, acUSDT, acEURC, acGBPC, acAUDC, acJPYC, acCHFC, acCADC, acXAUC
- **Debt (vdc* / sdc*):** vdcUSDC, sdcUSDC, vdcEURC, sdcEURC, vdcGBPC, sdcGBPC, vdcAUDC, sdcAUDC, vdcJPYC, sdcJPYC, vdcCHFC, sdcCHFC, vdcCADC, sdcCADC, vdcXAUC, sdcXAUC
Addresses per chain can be:
1. Set via env: `CUSDC_ADDRESS_138`, `CUSDT_ADDRESS_138`, etc.
2. Filled by the indexer when tokens are discovered on-chain.
3. Updated in the canonical config after deployment.
Only tokens with a non-empty address for the requested chain appear in `/report/coingecko`, `/report/cmc`, and in the per-chain sections of `/report/all` and `/report/token-list`.
## ERC-20 and DEX Compatibility
All canonical tokens are designed to be **ERC-20 compliant** and usable in DEX liquidity pools (see `smom-dbis-138/docs/tokenization/TOKEN_SCOPE_GRU.md`). Base and asset tokens are fully transferable; debt tokens can be deployed with optional transferability. The report API does not enforce on-chain checks; it reports whatever is in the canonical list and DB (addresses, market data, pools).
## 6. Cross-Chain Report — `GET /api/v1/report/cross-chain`
Returns cross-chain liquidity pools, bridge volume by lane, and atomic swap volume for Chain 138 and ALL Mainnet (651940). Covers CCIP (WETH9/WETH10), Alltra (138↔651940), Trustless (Lockbox), and Universal CCIP Bridge.
**Example:**
```bash
curl "http://localhost:3000/api/v1/report/cross-chain?chainId=138"
```
**Response shape:**
- `crossChainPools`: Array of `{ type, sourceChainId, destChainId, destChainName, bridgeAddress, tokenSymbol, bridgeType, isActive }`
- `volumeByLane`: Array of `{ sourceChainId, destChainId, destChainName, bridgeType, tokenSymbol, volume24hWei, volume7dWei, volume30dWei, txCount24h, txCount7d, txCount30d }`
- `atomicSwapVolume24h`: Total atomic swap volume (swap-then-bridge) in last 24h
- `bridgeVolume24hTotal`: Total bridge volume in last 24h
- `events`: Raw cross-chain events (limited to 500)
**Use for:** CMC/CoinGecko submission alongside single-chain reports. Include as `crossChain` attachment or reference URL when platforms support custom chain data.
**Note:** `/report/coingecko` and `/report/cmc` now include a `crossChain` field when cross-chain data is available.
## CMC / CoinGecko Listing Checklist
1. **Deploy and index:** Ensure tokens are deployed and the indexer has run so DB has addresses and (if applicable) market/pool data.
2. **Set addresses:** Configure canonical addresses via env or config so `/report/coingecko` and `/report/cmc` include all desired tokens.
3. **Export:** Use `GET /api/v1/report/coingecko` and `GET /api/v1/report/cmc` to obtain the payload for submission or sync.
4. **Submit:** Use each platforms official process (e.g. CoinGecko “Submit coin”, CMC DEX/listing forms) and attach or map the exported data as required.
5. **Ongoing:** Re-export periodically or run a scheduled job that pushes liquidity/volume to CMC/CoinGecko if they provide an API for custom chains.
## Related
- **GRU M1 listing prep (dry-runs, dominance simulation, peg stress-tests):** `../../../../docs/gru-m1/README.md`
- **Token scope (base, W, asset, debt):** `smom-dbis-138/docs/tokenization/TOKEN_SCOPE_GRU.md`
- **CoinGecko submission (cUSDC/cUSDT):** `docs/04-configuration/coingecko/COINGECKO_SUBMISSION_GUIDE.md`
- **Token Aggregation API:** `README.md` and `GET /api/v1/tokens`, `GET /api/v1/tokens/:address`, `GET /api/v1/pools/:poolAddress`

View File

@@ -0,0 +1,23 @@
# CoinGecko Listing Submission (Chain 138 / 651940)
**Purpose:** Steps to submit Chain 138 (and 651940) to CoinGecko for listing and price/volume display.
## Prerequisites
- Token Aggregation Service running and reachable (e.g. `https://your-api/report/coingecko`).
- Public URL for the report API (CoinGecko may need to fetch it).
## Steps
1. **Expose report API:** Ensure `GET /api/v1/report/coingecko` is publicly reachable or use CoinGecko's allowed submission method (e.g. API URL or file upload).
2. **CoinGecko request form:** Use [CoinGecko listing request](https://www.coingecko.com/en/coins/new) or partner/API submission if available. Provide:
- Chain name / ChainID (e.g. "Sankofa Chain" / 138).
- API URL returning CoinGecko-style data: `https://<your-domain>/api/v1/report/coingecko?chainId=138`.
- Or export JSON from `/api/v1/report/coingecko` and attach if they accept file upload.
3. **Response format:** Our `/report/coingecko` returns tokens with `contract_address`, `symbol`, `name`, `decimals`, `market_data`, `liquidity_pools` as per [CMC_COINGECKO_REPORTING.md](CMC_COINGECKO_REPORTING.md).
4. **Chain support:** CoinGecko may not yet support ChainID 138/651940; external price/volume may show empty until they add the chain. The report API still provides chain-native data.
## See also
- [CMC_COINGECKO_REPORTING.md](CMC_COINGECKO_REPORTING.md) — API reference
- [README.md](../README.md) — Service setup and env (canonical tokens, API keys)

View File

@@ -0,0 +1,268 @@
# Deployment Guide
## Prerequisites
1. **Database**: PostgreSQL 14+ with TimescaleDB extension
2. **Node.js**: Version 20 or higher
3. **Docker**: (Optional) For containerized deployment
4. **RPC Access**: Access to RPC endpoints for ChainID 138 and 651940
## Database Setup
1. Ensure PostgreSQL is running with TimescaleDB extension enabled:
```sql
CREATE EXTENSION IF NOT EXISTS timescaledb;
```
2. Run the migration from the explorer database:
```bash
# The migration file is located at:
# explorer-monorepo/backend/database/migrations/0011_token_aggregation_schema.up.sql
```
3. Verify tables were created:
```sql
\dt token_market_data
\dt liquidity_pools
\dt token_ohlcv
```
## Environment Configuration
1. Copy the example environment file:
```bash
cp .env.example .env
```
2. Configure required variables:
```bash
# Required
CHAIN_138_RPC_URL=https://rpc-http-pub.d-bis.org
CHAIN_651940_RPC_URL=https://mainnet-rpc.alltra.global
DATABASE_URL=postgresql://user:password@localhost:5432/explorer_db
# Optional (for external API enrichment)
COINGECKO_API_KEY=your_key_here
COINMARKETCAP_API_KEY=your_key_here
DEXSCREENER_API_KEY=your_key_here
```
## Local Deployment
### Using npm
1. Install dependencies:
```bash
npm install
```
2. Build the project:
```bash
npm run build
```
3. Start the service:
```bash
npm start
```
### Using Docker
1. Build the image:
```bash
docker build -t token-aggregation-service .
```
2. Run the container:
```bash
docker run -d \
--name token-aggregation \
-p 3000:3000 \
--env-file .env \
token-aggregation-service
```
### Using Docker Compose
1. Start all services:
```bash
docker-compose up -d
```
2. View logs:
```bash
docker-compose logs -f token-aggregation
```
## Production Deployment
### Kubernetes
1. Create a ConfigMap for environment variables:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: token-aggregation-config
data:
CHAIN_138_RPC_URL: "https://rpc-http-pub.d-bis.org"
CHAIN_651940_RPC_URL: "https://mainnet-rpc.alltra.global"
INDEXING_INTERVAL: "5000"
LOG_LEVEL: "info"
```
2. Create a Secret for sensitive data:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: token-aggregation-secrets
type: Opaque
stringData:
DATABASE_URL: "postgresql://..."
COINGECKO_API_KEY: "..."
COINMARKETCAP_API_KEY: "..."
```
3. Deploy the service:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: token-aggregation
spec:
replicas: 2
selector:
matchLabels:
app: token-aggregation
template:
metadata:
labels:
app: token-aggregation
spec:
containers:
- name: token-aggregation
image: token-aggregation-service:latest
ports:
- containerPort: 3000
envFrom:
- configMapRef:
name: token-aggregation-config
- secretRef:
name: token-aggregation-secrets
livenessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 10
periodSeconds: 5
```
## DEX Factory Configuration
For ChainID 138, configure DODO PoolManager address:
```bash
CHAIN_138_DODO_POOL_MANAGER=0x...
```
For ChainID 651940, configure DEX factories as they are discovered:
```bash
CHAIN_651940_UNISWAP_V2_FACTORY=0x...
CHAIN_651940_UNISWAP_V3_FACTORY=0x...
```
## Monitoring
### Health Checks
The service exposes a health check endpoint:
```bash
curl http://localhost:3000/health
```
### Logs
View service logs:
```bash
# Docker
docker logs -f token-aggregation
# Kubernetes
kubectl logs -f deployment/token-aggregation
```
### Metrics
Monitor the following:
- Database connection pool usage
- Indexing progress (tokens indexed, pools discovered)
- API request rates
- External API call success rates
## Troubleshooting
### Database Connection Issues
1. Verify database is accessible:
```bash
psql $DATABASE_URL -c "SELECT 1"
```
2. Check connection pool settings in `.env`
### RPC Connection Issues
1. Test RPC endpoints:
```bash
curl -X POST $CHAIN_138_RPC_URL \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}'
```
2. Verify RPC URLs in `.env`
### Indexing Not Working
1. Check logs for errors
2. Verify DEX factory addresses are configured
3. Ensure RPC endpoints have required APIs enabled (ETH, NET, etc.)
## Scaling
### Horizontal Scaling
The service is stateless and can be scaled horizontally:
- Multiple instances can run simultaneously
- Each instance will index independently
- Database handles concurrent writes
### Vertical Scaling
For high-volume chains:
- Increase `INDEXING_INTERVAL` for less frequent updates
- Increase database connection pool size
- Use read replicas for database queries
## Backup and Recovery
### Database Backups
Regular backups of the following tables:
- `token_market_data`
- `liquidity_pools`
- `token_ohlcv`
- `swap_events`
### Recovery
1. Restore database from backup
2. Restart indexing service
3. Service will backfill missing data automatically

View File

@@ -0,0 +1,204 @@
# Token Aggregation Service — REST API Reference
Base path: `/api/v1`. Used for discovery of tokens, liquidity pools, prices, volume, and OHLCV by dApps and MetaMask Snap integrations. Supports Chain 138 (DeFi Oracle Meta Mainnet) and Chain 651940 (ALL Mainnet).
## Chains
### GET /api/v1/chains
Returns supported chains.
**Response:** `{ chains: [{ chainId, name, explorerUrl }] }`
---
## Networks and config (Snap)
### GET /api/v1/networks
Full EIP-3085 chain params for `wallet_addEthereumChain` (Chain 138, Ethereum Mainnet 1, ALL Mainnet 651940). Includes RPC URLs, block explorer URLs, native currency, and oracles per chain. Used by the MetaMask Snap to serve dynamic network and oracle data. If **NETWORKS_JSON_URL** is set (e.g. GitHub raw URL), the service fetches that URL and returns `{ version, networks }`; otherwise uses built-in networks.
**Response:** `{ version: string, networks: NetworkEntry[] }`
Each `NetworkEntry` has: `chainId` (hex), `chainIdDecimal`, `chainName`, `rpcUrls`, `nativeCurrency`, `blockExplorerUrls`, `iconUrls` (chain-specific logos; optional), `oracles: [{ name, address, decimals? }]`. Chain 138 and ALL Mainnet include explorer favicons; Ethereum includes standard ETH logo.
### GET /api/v1/config
Oracles (and config) per chain. Used by the Snap for USD price feeds (e.g. ETH/USD).
**Query:** `chainId` (optional) — if provided, returns config for that chain only.
**Response (no query):** `{ version: string, chains: [{ chainId, oracles: [{ name, address }] }] }`
**Response (chainId=138):** `{ version, chainId: 138, oracles: [{ name, address }] }`
---
## Token list (report)
**GET /api/v1/report/token-list** returns a Uniswap-style token list with **logoURI** per token and a list-level **logoURI**. Each token has: `chainId`, `address`, `symbol`, `name`, `decimals`, `type`, `logoURI`. Use for MetaMask token list URL or Snap `get_token_list`. Optional query `?chainId=138` filters by chain. If **TOKEN_LIST_JSON_URL** is set (e.g. GitHub raw URL), the service fetches that JSON and returns it (with optional chainId filter); otherwise uses the built-in canonical token list.
---
## Quote
### GET /api/v1/quote
Returns an estimated swap output amount (constant-product from first available pool for the token pair).
**Query:**
| Param | Type | Required | Description |
|----------|--------|----------|--------------------------------------------------|
| chainId | number | yes | 138 or 651940 |
| tokenIn | string | yes | Token address (in) |
| tokenOut | string | yes | Token address (out) |
| amountIn | string | yes | Raw amount in token's smallest unit (integer) |
**Response:** `{ amountOut: string | null, error?: string, poolAddress?: string | null, dexType?: string }`
If no pool is found, `amountOut` is `null` and `error` describes. Used by the MetaMask Snap for swap quotes.
---
## Bridge routes
### GET /api/v1/bridge/routes
Returns CCIP bridge routes for WETH9 and WETH10 (Chain 138 and Ethereum Mainnet). Used by the MetaMask Snap and dApps for bridge discovery. If **BRIDGE_LIST_JSON_URL** is set (e.g. GitHub raw URL), the service fetches that JSON and returns `{ routes, chain138Bridges }`; otherwise uses built-in routes.
**Response:** `{ routes, chain138Bridges, tokenMappingApi? }`
- `routes`: `{ weth9: Record<string, string>, weth10: Record<string, string> }` — destination chain name → bridge address.
- `chain138Bridges`: `{ weth9: string, weth10: string }` — Chain 138 bridge addresses.
- `tokenMappingApi`: When the service runs from the monorepo, `{ basePath, pairs, resolve, note }` — use the same host and these paths for **cross-chain token address resolution** (138↔651940, 651940↔other chains). Bridge UIs should call `GET /api/v1/token-mapping?fromChain=&toChain=` or `GET /api/v1/token-mapping/resolve?fromChain=&toChain=&address=` when resolving token addresses on destination chains.
---
## Token mapping (multichain)
When run from the monorepo (with `config/token-mapping-multichain.json` available):
| Endpoint | Description |
|----------|-------------|
| `GET /api/v1/token-mapping?fromChain=138&toChain=651940` | Token mapping for a chain pair (tokens, addressMapFromTo, addressMapToFrom). |
| `GET /api/v1/token-mapping/pairs` | All defined chain pairs. |
| `GET /api/v1/token-mapping/resolve?fromChain=&toChain=&address=` | Resolve token address on target chain. |
Use these for bridge UIs and cross-chain address resolution. See token-aggregation README § Token mapping.
---
## Tokens
### GET /api/v1/tokens
List tokens for a chain with optional market data.
**Query:**
| Param | Type | Required | Description |
|-------|------|----------|-------------|
| chainId | number | yes | 138 or 651940 |
| limit | number | no | Default 50 |
| offset | number | no | Default 0 |
| includeDodoPool | boolean | no | Include DODO pool flag per token |
**Response:** `{ tokens: Token[], pagination: { limit, offset, count } }`
Each token may include `market` (price, volume, TVL) and `hasDodoPool` / `pmmPool` when requested.
### GET /api/v1/tokens/:address
Token detail by chain and address. Includes market data, pools, and external enrichment (CoinGecko, CMC, DexScreener).
**Query:** `chainId` (required) — 138 or 651940.
**Response:** `{ token: { ...token, onChain, market, external: { coingecko, cmc, dexscreener }, pools[], hasDodoPool, pmmPool } }`
### GET /api/v1/tokens/:address/pools
Liquidity pools for a token.
**Query:** `chainId` (required).
**Response:** `{ pools: [{ address, dex, token0, token1, reserves, tvl, volume24h, feeTier }] }`
### GET /api/v1/tokens/:address/ohlcv
OHLCV (candlestick) data for charts.
**Query:**
| Param | Type | Required | Description |
|-------|------|----------|-------------|
| chainId | number | yes | 138 or 651940 |
| interval | string | no | One of: 5m, 15m, 1h, 4h, 24h. Default 1h |
| from | ISO date | no | Start time (default: 7 days ago) |
| to | ISO date | no | End time (default: now) |
| poolAddress | string | no | Specific pool for price source |
**Response:** `{ chainId, tokenAddress, interval, data: OHLCV[] }`
### GET /api/v1/tokens/:address/signals
Trending/signals (e.g. CoinGecko trending rank) for a token.
**Query:** `chainId` (required).
**Response:** `{ chainId, tokenAddress, signals: { trendingRank } }`
---
## Pools
### GET /api/v1/pools/:poolAddress
Single pool by chain and pool address.
**Query:** `chainId` (required).
**Response:** `{ pool: { address, dex, token0, token1, reserves, tvl, volume24h, feeTier } }`
---
## Search
### GET /api/v1/search
Search tokens by symbol or name.
**Query:** `chainId` (required), `q` (required) — search string.
**Response:** `{ query, chainId, results: Token[] }`
---
## Pricing and market data
- **Per-token price/volume:** Use `GET /api/v1/tokens/:address`; the `market` and `external` fields provide prices and volume (from indexer and CoinGecko/CMC/DexScreener).
- **Liquidity (TVL):** Use `GET /api/v1/tokens/:address/pools` or `GET /api/v1/pools/:poolAddress`; each pool includes `tvl` and `volume24h`.
- **OHLCV:** Use `GET /api/v1/tokens/:address/ohlcv` for time-series price data.
---
## Health
### GET /health
Returns service and database health. Response: `{ status: 'healthy'|'unhealthy', timestamp, services: { database } }`.
---
## Caching and rate limiting
- Endpoints use short-lived cache (e.g. 15 minutes). Use `Cache-Control` headers from responses.
- `/api/v1` is rate-limited; see server configuration for limits.
---
## Use for discovery (no CMC/CoinGecko submission)
This API is the primary source for **token discovery**, **liquidity pool discovery**, and **pricing/market data** for Chain 138 and ALL Mainnet. dApps and a custom MetaMask Snap can consume it for:
- Token list and metadata
- Liquidity pools and TVL
- Prices and volume
- OHLCV for charts
Combine with the **explorer-hosted token list** (`https://explorer.d-bis.org/api/config/token-list`) for MetaMask token list URL discovery. For dynamic Snap integration, use **GET /api/v1/networks** and **GET /api/v1/config** (see "Networks and config (Snap)" above); the Snap can return the token list URL as `{apiBaseUrl}/api/v1/report/token-list` for MetaMask Settings.