Files
proxmox/docs/06-besu/VALIDATOR_TXPOOL_FIX_STATUS.md
defiQUG fbda1b4beb
Some checks failed
Deploy to Phoenix / deploy (push) Has been cancelled
docs: Ledger Live integration, contract deploy learnings, NEXT_STEPS updates
- ADD_CHAIN138_TO_LEDGER_LIVE: Ledger form done; public code review repo bis-innovations/LedgerLive; init/push commands
- CONTRACT_DEPLOYMENT_RUNBOOK: Chain 138 gas price 1 gwei, 36-addr check, TransactionMirror workaround
- CONTRACT_*: AddressMapper, MirrorManager deployed 2026-02-12; 36-address on-chain check
- NEXT_STEPS_FOR_YOU: Ledger done; steps completable now (no LAN); run-completable-tasks-from-anywhere
- MASTER_INDEX, OPERATOR_OPTIONAL, SMART_CONTRACTS_INVENTORY_SIMPLE: updates
- LEDGER_BLOCKCHAIN_INTEGRATION_COMPLETE: bis-innovations/LedgerLive reference

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-12 15:46:57 -08:00

6.5 KiB

Validator Transaction Pool Fix - Current Status

Last Updated: 2026-01-31
Document Version: 1.0
Status: Active Documentation


Date: 2026-01-24
Status: VALIDATORS CONFIGURED | ⚠️ STUCK TRANSACTIONS ISSUE


Executive Summary

The Validator Transaction Pool Configuration is correctly applied, but there's a separate issue with stuck transactions in the RPC mempool that's preventing bridge operations from completing.


What's Been Fixed

1. Validator TX-Pool Configuration

Both active validators (1003 and 1004) have the correct layered tx-pool configuration:

# DO NOT use legacy options (tx-pool-max-size, tx-pool-limit-by-account-percentage)
tx-pool-max-future-by-sender=200
tx-pool-layer-max-capacity=12500000
tx-pool-max-prioritized=2000

Status: COMPLETE

  • Validators 1003 & 1004: Running with correct configuration
  • Validators 1000-1002: Do not exist (were likely migrated or removed)
  • Network: 12 peers connected
  • Blocks: Producing regularly (block 1,301,111+)

⚠️ Remaining Issue: Stuck Transactions

Problem

  • Nonce stuck at: 13104
  • Blocks status: Empty (0 transactions)
  • Root cause: Transactions stuck in RPC mempool
  • Account: 0x4A666F96fC8764181194447A7dFdb7d471b301C8
  • Account balance: 999,610,163+ ETH (sufficient)

What Was Tried

  1. Restarted 7 RPC containers (VMIDs: 2304, 2305, 2306, 2307, 2308, 2402, 2403)
  2. ⚠️ Could not locate the specific node serving 192.168.11.211
  3. ⚠️ Nonce remains stuck after RPC restarts

Analysis

  • The RPC at 192.168.11.211 is responding (Besu v23.10.0)
  • MAC address: bc:24:11:16:e7:02
  • Could be running on different Proxmox host or as a VM
  • RPC may be persisting mempool to disk (surviving restarts)

🎯 Solution: Skip Stuck Transactions

Since we can't clear the stuck transactions, we'll bypass them by using the correct nonce for new transactions.

Created Tools

1. Test Bridge Script

Location: /home/intlc/projects/proxmox/scripts/test-bridge-with-fresh-nonce.sh

What it does:

  • Checks current nonce
  • Wraps ETH to WETH9 (if needed)
  • Approves WETH9 for bridge
  • Provides command to execute bridge transaction

Usage:

cd /home/intlc/projects/proxmox
export PRIVATE_KEY="0x..."
./scripts/test-bridge-with-fresh-nonce.sh

2. Skip Stuck Transactions Script

Location: /home/intlc/projects/proxmox/scripts/skip-stuck-transactions.sh

What it does:

  • Shows current nonce status
  • Calculates next available nonce
  • Provides guidance for bypassing stuck transactions

📋 Next Steps to Complete Bridge

Step 1: Run Test Script

cd /home/intlc/projects/proxmox
export PRIVATE_KEY="0xYourPrivateKeyHere"
./scripts/test-bridge-with-fresh-nonce.sh

This will:

  1. Check current state
  2. Wrap ETH to WETH9 (using next available nonce)
  3. Approve bridge (using next available nonce)
  4. Provide exact command to execute bridge

Step 2: Execute Bridge Transaction

The script will output the exact command to run. It will look like:

cast send 0x89dd12025bfCD38A168455A44B400e913ED33BE2 \
  'sendCrossChain(uint64,address,uint256)' \
  5009297550715157269 \
  0x4A666F96fC8764181194447A7dFdb7d471b301C8 \
  1000000000000000 \
  --private-key $PRIVATE_KEY \
  --rpc-url http://192.168.11.211:8545 \
  --gas-limit 200000 \
  --gas-price 1000000000 \
  --nonce <CORRECT_NONCE>

Step 3: Monitor Bridge Transfer

# Check transaction status
cast tx <TRANSACTION_HASH> --rpc-url http://192.168.11.211:8545

# Check if WETH9 arrived on Ethereum Mainnet (wait 1-5 minutes)
cast balance <YOUR_ADDRESS> --rpc-url https://eth.llamarpc.com --ether

🔍 Monitoring Commands

Check Current Nonce

cast nonce 0x4A666F96fC8764181194447A7dFdb7d471b301C8 \
  --rpc-url http://192.168.11.211:8545

Check Block Transactions

LATEST=$(cast block-number --rpc-url http://192.168.11.211:8545)
cast block $LATEST --rpc-url http://192.168.11.211:8545 --json | jq '.transactions | length'

Check Validator Status

ssh root@192.168.11.10 "pct exec 1003 -- systemctl status besu-validator"
ssh root@192.168.11.10 "pct exec 1004 -- systemctl status besu-validator"

Check Validator Logs

ssh root@192.168.11.10 "pct exec 1003 -- journalctl -u besu-validator -f"

📊 Current Network Status

Component Status Details
Validator 1003 Running Layered TX-pool configured
Validator 1004 Running Layered TX-pool configured
Block Production Working Block 1,301,111+
Peer Connections Good 12 peers
Transaction Processing Stuck Nonce 13104
RPC Accessibility Working 192.168.11.211:8545
Bridge Contracts Deployed Ready to use

🎯 Success Criteria

For bridge to work, we need:

  • Validators with layered TX-pool config
  • Blocks being produced
  • RPC accessible
  • Bridge contracts deployed
  • Transactions processing ⬅️ Use bypass method
  • First successful bridge transfer ⬅️ Execute with script

💡 Key Insights

Why Stuck Transactions Don't Block Everything

  • Stuck transactions are in RPC mempool only
  • They're not on-chain
  • We can bypass them by using correct nonce
  • New transactions will process normally

Why Restarting RPC Didn't Help

  • The specific RPC at 192.168.11.211 wasn't restarted
  • Or it's persisting mempool to disk
  • Or it's on a different host entirely

Why This Solution Works

  • Validators are configured correctly
  • Network is functional
  • We just need to use the right nonce
  • Bypass stuck transactions, don't try to clear them

🚀 Ready to Execute

All prerequisites are met. The test script will handle:

  1. Checking current state
  2. Wrapping ETH if needed
  3. Approving tokens if needed
  4. Providing exact bridge command

Just run:

export PRIVATE_KEY="0x..."
./scripts/test-bridge-with-fresh-nonce.sh


Last Updated: 2026-01-24
Status: Ready for bridge testing with nonce bypass method