Files
mev-beta/docs/FAST_MVP_PLAN.md
Gemini Agent e5bc5969f1 docs: add critical bug - stale reserve data causing zero opportunities
CRITICAL BUG DISCOVERED:
- Bot ran 17+ hours finding ZERO opportunities
- Root cause: Reserves fetched ONCE at startup, never refreshed
- Arbitrage detection uses stale data = misses all real opportunities

SOLUTION DOCUMENTED:
- Implement RefreshReserves() before each scan
- ~2 hours implementation time
- P0 priority - bot is non-functional without this

Lesson learned: Always test with LIVE data, not just unit tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 17:36:17 -06:00

21 KiB
Raw Blame History

MEV Bot - Fast MVP Plan (4-5 Weeks)

Objective: Deploy profitable arbitrage bot in 4-5 weeks with minimal viable features Strategy: Validate business model quickly, then decide whether to scale Capital: Start with 0.5-1 ETH, scale if profitable


🎯 Core Philosophy

Ship fast, validate profitability, iterate based on real data.

We're cutting everything that's not essential:

  • No 13+ protocols (just UniswapV2 + UniswapV3)
  • No 4-hop arbitrage (just 2-hop)
  • No sequencer integration yet (regular RPC first)
  • No batch execution (single trades only)
  • No fancy dashboard (basic metrics only)

We CAN add these later if the bot is profitable.


📅 4-Week Timeline

Week 1: Core Parsers (Dec 1-7)

Goal: Parse UniswapV2 and UniswapV3 swaps accurately

Day 1-2: UniswapV2 Parser

  • Implement Swap event parsing
  • Handle Mint/Burn events
  • Extract tokens from pool cache
  • Decimal scaling (USDC 6, WBTC 8, WETH 18)
  • 100% test coverage
  • Integration test with real Arbitrum tx

Success Criteria:

  • Parse any UniswapV2 swap on Arbitrum
  • No zero addresses
  • Correct decimal handling
  • Tests pass: make test-coverage

Day 3-5: UniswapV3 Parser

  • Parse V3 Swap events (signed amounts)
  • Handle sqrtPriceX96 and tick
  • Concentrated liquidity basics
  • Multi-hop swap support
  • 100% test coverage
  • Integration tests

Success Criteria:

  • Parse any UniswapV3 swap on Arbitrum
  • Handle negative amounts correctly
  • Proper tick/liquidity tracking

Day 6-7: Pool Discovery

  • Fetch all UniswapV2 pools on Arbitrum
  • Fetch all UniswapV3 pools on Arbitrum
  • Populate pool cache
  • Get initial reserves
  • ~500-1000 pools total

Success Criteria:

  • Pool cache has major trading pairs (WETH/USDC, WETH/ARB, etc.)
  • Reserves are accurate
  • Cache lookup is fast (<1ms)

Week 2: Arbitrage Detection (Dec 8-14)

Goal: Find profitable 2-hop arbitrage opportunities

Day 1-2: Market Graph

  • Build graph from pool cache
  • Nodes = tokens, Edges = pools
  • Efficient adjacency list representation
  • Unit tests

Example:

WETH --[UniV2 Pool]-- USDC
WETH --[UniV3 Pool]-- USDC
ARB  --[UniV2 Pool]-- WETH

Day 3-4: Path Finder (2-Hop Only)

  • Find circular paths: Token A → Token B → Token A
  • BFS algorithm (simple, fast)
  • Limit to 2 hops maximum
  • Filter by minimum liquidity ($10k+)
  • Unit tests with mock graph

Example Arbitrage:

Buy USDC with WETH on UniV2 (cheaper)
Sell USDC for WETH on UniV3 (more expensive)
Profit = difference - gas

Day 5-6: Profitability Calculator

  • Calculate swap output (AMM formula)
  • Account for fees (0.3% UniV2, 0.05-1% UniV3)
  • Estimate gas cost (~150k gas per hop)
  • Calculate net profit
  • Filter: only opportunities >0.01 ETH profit

Formula:

Gross Profit = Output Amount - Input Amount
Gas Cost = Gas Used × Gas Price
Net Profit = Gross Profit - Gas Cost

Only execute if Net Profit > 0.01 ETH

Day 7: Integration & Testing

  • End-to-end test: Pool cache → Graph → Paths → Profit
  • Test with historical Arbitrum data
  • Verify calculations match real outcomes
  • Performance: <50ms per detection

Week 3: Execution Engine (Dec 15-21)

Goal: Execute profitable arbitrage trades

Day 1-2: Transaction Builder

  • Build swap transaction for UniswapV2
  • Build swap transaction for UniswapV3
  • Calculate exact input/output amounts
  • Add slippage tolerance (0.5%)
  • Unit tests

Day 3-4: Execution Logic

  • Connect to Arbitrum RPC (Alchemy/Infura)
  • Get wallet nonce
  • Estimate gas
  • Set gas price (current + 10%)
  • Sign transaction
  • Submit to network
  • Wait for confirmation
  • Handle reverts gracefully

Simple Flow:

func (e *Executor) Execute(opportunity *Opportunity) error {
    // 1. Build transaction
    tx := e.buildArbitrageTx(opportunity)

    // 2. Estimate gas
    gas, err := e.client.EstimateGas(tx)
    if err != nil {
        return err
    }
    tx.Gas = gas * 1.1 // 10% buffer

    // 3. Get gas price
    gasPrice, _ := e.client.SuggestGasPrice()
    tx.GasPrice = gasPrice * 1.1 // 10% higher to ensure inclusion

    // 4. Check still profitable after gas
    netProfit := opportunity.GrossProfit - (gas * gasPrice)
    if netProfit < minProfit {
        return ErrNotProfitable
    }

    // 5. Sign and send
    signedTx, _ := types.SignTx(tx, e.signer, e.privateKey)
    err = e.client.SendTransaction(ctx, signedTx)

    // 6. Wait for confirmation
    receipt, err := e.waitForReceipt(signedTx.Hash())

    // 7. Record result
    e.recordTrade(opportunity, receipt)

    return err
}

Day 5-6: Basic Risk Management

  • Circuit breaker: Stop after 3 failed trades in a row
  • Max loss limit: Stop after -0.1 ETH total loss
  • Slippage protection: Revert if output < expected × 99.5%
  • Position limit: Max 0.5 ETH per trade
  • Cooldown: Wait 1 minute after circuit breaker trips

Day 7: Testnet Testing

  • Deploy to Arbitrum Sepolia testnet
  • Test full flow: Detect → Execute → Confirm
  • Verify trades succeed
  • Check profit calculations
  • Fix any bugs

Week 4: Monitoring & Deployment (Dec 22-28)

Goal: Deploy to mainnet with real capital

Day 1-2: Basic Metrics

  • Count opportunities found
  • Count trades executed
  • Count successful vs failed
  • Track total profit/loss
  • Track gas spent
  • Simple logging to file

Metrics to track:

Opportunities Found: 127
Trades Executed: 23
Successful: 18 (78%)
Failed: 5 (22%)

Gross Profit: 0.42 ETH
Gas Spent: 0.15 ETH
Net Profit: 0.27 ETH
ROI: 54% (on 0.5 ETH capital)

Day 3: Mainnet Preparation

  • Security audit of wallet handling
  • Create dedicated bot wallet
  • Fund with 0.5 ETH test capital
  • Set conservative limits:
    • Max 0.1 ETH per trade
    • Max 10 trades per day
    • Stop after -0.05 ETH loss
  • Backup private key securely

Day 4-5: Shadow Mode (24 hours)

  • Run bot in shadow mode (detect but don't execute)
  • Log all opportunities
  • Calculate theoretical profit
  • Verify no false positives
  • Check for edge cases

Example Log:

[2024-12-22 10:15:23] OPPORTUNITY FOUND
Path: WETH → USDC (UniV2) → WETH (UniV3)
Input: 0.1 ETH
Expected Output: 0.1023 ETH
Gross Profit: 0.0023 ETH ($4.60)
Gas Cost: 0.0008 ETH ($1.60)
Net Profit: 0.0015 ETH ($3.00)
Action: WOULD EXECUTE (shadow mode)

Day 6-7: Live Deployment (Carefully!)

  • Switch to live mode
  • Start with VERY conservative limits
  • Monitor constantly (first 24 hours)
  • Be ready to kill switch if needed

First Day Checklist:

  • Bot detects opportunities
  • Profit calculations are accurate
  • Trades execute successfully
  • No unexpected reverts
  • Gas costs as expected
  • Net profit is positive

🎯 Success Criteria for Fast MVP

Minimum Viable Success (Worth continuing):

  • Bot runs for 7 days without crashes
  • Executes at least 5 trades
  • Success rate >50%
  • Net profit >0 (even $10 is validation)
  • No major bugs or losses

Strong Success (Scale up immediately):

  • Net profit >10% in first week
  • Success rate >70%
  • Consistent daily opportunities
  • No circuit breaker trips
  • Clear path to scaling

Failure (Pivot or abandon):

  • Net loss after 7 days
  • Success rate <30%
  • Circuit breaker trips repeatedly
  • No arbitrage opportunities
  • Competition too fierce

🔍 What We're NOT Building (Yet)

These are explicitly OUT OF SCOPE for Fast MVP:

Deferred to "Scale-Up Phase" (if MVP is profitable):

  1. More DEX Protocols

    • Curve, Balancer, SushiSwap, Camelot
    • Add in Week 5-6 if MVP works
  2. Sequencer Integration

    • Front-running via sequencer feed
    • Add in Week 6-7 if needed for competitiveness
  3. Multi-Hop Arbitrage

    • 3-hop and 4-hop paths
    • Add if 2-hop is saturated
  4. Batch Execution

    • Multicall for gas savings
    • Add when trade volume justifies it
  5. Advanced Gas Optimization

    • Dynamic gas strategies
    • EIP-1559 optimization
    • Add if gas is eating profits
  6. Fancy Dashboard

    • Grafana, Prometheus
    • Real-time monitoring
    • Add when operating at scale
  7. Flashbots Integration

    • Not available on Arbitrum anyway
    • May never be needed

📊 Realistic Expectations

Week 1 Projections (Very Conservative):

Capital: 0.5 ETH
Trades: 3-5 per day
Success Rate: 50% (learning phase)
Avg Profit per Success: 0.005 ETH
Avg Gas per Trade: 0.002 ETH

Daily Net:
  Success: 2 × 0.005 = 0.01 ETH
  Failed Gas: 3 × 0.002 = 0.006 ETH
  Net: 0.004 ETH per day

Weekly: 0.028 ETH (5.6% ROI)

If This Works, Week 2-4:

Capital: 1.0 ETH (increase after validation)
Trades: 10 per day (more confidence)
Success Rate: 70% (optimization)
Avg Profit: 0.008 ETH

Daily Net: 0.035 ETH
Weekly: 0.245 ETH (24.5% ROI)

If This REALLY Works (Month 2-3):

  • Add more DEX protocols → more opportunities
  • Add sequencer → better execution
  • Scale capital to 5-10 ETH
  • Target: 50-100% monthly ROI

🚨 Risk Management

Hard Limits (Circuit Breakers):

max_loss_per_day: 0.05 ETH
max_loss_per_week: 0.1 ETH
max_consecutive_failures: 3
max_trade_size: 0.1 ETH
max_trades_per_day: 20
max_gas_price: 0.5 gwei (Arbitrum is cheap)

circuit_breaker_cooldown: 1 hour
emergency_stop_loss: 0.2 ETH total

Manual Oversight:

  • Check metrics every 6 hours (first week)
  • Review all failed trades
  • Adjust limits based on results
  • Be ready to pause if needed

Emergency Stop:

# Kill switch command
pkill -f mev-bot

# Or via API
curl -X POST http://localhost:8080/emergency-stop

📦 Minimal Tech Stack

Core Dependencies:

  • Language: Go 1.21+
  • Ethereum Client: go-ethereum (geth)
  • RPC Provider: Alchemy or Infura (free tier is fine)
  • Database: SQLite (simple, no postgres needed yet)
  • Logging: Standard Go log package
  • Metrics: Simple file-based logs

Infrastructure:

  • Hosting: Local machine or cheap VPS ($5/month)
  • Monitoring: Tail logs + manual checks
  • Alerts: None (you'll check manually)

Later (if profitable):

  • Upgrade to dedicated server
  • Add Prometheus + Grafana
  • Set up PagerDuty alerts
  • Use PostgreSQL for analytics

🎬 Implementation Order (Detailed)

Week 1: Days 1-2 (UniswapV2 Parser)

# Create feature branch
git checkout -b feature/v2/parsers/uniswap-v2-mvp

# Implement
touch pkg/parsers/uniswap_v2.go
touch pkg/parsers/uniswap_v2_test.go

# Focus areas:
1. Swap event signature: 0xd78ad95f...
2. ABI decoding: amount0In, amount1In, amount0Out, amount1Out
3. Token extraction from pool cache
4. Decimal scaling (critical!)
5. Validation (no zero addresses)

# Test with real data
# Example: https://arbiscan.io/tx/0x...
# Parse real Uniswap V2 swap on Arbitrum

# Achieve 100% coverage
go test ./pkg/parsers/... -coverprofile=coverage.out
go tool cover -html=coverage.out

Week 1: Days 3-5 (UniswapV3 Parser)

git checkout -b feature/v2/parsers/uniswap-v3-mvp

# Key differences from V2:
1. Signed amounts (int256, not uint256)
2. sqrtPriceX96 (Q64.96 fixed point)
3. Tick and liquidity
4. Fee tiers (0.05%, 0.3%, 1%)

# Math helpers needed:
func sqrtPriceX96ToPrice(sqrtPriceX96 *big.Int) *big.Float
func calculateSwapOutput(pool *Pool, amountIn *big.Int) *big.Int

Week 2: Days 1-4 (Arbitrage Detection)

git checkout -b feature/v2/arbitrage/basic-detection

# File structure:
pkg/arbitrage/
├── graph.go           # Market graph
├── pathfinder.go      # 2-hop BFS
├── profitability.go   # Profit calculation
└── detector.go        # Main detector

# Key algorithm:
For each token pair (A, B):
  Find all pools: A → B
  For each pool P1:
    For each other pool P2:
      If P1.price != P2.price:
        Calculate arbitrage profit
        If profit > minProfit:
          Emit opportunity

Week 3: Days 1-6 (Execution Engine)

git checkout -b feature/v2/execution/basic-executor

# Critical path:
1. Build swap calldata
2. Estimate gas
3. Calculate gas cost
4. Verify still profitable
5. Sign transaction
6. Send to network
7. Wait for receipt
8. Handle success/failure

# Test on testnet FIRST!
ARBITRUM_RPC=https://sepolia-rollup.arbitrum.io/rpc

Week 4: Days 1-7 (Deployment)

# Shadow mode config:
SHADOW_MODE=true
LOG_LEVEL=debug
MIN_PROFIT=0.01

# Go live:
SHADOW_MODE=false
MAX_TRADE_SIZE=0.1
MAX_TRADES_PER_DAY=10
CIRCUIT_BREAKER_LOSS=0.05

🎯 Decision Points

After Week 2 (Arbitrage Detection):

Question: Are there enough profitable opportunities?

Run detection against historical data:

go run cmd/historical-analysis/main.go \
  --start-block 150000000 \
  --end-block 150001000 \
  --min-profit 0.01

If <5 opportunities per day:

  • Stop and reconsider strategy
  • Maybe try different DEXs
  • Maybe lower profit threshold

If >10 opportunities per day:

  • Continue to execution phase

After Week 3 (Testnet):

Question: Do trades execute successfully?

If success rate <50%:

  • Debug execution logic
  • Check gas estimation
  • Verify slippage calculations

If success rate >70%:

  • Proceed to mainnet

After Week 4 Day 7 (First Week Live):

Question: Is this profitable?

If net profit >0:

  • Continue for another week
  • Consider scaling capital

If net profit <0:

  • Analyze why:
    • Competition too fierce?
    • Gas too expensive?
    • Calculations wrong?
  • Decide: Fix and retry, or pivot?

🔄 What Happens After 4 Weeks?

Scenario A: MVP is Profitable

Next Steps:

  1. Increase capital to 2-5 ETH
  2. Add more DEX protocols (Curve, Balancer)
  3. Implement 3-hop arbitrage
  4. Add sequencer integration (for speed)
  5. Build proper monitoring dashboard
  6. Scale to 20-50% monthly ROI

Timeline: 4 more weeks to "Full MVP"

Scenario B: MVP is Break-Even ⚖️

Next Steps:

  1. Optimize for 2 more weeks
  2. Add sequencer (may be the missing piece)
  3. Reduce gas costs (batch execution)
  4. If still break-even, reconsider

Scenario C: MVP is Unprofitable

Analysis:

  • Is Arbitrum too competitive?
  • Are opportunities too rare?
  • Is our execution too slow?

Options:

  1. Pivot to different chain (Polygon? Base?)
  2. Try different MEV strategy (liquidations?)
  3. Abandon and move on

Key: We only invested 4 weeks, not 10!


📋 Week-by-Week Checklist

Week 1 Checklist:

  • UniswapV2 parser complete (100% coverage)
  • UniswapV3 parser complete (100% coverage)
  • Pool cache populated with major pairs
  • Can parse any swap on Arbitrum
  • All tests passing

Week 2 Checklist:

  • Market graph built from pools
  • 2-hop pathfinder working
  • Profitability calculator accurate
  • Finding >5 opportunities per day (historical)
  • Detection latency <100ms

Week 3 Checklist:

  • Can execute swaps on testnet
  • Gas estimation accurate
  • Slippage protection working
  • Circuit breaker tested
  • Success rate >70% on testnet

Week 4 Checklist:

  • Metrics collection working
  • Shadow mode validated (24 hours)
  • First live trade successful
  • Circuit breaker hasn't tripped
  • Net profit >0 after 7 days

💡 Key Principles for Fast MVP

1. Simple Over Perfect

❌ Don't: Build a sophisticated gas optimization system
✅ Do: Just use current gas price + 10%

❌ Don't: Support 13 DEX protocols
✅ Do: Start with 2, add more if profitable

❌ Don't: Build a ML model for profit prediction
✅ Do: Simple math: output - input - gas

2. Validate Assumptions Fast

Week 1: Can we parse swaps correctly?
Week 2: Are there arbitrage opportunities?
Week 3: Can we execute trades?
Week 4: Is it profitable?

Each week answers ONE key question.

3. Fail Fast, Pivot Faster

If Week 2 shows no opportunities → STOP
If Week 3 shows trades fail → FIX or STOP
If Week 4 shows losses → PIVOT or STOP

Don't throw good time after bad.

4. Real Data Over Assumptions

Don't assume profitability → TEST IT
Don't assume opportunities exist → MEASURE THEM
Don't assume execution works → VERIFY IT

Shadow mode + small capital = real data

🚀 Let's Start!

Your next immediate action:

# 1. Review this plan
# 2. If approved, start Week 1 Day 1:

git checkout -b feature/v2/parsers/uniswap-v2-mvp

# Create the parser
touch pkg/parsers/uniswap_v2.go
touch pkg/parsers/uniswap_v2_test.go

# Let's build! 🏗️

This plan gets you to profitability validation in 4 weeks with minimal capital risk. After that, you have REAL DATA to decide whether to scale, pivot, or stop.

Ready to start Week 1? 🎬


🚨 CRITICAL BUG DISCOVERED (Nov 2024)

Problem: Bot Uses STALE Reserve Data

Symptom: Bot ran for 17+ hours, found ZERO arbitrage opportunities.

Root Cause: The arbitrage detector uses pool reserves fetched ONCE at startup and never refreshes them.

// CURRENT (BROKEN) FLOW:
1. Bot starts  DiscoverMajorPools() fetches reserves ONCE
2. Bot scans every 30s  Uses SAME stale reserves
3. Real prices change constantly  Bot sees old data
4. Result: ZERO opportunities found (markets look balanced with old data)

Why This Matters:

  • Pool reserves change with EVERY swap on-chain
  • Arbitrage opportunities exist for seconds/milliseconds
  • Using stale data = guaranteed to miss every opportunity
  • 17 hours of runtime = 17 hours of wasted scanning

Solution: Live Reserve Refresh

MUST IMPLEMENT before bot can find real opportunities:

// pkg/discovery/reserve_refresh.go

// RefreshReserves fetches latest reserves from chain for all cached pools
func (d *UniswapV2PoolDiscovery) RefreshReserves(ctx context.Context) error {
    pools, _ := d.poolCache.GetByLiquidity(ctx, big.NewInt(0), 10000)

    for _, pool := range pools {
        // Call getReserves() on each pool contract
        reserves, err := d.fetchReservesFromChain(ctx, pool.Address)
        if err != nil {
            continue // Skip failed pools
        }

        // Update cache with fresh reserves
        pool.Reserve0 = reserves.Reserve0
        pool.Reserve1 = reserves.Reserve1
        pool.LastUpdated = time.Now()

        d.poolCache.Update(ctx, pool)
    }
    return nil
}

Integration in main.go:

// BEFORE each scan, refresh reserves
for range ticker.C {
    // NEW: Refresh reserves from chain
    if err := poolDiscovery.RefreshReserves(ctx); err != nil {
        logger.Error("failed to refresh reserves", "error", err)
    }

    // Then scan for opportunities (with fresh data!)
    opportunities, err := detector.ScanForOpportunities(ctx, blockNumber)
    // ...
}

Implementation Priority: HIGHEST

Task Priority Estimated Time
Create reserve_refresh.go P0 30 min
Add RefreshReserves() method P0 30 min
Call refresh before each scan P0 15 min
Test with live data P0 30 min
Total CRITICAL ~2 hours

Verification Test

After implementing reserve refresh:

# Run bot with verbose logging
./bin/mev-flashloan --min-profit 5 --interval 10s --verbose

# Expected output (after fix):
# - Reserve values should CHANGE between scans
# - Some opportunities should be found (even if small)
# - If still 0 opportunities after 1 hour, lower min-profit further

Why This Wasn't Caught Earlier

  1. Unit tests use mock data (don't need live reserves)
  2. Integration tests check parsing, not live detection
  3. Bot "worked" (no crashes) but with stale data
  4. Need live mainnet testing with reserve updates

Lesson Learned

Always test with LIVE data before declaring production-ready.

Static test data can hide critical bugs like stale caches.


📋 Updated Implementation Checklist

Immediate Fixes Required:

  • P0: Implement RefreshReserves() in pool discovery
  • P0: Call reserve refresh before each arbitrage scan
  • P0: Add logging to show reserve changes between scans
  • P0: Test on mainnet - verify opportunities are found
  • P1: Add reserve age check (skip pools not updated in >60s)
  • P1: Batch RPC calls for efficiency (multicall)
  • P2: Add WebSocket subscription for real-time reserve updates

Performance Considerations:

With 9 pools, refreshing reserves adds:

  • ~9 RPC calls per scan (one per pool)
  • ~500ms latency (with public RPC)
  • Acceptable for 30s scan interval

For scaling to 100+ pools:

  • Use multicall to batch reserve fetches
  • Consider WebSocket subscriptions
  • Target <100ms refresh time