mev-beta/docs/CACHING_ANALYSIS_20251030.md

Caching & Price Tracking Analysis - MEV Bot

Date: October 30, 2025 Status: ⚠️ CRITICAL ISSUES IDENTIFIED

---

Executive Summary

Analysis of the MEV bot's caching and price tracking mechanisms reveals 3 critical issues that are preventing effective arbitrage detection:

1. Pool cache is NOT being persisted to disk
2. NO swap events are being processed from the sequencer feed
3. Prices are NOT being updated from on-chain swaps

---

🔴 Critical Issue #1: Pool Cache Not Persisted

Current Behavior

Discovery IS working ✅

• Bot discovered 96 pools across 32 pairs (up from 10)
• All 20 new tokens found pools
• Discovery took ~3 minutes

Cache NOT saved ❌

• Pools stored in memory only
• data/pools.json shows old data (Oct 27 23:59)
• Discovered pools lost on restart

Evidence

# Pool cache file (outdated)
-rw-r--r-- 1 administrator administrator 5.1K Oct 27 23:59 data/pools.json

# Current pools in cache
$ jq 'length' data/pools.json
10  # Should be 96!

# Latest discovery logs (17:59-18:00)
[INFO] ✅ Found 96 pools across 32 pairs
[INFO] Discovery complete! Monitoring 96 pools

Root Cause

Missing or not-called SaveCache function:

• poolsFile: "data/pools.json" defined in discovery.go
• No evidence of SaveCache() being called after discovery
• Pools kept in memory but never persisted

Impact

• High: Bot loses all discoveries on restart
• High: Must re-discover 96 pools every startup (3+ minutes)
• Medium: Increased RPC calls and startup time

---

🔴 Critical Issue #2: Zero Swap Events Processed

Current Behavior

Bot IS monitoring blocks ✅

• Processing blocks every 3 seconds
• Checking for Uniswap V3 swap events
• Arbitrage service stats updating every 10s

NO swaps detected ❌

• 0 swap events found in 100+ blocks
• Every block shows: "NO SWAP EVENTS FOUND"
• 0 arbitrage opportunities detected

Evidence

# Swap event count
$ grep "swap event" logs/mev_bot.log | wc -l
0  # Should be hundreds/thousands

# Block monitoring (17:13-17:14)
[INFO] 📦 Block 395116600: NO SWAP EVENTS FOUND
[INFO] 📦 Block 395116612: NO SWAP EVENTS FOUND
[INFO] 📦 Block 395116624: NO SWAP EVENTS FOUND
... (continues for all blocks)

# Arbitrage stats
[INFO] Arbitrage Service Stats - Detected: 0, Executed: 0

Root Cause Analysis

Possible causes:

1. Wrong event signature - Uniswap V3 Swap signature mismatch
2. Pool address filter - Only monitoring 96 pools, swaps happening elsewhere
3. Block processing lag - Getting stale blocks after swaps already settled
4. Log parsing error - Swap logs exist but not being decoded correctly

Expected Behavior

On Arbitrum mainnet:

• ~500-1000 swaps per minute across all DEXes
• ~5-10 swaps per block on monitored pools
• Bot should detect 50+ swaps/minute minimum

Impact

• CRITICAL: Cannot detect arbitrage without swap data
• CRITICAL: No price updates = stale pricing
• HIGH: Bot is effectively blind to market activity

---

🔴 Critical Issue #3: No Price Updates

Current Behavior

Price tracking exists ✅

• ExchangePricer with priceCache map
• Market.UpdatePriceData() function available
• Price cache initialized at startup

Prices NOT being updated ❌

• No swap events = no price data
• Pools have initial liquidity but no current prices
• Cache shows 0 entries

Evidence

# Price-related logs
$ grep -i "price.*update" logs/mev_bot.log
(only gas price updates found, no token price updates)

# Price cache stats
priceCache: map[string]*PriceEntry  # Empty
cached_prices: 0

# Market price updates
$ grep "UpdatePriceData" logs/mev_bot.log
(no results)

How Pricing SHOULD Work

1. Sequencer feed → New transaction
2. Decode Swap event → Extract amounts, pool
3. Calculate new price → sqrtPriceX96, tick
4. Update pool cache → Market.UpdatePriceData()
5. Compare prices → Detect arbitrage

Current Broken Flow

1. Sequencer feed → New transaction ✅
2. Check for swaps → NONE FOUND ❌
3. No price calculation ❌
4. No cache update ❌
5. No arbitrage detection ❌

Impact

• CRITICAL: Arbitrage detection requires real-time prices
• HIGH: Using stale/initial prices leads to false opportunities
• HIGH: Cannot react to market movements

---

📊 Token Cache Analysis

Current State

{
  "tokens_cached": 6,
  "expected": 20,
  "file": "data/tokens.json",
  "last_updated": "Oct 27 16:26"
}

Issues

1. Only 6 tokens cached (should be 20)
2. Missing new tokens: PENDLE, RDNT, MAGIC, GRAIL, AAVE, CRV, BAL, COMP, MKR, USDC.e
3. Stale data - 3 days old

Impact

• MEDIUM: Token metadata lookups may fail for new tokens
• LOW: Doesn't block trading, just missing metadata

---

🔍 Detailed Investigation

Pool Discovery Process

What's Working:

✅ Discovery scans 190 pairs
✅ Finds pools via CREATE2 calculation
✅ Validates pools on-chain
✅ Stores pools in memory
✅ Logs discovery progress

What's NOT Working:

❌ SaveCache() not called after discovery
❌ data/pools.json not updated
❌ Pools lost on restart

Swap Processing Pipeline

What's Working:

✅ Connects to Arbitrum sequencer feed
✅ Receives new blocks every 3 seconds
✅ Checks blocks for Uniswap V3 logs
✅ Monitors 96 pools

What's NOT Working:

❌ No swap events detected (0 in 1000+ blocks)
❌ No logs match Uniswap V3 Swap signature
❌ Either wrong signature OR pools inactive

Price Caching System

Components:

• ExchangePricer - Manages cross-exchange pricing
• PriceEntry - Cache entry with timestamp & validity
• Market.UpdatePriceData() - Updates price per pool
• priceCache map - In-memory price storage

Status:

• ✅ System initialized
• ✅ Data structures ready
• ❌ Never receives swap data
• ❌ Never updates prices

---

🔧 Required Fixes

Fix #1: Implement Pool Cache Persistence

Add SaveCache call after discovery:

// In cmd/mev-bot/main.go after discovery complete
log.Info(fmt.Sprintf("🎉 Pool discovery complete! Monitoring %d pools", totalPools))

// ADD THIS: Save discovered pools to disk
if err := poolDiscovery.SaveCache(); err != nil {
    log.Error(fmt.Sprintf("Failed to save pool cache: %v", err))
} else {
    log.Info("✅ Pool cache saved to data/pools.json")
}

Implement SaveCache if missing:

// In pkg/pools/discovery.go
func (pd *PoolDiscovery) SaveCache() error {
    pd.mu.RLock()
    defer pd.mu.RUnlock()

    // Convert map to slice
    pools := make([]*Pool, 0, len(pd.pools))
    for _, pool := range pd.pools {
        pools = append(pools, pool)
    }

    // Marshal to JSON
    data, err := json.MarshalIndent(pools, "", "  ")
    if err != nil {
        return fmt.Errorf("failed to marshal pools: %w", err)
    }

    // Write to file
    if err := os.WriteFile(pd.poolsFile, data, 0644); err != nil {
        return fmt.Errorf("failed to write pools file: %w", err)
    }

    return nil
}

Fix #2: Debug Swap Event Detection

Check event signature:

# Uniswap V3 Swap event signature
keccak256("Swap(address,address,int256,int256,uint160,uint128,int24)")
= 0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67

Add diagnostic logging:

// In swap processing
log.Info(fmt.Sprintf("Checking block %d for swaps", blockNumber))
log.Debug(fmt.Sprintf("Monitoring %d pools", len(monitoredPools)))

for i, log := range logs {
    log.Debug(fmt.Sprintf("Log %d: topics=%v", i, log.Topics))

    if log.Topics[0].Hex() == SwapEventSignature {
        log.Info("✅ SWAP FOUND!")
        // Process swap
    }
}

Verify pool addresses:

// Check if we're monitoring the right pools
for pool := range monitoredPools {
    log.Debug(fmt.Sprintf("Monitoring pool: %s", pool.Hex()))
}

Fix #3: Implement Price Updates from Swaps

Add price update to swap handler:

// After processing swap event
func handleSwap(pool common.Address, sqrtPriceX96 *big.Int, tick int32) {
    // Calculate price from sqrtPriceX96
    price := uniswap.SqrtPriceX96ToPrice(sqrtPriceX96)

    // Update market data
    if market, exists := markets[pool]; exists {
        market.UpdatePriceData(price, liquidity, sqrtPriceX96, tick)
        log.Debug(fmt.Sprintf("Updated price for pool %s: %s",
            pool.Hex()[:8], price.String()))
    }

    // Update price cache
    cacheKey := fmt.Sprintf("%s/%s", tokenIn, tokenOut)
    priceCache[cacheKey] = &PriceEntry{
        Price:     price,
        Timestamp: time.Now(),
        Validity:  5 * time.Minute,
    }
}

---

🎯 Verification Steps

After Fix #1 (Pool Persistence)

# 1. Start bot
./mev-bot start

# 2. Wait for discovery (3 minutes)
# Should see: "✅ Pool cache saved"

# 3. Check file updated
ls -lh data/pools.json
# Should show current timestamp

# 4. Verify contents
jq 'length' data/pools.json
# Should show 96 (not 10)

# 5. Restart bot
# Should load 96 pools instantly

After Fix #2 (Swap Detection)

# Monitor logs for swaps
tail -f logs/mev_bot.log | grep -i swap

# Expected output:
[INFO] ✅ SWAP FOUND on pool 0x82af...
[INFO] Processing swap: 1000 USDC → 0.5 WETH
[INFO] Updated price for pool 0x82af
[INFO] Checking for arbitrage opportunities...

After Fix #3 (Price Updates)

# Check price cache stats
grep "price.*cache\|cached.*price" logs/mev_bot.log

# Expected:
[INFO] Price cache stats: 32 prices cached
[INFO] Updated 15 prices in last minute
[INFO] Oldest price age: 45 seconds

---

📈 Expected Performance After Fixes

Pool Management

• ✅ 96 pools persisted to disk
• ✅ Instant load on restart (<1 second)
• ✅ Discovery runs once, cache reused

Swap Processing

• ✅ 50-100 swaps/minute detected
• ✅ Real-time price updates
• ✅ <100ms latency from swap to price update

Arbitrage Detection

• ✅ Compare prices across 96 pools
• ✅ Detect opportunities within 1 block
• ✅ 5-10+ opportunities/hour expected

---

🚨 Priority Order

1. URGENT: Fix swap event detection (Fix #2)

   • Why: Bot is blind without swap data
   • Impact: Enables all arbitrage detection

2. HIGH: Implement price updates (Fix #3)

   • Why: Required for arbitrage calculation
   • Impact: Real-time opportunity detection

3. MEDIUM: Add pool cache persistence (Fix #1)

   • Why: Improves startup time
   • Impact: UX improvement, reduced RPC load

---

📝 Additional Recommendations

Monitoring Enhancements

# Add metrics for debugging
- swap_events_processed (counter)
- price_updates_count (counter)
- cache_hit_rate (gauge)
- pool_count (gauge)
- arbitrage_opportunities_detected (counter)

Logging Improvements

// Add periodic stats logging
Every 60 seconds:
  - Pools monitored: 96
  - Swap events (last min): 47
  - Price updates (last min): 32
  - Arbitrage opportunities: 3
  - Cache size: 5.2KB

Testing Strategy

1. Unit Tests: Test SaveCache() function
2. Integration Tests: Verify swap event parsing
3. End-to-End Tests: Full flow from swap → arbitrage
4. Load Tests: Handle 1000+ swaps/minute

---

🎯 Success Criteria

After fixes are applied, the bot should:

1. ✅ Discover 96 pools and save to disk
2. ✅ Detect 50+ swaps per minute
3. ✅ Update prices within 100ms of swap
4. ✅ Cache 20-30 active prices
5. ✅ Detect 5-10 arbitrage opportunities per hour
6. ✅ Restart in <5 seconds (load from cache)

---

📚 Code References

• Pool Discovery: pkg/pools/discovery.go
• Price Engine: pkg/pricing/engine.go
• Market Updates: pkg/marketmanager/types.go:UpdatePriceData()
• Swap Processing: pkg/scanner/market/scanner.go
• Main Entry: cmd/mev-bot/main.go:256-323

---

Status: ⚠️ CRITICAL FIXES REQUIRED Next Action: Implement Fix #2 (Swap Detection) as highest priority Estimated Time: 2-4 hours for all fixes Expected Value: Enable arbitrage detection capability

---

Document Version: 1.0 Last Updated: October 30, 2025 18:15 UTC Analysis Duration: 30 minutes Evidence Reviewed: 10+ log files, 5 code modules, 3 cache files