mev-beta/docs/LOG_ANALYSIS_20251027_090300.md

# MEV Bot Log Analysis Report
## Generated: October 27, 2025 09:01 AM

### 🚨 CRITICAL ISSUES

#### 1. **RPS LIMIT EXCEEDED (CRITICAL)**
- **Count**: 29,762 occurrences in error log
- **Impact**: Bot cannot fetch blocks or pool data
- **Root Cause**: Exceeding Chainstack RPS limits
- **Status**: BLOCKING - Prevents normal operation

**Error Message:**
```
"You've exceeded the RPS limit available on the current plan"
```

**Affected Operations:**
- Block fetching (GetBlockByNumber)
- Pool state queries (slot0, liquidity, token0, token1, fee)
- Token data retrieval

**Impact on Performance:**
- Missing blocks entirely during rate limit periods
- Unable to fetch pool reserves for profit calculations
- Cascading failures in arbitrage detection

---

#### 2. **ZERO ARBITRAGE OPPORTUNITIES DETECTED**
- **Detected**: 0
- **Executed**: 0  
- **Successful**: 0
- **Total Profit**: 0.000000 ETH

**Root Causes:**
1. RPS limiting preventing pool data fetch
2. Most blocks showing "No DEX transactions found"
3. Blacklisted pools reducing detection surface

---

#### 3. **MATHEMATICAL CALCULATION ERRORS**
- **Count**: 2,353 instances of "sqrtPrice is non-positive"
- **Impact**: Using fallback "price before" which may be inaccurate

**Example:**
```
Calculated sqrtPrice is non-positive (-1092.786631), using price before
Calculated sqrtPrice is non-positive (-0.333409), using price before
```

**Analysis:**
- Negative square root calculations indicate formula issues
- Falling back to previous price may miss actual arbitrage opportunities
- Related to price impact calculations in analyzer.go

---

#### 4. **POOL BLACKLISTING**
- **Blacklisted Pools**: 9 unique pools
- **Primary Issue**: Pool `0xB1026b8e...` referenced 7 times
- **Reason**: "slot0() consistently reverts - invalid pool contract"

**Blacklisted Pool Example:**
```
Pool: 0xB1026b8e7276e7AC75410F1fcbbe21796e8f7526
Issue: slot0() call reverts consistently
Impact: Cannot fetch pool state, events rejected
```

**Newly Blacklisted:**
```
Pool: 0x13BC35D101B646Cf1F566f95077E67a9f5b301a3
Reason: execution reverted on slot0()
```

---

#### 5. **PRICE IMPACT TOO LARGE**
- **Count**: 1,287 warnings
- **Message**: "Price impact too large (X.XX), capping at 1.0"

**Examples:**
```
Price impact too large (1.48), capping at 1.0
```

**Analysis:**
- Price impact exceeding 100% (1.0) suggests calculation issues
- May indicate abnormal swap sizes or liquidity depth problems
- Capping may hide real arbitrage signals

---

### 📊 PERFORMANCE METRICS

#### Block Processing
- **Block Range**: 393955078 - 393957767 (~2,689 blocks)
- **Average Processing Time**: ~100-600ms per block
- **Parse Rate**: 25,000 - 572,000 tx/sec (highly variable)

#### DEX Transaction Detection
- **Most Blocks**: 0 DEX transactions found
- **Occasional Blocks**: 1-2 DEX transactions
- **Detection Rate**: <5% of blocks contain DEX transactions

#### RPC Performance
- **Successful Calls**: Many showing "SUCCESS" status
- **Call Duration**: 79ms - 623ms per call
- **Issue**: Success rate drops dramatically during RPS limit periods

---

### 🔍 ERROR BREAKDOWN

| Error Type | Count | Severity | Status |
|------------|-------|----------|--------|
| RPS Limit Exceeded | 29,762 | CRITICAL | Blocking |
| Negative sqrtPrice | 2,353 | HIGH | Affecting accuracy |
| Price Impact Too Large | 1,287 | MEDIUM | May hide opportunities |
| Pool Blacklisted | 9 | MEDIUM | Reducing coverage |
| No Arbitrage Detected | All scans | HIGH | No profit |

**Total Error Log Size:** 13MB (41,707 lines)

---

### 💡 ROOT CAUSE ANALYSIS

#### Primary Issue: RPS Limiting
The Chainstack RPC endpoint is being hit too aggressively:

**Current Configuration:**
```env
RPC_REQUESTS_PER_SECOND=5
RPC_MAX_CONCURRENT=3
```

**Actual Behavior:**
- Bot attempts to fetch block + multiple pool states per block
- Each DEX transaction triggers multiple pool queries
- No effective rate limiting or request batching
- Reserve cache exists but RPS limits hit before cache helps

**Calculation:**
- ~2 blocks/second * (1 block fetch + 5-10 pool queries) = 12-22 RPS
- This exceeds the 5 RPS limit configured in .env

---

#### Secondary Issue: Detection Coverage
**Very Few DEX Transactions Detected:**
- Most blocks: "No DEX transactions found"
- Possible reasons:
  1. ABI decoder not recognizing all DEX protocols
  2. TraderJoe and other DEXes not in detection list
  3. Multicall transactions not fully parsed

**Example Missed:**
```
0x7cb42dcf: TraderJoeRouter calling multicall (Multicall)
Status: Detected but not analyzed for arbitrage
```

---

#### Tertiary Issue: Mathematical Errors
**Negative Square Root Calculations:**
Location: `pkg/scanner/swap/analyzer.go`

The formula for calculating `priceAfter` produces negative values under certain conditions:
```go
// Somewhere in analyzer.go
sqrtPrice = calculatePriceAfterSwap(...)
// Result: negative value (mathematically impossible for sqrt)
```

**Likely Causes:**
1. Division by zero or near-zero liquidity
2. Incorrect handling of swap direction (token0 → token1 vs token1 → token0)
3. Overflow/underflow in big.Int arithmetic

---

### ⚠️ INCONSISTENCIES DETECTED

#### 1. **Configuration vs Reality**
- **Config**: `RPC_REQUESTS_PER_SECOND=5`
- **Reality**: Bot makes 12-22 RPS during active blocks
- **Result**: Constant RPS limit errors

#### 2. **Cache Implementation**
- **Implemented**: Reserve cache with 45s TTL
- **Issue**: Cache not preventing RPS limits
- **Likely**: Cache misses due to unique pool addresses per transaction

#### 3. **Zero Address Detection**
Several events show zero addresses for tokens:
```
📤 Submitting event: Tokens=0x00000000↔0x00000000
```

This indicates:
- Token address extraction failing
- Events being processed with invalid data
- Wasting resources on invalid events

---

### 📈 STATISTICAL SUMMARY

**Time Period Analyzed:** ~1 hour (08:55 - 09:01)
**Blocks Processed:** ~2,689 blocks
**Error Rate:** 29,762 errors / 2,689 blocks = ~11 errors per block
**Success Rate:** 0% (no arbitrage detected/executed)

**Most Common Errors (by type):**
1. RPS Limit: 71.3% of all errors
2. Negative sqrtPrice: 5.6%
3. Price Impact Too Large: 3.1%
4. Other: 20%

---

### 🎯 RECOMMENDATIONS

#### Immediate Actions (Within 24 Hours)

**1. Reduce RPC Request Rate**
```diff
# .env
- RPC_REQUESTS_PER_SECOND=5
+ RPC_REQUESTS_PER_SECOND=2
- RPC_MAX_CONCURRENT=3
+ RPC_MAX_CONCURRENT=1
```

**2. Implement Request Batching**
- Batch multiple pool queries into single multicall
- Use existing Multicall3 contract on Arbitrum
- Reduce RPC calls by 80-90%

**3. Fix sqrtPrice Calculation**
- Add bounds checking before sqrt operations
- Handle zero/negative liquidity gracefully
- Add logging for calculation inputs

#### Short-Term Fixes (Within 1 Week)

**1. Improve DEX Detection**
- Add TraderJoe protocol support
- Enhance multicall transaction parsing
- Add more DEX router addresses

**2. Implement Exponential Backoff**
- Add retry logic for RPC failures
- Exponential backoff on rate limits
- Circuit breaker pattern

**3. Optimize Cache Strategy**
- Increase cache TTL to 60-90 seconds
- Add pool address normalization
- Pre-fetch common pool data

#### Long-Term Improvements (Within 1 Month)

**1. Upgrade RPC Provider**
- Consider higher-tier Chainstack plan
- Add fallback RPC endpoints
- Implement load balancing

**2. Add Request Queue**
- Priority queue for critical requests
- Throttle non-critical queries
- Smart request scheduling

**3. Enhance Mathematical Robustness**
- Add unit tests for edge cases
- Implement SafeMath patterns
- Add assertion checks

---

### 🔧 CONFIGURATION CHANGES NEEDED

**File: `.env`**
```bash
# Reduce RPC rate to avoid limits
RPC_REQUESTS_PER_SECOND=2
RPC_MAX_CONCURRENT=1

# Increase cache TTL
CACHE_TTL_SECONDS=90

# Enable request batching
ENABLE_MULTICALL_BATCHING=true
MULTICALL_BATCH_SIZE=20
```

**File: `config/config.yaml`**
```yaml
arbitrum:
  rpc_rate_limit: 2
  max_concurrent_requests: 1
  enable_retry: true
  retry_max_attempts: 3
  retry_backoff_ms: 1000

cache:
  reserve_ttl_seconds: 90
  pool_ttl_seconds: 300
  
scanner:
  enable_multicall_batching: true
  batch_size: 20
```

---

### ✅ CONCLUSION

**Critical Issues:**
1. ⛔ RPS limiting preventing normal operation
2. ⛔ Zero arbitrage opportunities detected
3. ⚠️ Mathematical calculation errors

**Immediate Impact:**
- Bot is essentially non-functional for arbitrage detection
- Cannot fetch necessary pool data due to rate limits
- No revenue generation possible in current state

**Priority Actions:**
1. **NOW**: Reduce RPC request rate to 2 RPS
2. **TODAY**: Implement request batching with multicall
3. **THIS WEEK**: Fix sqrtPrice calculation errors
4. **THIS MONTH**: Upgrade RPC tier or add fallbacks

**Expected Improvement:**
- Reduce errors by 90%+ with proper rate limiting
- Enable arbitrage detection with available pool data
- Generate first profitable opportunities within 24-48 hours