docs: add critical bug - stale reserve data causing zero opportunities
CRITICAL BUG DISCOVERED: - Bot ran 17+ hours finding ZERO opportunities - Root cause: Reserves fetched ONCE at startup, never refreshed - Arbitrage detection uses stale data = misses all real opportunities SOLUTION DOCUMENTED: - Implement RefreshReserves() before each scan - ~2 hours implementation time - P0 priority - bot is non-functional without this Lesson learned: Always test with LIVE data, not just unit tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Gemini Agent
829
docs/FAST_MVP_PLAN.md
Normal file
829
docs/FAST_MVP_PLAN.md
Normal file
@@ -0,0 +1,829 @@
|
|||||||
|
# MEV Bot - Fast MVP Plan (4-5 Weeks)
|
||||||
|
|
||||||
|
**Objective**: Deploy profitable arbitrage bot in 4-5 weeks with minimal viable features
|
||||||
|
**Strategy**: Validate business model quickly, then decide whether to scale
|
||||||
|
**Capital**: Start with 0.5-1 ETH, scale if profitable
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Core Philosophy
|
||||||
|
|
||||||
|
**Ship fast, validate profitability, iterate based on real data.**
|
||||||
|
|
||||||
|
We're cutting everything that's not essential:
|
||||||
|
- ❌ No 13+ protocols (just UniswapV2 + UniswapV3)
|
||||||
|
- ❌ No 4-hop arbitrage (just 2-hop)
|
||||||
|
- ❌ No sequencer integration yet (regular RPC first)
|
||||||
|
- ❌ No batch execution (single trades only)
|
||||||
|
- ❌ No fancy dashboard (basic metrics only)
|
||||||
|
|
||||||
|
**We CAN add these later if the bot is profitable.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📅 4-Week Timeline
|
||||||
|
|
||||||
|
### **Week 1: Core Parsers** (Dec 1-7)
|
||||||
|
**Goal**: Parse UniswapV2 and UniswapV3 swaps accurately
|
||||||
|
|
||||||
|
#### Day 1-2: UniswapV2 Parser
|
||||||
|
- [ ] Implement Swap event parsing
|
||||||
|
- [ ] Handle Mint/Burn events
|
||||||
|
- [ ] Extract tokens from pool cache
|
||||||
|
- [ ] Decimal scaling (USDC 6, WBTC 8, WETH 18)
|
||||||
|
- [ ] 100% test coverage
|
||||||
|
- [ ] Integration test with real Arbitrum tx
|
||||||
|
|
||||||
|
**Success Criteria**:
|
||||||
|
- Parse any UniswapV2 swap on Arbitrum
|
||||||
|
- No zero addresses
|
||||||
|
- Correct decimal handling
|
||||||
|
- Tests pass: `make test-coverage`
|
||||||
|
|
||||||
|
#### Day 3-5: UniswapV3 Parser
|
||||||
|
- [ ] Parse V3 Swap events (signed amounts)
|
||||||
|
- [ ] Handle sqrtPriceX96 and tick
|
||||||
|
- [ ] Concentrated liquidity basics
|
||||||
|
- [ ] Multi-hop swap support
|
||||||
|
- [ ] 100% test coverage
|
||||||
|
- [ ] Integration tests
|
||||||
|
|
||||||
|
**Success Criteria**:
|
||||||
|
- Parse any UniswapV3 swap on Arbitrum
|
||||||
|
- Handle negative amounts correctly
|
||||||
|
- Proper tick/liquidity tracking
|
||||||
|
|
||||||
|
#### Day 6-7: Pool Discovery
|
||||||
|
- [ ] Fetch all UniswapV2 pools on Arbitrum
|
||||||
|
- [ ] Fetch all UniswapV3 pools on Arbitrum
|
||||||
|
- [ ] Populate pool cache
|
||||||
|
- [ ] Get initial reserves
|
||||||
|
- [ ] ~500-1000 pools total
|
||||||
|
|
||||||
|
**Success Criteria**:
|
||||||
|
- Pool cache has major trading pairs (WETH/USDC, WETH/ARB, etc.)
|
||||||
|
- Reserves are accurate
|
||||||
|
- Cache lookup is fast (<1ms)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### **Week 2: Arbitrage Detection** (Dec 8-14)
|
||||||
|
**Goal**: Find profitable 2-hop arbitrage opportunities
|
||||||
|
|
||||||
|
#### Day 1-2: Market Graph
|
||||||
|
- [ ] Build graph from pool cache
|
||||||
|
- [ ] Nodes = tokens, Edges = pools
|
||||||
|
- [ ] Efficient adjacency list representation
|
||||||
|
- [ ] Unit tests
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
```
|
||||||
|
WETH --[UniV2 Pool]-- USDC
|
||||||
|
WETH --[UniV3 Pool]-- USDC
|
||||||
|
ARB --[UniV2 Pool]-- WETH
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Day 3-4: Path Finder (2-Hop Only)
|
||||||
|
- [ ] Find circular paths: Token A → Token B → Token A
|
||||||
|
- [ ] BFS algorithm (simple, fast)
|
||||||
|
- [ ] Limit to 2 hops maximum
|
||||||
|
- [ ] Filter by minimum liquidity ($10k+)
|
||||||
|
- [ ] Unit tests with mock graph
|
||||||
|
|
||||||
|
**Example Arbitrage**:
|
||||||
|
```
|
||||||
|
Buy USDC with WETH on UniV2 (cheaper)
|
||||||
|
Sell USDC for WETH on UniV3 (more expensive)
|
||||||
|
Profit = difference - gas
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Day 5-6: Profitability Calculator
|
||||||
|
- [ ] Calculate swap output (AMM formula)
|
||||||
|
- [ ] Account for fees (0.3% UniV2, 0.05-1% UniV3)
|
||||||
|
- [ ] Estimate gas cost (~150k gas per hop)
|
||||||
|
- [ ] Calculate net profit
|
||||||
|
- [ ] Filter: only opportunities >0.01 ETH profit
|
||||||
|
|
||||||
|
**Formula**:
|
||||||
|
```
|
||||||
|
Gross Profit = Output Amount - Input Amount
|
||||||
|
Gas Cost = Gas Used × Gas Price
|
||||||
|
Net Profit = Gross Profit - Gas Cost
|
||||||
|
|
||||||
|
Only execute if Net Profit > 0.01 ETH
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Day 7: Integration & Testing
|
||||||
|
- [ ] End-to-end test: Pool cache → Graph → Paths → Profit
|
||||||
|
- [ ] Test with historical Arbitrum data
|
||||||
|
- [ ] Verify calculations match real outcomes
|
||||||
|
- [ ] Performance: <50ms per detection
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### **Week 3: Execution Engine** (Dec 15-21)
|
||||||
|
**Goal**: Execute profitable arbitrage trades
|
||||||
|
|
||||||
|
#### Day 1-2: Transaction Builder
|
||||||
|
- [ ] Build swap transaction for UniswapV2
|
||||||
|
- [ ] Build swap transaction for UniswapV3
|
||||||
|
- [ ] Calculate exact input/output amounts
|
||||||
|
- [ ] Add slippage tolerance (0.5%)
|
||||||
|
- [ ] Unit tests
|
||||||
|
|
||||||
|
#### Day 3-4: Execution Logic
|
||||||
|
- [ ] Connect to Arbitrum RPC (Alchemy/Infura)
|
||||||
|
- [ ] Get wallet nonce
|
||||||
|
- [ ] Estimate gas
|
||||||
|
- [ ] Set gas price (current + 10%)
|
||||||
|
- [ ] Sign transaction
|
||||||
|
- [ ] Submit to network
|
||||||
|
- [ ] Wait for confirmation
|
||||||
|
- [ ] Handle reverts gracefully
|
||||||
|
|
||||||
|
**Simple Flow**:
|
||||||
|
```go
|
||||||
|
func (e *Executor) Execute(opportunity *Opportunity) error {
|
||||||
|
// 1. Build transaction
|
||||||
|
tx := e.buildArbitrageTx(opportunity)
|
||||||
|
|
||||||
|
// 2. Estimate gas
|
||||||
|
gas, err := e.client.EstimateGas(tx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
tx.Gas = gas * 1.1 // 10% buffer
|
||||||
|
|
||||||
|
// 3. Get gas price
|
||||||
|
gasPrice, _ := e.client.SuggestGasPrice()
|
||||||
|
tx.GasPrice = gasPrice * 1.1 // 10% higher to ensure inclusion
|
||||||
|
|
||||||
|
// 4. Check still profitable after gas
|
||||||
|
netProfit := opportunity.GrossProfit - (gas * gasPrice)
|
||||||
|
if netProfit < minProfit {
|
||||||
|
return ErrNotProfitable
|
||||||
|
}
|
||||||
|
|
||||||
|
// 5. Sign and send
|
||||||
|
signedTx, _ := types.SignTx(tx, e.signer, e.privateKey)
|
||||||
|
err = e.client.SendTransaction(ctx, signedTx)
|
||||||
|
|
||||||
|
// 6. Wait for confirmation
|
||||||
|
receipt, err := e.waitForReceipt(signedTx.Hash())
|
||||||
|
|
||||||
|
// 7. Record result
|
||||||
|
e.recordTrade(opportunity, receipt)
|
||||||
|
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Day 5-6: Basic Risk Management
|
||||||
|
- [ ] Circuit breaker: Stop after 3 failed trades in a row
|
||||||
|
- [ ] Max loss limit: Stop after -0.1 ETH total loss
|
||||||
|
- [ ] Slippage protection: Revert if output < expected × 99.5%
|
||||||
|
- [ ] Position limit: Max 0.5 ETH per trade
|
||||||
|
- [ ] Cooldown: Wait 1 minute after circuit breaker trips
|
||||||
|
|
||||||
|
#### Day 7: Testnet Testing
|
||||||
|
- [ ] Deploy to Arbitrum Sepolia testnet
|
||||||
|
- [ ] Test full flow: Detect → Execute → Confirm
|
||||||
|
- [ ] Verify trades succeed
|
||||||
|
- [ ] Check profit calculations
|
||||||
|
- [ ] Fix any bugs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### **Week 4: Monitoring & Deployment** (Dec 22-28)
|
||||||
|
**Goal**: Deploy to mainnet with real capital
|
||||||
|
|
||||||
|
#### Day 1-2: Basic Metrics
|
||||||
|
- [ ] Count opportunities found
|
||||||
|
- [ ] Count trades executed
|
||||||
|
- [ ] Count successful vs failed
|
||||||
|
- [ ] Track total profit/loss
|
||||||
|
- [ ] Track gas spent
|
||||||
|
- [ ] Simple logging to file
|
||||||
|
|
||||||
|
**Metrics to track**:
|
||||||
|
```
|
||||||
|
Opportunities Found: 127
|
||||||
|
Trades Executed: 23
|
||||||
|
Successful: 18 (78%)
|
||||||
|
Failed: 5 (22%)
|
||||||
|
|
||||||
|
Gross Profit: 0.42 ETH
|
||||||
|
Gas Spent: 0.15 ETH
|
||||||
|
Net Profit: 0.27 ETH
|
||||||
|
ROI: 54% (on 0.5 ETH capital)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Day 3: Mainnet Preparation
|
||||||
|
- [ ] Security audit of wallet handling
|
||||||
|
- [ ] Create dedicated bot wallet
|
||||||
|
- [ ] Fund with 0.5 ETH test capital
|
||||||
|
- [ ] Set conservative limits:
|
||||||
|
- Max 0.1 ETH per trade
|
||||||
|
- Max 10 trades per day
|
||||||
|
- Stop after -0.05 ETH loss
|
||||||
|
- [ ] Backup private key securely
|
||||||
|
|
||||||
|
#### Day 4-5: Shadow Mode (24 hours)
|
||||||
|
- [ ] Run bot in shadow mode (detect but don't execute)
|
||||||
|
- [ ] Log all opportunities
|
||||||
|
- [ ] Calculate theoretical profit
|
||||||
|
- [ ] Verify no false positives
|
||||||
|
- [ ] Check for edge cases
|
||||||
|
|
||||||
|
**Example Log**:
|
||||||
|
```
|
||||||
|
[2024-12-22 10:15:23] OPPORTUNITY FOUND
|
||||||
|
Path: WETH → USDC (UniV2) → WETH (UniV3)
|
||||||
|
Input: 0.1 ETH
|
||||||
|
Expected Output: 0.1023 ETH
|
||||||
|
Gross Profit: 0.0023 ETH ($4.60)
|
||||||
|
Gas Cost: 0.0008 ETH ($1.60)
|
||||||
|
Net Profit: 0.0015 ETH ($3.00)
|
||||||
|
Action: WOULD EXECUTE (shadow mode)
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Day 6-7: Live Deployment (Carefully!)
|
||||||
|
- [ ] Switch to live mode
|
||||||
|
- [ ] Start with VERY conservative limits
|
||||||
|
- [ ] Monitor constantly (first 24 hours)
|
||||||
|
- [ ] Be ready to kill switch if needed
|
||||||
|
|
||||||
|
**First Day Checklist**:
|
||||||
|
- [ ] Bot detects opportunities ✅
|
||||||
|
- [ ] Profit calculations are accurate ✅
|
||||||
|
- [ ] Trades execute successfully ✅
|
||||||
|
- [ ] No unexpected reverts ✅
|
||||||
|
- [ ] Gas costs as expected ✅
|
||||||
|
- [ ] Net profit is positive ✅
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Success Criteria for Fast MVP
|
||||||
|
|
||||||
|
### **Minimum Viable Success** (Worth continuing):
|
||||||
|
- ✅ Bot runs for 7 days without crashes
|
||||||
|
- ✅ Executes at least 5 trades
|
||||||
|
- ✅ Success rate >50%
|
||||||
|
- ✅ Net profit >0 (even $10 is validation)
|
||||||
|
- ✅ No major bugs or losses
|
||||||
|
|
||||||
|
### **Strong Success** (Scale up immediately):
|
||||||
|
- ✅ Net profit >10% in first week
|
||||||
|
- ✅ Success rate >70%
|
||||||
|
- ✅ Consistent daily opportunities
|
||||||
|
- ✅ No circuit breaker trips
|
||||||
|
- ✅ Clear path to scaling
|
||||||
|
|
||||||
|
### **Failure** (Pivot or abandon):
|
||||||
|
- ❌ Net loss after 7 days
|
||||||
|
- ❌ Success rate <30%
|
||||||
|
- ❌ Circuit breaker trips repeatedly
|
||||||
|
- ❌ No arbitrage opportunities
|
||||||
|
- ❌ Competition too fierce
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 What We're NOT Building (Yet)
|
||||||
|
|
||||||
|
These are explicitly OUT OF SCOPE for Fast MVP:
|
||||||
|
|
||||||
|
### Deferred to "Scale-Up Phase" (if MVP is profitable):
|
||||||
|
1. **More DEX Protocols**
|
||||||
|
- Curve, Balancer, SushiSwap, Camelot
|
||||||
|
- Add in Week 5-6 if MVP works
|
||||||
|
|
||||||
|
2. **Sequencer Integration**
|
||||||
|
- Front-running via sequencer feed
|
||||||
|
- Add in Week 6-7 if needed for competitiveness
|
||||||
|
|
||||||
|
3. **Multi-Hop Arbitrage**
|
||||||
|
- 3-hop and 4-hop paths
|
||||||
|
- Add if 2-hop is saturated
|
||||||
|
|
||||||
|
4. **Batch Execution**
|
||||||
|
- Multicall for gas savings
|
||||||
|
- Add when trade volume justifies it
|
||||||
|
|
||||||
|
5. **Advanced Gas Optimization**
|
||||||
|
- Dynamic gas strategies
|
||||||
|
- EIP-1559 optimization
|
||||||
|
- Add if gas is eating profits
|
||||||
|
|
||||||
|
6. **Fancy Dashboard**
|
||||||
|
- Grafana, Prometheus
|
||||||
|
- Real-time monitoring
|
||||||
|
- Add when operating at scale
|
||||||
|
|
||||||
|
7. **Flashbots Integration**
|
||||||
|
- Not available on Arbitrum anyway
|
||||||
|
- May never be needed
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Realistic Expectations
|
||||||
|
|
||||||
|
### Week 1 Projections (Very Conservative):
|
||||||
|
```
|
||||||
|
Capital: 0.5 ETH
|
||||||
|
Trades: 3-5 per day
|
||||||
|
Success Rate: 50% (learning phase)
|
||||||
|
Avg Profit per Success: 0.005 ETH
|
||||||
|
Avg Gas per Trade: 0.002 ETH
|
||||||
|
|
||||||
|
Daily Net:
|
||||||
|
Success: 2 × 0.005 = 0.01 ETH
|
||||||
|
Failed Gas: 3 × 0.002 = 0.006 ETH
|
||||||
|
Net: 0.004 ETH per day
|
||||||
|
|
||||||
|
Weekly: 0.028 ETH (5.6% ROI)
|
||||||
|
```
|
||||||
|
|
||||||
|
### If This Works, Week 2-4:
|
||||||
|
```
|
||||||
|
Capital: 1.0 ETH (increase after validation)
|
||||||
|
Trades: 10 per day (more confidence)
|
||||||
|
Success Rate: 70% (optimization)
|
||||||
|
Avg Profit: 0.008 ETH
|
||||||
|
|
||||||
|
Daily Net: 0.035 ETH
|
||||||
|
Weekly: 0.245 ETH (24.5% ROI)
|
||||||
|
```
|
||||||
|
|
||||||
|
### If This REALLY Works (Month 2-3):
|
||||||
|
- Add more DEX protocols → more opportunities
|
||||||
|
- Add sequencer → better execution
|
||||||
|
- Scale capital to 5-10 ETH
|
||||||
|
- Target: 50-100% monthly ROI
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚨 Risk Management
|
||||||
|
|
||||||
|
### Hard Limits (Circuit Breakers):
|
||||||
|
```yaml
|
||||||
|
max_loss_per_day: 0.05 ETH
|
||||||
|
max_loss_per_week: 0.1 ETH
|
||||||
|
max_consecutive_failures: 3
|
||||||
|
max_trade_size: 0.1 ETH
|
||||||
|
max_trades_per_day: 20
|
||||||
|
max_gas_price: 0.5 gwei (Arbitrum is cheap)
|
||||||
|
|
||||||
|
circuit_breaker_cooldown: 1 hour
|
||||||
|
emergency_stop_loss: 0.2 ETH total
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manual Oversight:
|
||||||
|
- [ ] Check metrics every 6 hours (first week)
|
||||||
|
- [ ] Review all failed trades
|
||||||
|
- [ ] Adjust limits based on results
|
||||||
|
- [ ] Be ready to pause if needed
|
||||||
|
|
||||||
|
### Emergency Stop:
|
||||||
|
```bash
|
||||||
|
# Kill switch command
|
||||||
|
pkill -f mev-bot
|
||||||
|
|
||||||
|
# Or via API
|
||||||
|
curl -X POST http://localhost:8080/emergency-stop
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📦 Minimal Tech Stack
|
||||||
|
|
||||||
|
### Core Dependencies:
|
||||||
|
- **Language**: Go 1.21+
|
||||||
|
- **Ethereum Client**: go-ethereum (geth)
|
||||||
|
- **RPC Provider**: Alchemy or Infura (free tier is fine)
|
||||||
|
- **Database**: SQLite (simple, no postgres needed yet)
|
||||||
|
- **Logging**: Standard Go `log` package
|
||||||
|
- **Metrics**: Simple file-based logs
|
||||||
|
|
||||||
|
### Infrastructure:
|
||||||
|
- **Hosting**: Local machine or cheap VPS ($5/month)
|
||||||
|
- **Monitoring**: Tail logs + manual checks
|
||||||
|
- **Alerts**: None (you'll check manually)
|
||||||
|
|
||||||
|
### Later (if profitable):
|
||||||
|
- Upgrade to dedicated server
|
||||||
|
- Add Prometheus + Grafana
|
||||||
|
- Set up PagerDuty alerts
|
||||||
|
- Use PostgreSQL for analytics
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎬 Implementation Order (Detailed)
|
||||||
|
|
||||||
|
### Week 1: Days 1-2 (UniswapV2 Parser)
|
||||||
|
```bash
|
||||||
|
# Create feature branch
|
||||||
|
git checkout -b feature/v2/parsers/uniswap-v2-mvp
|
||||||
|
|
||||||
|
# Implement
|
||||||
|
touch pkg/parsers/uniswap_v2.go
|
||||||
|
touch pkg/parsers/uniswap_v2_test.go
|
||||||
|
|
||||||
|
# Focus areas:
|
||||||
|
1. Swap event signature: 0xd78ad95f...
|
||||||
|
2. ABI decoding: amount0In, amount1In, amount0Out, amount1Out
|
||||||
|
3. Token extraction from pool cache
|
||||||
|
4. Decimal scaling (critical!)
|
||||||
|
5. Validation (no zero addresses)
|
||||||
|
|
||||||
|
# Test with real data
|
||||||
|
# Example: https://arbiscan.io/tx/0x...
|
||||||
|
# Parse real Uniswap V2 swap on Arbitrum
|
||||||
|
|
||||||
|
# Achieve 100% coverage
|
||||||
|
go test ./pkg/parsers/... -coverprofile=coverage.out
|
||||||
|
go tool cover -html=coverage.out
|
||||||
|
```
|
||||||
|
|
||||||
|
### Week 1: Days 3-5 (UniswapV3 Parser)
|
||||||
|
```bash
|
||||||
|
git checkout -b feature/v2/parsers/uniswap-v3-mvp
|
||||||
|
|
||||||
|
# Key differences from V2:
|
||||||
|
1. Signed amounts (int256, not uint256)
|
||||||
|
2. sqrtPriceX96 (Q64.96 fixed point)
|
||||||
|
3. Tick and liquidity
|
||||||
|
4. Fee tiers (0.05%, 0.3%, 1%)
|
||||||
|
|
||||||
|
# Math helpers needed:
|
||||||
|
func sqrtPriceX96ToPrice(sqrtPriceX96 *big.Int) *big.Float
|
||||||
|
func calculateSwapOutput(pool *Pool, amountIn *big.Int) *big.Int
|
||||||
|
```
|
||||||
|
|
||||||
|
### Week 2: Days 1-4 (Arbitrage Detection)
|
||||||
|
```bash
|
||||||
|
git checkout -b feature/v2/arbitrage/basic-detection
|
||||||
|
|
||||||
|
# File structure:
|
||||||
|
pkg/arbitrage/
|
||||||
|
├── graph.go # Market graph
|
||||||
|
├── pathfinder.go # 2-hop BFS
|
||||||
|
├── profitability.go # Profit calculation
|
||||||
|
└── detector.go # Main detector
|
||||||
|
|
||||||
|
# Key algorithm:
|
||||||
|
For each token pair (A, B):
|
||||||
|
Find all pools: A → B
|
||||||
|
For each pool P1:
|
||||||
|
For each other pool P2:
|
||||||
|
If P1.price != P2.price:
|
||||||
|
Calculate arbitrage profit
|
||||||
|
If profit > minProfit:
|
||||||
|
Emit opportunity
|
||||||
|
```
|
||||||
|
|
||||||
|
### Week 3: Days 1-6 (Execution Engine)
|
||||||
|
```bash
|
||||||
|
git checkout -b feature/v2/execution/basic-executor
|
||||||
|
|
||||||
|
# Critical path:
|
||||||
|
1. Build swap calldata
|
||||||
|
2. Estimate gas
|
||||||
|
3. Calculate gas cost
|
||||||
|
4. Verify still profitable
|
||||||
|
5. Sign transaction
|
||||||
|
6. Send to network
|
||||||
|
7. Wait for receipt
|
||||||
|
8. Handle success/failure
|
||||||
|
|
||||||
|
# Test on testnet FIRST!
|
||||||
|
ARBITRUM_RPC=https://sepolia-rollup.arbitrum.io/rpc
|
||||||
|
```
|
||||||
|
|
||||||
|
### Week 4: Days 1-7 (Deployment)
|
||||||
|
```bash
|
||||||
|
# Shadow mode config:
|
||||||
|
SHADOW_MODE=true
|
||||||
|
LOG_LEVEL=debug
|
||||||
|
MIN_PROFIT=0.01
|
||||||
|
|
||||||
|
# Go live:
|
||||||
|
SHADOW_MODE=false
|
||||||
|
MAX_TRADE_SIZE=0.1
|
||||||
|
MAX_TRADES_PER_DAY=10
|
||||||
|
CIRCUIT_BREAKER_LOSS=0.05
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Decision Points
|
||||||
|
|
||||||
|
### After Week 2 (Arbitrage Detection):
|
||||||
|
**Question**: Are there enough profitable opportunities?
|
||||||
|
|
||||||
|
Run detection against historical data:
|
||||||
|
```bash
|
||||||
|
go run cmd/historical-analysis/main.go \
|
||||||
|
--start-block 150000000 \
|
||||||
|
--end-block 150001000 \
|
||||||
|
--min-profit 0.01
|
||||||
|
```
|
||||||
|
|
||||||
|
**If <5 opportunities per day**:
|
||||||
|
- ❌ Stop and reconsider strategy
|
||||||
|
- Maybe try different DEXs
|
||||||
|
- Maybe lower profit threshold
|
||||||
|
|
||||||
|
**If >10 opportunities per day**:
|
||||||
|
- ✅ Continue to execution phase
|
||||||
|
|
||||||
|
### After Week 3 (Testnet):
|
||||||
|
**Question**: Do trades execute successfully?
|
||||||
|
|
||||||
|
**If success rate <50%**:
|
||||||
|
- Debug execution logic
|
||||||
|
- Check gas estimation
|
||||||
|
- Verify slippage calculations
|
||||||
|
|
||||||
|
**If success rate >70%**:
|
||||||
|
- ✅ Proceed to mainnet
|
||||||
|
|
||||||
|
### After Week 4 Day 7 (First Week Live):
|
||||||
|
**Question**: Is this profitable?
|
||||||
|
|
||||||
|
**If net profit >0**:
|
||||||
|
- ✅ Continue for another week
|
||||||
|
- Consider scaling capital
|
||||||
|
|
||||||
|
**If net profit <0**:
|
||||||
|
- Analyze why:
|
||||||
|
- Competition too fierce?
|
||||||
|
- Gas too expensive?
|
||||||
|
- Calculations wrong?
|
||||||
|
- Decide: Fix and retry, or pivot?
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔄 What Happens After 4 Weeks?
|
||||||
|
|
||||||
|
### Scenario A: MVP is Profitable ✅
|
||||||
|
**Next Steps**:
|
||||||
|
1. Increase capital to 2-5 ETH
|
||||||
|
2. Add more DEX protocols (Curve, Balancer)
|
||||||
|
3. Implement 3-hop arbitrage
|
||||||
|
4. Add sequencer integration (for speed)
|
||||||
|
5. Build proper monitoring dashboard
|
||||||
|
6. Scale to 20-50% monthly ROI
|
||||||
|
|
||||||
|
**Timeline**: 4 more weeks to "Full MVP"
|
||||||
|
|
||||||
|
### Scenario B: MVP is Break-Even ⚖️
|
||||||
|
**Next Steps**:
|
||||||
|
1. Optimize for 2 more weeks
|
||||||
|
2. Add sequencer (may be the missing piece)
|
||||||
|
3. Reduce gas costs (batch execution)
|
||||||
|
4. If still break-even, reconsider
|
||||||
|
|
||||||
|
### Scenario C: MVP is Unprofitable ❌
|
||||||
|
**Analysis**:
|
||||||
|
- Is Arbitrum too competitive?
|
||||||
|
- Are opportunities too rare?
|
||||||
|
- Is our execution too slow?
|
||||||
|
|
||||||
|
**Options**:
|
||||||
|
1. Pivot to different chain (Polygon? Base?)
|
||||||
|
2. Try different MEV strategy (liquidations?)
|
||||||
|
3. Abandon and move on
|
||||||
|
|
||||||
|
**Key**: We only invested 4 weeks, not 10!
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Week-by-Week Checklist
|
||||||
|
|
||||||
|
### Week 1 Checklist:
|
||||||
|
- [ ] UniswapV2 parser complete (100% coverage)
|
||||||
|
- [ ] UniswapV3 parser complete (100% coverage)
|
||||||
|
- [ ] Pool cache populated with major pairs
|
||||||
|
- [ ] Can parse any swap on Arbitrum
|
||||||
|
- [ ] All tests passing
|
||||||
|
|
||||||
|
### Week 2 Checklist:
|
||||||
|
- [ ] Market graph built from pools
|
||||||
|
- [ ] 2-hop pathfinder working
|
||||||
|
- [ ] Profitability calculator accurate
|
||||||
|
- [ ] Finding >5 opportunities per day (historical)
|
||||||
|
- [ ] Detection latency <100ms
|
||||||
|
|
||||||
|
### Week 3 Checklist:
|
||||||
|
- [ ] Can execute swaps on testnet
|
||||||
|
- [ ] Gas estimation accurate
|
||||||
|
- [ ] Slippage protection working
|
||||||
|
- [ ] Circuit breaker tested
|
||||||
|
- [ ] Success rate >70% on testnet
|
||||||
|
|
||||||
|
### Week 4 Checklist:
|
||||||
|
- [ ] Metrics collection working
|
||||||
|
- [ ] Shadow mode validated (24 hours)
|
||||||
|
- [ ] First live trade successful
|
||||||
|
- [ ] Circuit breaker hasn't tripped
|
||||||
|
- [ ] Net profit >0 after 7 days
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 💡 Key Principles for Fast MVP
|
||||||
|
|
||||||
|
### 1. **Simple Over Perfect**
|
||||||
|
```
|
||||||
|
❌ Don't: Build a sophisticated gas optimization system
|
||||||
|
✅ Do: Just use current gas price + 10%
|
||||||
|
|
||||||
|
❌ Don't: Support 13 DEX protocols
|
||||||
|
✅ Do: Start with 2, add more if profitable
|
||||||
|
|
||||||
|
❌ Don't: Build a ML model for profit prediction
|
||||||
|
✅ Do: Simple math: output - input - gas
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. **Validate Assumptions Fast**
|
||||||
|
```
|
||||||
|
Week 1: Can we parse swaps correctly?
|
||||||
|
Week 2: Are there arbitrage opportunities?
|
||||||
|
Week 3: Can we execute trades?
|
||||||
|
Week 4: Is it profitable?
|
||||||
|
|
||||||
|
Each week answers ONE key question.
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. **Fail Fast, Pivot Faster**
|
||||||
|
```
|
||||||
|
If Week 2 shows no opportunities → STOP
|
||||||
|
If Week 3 shows trades fail → FIX or STOP
|
||||||
|
If Week 4 shows losses → PIVOT or STOP
|
||||||
|
|
||||||
|
Don't throw good time after bad.
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. **Real Data Over Assumptions**
|
||||||
|
```
|
||||||
|
Don't assume profitability → TEST IT
|
||||||
|
Don't assume opportunities exist → MEASURE THEM
|
||||||
|
Don't assume execution works → VERIFY IT
|
||||||
|
|
||||||
|
Shadow mode + small capital = real data
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Let's Start!
|
||||||
|
|
||||||
|
Your next immediate action:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Review this plan
|
||||||
|
# 2. If approved, start Week 1 Day 1:
|
||||||
|
|
||||||
|
git checkout -b feature/v2/parsers/uniswap-v2-mvp
|
||||||
|
|
||||||
|
# Create the parser
|
||||||
|
touch pkg/parsers/uniswap_v2.go
|
||||||
|
touch pkg/parsers/uniswap_v2_test.go
|
||||||
|
|
||||||
|
# Let's build! 🏗️
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**This plan gets you to profitability validation in 4 weeks with minimal capital risk. After that, you have REAL DATA to decide whether to scale, pivot, or stop.**
|
||||||
|
|
||||||
|
**Ready to start Week 1? 🎬**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚨 CRITICAL BUG DISCOVERED (Nov 2024)
|
||||||
|
|
||||||
|
### Problem: Bot Uses STALE Reserve Data
|
||||||
|
|
||||||
|
**Symptom**: Bot ran for 17+ hours, found ZERO arbitrage opportunities.
|
||||||
|
|
||||||
|
**Root Cause**: The arbitrage detector uses pool reserves fetched ONCE at startup and never refreshes them.
|
||||||
|
|
||||||
|
```go
|
||||||
|
// CURRENT (BROKEN) FLOW:
|
||||||
|
1. Bot starts → DiscoverMajorPools() fetches reserves ONCE
|
||||||
|
2. Bot scans every 30s → Uses SAME stale reserves
|
||||||
|
3. Real prices change constantly → Bot sees old data
|
||||||
|
4. Result: ZERO opportunities found (markets look balanced with old data)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why This Matters**:
|
||||||
|
- Pool reserves change with EVERY swap on-chain
|
||||||
|
- Arbitrage opportunities exist for seconds/milliseconds
|
||||||
|
- Using stale data = guaranteed to miss every opportunity
|
||||||
|
- 17 hours of runtime = 17 hours of wasted scanning
|
||||||
|
|
||||||
|
### Solution: Live Reserve Refresh
|
||||||
|
|
||||||
|
**MUST IMPLEMENT** before bot can find real opportunities:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// pkg/discovery/reserve_refresh.go
|
||||||
|
|
||||||
|
// RefreshReserves fetches latest reserves from chain for all cached pools
|
||||||
|
func (d *UniswapV2PoolDiscovery) RefreshReserves(ctx context.Context) error {
|
||||||
|
pools, _ := d.poolCache.GetByLiquidity(ctx, big.NewInt(0), 10000)
|
||||||
|
|
||||||
|
for _, pool := range pools {
|
||||||
|
// Call getReserves() on each pool contract
|
||||||
|
reserves, err := d.fetchReservesFromChain(ctx, pool.Address)
|
||||||
|
if err != nil {
|
||||||
|
continue // Skip failed pools
|
||||||
|
}
|
||||||
|
|
||||||
|
// Update cache with fresh reserves
|
||||||
|
pool.Reserve0 = reserves.Reserve0
|
||||||
|
pool.Reserve1 = reserves.Reserve1
|
||||||
|
pool.LastUpdated = time.Now()
|
||||||
|
|
||||||
|
d.poolCache.Update(ctx, pool)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Integration in main.go**:
|
||||||
|
```go
|
||||||
|
// BEFORE each scan, refresh reserves
|
||||||
|
for range ticker.C {
|
||||||
|
// NEW: Refresh reserves from chain
|
||||||
|
if err := poolDiscovery.RefreshReserves(ctx); err != nil {
|
||||||
|
logger.Error("failed to refresh reserves", "error", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Then scan for opportunities (with fresh data!)
|
||||||
|
opportunities, err := detector.ScanForOpportunities(ctx, blockNumber)
|
||||||
|
// ...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Implementation Priority: HIGHEST
|
||||||
|
|
||||||
|
| Task | Priority | Estimated Time |
|
||||||
|
|------|----------|----------------|
|
||||||
|
| Create `reserve_refresh.go` | P0 | 30 min |
|
||||||
|
| Add `RefreshReserves()` method | P0 | 30 min |
|
||||||
|
| Call refresh before each scan | P0 | 15 min |
|
||||||
|
| Test with live data | P0 | 30 min |
|
||||||
|
| **Total** | **CRITICAL** | **~2 hours** |
|
||||||
|
|
||||||
|
### Verification Test
|
||||||
|
|
||||||
|
After implementing reserve refresh:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run bot with verbose logging
|
||||||
|
./bin/mev-flashloan --min-profit 5 --interval 10s --verbose
|
||||||
|
|
||||||
|
# Expected output (after fix):
|
||||||
|
# - Reserve values should CHANGE between scans
|
||||||
|
# - Some opportunities should be found (even if small)
|
||||||
|
# - If still 0 opportunities after 1 hour, lower min-profit further
|
||||||
|
```
|
||||||
|
|
||||||
|
### Why This Wasn't Caught Earlier
|
||||||
|
|
||||||
|
1. Unit tests use mock data (don't need live reserves)
|
||||||
|
2. Integration tests check parsing, not live detection
|
||||||
|
3. Bot "worked" (no crashes) but with stale data
|
||||||
|
4. Need live mainnet testing with reserve updates
|
||||||
|
|
||||||
|
### Lesson Learned
|
||||||
|
|
||||||
|
**Always test with LIVE data before declaring production-ready.**
|
||||||
|
|
||||||
|
Static test data can hide critical bugs like stale caches.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Updated Implementation Checklist
|
||||||
|
|
||||||
|
### Immediate Fixes Required:
|
||||||
|
|
||||||
|
- [ ] **P0**: Implement `RefreshReserves()` in pool discovery
|
||||||
|
- [ ] **P0**: Call reserve refresh before each arbitrage scan
|
||||||
|
- [ ] **P0**: Add logging to show reserve changes between scans
|
||||||
|
- [ ] **P0**: Test on mainnet - verify opportunities are found
|
||||||
|
- [ ] **P1**: Add reserve age check (skip pools not updated in >60s)
|
||||||
|
- [ ] **P1**: Batch RPC calls for efficiency (multicall)
|
||||||
|
- [ ] **P2**: Add WebSocket subscription for real-time reserve updates
|
||||||
|
|
||||||
|
### Performance Considerations:
|
||||||
|
|
||||||
|
With 9 pools, refreshing reserves adds:
|
||||||
|
- ~9 RPC calls per scan (one per pool)
|
||||||
|
- ~500ms latency (with public RPC)
|
||||||
|
- Acceptable for 30s scan interval
|
||||||
|
|
||||||
|
For scaling to 100+ pools:
|
||||||
|
- Use multicall to batch reserve fetches
|
||||||
|
- Consider WebSocket subscriptions
|
||||||
|
- Target <100ms refresh time
|
||||||
Reference in New Issue
Block a user