mev-beta/docs/CODEBASE_QUICK_REFERENCE.md

# MEV Bot Codebase - Quick Reference Guide

## Project Snapshot
- **Language:** Go 1.25
- **Module:** github.com/fraktal/mev-beta
- **Total Files:** 362 Go files (253 in pkg/, 31 in internal/, 4 in cmd/)
- **Total LOC:** ~100,000+ (pkg/ only)
- **Purpose:** Production-grade MEV (Maximal Extractable Value) arbitrage bot for Arbitrum

---

## Directory Structure at a Glance

```
mev-beta/
├── cmd/mev-bot/           → Entry point (786 lines) + DEX integration
├── pkg/                   → 47 packages with core logic
│   ├── arbitrage/         → Arbitrage detection & execution (7000+ LOC)
│   ├── arbitrum/          → Arbitrum L2 integration (8000+ LOC)
│   ├── monitor/           → Sequencer monitoring (1351 LOC)
│   ├── scanner/           → Event processing with worker pools
│   ├── dex/               → Protocol implementations (V3, V2, Curve, Balancer)
│   ├── security/          → Key management & encryption (5000+ LOC)
│   ├── events/            → Event parsing (1806 LOC)
│   ├── transport/         → RPC provider management
│   ├── pools/             → Pool discovery (1065 LOC)
│   ├── math/              → Mathematical operations
│   └── [35 more]          → Specialized packages
├── internal/              → 14 private packages (config, logger, etc.)
├── bindings/              → Go-ethereum generated smart contract ABIs
├── config/                → YAML configuration files
├── tests/                 → Integration & unit tests
└── scripts/               → Build & utility scripts
```

---

## Core Components (Top 10 by Impact)

| # | Component | File | LOC | Role |
|---|-----------|------|-----|------|
| 1 | ArbitrageService | pkg/arbitrage/service.go | 1,995 | Main orchestration |
| 2 | L2Parser | pkg/arbitrum/l2_parser.go | 1,985 | Transaction parsing |
| 3 | KeyManager | pkg/security/keymanager.go | 1,841 | Key & signing |
| 4 | EventParser | pkg/events/parser.go | 1,806 | Event extraction |
| 5 | AuditAnalyzer | pkg/security/audit_analyzer.go | 1,646 | Audit logs |
| 6 | ArbitrageExecutor | pkg/arbitrage/executor.go | 1,641 | Execution |
| 7 | FlashSwapExecutor | pkg/arbitrage/flash_executor.go | 1,462 | Flash swaps |
| 8 | RateLimiter | pkg/security/rate_limiter.go | 1,411 | Rate limiting |
| 9 | ArbitrumMonitor | pkg/monitor/concurrent.go | 1,351 | Monitoring |
| 10 | PerformanceProfiler | pkg/security/performance_profiler.go | 1,316 | Metrics |

---

## Data Flow (Simple)

```
Arbitrum Blocks (WebSocket)
    ↓
ArbitrumMonitor → L2Parser → EventParser
    ↓
Scanner (with Worker Pool)
    ↓
MarketScanner → SwapAnalyzer → LiquidityAnalyzer
    ↓
ArbitrageService → MultiHopScanner → DetectionEngine
    ↓
ArbitrageExecutor → Flash Swap Contract → Arbitrum
    ↓
KeyManager (Signing) → Transaction Submission
```

---

## Key Architectural Patterns

### 1. **Layered Architecture**
- Smart Contract Layer (bindings/)
- Execution Layer (executor, flash_executor)
- Detection Layer (detection_engine, scanner)
- Event Collection Layer (monitor, parser)
- Infrastructure Layer (transport, pools, security)

### 2. **Worker Pool Pattern**
- Event processing with configurable workers
- Non-blocking channel communication
- Graceful shutdown with WaitGroup

### 3. **Rate Limiting**
- Per-endpoint rate limiters
- Global transaction limits
- Burst allowances
- Automatic backoff

### 4. **Multi-Pool Provider Architecture**
- Read-only pool (high RPS tolerance)
- Execution pool (reliable, limited RPS)
- Testing pool (isolated, safe)

### 5. **Configuration Hierarchy**
- Base: YAML files (config/arbitrum_production.yaml)
- Override: Environment variables (GO_ENV, etc.)
- Runtime: Per-endpoint config (providers.yaml)

---

## Entry Points & Main Functions

### Startup Sequence (main.go - 786 lines)

```
main()
  ↓ (20 debug checkpoints)
  ├─ Load config (arbitrum_production.yaml)
  ├─ Initialize logger
  ├─ Create provider manager (3-pool architecture)
  ├─ Create key manager (transaction signing)
  ├─ Initialize pool discovery (cache-only)
  ├─ Create market manager
  ├─ Create scanner (worker pools)
  ├─ Create arbitrage service
  ├─ Start monitor loop
  └─ Handle shutdown
```

### Real-Time Loop

**ArbitrumMonitor.Start()**
```go
for {
    block := subscribe.Next()
    for _, tx := range block.Transactions {
        parsed := l2Parser.ParseTransaction(tx)
        events := eventParser.ParseEvents(receipt)
        scanner.ProcessEvent(events)
    }
}
```

---

## DEX Protocols Supported

| Protocol | File | Status |
|----------|------|--------|
| Uniswap V3 | pkg/dex/uniswap_v3.go | Active |
| Uniswap V2 | pkg/dex/ | Active |
| SushiSwap | pkg/dex/sushiswap.go | Active |
| Curve | pkg/dex/curve.go | Active |
| Balancer | pkg/dex/balancer.go | Active |

---

## Configuration Examples

### config/arbitrum_production.yaml
```yaml
tokens:
  weth: {address: 0x82af..., decimals: 18}
  usdc: {address: 0xaf88..., decimals: 6}
  [20+ tokens defined]

arbitrage:
  min_profit_threshold: 0.1%
  max_slippage: 0.5%
  max_gas_price: 50 gwei
```

### config/providers.yaml
```yaml
read_only_pool:
  endpoints:
    - url: "https://..."
      max_rps: 50
execution_pool:
  endpoints:
    - url: "https://..."
      max_rps: 20
```

### Environment Variables (Required)
```bash
GO_ENV=production
MEV_BOT_ENCRYPTION_KEY=<32-byte hex key>
MEV_BOT_KEYSTORE_PATH=keystore
ARBITRUM_RPC_ENDPOINT=wss://...
ARBITRUM_WS_ENDPOINT=wss://...
```

---

## Common Build Commands

```bash
# Build the binary
make build
cd bin && ./mev-bot start

# Run with specific environment
GO_ENV=production ./bin/mev-bot start

# Run tests
make test                 # All tests
make test-integration     # Integration only
make test-coverage        # With coverage

# Security & Quality
make lint                 # Run linter
make security-scan        # Security audit
```

---

## Type Definitions (Most Important)

### ArbitrageOpportunity
```go
type ArbitrageOpportunity struct {
    ID string
    Path []string              // Token path
    Pools []string             // Pool addresses
    AmountIn *big.Int          // Input amount (wei)
    Profit *big.Int            // Gross profit
    NetProfit *big.Int         // After gas
    GasEstimate *big.Int       // Gas cost
    ROI float64                // Return %
    Confidence float64         // 0-1 score
    TokenIn, TokenOut common.Address
    Timestamp int64
    DetectedAt, ExpiresAt time.Time
}
```

### ArbitrageService
```go
type ArbitrageService struct {
    client *ethclient.Client
    keyManager *security.KeyManager
    executor *ArbitrageExecutor
    detectionEngine *ArbitrageDetectionEngine
    flashExecutor *FlashSwapExecutor
    poolDiscovery *pools.PoolDiscovery
    marketManager *market.MarketManager
}
```

### ArbitrumMonitor
```go
type ArbitrumMonitor struct {
    client *ethclient.Client
    l2Parser *arbitrum.ArbitrumL2Parser
    scanner *scanner.Scanner
    pipeline *market.Pipeline
    eventParser *events.EventParser
}
```

---

## Execution Flow (Simple)

```
Opportunity Detected
    ↓
Validate (amount, slippage, confidence)
    ↓
Get execution client
    ↓
Prepare transaction:
    ├─ Select path
    ├─ Estimate gas
    ├─ Calculate slippage
    ├─ Verify profitability
    └─ Sign with KeyManager
    ↓
Submit to Arbitrum
    ↓
Wait for receipt (60s max)
    ↓
Log results & update stats
```

---

## Known Issues & Workarounds

| Issue | Cause | Workaround |
|-------|-------|-----------|
| Startup hang | Pool discovery RPC calls timeout | Skip discovery, use cache |
| Security manager off | Debugging hang issue | Commented out in main.go |
| Multicall parsing | Complex transaction structures | AbiDecoder with fallbacks |

---

## Database & Persistence

### Current Implementation
- In-memory caches for pools and opportunities
- Optional PostgreSQL/SQLite for historical data
- JSON persistence for pool discovery cache

### Key Data
- Pool cache: ~314 pools loaded at startup
- Opportunity history: In-memory window
- Execution history: Configurable retention

---

## Monitoring & Logging

### Log Locations
```
logs/
├── mev_bot.log              # Main application
├── mev_bot_errors.log       # Errors only
├── mev_bot_performance.log  # Metrics
├── analytics/               # Real-time analysis
├── health/                  # Health reports
└── archives/                # Compressed backups
```

### Metrics Available
- Health score: 97.97/100
- Error rate: 2.03%
- MEV opportunities: Count
- Processing latency: ~10-50ms
- Memory usage: ~200-500MB

---

## Testing

### Test Organization
```
tests/integration/       # Fork testing, real scenarios
tests/cache/            # Cache functionality
tests/contracts/        # Contract interactions
pkg/*_test.go          # Unit tests (colocated)
```

### Running Tests
```bash
make test              # All
make test-integration  # Integration only
make test-coverage     # With coverage report
go test -v ./...       # Verbose output
```

---

## Key Files to Understand First

**In Order of Importance:**
1. `cmd/mev-bot/main.go` - Understand startup flow
2. `pkg/arbitrage/service.go` - Core orchestration
3. `pkg/monitor/concurrent.go` - How monitoring works
4. `pkg/scanner/concurrent.go` - Event processing
5. `pkg/arbitrage/executor.go` - Execution logic
6. `pkg/arbitrum/l2_parser.go` - Transaction parsing
7. `internal/config/config.go` - Configuration system
8. `pkg/security/keymanager.go` - Key management

---

## Performance Notes

- **Throughput:** ~100+ events/sec (with 4-8 workers)
- **Latency:** Detection <50ms, execution <5s
- **Rate Limits:** Read 50 RPS, Execute 20 RPS
- **Memory:** ~200-500MB baseline
- **Disk:** Logs rotate, archives compress

---

## Security Notes

- Keys encrypted in keystore
- All RPC endpoints validated
- Rate limiting prevents DoS
- Audit logging enabled
- Anomaly detection available
- Key rotation supported (30-day default)

---

## See Also

- Full analysis: `docs/CODEBASE_EXPLORATION_COMPLETE.md` (1140 lines)
- Configuration guide: `docs/` directory
- Smart contracts: `contracts/` (Solidity source)
- Examples: `examples/` directory
- Tests: `tests/` directory

---

**Last Updated:** November 1, 2025  
**Status:** Feature/production-profit-optimization branch  
**Maintainer:** MEV Bot Team