mev-beta/orig/tests/calculation-validation/README.md

# Arbitrage Calculation Validation Framework

This testing framework extracts arbitrage opportunity data from MEV bot logs and validates profit calculations to ensure accuracy and detect bugs.

## Overview

The framework consists of three main components:

1. **Log Extraction** (`scripts/test-calculations.sh`) - Extracts opportunities, calculations, and threshold checks from logs
2. **Validation Library** (Go packages) - Parses and validates calculation correctness
3. **Replay Tool** (`replay.go`) - Replays calculations and generates validation reports

## Quick Start

### Extract and Analyze Logs

```bash
# Extract from latest container logs
./scripts/test-calculations.sh

# Extract from specific log file
./scripts/test-calculations.sh /path/to/logs.txt
```

### Run Validation Tests

```bash
# Run unit tests
go test ./tests/calculation-validation/... -v

# Or in container
podman exec mev-bot-dev-master-dev go test ./tests/calculation-validation/... -v
```

### Replay Calculations

```bash
# Replay and validate all calculations
go run ./tests/calculation-validation/replay.go

# With verbose output
go run ./tests/calculation-validation/replay.go -verbose

# Custom test directory
go run ./tests/calculation-validation/replay.go -dir tests/calculation-validation
```

## What It Validates

### ✅ Profit Threshold Checks

Validates that opportunities are correctly marked as executable based on profit thresholds:

```go
netProfit >= minThreshold  → executable = true
netProfit < minThreshold   → executable = false
```

**Critical Bug Fixed:** Before the fix, the code was comparing the integer part of ETH (834) to wei threshold (100000000000000), causing all opportunities to be rejected.

**After Fix:** Properly converts both values to same units (ETH) for comparison:
```go
834.210302 ETH >= 0.0001 ETH  → EXECUTABLE ✅
```

### ✅ ETH to Wei Conversions

Validates that ETH amounts are correctly converted to wei:

```go
1 ETH = 1,000,000,000,000,000,000 wei (1e18)
```

### ✅ V3 Swap Calculations

Validates Uniswap V3 swap calculations including:
- Fee deductions (0.05%, 0.3%, 1%)
- Output amounts
- Zero output detection

### ✅ Opportunity Consistency

Validates logical consistency:
- Executable opportunities have positive profit
- Executable opportunities meet minimum threshold
- Rejected opportunities have valid reject reasons

## Output Files

### Extraction Phase

```
tests/calculation-validation/
├── extracted/
│   ├── executable_opportunities.log  # Opportunities marked executable
│   ├── opportunity_details.log       # Full opportunity records
│   ├── v3_calculations.log           # V3 swap calculations
│   ├── threshold_checks.log          # Profit threshold validations
│   ├── rejections.log                # Rejected opportunities
│   ├── profit_values.txt             # Extracted profit amounts
│   └── test_data.json                # Structured test data
└── reports/
    └── validation_report_*.md        # Summary validation report
```

## Example Report

```
═══════════════════════════════════════════════════════════════════════
  Extraction Complete!
═══════════════════════════════════════════════════════════════════════

Summary Statistics:
  ✅ Executable Opportunities: 11
  📊 Total Opportunity Records: 23
  🔢 V3 Calculations: 600
  ✔️  Threshold Checks: 11
  ❌ Rejections: 23

Profit Statistics:
  💰 Total Profit: 4,053.69 ETH
  📈 Average Profit: 368.517 ETH
  🔝 Max Profit: 1,363.86 ETH
  📉 Min Profit: 0.003708 ETH
```

## Validation Results

After running the replay tool, you'll get:

```
━━━ Profit Threshold Checks ━━━
  Total: 11
  ✅ Valid: 11
  ❌ Invalid: 0
  Success Rate: 100.00%

━━━ Critical Bug Fix Validation ━━━
  Testing bug fix scenario:
    Net Profit: 834.210302 ETH
    Min Threshold: 0.0001 ETH
    ✅ BUG FIX VALIDATED: Correctly marked as executable

  Testing edge case (profit == threshold):
    Net Profit: 0.0001 ETH
    ✅ Edge case handled correctly

  Testing rejection case (profit < threshold):
    Net Profit: 0.00001 ETH
    ✅ Correctly rejected
```

## Test Coverage

The framework includes comprehensive unit tests:

### `validator_test.go`

- ✅ `TestValidateThresholdCheck` - Threshold validation logic
- ✅ `TestValidateThresholdComparison` - Critical bug fix validation
- ✅ `TestValidateV3Calculation` - Uniswap V3 math
- ✅ `TestCompareETHtoWei` - Unit conversion
- ✅ `TestCalculateStatistics` - Profit statistics
- ✅ `TestValidateOpportunity` - Full opportunity validation

Run tests:
```bash
go test ./tests/calculation-validation/... -v -cover
```

## Architecture

### Data Flow

```
┌─────────────────┐
│   Log Files     │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Log Parser     │  (parser.go)
│  - Regex        │
│  - Extraction   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Test Data      │  (types.go)
│  - Opportunities│
│  - Calculations │
│  - Thresholds   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   Validator     │  (validator.go)
│  - Comparison   │
│  - Validation   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Test Report    │
│  - Statistics   │
│  - Errors       │
│  - Warnings     │
└─────────────────┘
```

## Use Cases

### 1. Verify Bug Fixes

After applying a fix to profit calculations, run the framework to validate:

```bash
./scripts/test-calculations.sh
go run ./tests/calculation-validation/replay.go -verbose
```

### 2. Continuous Integration

Add to CI pipeline to catch calculation regressions:

```bash
#!/bin/bash
# Get logs from test run
./mev-bot start &
BOT_PID=$!
sleep 60
kill $BOT_PID

# Extract and validate
./scripts/test-calculations.sh logs/mev_bot.log
go test ./tests/calculation-validation/... || exit 1
```

### 3. Performance Analysis

Extract profit statistics over time:

```bash
for log in logs/archive/*.log; do
    ./scripts/test-calculations.sh "$log"
done

# Aggregate results
find tests/calculation-validation/reports -name "*.md" -exec cat {} \;
```

## Critical Metrics

The framework tracks key metrics to ensure bot health:

| Metric | Healthy Range | Warning | Critical |
|--------|---------------|---------|----------|
| Executable % | > 5% | 1-5% | < 1% |
| Threshold Pass Rate | 100% | 95-99% | < 95% |
| V3 Zero Outputs | < 10% | 10-25% | > 25% |
| Avg Profit | > 0.1 ETH | 0.01-0.1 ETH | < 0.01 ETH |

## Troubleshooting

### No Executable Opportunities Found

**Symptom:** `✓ Found 0 executable opportunities`

**Possible Causes:**
1. Log file is from before the bug fix was applied
2. Bot isn't detecting any profitable opportunities
3. All opportunities are being rejected

**Solution:**
```bash
# Get fresh logs
podman logs mev-bot-dev-master-dev 2>&1 > /tmp/fresh_logs.txt
./scripts/test-calculations.sh /tmp/fresh_logs.txt

# Check for threshold validation
grep "Profit threshold check" /tmp/fresh_logs.txt
```

### Tests Failing

**Symptom:** `go test` failures

**Solution:**
```bash
# Check if modules are initialized
cd tests/calculation-validation
go mod init mev-bot/tests/calculation-validation 2>/dev/null || true
go mod tidy

# Run with verbose output
go test -v
```

### Extraction Script Not Found

**Symptom:** `bash: ./scripts/test-calculations.sh: Permission denied`

**Solution:**
```bash
chmod +x ./scripts/test-calculations.sh
```

## Future Enhancements

- [ ] Real-time validation during bot operation
- [ ] Alerting on calculation anomalies
- [ ] Historical trend analysis
- [ ] Multi-DEX calculation validation
- [ ] Gas cost accuracy validation
- [ ] Slippage calculation validation

## Related Documentation

- [Bug Fix Solution](/docker/mev-beta/logs/BUG_FIX_SOLUTION_20251109.md) - Detailed bug analysis
- [Critical Bugs Found](/docker/mev-beta/logs/CRITICAL_BUGS_FOUND_20251109.md) - Initial bug discovery
- [Profit Calculator](/docker/mev-beta/pkg/profitcalc/profit_calc.go) - Source code

## License

Part of MEV Bot project - see main LICENSE file.