copper-tone-tech/mev-beta
1
0
Fork 0
Code Issues Pull Requests 7 Actions Packages Projects Releases Wiki Activity

docs: add comprehensive V2 requirements documentation

Browse Source 
- Created MODULARITY_REQUIREMENTS.md with component independence rules
- Created PROTOCOL_SUPPORT_REQUIREMENTS.md covering 13+ protocols
- Created TESTING_REQUIREMENTS.md enforcing 100% coverage
- Updated CLAUDE.md with strict feature/v2/* branch strategy

Requirements documented:
- Component modularity (standalone + integrated)
- 100% test coverage enforcement (non-negotiable)
- All DEX protocols (Uniswap V2/V3/V4, Curve, Balancer V2/V3, Kyber Classic/Elastic, Camelot V2/V3 with all Algebra variants)
- Proper decimal handling (critical for calculations)
- Pool caching with multi-index and O(1) mappings
- Market building with essential arbitrage detection values
- Price movement detection with decimal precision
- Transaction building (single and batch execution)
- Pool discovery and caching
- Comprehensive validation at all layers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Administrator
2025-11-10 14:26:56 +01:00
parent c54c569f30
commit f41adbe575
4 changed files with 2063 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

714
docs/planning/03_TESTING_REQUIREMENTS.md Normal file
View File

@@ -0,0 +1,714 @@
# V2 Testing Requirements
## Non-Negotiable Standards
**100% Test Coverage Required**
**100% Test Passage Required**
**Zero Tolerance for Failures**
## Testing Philosophy
Every line of code MUST be tested. Every edge case MUST be covered. Every failure MUST be fixed before merging.
## Coverage Requirements
### Code Coverage Targets
```bash
# Minimum coverage requirements (ENFORCED)
Overall Project: 100%
Per Package: 100%
Per File: 100%
Branch Coverage: 100%
```
### Coverage Verification
```bash
# Run coverage report
go test ./... -coverprofile=coverage.out -covermode=atomic
# View coverage by package
go tool cover -func=coverage.out
# MUST show 100% for every file
pkg/parsers/uniswap_v2/parser.go:100.0%
pkg/parsers/uniswap_v3/parser.go:100.0%
pkg/cache/pool_cache.go:100.0%
# ... etc
# Generate HTML report
go tool cover -html=coverage.out -o coverage.html
# CI/CD enforcement
if [ $(go tool cover -func=coverage.out | grep total | awk '{print $3}' | sed 's/%//') -lt 100 ]; then
echo "FAILED: Coverage below 100%"
exit 1
fi
```
## Test Types and Requirements
### 1. Unit Tests (100% Coverage Required)
Every function, every method, every code path MUST be tested.
```go
// Example: Complete unit test coverage
package uniswap_v2
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestParser_ParseSwapEvent_Success(t *testing.T) {
// Test successful parsing
}
func TestParser_ParseSwapEvent_ZeroAddress(t *testing.T) {
// Test zero address rejection
}
func TestParser_ParseSwapEvent_ZeroAmounts(t *testing.T) {
// Test zero amount rejection
}
func TestParser_ParseSwapEvent_MaxValues(t *testing.T) {
// Test maximum value handling
}
func TestParser_ParseSwapEvent_MinValues(t *testing.T) {
// Test minimum value handling
}
func TestParser_ParseSwapEvent_InvalidLog(t *testing.T) {
// Test invalid log handling
}
func TestParser_ParseSwapEvent_NilTransaction(t *testing.T) {
// Test nil transaction handling
}
func TestParser_ParseSwapEvent_Decimals(t *testing.T) {
// Test decimal precision handling
tests := []struct{
name string
token0Dec uint8
token1Dec uint8
amount0In *big.Int
expected *big.Int
}{
{"USDC-WETH", 6, 18, big.NewInt(1000000), ...},
{"DAI-USDC", 18, 6, big.NewInt(1000000000000000000), ...},
{"WBTC-WETH", 8, 18, big.NewInt(100000000), ...},
}
// Test all combinations
}
// MUST test ALL error paths
func TestParser_ParseSwapEvent_AllErrors(t *testing.T) {
tests := []struct{
name string
log *types.Log
wantErr string
}{
{"nil log", nil, "log is nil"},
{"wrong signature", wrongSigLog, "invalid signature"},
{"malformed data", malformedLog, "failed to decode"},
// ... every possible error
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := parser.ParseSwapEvent(tt.log)
require.Error(t, err)
assert.Contains(t, err.Error(), tt.wantErr)
})
}
}
```
### 2. Integration Tests (Real Data Required)
Test with REAL transactions from Arbiscan.
```go
// Example: Integration test with real Arbiscan data
func TestUniswapV2Parser_RealTransaction(t *testing.T) {
// Load real transaction from Arbiscan
// txHash: 0x1234...
realTx := loadTransaction("testdata/uniswap_v2_swap_0x1234.json")
realReceipt := loadReceipt("testdata/uniswap_v2_receipt_0x1234.json")
parser := NewParser(logger, cache)
events, err := parser.ParseReceipt(realReceipt, realTx)
require.NoError(t, err)
require.Len(t, events, 1)
event := events[0]
// Verify against known values from Arbiscan
assert.Equal(t, "0x...", event.Token0.Hex())
assert.Equal(t, "0x...", event.Token1.Hex())
assert.Equal(t, big.NewInt(1000000), event.Amount0In)
// ... verify all fields match Arbiscan
}
// MUST test multiple real transactions per protocol
func TestUniswapV2Parser_RealTransactions_Comprehensive(t *testing.T) {
testCases := []string{
"0x1234", // Basic swap
"0x5678", // Swap with ETH
"0x9abc", // Multi-hop swap
"0xdef0", // Large amount swap
"0x2468", // Small amount swap
}
for _, txHash := range testCases {
t.Run(txHash, func(t *testing.T) {
// Load and test real transaction
})
}
}
```
### 3. Edge Case Tests (Comprehensive)
Test EVERY edge case imaginable.
```go
func TestParser_EdgeCases(t *testing.T) {
tests := []struct{
name string
setupFunc func() *types.Log
expectError bool
errorMsg string
}{
// Boundary values
{"max uint256", setupMaxUint256, false, ""},
{"min uint256", setupMinUint256, false, ""},
{"zero amount", setupZeroAmount, true, "zero amount"},
// Token edge cases
{"same token0 and token1", setupSameTokens, true, "same token"},
{"token0 > token1 (not sorted)", setupUnsorted, false, ""},
// Decimal edge cases
{"0 decimals", setupZeroDecimals, true, "invalid decimals"},
{"19 decimals", setup19Decimals, true, "invalid decimals"},
{"different decimals", setupDifferentDecimals, false, ""},
// Amount edge cases
{"both in amounts zero", setupBothInZero, true, "zero amount"},
{"both out amounts zero", setupBothOutZero, true, "zero amount"},
{"negative amount (V3)", setupNegativeAmount, false, ""},
// Address edge cases
{"zero token0 address", setupZeroToken0, true, "zero address"},
{"zero token1 address", setupZeroToken1, true, "zero address"},
{"zero pool address", setupZeroPool, true, "zero address"},
{"zero sender", setupZeroSender, true, "zero address"},
// Data edge cases
{"empty log data", setupEmptyData, true, "empty data"},
{"truncated log data", setupTruncatedData, true, "invalid data"},
{"extra log data", setupExtraData, false, ""},
// Overflow cases
{"amount overflow", setupOverflow, false, ""},
{"price overflow", setupPriceOverflow, false, ""},
{"liquidity overflow", setupLiquidityOverflow, false, ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
log := tt.setupFunc()
event, err := parser.ParseSwapEvent(log)
if tt.expectError {
require.Error(t, err)
assert.Contains(t, err.Error(), tt.errorMsg)
assert.Nil(t, event)
} else {
require.NoError(t, err)
assert.NotNil(t, event)
}
})
}
}
```
### 4. Decimal Precision Tests (Critical)
MUST test decimal handling with EXACT precision.
```go
func TestDecimalPrecision(t *testing.T) {
tests := []struct{
name string
token0Decimals uint8
token1Decimals uint8
reserve0 *big.Int
reserve1 *big.Int
expectedPrice string // Exact decimal string
}{
{
name: "USDC/WETH (6/18)",
token0Decimals: 6,
token1Decimals: 18,
reserve0: big.NewInt(1000000), // 1 USDC
reserve1: big.NewInt(1e18), // 1 WETH
expectedPrice: "1.000000000000000000", // Exact
},
{
name: "WBTC/WETH (8/18)",
token0Decimals: 8,
token1Decimals: 18,
reserve0: big.NewInt(100000000), // 1 WBTC
reserve1: big.NewInt(15.5 * 1e18), // 15.5 WETH
expectedPrice: "15.500000000000000000", // Exact
},
{
name: "DAI/USDC (18/6)",
token0Decimals: 18,
token1Decimals: 6,
reserve0: big.NewInt(1e18), // 1 DAI
reserve1: big.NewInt(999000), // 0.999 USDC
expectedPrice: "0.999000000000000000", // Exact
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
price := CalculatePrice(
tt.reserve0,
tt.reserve1,
tt.token0Decimals,
tt.token1Decimals,
)
// MUST match exactly
assert.Equal(t, tt.expectedPrice, price.Text('f', 18))
})
}
}
// Test rounding errors
func TestDecimalRounding(t *testing.T) {
// Ensure no precision loss through multiple operations
initial := new(big.Float).SetPrec(256).SetFloat64(1.123456789012345678)
// Simulate multiple swaps
result := initial
for i := 0; i < 1000; i++ {
result = simulateSwap(result)
}
// Should maintain precision
diff := new(big.Float).Sub(initial, result)
tolerance := new(big.Float).SetFloat64(1e-15)
assert.True(t, diff.Cmp(tolerance) < 0, "precision loss detected")
}
```
### 5. Concurrency Tests (Thread Safety)
Test concurrent access to shared resources.
```go
func TestPoolCache_Concurrency(t *testing.T) {
cache := NewPoolCache()
// Concurrent writes
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
pool := createTestPool(id)
err := cache.Add(pool)
assert.NoError(t, err)
}(i)
}
// Concurrent reads
for i := 0; i < 100; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
pool, err := cache.Get(testAddress(id))
assert.NoError(t, err)
if pool != nil {
assert.NotNil(t, pool.Token0)
}
}(i)
}
wg.Wait()
// Verify no data corruption
for i := 0; i < 100; i++ {
pool, err := cache.Get(testAddress(i))
require.NoError(t, err)
if pool != nil {
ValidatePoolInfo(pool) // MUST pass validation
}
}
}
// Test race conditions
func TestPoolCache_RaceConditions(t *testing.T) {
// Run with: go test -race
cache := NewPoolCache()
done := make(chan bool)
// Writer goroutine
go func() {
for i := 0; i < 1000; i++ {
cache.Add(createTestPool(i))
}
done <- true
}()
// Reader goroutines
for i := 0; i < 10; i++ {
go func() {
for j := 0; j < 1000; j++ {
cache.Get(testAddress(j % 100))
}
done <- true
}()
}
// Wait for completion
for i := 0; i < 11; i++ {
<-done
}
}
```
### 6. Performance Tests (Benchmarks Required)
MUST benchmark all critical paths.
```go
func BenchmarkParser_ParseSwapEvent(b *testing.B) {
parser := setupParser()
log := createTestLog()
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, _ = parser.ParseSwapEvent(log)
}
}
// MUST meet performance targets
func BenchmarkParser_ParseSwapEvent_Target(b *testing.B) {
parser := setupParser()
log := createTestLog()
b.ResetTimer()
start := time.Now()
for i := 0; i < b.N; i++ {
_, _ = parser.ParseSwapEvent(log)
}
elapsed := time.Since(start)
avgTime := elapsed / time.Duration(b.N)
// MUST complete in < 1ms
if avgTime > time.Millisecond {
b.Fatalf("Too slow: %v per operation (target: < 1ms)", avgTime)
}
}
// Memory allocation benchmarks
func BenchmarkParser_ParseSwapEvent_Allocs(b *testing.B) {
parser := setupParser()
log := createTestLog()
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, _ = parser.ParseSwapEvent(log)
}
// Check allocation count
// Should minimize allocations
}
```
### 7. Protocol-Specific Tests
Each protocol MUST have comprehensive test suite.
```go
// Uniswap V2
func TestUniswapV2_AllEventTypes(t *testing.T) {
tests := []string{"Swap", "Mint", "Burn", "Sync"}
// Test all event types
}
// Uniswap V3
func TestUniswapV3_AllEventTypes(t *testing.T) {
tests := []string{"Swap", "Mint", "Burn", "Flash", "Collect"}
// Test all event types + V3 specific logic
}
// Curve
func TestCurve_AllPoolTypes(t *testing.T) {
tests := []string{"StableSwap", "CryptoSwap", "Tricrypto"}
// Test all Curve variants
}
// Camelot Algebra versions
func TestCamelot_AllAlgebraVersions(t *testing.T) {
tests := []string{"AlgebraV1", "AlgebraV1.9", "AlgebraIntegral", "AlgebraDirectional"}
// Test all Algebra variants with different fee structures
}
```
### 8. Integration Test Suite
Complete end-to-end testing.
```go
func TestE2E_FullPipeline(t *testing.T) {
// Setup full system
monitor := setupMonitor()
factory := setupParserFactory()
cache := setupCache()
validator := setupValidator()
arbDetector := setupArbitrageDetector()
// Feed real block data
block := loadRealBlock("testdata/block_12345.json")
// Process through full pipeline
txs := monitor.ParseBlock(block)
for _, tx := range txs {
events, err := factory.ParseTransaction(tx)
require.NoError(t, err)
for _, event := range events {
// Validate
err = validator.Validate(event)
require.NoError(t, err)
// Update cache
cache.UpdateFromEvent(event)
// Check for arbitrage
opps, err := arbDetector.FindOpportunities(event)
require.NoError(t, err)
for _, opp := range opps {
// Verify opportunity is valid
ValidateOpportunity(opp)
}
}
}
// Verify final state
assert.True(t, cache.Size() > 0)
assert.True(t, validator.GetStats().TotalValidated > 0)
}
```
## Test Data Requirements
### 1. Real Transaction Data
Store real Arbiscan data in `testdata/`:
```
testdata/
├── uniswap_v2/
│ ├── swap_0x1234.json
│ ├── mint_0x5678.json
│ └── burn_0x9abc.json
├── uniswap_v3/
│ ├── swap_0xdef0.json
│ ├── mint_0x2468.json
│ └── flash_0x1357.json
├── curve/
│ └── exchange_0xace0.json
└── camelot/
├── algebra_v1_swap_0xfff0.json
├── algebra_integral_swap_0xeee0.json
└── algebra_directional_swap_0xddd0.json
```
### 2. Test Data Generation
```go
// Generate comprehensive test data
func GenerateTestData() {
protocols := []Protocol{
ProtocolUniswapV2,
ProtocolUniswapV3,
ProtocolCurve,
// ... all protocols
}
for _, protocol := range protocols {
// Generate edge cases
generateZeroAddressCases(protocol)
generateZeroAmountCases(protocol)
generateMaxValueCases(protocol)
generateDecimalCases(protocol)
generateOverflowCases(protocol)
}
}
```
## CI/CD Integration
### Pre-commit Hooks
```bash
#!/bin/bash
# .git/hooks/pre-commit
# Run tests
go test ./... -v
if [ $? -ne 0 ]; then
echo "Tests failed. Commit rejected."
exit 1
fi
# Check coverage
coverage=$(go test ./... -coverprofile=coverage.out -covermode=atomic | \
go tool cover -func=coverage.out | \
grep total | \
awk '{print $3}' | \
sed 's/%//')
if (( $(echo "$coverage < 100" | bc -l) )); then
echo "Coverage is ${coverage}% (required: 100%). Commit rejected."
exit 1
fi
echo "All tests passed with 100% coverage. ✓"
```
### GitHub Actions
```yaml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.24'
- name: Run tests
run: go test ./... -v -race -coverprofile=coverage.out -covermode=atomic
- name: Check coverage
run: |
coverage=$(go tool cover -func=coverage.out | grep total | awk '{print $3}' | sed 's/%//')
if (( $(echo "$coverage < 100" | bc -l) )); then
echo "Coverage is ${coverage}% (required: 100%)"
exit 1
fi
echo "Coverage: ${coverage}% ✓"
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
files: ./coverage.out
fail_ci_if_error: true
```
## Test Execution Commands
```bash
# Run all tests with coverage
go test ./... -v -race -coverprofile=coverage.out -covermode=atomic
# Run specific package tests
go test ./pkg/parsers/uniswap_v2/... -v
# Run with race detector
go test ./... -race
# Run benchmarks
go test ./... -bench=. -benchmem
# Run only unit tests
go test ./... -short
# Run integration tests
go test ./... -run Integration
# Generate coverage report
go tool cover -html=coverage.out -o coverage.html
# Check coverage percentage
go tool cover -func=coverage.out | grep total
```
## Coverage Enforcement Rules
1. **No merging without 100% coverage**
- PR CI/CD must show 100% coverage
- Manual override NOT allowed
2. **No skipping tests**
- `t.Skip()` NOT allowed except for known external dependencies
- Must document reason for skip
3. **No ignoring failures**
- All test failures MUST be fixed
- Cannot merge with failing tests
4. **Coverage for all code paths**
- Every `if` statement tested (both branches)
- Every `switch` case tested
- Every error path tested
- Every success path tested
## Test Documentation
Each test file MUST include:
```go
/*
Package uniswap_v2_test provides comprehensive test coverage for UniswapV2Parser.
Test Coverage:
- ParseSwapEvent: 100%
- ParseMintEvent: 100%
- ParseBurnEvent: 100%
- ValidateEvent: 100%
Edge Cases Tested:
- Zero addresses (rejected)
- Zero amounts (rejected)
- Maximum values (accepted)
- Decimal precision (verified)
- Concurrent access (safe)
Real Data Tests:
- 5 real Arbiscan transactions
- All event types covered
- Multiple pool types tested
Performance:
- Parse time: < 1ms (verified)
- Memory: < 1KB per parse (verified)
*/
```
---
**ABSOLUTE REQUIREMENT**: 100% coverage, 100% passage, zero tolerance for failures.