mev-beta/docs/planning/07_TASK_BREAKDOWN.md

MEV Bot V2 - Detailed Task Breakdown

How to Use This Document

Each task is broken down into the smallest possible actionable unit. Tasks are:

• Atomic: Can be completed in one sitting (< 2 hours)
• Testable: Has clear success criteria
• Independent: Minimal dependencies on other incomplete tasks

Task Format:

### [PHASE-ID-NUMBER] Task Title
**Est**: X hours
**Dependencies**: [ID, ID, ...]
**Success Criteria**:
- Criterion 1
- Criterion 2

**Steps**:
1. Step 1
2. Step 2

---

Phase 1: Foundation

[P1-001] Create V2 Directory Structure

Est: 0.5 hours Dependencies: None Success Criteria:

• All directories created as per master plan
• README.md in each major directory
• .gitkeep files where needed

Steps:

1. Create cmd/mev-bot/ directory
2. Create pkg/parsers/ directory
3. Create pkg/validation/ directory
4. Create pkg/cache/ directory
5. Create pkg/discovery/ directory
6. Create pkg/monitor/ directory
7. Create pkg/events/ directory
8. Create pkg/arbitrage/ directory
9. Create pkg/observability/ directory
10. Create tests/unit/ directory
11. Create tests/integration/ directory
12. Create tests/e2e/ directory
13. Add README.md to each directory

---

[P1-002] Define Event Types

Est: 1 hour Dependencies: [P1-001] Success Criteria:

• Event type constants defined
• Event struct with all fields
• String methods for debugging
• Unit tests for type conversions

Steps:

1. Create pkg/events/types.go
2. Define EventType constants (Swap, Mint, Burn, etc.)
3. Define Event struct with fields:
   • Type
   • Protocol
   • PoolAddress
   • Token0, Token1
   • Amount0In, Amount0Out
   • Amount1In, Amount1Out
   • Sender, Recipient
   • TxHash, BlockNumber, LogIndex
   • Timestamp
4. Add String() method to EventType
5. Add validation helper methods
6. Write unit tests

---

[P1-003] Define Parser Interface

Est: 0.5 hours Dependencies: [P1-002] Success Criteria:

• Interface defined with all methods
• Documentation for each method
• Mock implementation for testing

Steps:

1. Create pkg/parsers/interface.go
2. Define Parser interface:

   type Parser interface {
       ParseLog(log *types.Log, tx *types.Transaction) (*Event, error)
       ParseReceipt(receipt *types.Receipt, tx *types.Transaction) ([]*Event, error)
       SupportedProtocols() []Protocol
       ValidateEvent(event *Event) error
   }
   

3. Add comprehensive documentation
4. Create pkg/parsers/mock_parser_test.go
5. Implement mock for unit tests

---

[P1-004] Define PoolCache Interface

Est: 0.5 hours Dependencies: None Success Criteria:

• Interface defined
• All lookup methods specified
• Documentation complete

Steps:

1. Create pkg/cache/interface.go
2. Define PoolCache interface:

   type PoolCache interface {
       Get(address common.Address) (*PoolInfo, error)
       GetByTokenPair(token0, token1 common.Address) ([]*PoolInfo, error)
       GetByProtocol(protocol Protocol) ([]*PoolInfo, error)
       GetTopByLiquidity(limit int) ([]*PoolInfo, error)
       Add(pool *PoolInfo) error
       Update(address common.Address, pool *PoolInfo) error
       Remove(address common.Address) error
       Size() int
   }
   

3. Document each method
4. Create mock implementation

---

[P1-005] Define Validator Interface

Est: 0.5 hours Dependencies: [P1-002] Success Criteria:

• Interface defined
• Rule types specified
• Documentation complete

Steps:

1. Create pkg/validation/interface.go
2. Define Validator interface:

   type Validator interface {
       Validate(event *Event) ValidationResult
       AddRule(rule ValidationRule)
       RemoveRule(name string)
       GetMetrics() ValidationMetrics
   }
   

3. Define ValidationRule interface
4. Define ValidationResult struct
5. Document everything

---

[P1-006] Setup Logging Infrastructure

Est: 1.5 hours Dependencies: None Success Criteria:

• Structured logger wrapping slog
• Log levels (Debug, Info, Warn, Error)
• Context support
• JSON and text formatters
• Unit tests

Steps:

1. Create pkg/observability/logger.go
2. Define Logger struct wrapping *slog.Logger
3. Add methods: Debug(), Info(), Warn(), Error(), Fatal()
4. Add WithContext() for context support
5. Add WithFields() for structured fields
6. Create constructor NewLogger(level, format string)
7. Write unit tests for each log level
8. Write test for JSON vs text formatting

---

[P1-007] Setup Metrics Infrastructure

Est: 2 hours Dependencies: None Success Criteria:

• Prometheus metrics integration
• Counter, Gauge, Histogram, Summary types
• Metric registration
• HTTP endpoint for scraping
• Unit tests

Steps:

1. Create pkg/observability/metrics.go
2. Add prometheus dependency: go get github.com/prometheus/client_golang/prometheus
3. Create MetricsRegistry struct
4. Implement Counter registration and increment
5. Implement Gauge registration and set/inc/dec
6. Implement Histogram registration and observe
7. Implement Summary registration and observe
8. Create HTTP handler for /metrics endpoint
9. Write unit tests for each metric type
10. Write integration test for scraping

---

[P1-008] Define PoolInfo Struct

Est: 0.5 hours Dependencies: None Success Criteria:

• Complete struct definition
• All necessary fields
• Validation methods
• Unit tests

Steps:

1. Create pkg/cache/pool_info.go
2. Define PoolInfo struct:

   type PoolInfo struct {
       Address        common.Address
       Protocol       Protocol
       PoolType       PoolType
       Factory        common.Address
       Token0         common.Address
       Token1         common.Address
       Token0Decimals uint8
       Token1Decimals uint8
       Fee            uint32
       TickSpacing    uint32
       Liquidity      *big.Int
       CreatedBlock   uint64
       CreatedTx      common.Hash
       LastUpdated    time.Time
   }
   

3. Add Validate() method
4. Add IsValid() helper
5. Write unit tests

---

[P1-009] Setup Connection Manager

Est: 2 hours Dependencies: [P1-006] Success Criteria:

• RPC connection pooling
• Automatic reconnection
• Health checks
• Failover support
• Unit tests

Steps:

1. Create pkg/monitor/connection.go
2. Define ConnectionManager struct
3. Implement connection pool (max 5 connections)
4. Add GetConnection() with round-robin
5. Add HealthCheck() for connection testing
6. Implement automatic reconnection on failure
7. Add failover to backup endpoints
8. Add circuit breaker pattern
9. Write unit tests for each failure scenario
10. Write integration test with real endpoint

---

[P1-010] Create Base Test Framework

Est: 1 hour Dependencies: None Success Criteria:

• Test utilities created
• Mock data generators
• Assertion helpers
• Test fixtures

Steps:

1. Create tests/testutil/helpers.go
2. Add CreateMockLog() function
3. Add CreateMockTransaction() function
4. Add CreateMockReceipt() function
5. Add CreateMockEvent() function
6. Add AssertEventEquals() helper
7. Add AssertNoError() helper
8. Create tests/fixtures/ directory
9. Add sample log JSON files
10. Add sample transaction JSON files
11. Write tests for test helpers

---

Phase 2: Parser Refactor

[P2-001] Create Parser Factory

Est: 1 hour Dependencies: [P1-003] Success Criteria:

• Factory pattern implementation
• Parser registration
• Parser selection by protocol
• Unit tests

Steps:

1. Create pkg/parsers/factory.go
2. Define ParserFactory struct
3. Add Register(protocol Protocol, parser Parser) method
4. Add GetParser(protocol Protocol) (Parser, error) method
5. Add GetParserForLog(log *types.Log) (Parser, error) method
6. Create global registry instance
7. Write unit tests for registration
8. Write unit tests for retrieval
9. Write test for unknown protocol handling

---

[P2-002] Implement UniswapV2 Parser - Base Structure

Est: 1 hour Dependencies: [P1-003, P2-001] Success Criteria:

• Struct defined
• Constructor implemented
• Interface methods stubbed
• Compiles without errors

Steps:

1. Create pkg/parsers/uniswap_v2.go
2. Define UniswapV2Parser struct:

   type UniswapV2Parser struct {
       logger *Logger
       cache  PoolCache
   }
   

3. Implement NewUniswapV2Parser(logger, cache) constructor
4. Stub ParseLog(log, tx) - return nil, nil
5. Stub ParseReceipt(receipt, tx) - return nil, nil
6. Implement SupportedProtocols() - return []Protocol{ProtocolUniswapV2}
7. Stub ValidateEvent(event) - return nil
8. Register in factory init function
9. Write basic constructor test

---

[P2-003] UniswapV2 Parser - Parse Swap Event

Est: 2 hours Dependencies: [P2-002] Success Criteria:

• Swap event signature recognized
• Amounts decoded correctly
• Event struct populated
• Unit tests with real log data

Steps:

1. Define Swap event signature constant
2. Add signature recognition in ParseLog()
3. Implement ABI decoding for Swap event:
   • sender (address)
   • amount0In (uint256)
   • amount1In (uint256)
   • amount0Out (uint256)
   • amount1Out (uint256)
   • to (address)
4. Create Event struct from decoded data
5. Set event type to EventTypeSwap
6. Set protocol to ProtocolUniswapV2
7. Extract pool address from log.Address
8. Write unit test with sample Swap log
9. Write test with zero amounts (should handle)
10. Write test with invalid log (should error)

---

[P2-004] UniswapV2 Parser - Extract Token Addresses

Est: 1.5 hours Dependencies: [P2-003] Success Criteria:

• Tokens extracted from pool cache
• Fallback to RPC if cache miss
• Zero addresses handled
• Unit tests

Steps:

1. Add getPoolTokens(poolAddress) helper method
2. Check pool cache first: cache.Get(poolAddress)
3. If found, return cached Token0 and Token1
4. If not found, query RPC (use multicall):
   • Call pool.token0()
   • Call pool.token1()
5. Handle RPC errors gracefully
6. If both fail, return zero addresses with error
7. Update event with token addresses
8. Write unit test with cache hit
9. Write unit test with cache miss + RPC success
10. Write unit test with cache miss + RPC failure
11. Write test for zero address detection

---

[P2-005] UniswapV2 Parser - Validate Swap Event

Est: 1 hour Dependencies: [P2-004] Success Criteria:

• All validation rules implemented
• Invalid events rejected
• Unit tests for each rule

Steps:

1. Implement ValidateEvent(event *Event) error method
2. Check: Token0 != zero address
3. Check: Token1 != zero address
4. Check: PoolAddress != zero address
5. Check: NOT (Amount0In == 0 AND Amount0Out == 0)
6. Check: NOT (Amount1In == 0 AND Amount1Out == 0)
7. Check: Sender != zero address
8. Check: Recipient != zero address
9. Return detailed error for each failure
10. Write unit test for each validation rule
11. Write test for valid event (should pass)

---

[P2-006] UniswapV2 Parser - Parse Mint Event

Est: 1.5 hours Dependencies: [P2-003] Success Criteria:

• Mint event signature recognized
• Liquidity amounts decoded
• Event struct populated
• Unit tests

Steps:

1. Define Mint event signature constant
2. Add signature recognition in ParseLog()
3. Implement ABI decoding for Mint event:
   • sender (address)
   • amount0 (uint256)
   • amount1 (uint256)
4. Create Event struct from decoded data
5. Set event type to EventTypeMint
6. Extract token addresses using existing helper
7. Write unit test with sample Mint log
8. Write validation test
9. Integration test with Swap parsing

---

[P2-007] UniswapV2 Parser - Parse Burn Event

Est: 1.5 hours Dependencies: [P2-006] Success Criteria:

• Burn event signature recognized
• Withdrawal amounts decoded
• Event struct populated
• Unit tests

Steps:

1. Define Burn event signature constant
2. Add signature recognition in ParseLog()
3. Implement ABI decoding for Burn event:
   • sender (address)
   • amount0 (uint256)
   • amount1 (uint256)
   • to (address)
4. Create Event struct from decoded data
5. Set event type to EventTypeBurn
6. Extract token addresses
7. Write unit test with sample Burn log
8. Write validation test

---

[P2-008] UniswapV2 Parser - ParseReceipt Implementation

Est: 1 hour Dependencies: [P2-007] Success Criteria:

• All logs in receipt parsed
• Multiple events handled
• Transaction context available
• Unit tests

Steps:

1. Implement ParseReceipt(receipt, tx) method
2. Loop through receipt.Logs
3. Call ParseLog(log, tx) for each
4. Collect all non-nil events
5. Return event slice
6. Handle errors gracefully (log, don't fail)
7. Write test with single event
8. Write test with multiple events
9. Write test with mixed event types
10. Write test with invalid logs

---

[P2-009] UniswapV2 Parser - Integration Tests

Est: 1.5 hours Dependencies: [P2-008] Success Criteria:

• Real transaction tested
• All event types covered
• Edge cases tested
• Arbiscan data validated

Steps:

1. Find real UniswapV2 transaction on Arbiscan
2. Download transaction JSON
3. Download receipt JSON
4. Create integration test fixture
5. Parse transaction using parser
6. Assert correct number of events
7. Assert event details match Arbiscan
8. Test with multicall transaction
9. Test with failed transaction
10. Verify no zero addresses in output

---

[P2-010] Implement UniswapV3 Parser - Base Structure

Est: 1 hour Dependencies: [P1-003, P2-001] Success Criteria:

• Struct defined
• Constructor implemented
• Interface methods stubbed

Steps: (Similar to P2-002 but for UniswapV3)

1. Create pkg/parsers/uniswap_v3.go
2. Define UniswapV3Parser struct
3. Implement constructor
4. Stub interface methods
5. Register in factory
6. Write basic test

---

[P2-011] UniswapV3 Parser - Parse Swap Event

Est: 2.5 hours Dependencies: [P2-010] Success Criteria:

• V3 Swap event decoded
• Tick and sqrtPrice captured
• Amounts calculated correctly
• Unit tests

Steps:

1. Define V3 Swap event signature
2. Implement ABI decoding for V3 Swap:
   • sender (address)
   • recipient (address)
   • amount0 (int256) - note: signed!
   • amount1 (int256) - note: signed!
   • sqrtPriceX96 (uint160)
   • liquidity (uint128)
   • tick (int24)
3. Convert signed amounts to In/Out format
4. Handle negative amounts (indicate direction)
5. Extract token addresses from pool cache
6. Create Event struct
7. Write unit test with sample V3 Swap log
8. Write test for each swap direction
9. Write test for zero liquidity edge case
10. Validate against real Arbiscan transaction

---

[P2-012] UniswapV3 Parser - Extract Tokens from Transaction

Est: 2 hours Dependencies: [P2-011] Success Criteria:

• Tokens extracted from calldata
• Router calls decoded
• Multi-hop swaps handled
• Unit tests

Steps:

1. Add extractTokensFromCalldata(tx) helper
2. Recognize SwapRouter signatures:
   • exactInputSingle
   • exactOutputSingle
   • exactInput (multi-hop)
   • exactOutput (multi-hop)
3. Decode token path from params
4. Match pool address to correct token pair
5. Handle multi-hop by splitting path
6. Fallback to pool cache if calldata unavailable
7. Write test for single-hop swap
8. Write test for multi-hop swap
9. Write test for failed decoding
10. Integration test with real transaction

---

[P2-013] UniswapV3 Parser - Parse Mint Event

Est: 1.5 hours Dependencies: [P2-011] Success Criteria:

• V3 Mint event decoded
• Liquidity and tick range captured
• Unit tests

Steps:

1. Define V3 Mint event signature
2. Implement ABI decoding:
   • sender (address)
   • owner (address)
   • tickLower (int24)
   • tickUpper (int24)
   • amount (uint128)
   • amount0 (uint256)
   • amount1 (uint256)
3. Create Event struct
4. Write unit tests
5. Validate amounts match

---

[P2-014] UniswapV3 Parser - Parse Burn Event

Est: 1.5 hours Dependencies: [P2-013] Success Criteria:

• V3 Burn event decoded
• Liquidity removal captured
• Unit tests

Steps: (Similar structure to P2-013 but for Burn)

1. Define V3 Burn signature
2. Implement ABI decoding
3. Create Event struct
4. Write unit tests

---

[P2-015] UniswapV3 Parser - Integration Tests

Est: 2 hours Dependencies: [P2-014] Success Criteria:

• Real V3 transactions tested
• Multi-hop swaps validated
• No zero addresses

Steps:

1. Find real V3 swap on Arbiscan
2. Find real V3 multi-hop swap
3. Download fixtures
4. Parse and validate
5. Assert zero-free outputs
6. Performance benchmark

---

[P2-016-025] Implement SushiSwap, Camelot, Curve Parsers

Est: 8-10 hours each Dependencies: [P2-001, P2-009, P2-015] Success Criteria:

• Each parser follows same pattern as Uniswap
• All event types supported
• Comprehensive tests
• Zero address guarantees

Steps: (Replicate P2-002 through P2-009 for each exchange)

• SushiSwap is V2-like, reuse much of UniswapV2 logic
• Camelot has V2 and V3 variants
• Curve has unique stable swap math

---

Phase 3: Cache System

[P3-001] Implement Address Index

Est: 2 hours Dependencies: [P1-004, P1-008] Success Criteria:

• O(1) lookup by address
• Thread-safe
• Add/Update/Remove operations
• Unit tests

Steps:

1. Create pkg/cache/index_by_address.go
2. Define AddressIndex struct:

   type AddressIndex struct {
       pools map[common.Address]*PoolInfo
       mu    sync.RWMutex
   }
   

3. Implement Get(address) with RLock
4. Implement Add(pool) with Lock
5. Implement Update(address, pool) with Lock
6. Implement Remove(address) with Lock
7. Implement Size() method
8. Write unit test for each operation
9. Write concurrency test (multiple goroutines)
10. Benchmark lookup performance

---

[P3-002] Implement Token-Pair Index

Est: 2.5 hours Dependencies: [P3-001] Success Criteria:

• O(1) lookup by token pair
• Handles token order (A-B == B-A)
• Thread-safe
• Unit tests

Steps:

1. Create pkg/cache/index_by_tokens.go
2. Define TokenPairIndex struct:

   type TokenPairIndex struct {
       pools map[string][]*PoolInfo  // key: "token0-token1" sorted
       mu    sync.RWMutex
   }
   

3. Add makeKey(token0, token1) helper (sorts addresses)
4. Implement Get(token0, token1) with RLock
5. Implement Add(pool) - add to both token orders
6. Implement Update(pool)
7. Implement Remove(pool)
8. Write unit test for token order independence
9. Write test for multiple pools per pair
10. Benchmark lookup performance

---

[P3-003] Implement Liquidity Index

Est: 2 hours Dependencies: [P3-001] Success Criteria:

• Pools sorted by liquidity
• Fast top-N retrieval
• Auto-reorder on updates
• Unit tests

Steps:

1. Create pkg/cache/index_by_liquidity.go
2. Define LiquidityIndex struct:

   type LiquidityIndex struct {
       pools []*PoolInfo  // sorted by liquidity desc
       mu    sync.RWMutex
   }
   

3. Implement GetTop(limit) with RLock
4. Implement Add(pool) - insert in sorted position
5. Implement Update(pool) - remove and re-insert
6. Implement Remove(pool) - binary search and remove
7. Use binary search for efficient insertion
8. Write unit test for sorting
9. Write test for top-N retrieval
10. Benchmark insertion and lookup

---

[P3-004] Implement Protocol Index

Est: 1.5 hours Dependencies: [P3-001] Success Criteria:

• Fast lookup by protocol
• Multiple protocols per pool
• Thread-safe
• Unit tests

Steps:

1. Create pkg/cache/index_by_protocol.go
2. Define ProtocolIndex struct:

   type ProtocolIndex struct {
       pools map[Protocol][]*PoolInfo
       mu    sync.RWMutex
   }
   

3. Implement Get(protocol) with RLock
4. Implement Add(pool) with Lock
5. Implement Update(pool) with Lock
6. Implement Remove(pool) with Lock
7. Write unit test for each protocol
8. Write test for unknown protocol
9. Benchmark lookup

---

[P3-005] Implement Multi-Index PoolCache

Est: 3 hours Dependencies: [P3-001, P3-002, P3-003, P3-004] Success Criteria:

• All indexes integrated
• Consistent updates across all indexes
• Thread-safe
• Unit tests

Steps:

1. Create pkg/cache/pool_cache.go
2. Define PoolCache struct:

   type PoolCache struct {
       addressIndex   *AddressIndex
       tokenPairIndex *TokenPairIndex
       liquidityIndex *LiquidityIndex
       protocolIndex  *ProtocolIndex
       mu             sync.RWMutex
   }
   

3. Implement Get(address) - delegate to addressIndex
4. Implement GetByTokenPair(t0, t1) - delegate
5. Implement GetByProtocol(proto) - delegate
6. Implement GetTopByLiquidity(limit) - delegate
7. Implement Add(pool) - update ALL indexes atomically
8. Implement Update(address, pool) - update ALL indexes
9. Implement Remove(address) - remove from ALL indexes
10. Write unit test for consistency across indexes
11. Write concurrency test
12. Benchmark full cache operations

---

[P3-006] Implement Cache Persistence

Est: 2 hours Dependencies: [P3-005] Success Criteria:

• Save cache to disk
• Load cache from disk
• JSON format
• Corruption handling

Steps:

1. Add SaveToFile(path string) method
2. Marshal all pools to JSON
3. Write atomically (temp file + rename)
4. Add LoadFromFile(path string) method
5. Read JSON from disk
6. Unmarshal pools
7. Rebuild all indexes
8. Handle file not found gracefully
9. Handle corrupted JSON
10. Write unit test for save/load round-trip
11. Write test for corruption recovery

---

[P3-007] Implement Cache TTL and Eviction

Est: 2 hours Dependencies: [P3-005] Success Criteria:

• Pools have TTL
• Auto-eviction of expired pools
• Background cleanup goroutine
• Unit tests

Steps:

1. Add LastUpdated time.Time to PoolInfo
2. Add ttl time.Duration to PoolCache config
3. Add StartCleanup() method - launch goroutine
4. Implement cleanup loop (every 5 minutes):
   • Lock cache
   • Find expired pools
   • Remove from all indexes
   • Log evictions
5. Add StopCleanup() method - stop goroutine
6. Write unit test with short TTL
7. Write test for cleanup goroutine
8. Write test for manual cleanup trigger

---

Phase 4: Validation Pipeline

[P4-001] Implement Validation Rules

Est: 2 hours Dependencies: [P1-005] Success Criteria:

• Rule interface implemented
• Built-in rules created
• Composable rules
• Unit tests

Steps:

1. Create pkg/validation/rules.go
2. Implement ValidationRule interface:

   type ValidationRule interface {
       Name() string
       Validate(event *Event) error
   }
   

3. Implement ZeroAddressRule - check Token0, Token1, Pool
4. Implement ZeroAmountRule - check amounts not all zero
5. Implement PoolCacheRule - validate against cache
6. Implement CompositeRule - combine multiple rules
7. Write unit test for each rule
8. Write test for rule composition

---

[P4-002] Implement Event Validator

Est: 2 hours Dependencies: [P4-001] Success Criteria:

• Validator struct
• Rule registration
• Validation execution
• Detailed error messages

Steps:

1. Create pkg/validation/validator.go
2. Define EventValidator struct:

   type EventValidator struct {
       rules map[string]ValidationRule
       mu    sync.RWMutex
   }
   

3. Implement AddRule(rule)
4. Implement RemoveRule(name)
5. Implement Validate(event):
   • Run all rules
   • Collect errors
   • Return ValidationResult
6. Define ValidationResult struct:

   type ValidationResult struct {
       Valid  bool
       Errors []ValidationError
   }
   

7. Write unit test for validation
8. Write test for multiple rules
9. Write test for error collection

---

[P4-003] Implement Validation Metrics

Est: 1.5 hours Dependencies: [P4-002, P1-007] Success Criteria:

• Metrics for validation results
• Counters for each rule
• Histogram for validation time
• Unit tests

Steps:

1. Create pkg/validation/metrics.go
2. Define metrics:
   • validation_total counter (label: result=pass/fail)
   • validation_rule_failures counter (label: rule_name)
   • validation_duration_seconds histogram
3. Add RecordValidation(result, duration) method
4. Add RecordRuleFailure(ruleName) method
5. Integrate metrics into EventValidator
6. Write unit test for metrics recording
7. Write integration test with Prometheus

---

[P4-004] Implement Background Validation Channel

Est: 2 hours Dependencies: [P4-002] Success Criteria:

• Channel for async validation
• Background goroutine
• Discrepancy logging
• Unit tests

Steps:

1. Create pkg/validation/background.go
2. Define BackgroundValidator struct:

   type BackgroundValidator struct {
       validationChan chan *ValidationRequest
       cache          PoolCache
       logger         *Logger
       stopChan       chan struct{}
   }
   

3. Define ValidationRequest struct:

   type ValidationRequest struct {
       Event      *Event
       ParsedData *ParsedPoolData
       CachedData *PoolInfo
   }
   

4. Implement Start() method - launch goroutine
5. Implement validation loop:
   • Receive from validationChan
   • Compare parsed vs cached
   • Log discrepancies
   • Update metrics
6. Implement Stop() method
7. Implement Submit(req) method
8. Write unit test for comparison logic
9. Write test for goroutine lifecycle
10. Write test for channel backpressure

---

[P4-005] Implement Discrepancy Logger

Est: 1.5 hours Dependencies: [P4-004] Success Criteria:

• Structured discrepancy logs
• JSON output
• Queryable fields
• Unit tests

Steps:

1. Add LogDiscrepancy(event, field, parsed, cached) to BackgroundValidator
2. Create structured log entry:

   {
       "timestamp": "",
       "event_type": "",
       "pool": "",
       "field": "",
       "parsed_value": "",
       "cached_value": "",
       "tx_hash": ""
   }
   

3. Write to dedicated log file validation_discrepancies.log
4. Add log rotation
5. Write unit test for log format
6. Write test for file writing
7. Integration test reading log back

---

[P4-006] Implement Validation Alerts

Est: 2 hours Dependencies: [P4-003, P4-005] Success Criteria:

• Alert thresholds configurable
• Email/Slack notification
• Alert deduplication
• Unit tests

Steps:

1. Create pkg/validation/alerts.go
2. Define AlertManager struct:

   type AlertManager struct {
       thresholds map[string]float64
       notifiers  []Notifier
       mu         sync.RWMutex
   }
   

3. Define Notifier interface:

   type Notifier interface {
       Notify(alert Alert) error
   }
   

4. Implement EmailNotifier
5. Implement SlackNotifier
6. Add threshold checking in validator
7. Implement alert deduplication (1 per hour)
8. Write unit test for threshold detection
9. Write test for notifier integration
10. Write test for deduplication

---

[P4-007] Integration Test - Validation Pipeline

Est: 2 hours Dependencies: [P4-001 through P4-006] Success Criteria:

• End-to-end validation flow
• Real event data
• All components integrated
• Performance benchmarks

Steps:

1. Create test with real Uniswap transaction
2. Parse event with parser
3. Validate with EventValidator
4. Submit to BackgroundValidator
5. Assert discrepancies logged
6. Assert metrics recorded
7. Verify alerts not triggered (good data)
8. Test with intentionally bad data
9. Verify alerts triggered
10. Benchmark full pipeline

---

Phase 5: Migration & Testing

[P5-001] Create V1/V2 Comparison Harness

Est: 3 hours Dependencies: [P2-025, P4-007] Success Criteria:

• Run V1 and V2 parsers in parallel
• Compare outputs
• Log differences
• Statistical analysis

Steps:

1. Create tests/migration/comparison_test.go
2. Load 1000 real transactions
3. Parse with V1 parser
4. Parse with V2 parser
5. Compare events:
   • Count
   • Event types
   • Addresses
   • Amounts
6. Log all differences
7. Calculate metrics:
   • Agreement rate
   • V2 improvements (zero addresses eliminated)
   • Performance difference
8. Generate comparison report
9. Identify regressions
10. Create visual diff tool

---

[P5-002] Load Testing

Est: 2 hours Dependencies: [P5-001] Success Criteria:

• 10,000 events/second throughput
• Latency < 10ms p99
• No memory leaks
• CPU usage acceptable

Steps:

1. Create tests/load/load_test.go
2. Generate synthetic event stream
3. Feed to V2 pipeline
4. Measure throughput
5. Measure latency (p50, p95, p99)
6. Profile memory usage
7. Profile CPU usage
8. Identify bottlenecks
9. Optimize hot paths
10. Re-test and verify improvements

---

[P5-003] Chaos Testing

Est: 3 hours Dependencies: [P5-002] Success Criteria:

• System survives RPC failures
• Graceful degradation
• Automatic recovery
• No data loss

Steps:

1. Create tests/chaos/chaos_test.go
2. Test scenario: RPC connection drops
3. Test scenario: RPC returns errors
4. Test scenario: Cache corruption
5. Test scenario: High memory pressure
6. Test scenario: Concurrent load
7. Test scenario: Malformed event data
8. Verify circuit breakers activate
9. Verify automatic reconnection
10. Verify event replay from checkpoint

---

[P5-004] Create Migration Plan

Est: 2 hours Dependencies: [P5-003] Success Criteria:

• Step-by-step migration guide
• Rollback procedure
• Monitoring checklist
• Risk assessment

Steps:

1. Create docs/MIGRATION_PLAN.md
2. Document prerequisites
3. Document migration steps:
   • Deploy V2 alongside V1
   • Enable V2 shadow mode (no writes)
   • Monitor for 24 hours
   • Compare outputs
   • Gradually increase V2 traffic
   • Full cutover
4. Document rollback procedure
5. Create monitoring dashboard
6. Define success criteria
7. Define rollback triggers
8. Document post-migration tasks
9. Review with team
10. Get approval

---

[P5-005] Production Deployment

Est: 4 hours Dependencies: [P5-004] Success Criteria:

• V2 deployed to production
• Monitoring active
• No regressions
• Performance improved

Steps:

1. Create deployment branch
2. Run full test suite
3. Deploy to staging
4. Smoke test staging
5. Enable shadow mode in production
6. Monitor for 2 hours
7. Compare V1 vs V2 outputs
8. Gradual traffic shift (10% -> 50% -> 100%)
9. Full cutover to V2
10. Decommission V1 after 7 days

---

Task Dependencies Visualization

Phase 1: Foundation
[P1-001] -> [P1-002] -> [P1-003] -> [P2-001]
          -> [P1-005] -> [P4-001]
[P1-006] -> All phases
[P1-007] -> [P4-003]
[P1-004] -> [P1-008] -> [P3-001]
[P1-009] -> [P2-002]
[P1-010] -> All test tasks

Phase 2: Parsers
[P2-001] -> [P2-002] -> ... -> [P2-009] (UniswapV2)
         -> [P2-010] -> ... -> [P2-015] (UniswapV3)
         -> [P2-016] -> ... (SushiSwap, Camelot, Curve)

Phase 3: Cache
[P3-001] -> [P3-002], [P3-003], [P3-004]
         -> [P3-005] -> [P3-006], [P3-007]

Phase 4: Validation
[P4-001] -> [P4-002] -> [P4-003]
                     -> [P4-004] -> [P4-005]
[P4-003], [P4-005] -> [P4-006]
All Phase 4 -> [P4-007]

Phase 5: Migration
All phases -> [P5-001] -> [P5-002] -> [P5-003] -> [P5-004] -> [P5-005]

Estimation Summary

• Phase 1: ~11 hours
• Phase 2: ~45 hours (all parsers)
• Phase 3: ~16 hours
• Phase 4: ~13 hours
• Phase 5: ~14 hours

Total: ~99 hours (approximately 12-13 working days for 1 developer)

With parallel work and multiple developers, target 4-6 weeks total.

---

Document Status: Detailed Task Breakdown Created: 2025-11-10 Last Updated: 2025-11-10 Version: 1.0