mev-beta/COMMON.md

MEV Bot - Common Requirements and Integration Points

This document serves as a central reference for all AI assistants working on the MEV Bot project. It outlines the core requirements, integration points, and shared knowledge that should be consistent across all modules and components.

Project Overview

The MEV Bot is a high-frequency trading bot written in Go that monitors the Arbitrum sequencer for potential swap opportunities and identifies profitable arbitrage opportunities. The bot uses off-chain methods to calculate price movements using Uniswap V3 pricing functions.

Core Technologies

• Go 1.24+
• Ethereum/go-ethereum library
• Arbitrum sequencer monitoring
• Uniswap V3 pricing functions
• Concurrency patterns (worker pools, pipelines, fan-in/fan-out)
• Multiple transport mechanisms (shared memory, Unix sockets, TCP, WebSockets, gRPC)

Architecture Principles

1. Modularity: Each component should be independently deployable
2. Scalability: Design for high-throughput, low-latency processing
3. Resilience: Handle failures gracefully with proper error handling and recovery
4. Security: Protect sensitive data and implement secure communication
5. Observability: Comprehensive logging, metrics, and monitoring
6. Testability: Code should be easily testable with unit and integration tests

Integration Points

Configuration Management

• Centralized configuration in YAML format
• Environment variable overrides
• Hot reloading capability
• Validation of configuration parameters

Event Processing Pipeline

• Multi-stage processing with configurable stages
• Worker pools for concurrent processing
• Backpressure handling
• Error propagation and recovery

Communication Layer

• Universal message bus supporting multiple transports
• Smart routing based on message characteristics
• Automatic transport selection
• Module lifecycle management (START, STOP, PAUSE, RESUME)

Data Management

• In-memory caching for frequently accessed data
• Persistent storage for historical data
• Efficient data structures for high-frequency access
• Proper indexing for query performance

Security

• Secure key management
• Encrypted connections for RPC endpoints
• Input validation and sanitization
• Access controls and authentication

Key Data Structures

Event

type Event struct {
    Type           EventType
    Protocol       string
    PoolAddress    common.Address
    Token0         common.Address
    Token1         common.Address
    Amount0        *big.Int
    Amount1        *big.Int
    SqrtPriceX96   *uint256.Int
    Liquidity      *uint256.Int
    Tick           int
    Timestamp      uint64
    TransactionHash common.Hash
    BlockNumber    uint64
}

ArbitrageOpportunity

type ArbitrageOpportunity struct {
    Path        []string
    Pools       []string
    Profit      *big.Int
    GasEstimate *big.Int
    ROI         float64
    Protocol    string
}

MarketData

type MarketData struct {
    Address       common.Address
    Token0        common.Address
    Token1        common.Address
    Fee           int64
    Liquidity     *uint256.Int
    SqrtPriceX96  *uint256.Int
    Tick          int
    TickSpacing   int
    LastUpdated   time.Time
}

Performance Requirements

• Latency < 10 microseconds for critical path
• Throughput > 100,000 messages/second
• Sub-millisecond processing for arbitrage detection
• Deterministic transaction ordering
• Horizontal scalability

Error Handling Standards

• Use Go's error wrapping with context
• Implement retry mechanisms with exponential backoff
• Handle timeouts appropriately
• Log at appropriate levels (debug, info, warn, error)
• Include contextual information in log messages

Testing Standards

• Unit tests for all functions
• Integration tests for component interactions
• Property-based testing for mathematical functions
• Benchmarks for performance-critical code
• Mock external dependencies
• Test edge cases and boundary conditions

Documentation Standards

• Comprehensive comments for all exported functions
• Clear explanation of complex algorithms
• Usage examples for public APIs
• Architecture diagrams for complex components
• Performance characteristics documentation

Common Patterns to Use

Worker Pool Pattern

// Use worker pools for concurrent processing of transactions
type WorkerPool struct {
    workers    []*Worker
    jobQueue   chan Job
    quitChan   chan bool
}

Pipeline Pattern

// Use pipelines for multi-stage processing
type PipelineStage func(context.Context, <-chan Input, chan<- Output) error

Rate Limiting

// Use token bucket or leaky bucket algorithms for rate limiting
type RateLimiter struct {
    limiter *rate.Limiter
}

Caching

// Use LRU or similar caching strategies with TTL
type Cache struct {
    data map[string]*CachedItem
    ttl  time.Duration
}

Common Dependencies

• github.com/ethereum/go-ethereum - Ethereum client library
• github.com/holiman/uint256 - Uint256 arithmetic
• github.com/stretchr/testify - Testing utilities
• github.com/urfave/cli/v2 - CLI framework
• golang.org/x/time/rate - Rate limiting
• golang.org/x/sync - Extended concurrency primitives

Monitoring and Metrics

• Latency metrics for each component
• Throughput metrics for message processing
• Error rates and failure patterns
• Resource utilization (CPU, memory, disk)
• Custom business metrics (profitability, opportunities found)

Deployment Considerations

• Support for containerization (Docker)
• Configuration via environment variables
• Health check endpoints
• Graceful shutdown procedures
• Log aggregation and rotation