mev-beta/docs/3_core_packages/MONITOR_PACKAGE.md

Monitor Package Documentation

Overview

The monitor package provides comprehensive monitoring of the Arbitrum sequencer for MEV opportunities. It implements concurrent processing, rate limiting, DEX event subscription, and real-time transaction analysis to detect potential arbitrage opportunities in the mempool and confirmed blocks.

Core Components

ArbitrumMonitor Structure

The main monitor structure provides sequencer monitoring capabilities:

1. Configuration - Arbitrum and bot configuration
2. Client - Ethereum client for RPC calls
3. L2Parser - Arbitrum L2 transaction parser
4. Logger - Structured logging
5. RateLimiter - Rate limiting for RPC calls
6. MarketManager - Market data management
7. Scanner - Market scanner for opportunities
8. Pipeline - Processing pipeline for transactions
9. FanManager - Concurrent processing manager
10. Limiter - Additional rate limiting
11. PollInterval - Block polling interval
12. Running - Monitor state flag

Initialization

NewArbitrumMonitor() Function

Creates a new Arbitrum monitor with all required components:

1. Ethereum Client - Connects to Arbitrum node
2. Price Oracle - Creates price oracle for L2 parsing
3. L2 Parser - Initializes Arbitrum L2 transaction parser
4. Rate Limiter - Configures rate limiting based on settings
5. Pipeline - Creates market processing pipeline
6. Fan Manager - Initializes concurrent processing manager
7. Event Parser - Prepares DEX event parsing (future use)
8. Pool Discovery - Prepares pool discovery (future use)

Monitoring Functions

Start(ctx context.Context) error

Begins monitoring the Arbitrum sequencer:

1. State Management - Sets running flag
2. Block Tracking - Gets latest block number to start from
3. Event Subscription - Subscribes to DEX events
4. Polling Loop - Continuously polls for new blocks
5. Block Processing - Processes blocks for DEX transactions
6. Rate Limiting - Applies rate limits for RPC calls
7. Context Handling - Respects context cancellation

Stop()

Stops the monitor gracefully:

1. State Management - Clears running flag
2. Cleanup - Performs necessary cleanup operations
3. Logging - Logs stop event

Block Processing

processBlock(ctx context.Context, blockNumber uint64) error

Processes a single block for potential swap transactions:

1. Rate Limiting - Waits for rate limiter
2. Block Retrieval - Gets block using L2 parser
3. DEX Parsing - Parses DEX transactions from block
4. Performance Logging - Logs RPC and parsing metrics
5. Transaction Analysis - Analyzes DEX transactions
6. Empty Block Handling - Reports empty blocks

L2 Parsing Benefits

The L2 parser provides several advantages:

• Bypass Transaction Type Issues - Handles Arbitrum-specific transaction types
• Enhanced DEX Detection - Better identification of DEX transactions
• Improved Performance - More efficient block processing
• Reduced Errors - Fewer parsing failures

DEX Event Monitoring

subscribeToDEXEvents(ctx context.Context) error

Subscribes to DEX contract events for real-time monitoring:

1. Contract Definitions - Defines official DEX contracts:

   • Uniswap V2/V3 factories and routers
   • SushiSwap factories and routers
   • High-volume pools
   • Universal routers
   • GMX router

2. Event Signatures - Defines common DEX event signatures:

   • Uniswap V2 Swap, Mint, Burn
   • Uniswap V3 Swap, Mint, Burn

3. Filter Query - Creates filter for DEX events

4. Subscription - Subscribes to logs

5. Event Processing - Processes events in goroutine

processDEXEvent(ctx context.Context, log types.Log)

Processes a DEX event log:

1. Event Identification - Identifies event type by signature
2. Logging - Logs detected events
3. Receipt Fetching - Gets transaction receipt
4. Transaction Processing - Processes receipt for opportunities

processTransactionReceipt(ctx context.Context, receipt *types.Receipt, blockNumber uint64, blockHash common.Hash)

Processes a transaction receipt for DEX events:

1. Log Analysis - Analyzes receipt logs for DEX events
2. Event Counting - Counts DEX events in transaction
3. Pipeline Processing - Processes through market pipeline
4. Legacy Compatibility - Maintains compatibility with legacy systems

Transaction Processing

processTransaction(ctx context.Context, tx *types.Transaction) error

Analyzes a transaction for potential swap opportunities:

1. Sender Identification - Gets transaction sender
2. Basic Logging - Logs transaction details
3. Swap Detection - (TODO) Detects swap transactions
4. Price Impact - (TODO) Calculates potential price impact

Future Enhancements

The transaction processing includes placeholders for:

• Swap transaction detection
• Function signature checking
• Token address extraction
• Amount parsing
• Price impact calculation

Rate Limiting

Integrated Rate Limiting

The monitor implements comprehensive rate limiting:

1. Configuration-Based - Uses config.RateLimit settings
2. Token Bucket - Implements token bucket algorithm
3. Per-Endpoint - Rate limits per RPC endpoint
4. Wait Mechanism - Blocks until rate limit allows
5. Error Handling - Handles rate limit errors gracefully

Rate Limit Configuration

• RequestsPerSecond - Maximum requests per second
• MaxConcurrent - Maximum concurrent requests
• Burst - Burst size for rate limiting

Performance Monitoring

RPC Performance Logging

The monitor logs RPC call performance:

• Duration Tracking - Measures call durations
• Success/Failure - Tracks call success rates
• Error Reporting - Logs RPC errors
• Endpoint Monitoring - Tracks per-endpoint performance

Block Processing Metrics

The monitor logs block processing metrics:

• Transaction Counts - Total and DEX transaction counts
• Processing Time - Block processing duration
• Parse Rates - Transactions per second parsing rate
• DEX Detection - DEX transaction identification rates

Concurrency Support

Fan Manager

The monitor uses a fan manager for concurrent processing:

• Parallel Processing - Processes multiple items concurrently
• Resource Management - Manages worker pool resources
• Load Distribution - Distributes load across workers
• Backpressure Handling - Handles processing backpressure

Pipeline Processing

The monitor implements a processing pipeline:

• Stage Management - Manages processing stages
• Data Flow - Controls data flow through pipeline
• Error Handling - Handles pipeline errors
• Performance Monitoring - Monitors pipeline performance

Error Handling

Retry Mechanisms

The monitor implements retry mechanisms:

• Exponential Backoff - Increases delay between retries
• Max Retry Limits - Limits retry attempts
• Context Respect - Respects context cancellation
• Error Classification - Classifies retryable errors

getTransactionReceiptWithRetry() Function

Implements retry logic for receipt fetching:

1. Attempt Tracking - Tracks retry attempts
2. Exponential Backoff - Increases delay between attempts
3. Context Checking - Respects context cancellation
4. Error Logging - Logs retry attempts and failures

Security Features

Rate Limiting

Prevents abuse of RPC endpoints:

• Configurable Limits - Adjustable rate limits
• Burst Handling - Handles request bursts
• Endpoint Protection - Protects individual endpoints
• Fair Usage - Ensures fair RPC usage

Event Validation

Validates DEX events:

• Signature Checking - Verifies event signatures
• Contract Verification - Verifies contract addresses
• Data Validation - Validates event data
• Error Prevention - Prevents invalid data processing

Best Practices

Monitoring Optimization

1. Rate Limiting - Always respect RPC rate limits
2. Efficient Parsing - Use optimized parsing methods
3. Selective Logging - Log only necessary information
4. Resource Management - Manage memory and goroutines
5. Error Recovery - Handle errors gracefully
6. Performance Monitoring - Monitor performance metrics

Event Processing

1. Event Identification - Quickly identify relevant events
2. Data Extraction - Efficiently extract required data
3. Validation - Validate all extracted data
4. Processing - Process events through appropriate pipelines
5. Logging - Log important events and metrics

Concurrency Management

1. Worker Pools - Use appropriate worker pool sizes
2. Resource Limits - Limit concurrent resource usage
3. Backpressure - Handle processing backpressure
4. Cleanup - Clean up resources properly
5. Monitoring - Monitor concurrent operations

Future Improvements

Enhanced Features

1. Advanced DEX Detection - Improved DEX transaction identification
2. Mempool Monitoring - Real-time mempool transaction analysis
3. Machine Learning - ML-based opportunity prediction
4. Cross-Chain - Multi-chain monitoring support
5. Advanced Analytics - Enhanced performance analytics

Performance Enhancements

1. Optimized Parsing - Further L2 parsing optimizations
2. Caching - Intelligent caching strategies
3. Parallel Processing - Enhanced concurrent processing
4. Memory Management - Improved memory usage
5. Network Optimization - Optimized RPC usage

Security Improvements

1. Enhanced Validation - More thorough data validation
2. Rate Limiting - Advanced rate limiting strategies
3. Access Control - Enhanced access controls
4. Audit Logging - Comprehensive audit trails
5. Threat Detection - Advanced threat detection