mev-beta/docs/3_core_packages/LOGGER_PACKAGE.md

Logger Package Documentation

Overview

The logger package provides a sophisticated logging system with multiple specialized loggers, security filtering, and separation of concerns for different types of log messages. It's designed specifically for MEV bot applications with detailed logging for opportunities, transactions, performance metrics, and security-sensitive data filtering.

Core Components

Logger Structure

The main logger structure contains multiple specialized loggers:

1. Main Logger - General application logging
2. Opportunity Logger - MEV opportunities and arbitrage attempts
3. Error Logger - Errors and warnings only
4. Performance Logger - Performance metrics and RPC calls
5. Transaction Logger - Detailed transaction analysis

LogLevel Enum

The logger supports multiple log levels:

• DEBUG - Detailed debugging information
• INFO - General operational information
• WARN - Warning conditions
• ERROR - Error conditions
• OPPORTUNITY - Special level for arbitrage opportunities

SecureFilter

The secure filter provides security filtering for sensitive data:

• SecurityLevelDebug - Log everything (development only)
• SecurityLevelInfo - Log basic info, filter amounts
• SecurityLevelProduction - Log minimal info, filter sensitive data

Initialization

New(level string, format string, file string) *Logger

Creates a new multi-file logger with separation of concerns:

1. Parses the log level from string
2. Creates specialized log files for different concerns
3. Initializes security filtering based on environment
4. Sets up all specialized loggers

Log File Organization

The logger automatically creates specialized log files:

• Main log file - General application logs
• opportunities.log - MEV opportunities and arbitrage attempts
• errors.log - Errors and warnings only
• performance.log - Performance metrics and RPC calls
• transactions.log - Detailed transaction analysis

Logging Functions

General Logging

Debug(v ...interface{})

Logs debug messages when log level is DEBUG or higher.

Info(v ...interface{})

Logs info messages when log level is INFO or higher.

Warn(v ...interface{})

Logs warning messages when log level is WARN or higher. Also logs to the error file for alerting.

Error(v ...interface{})

Logs error messages when log level is ERROR or higher. Also logs to the error file for alerting.

Specialized Logging

Opportunity(txHash, from, to, method, protocol string, amountIn, amountOut, minOut, profitUSD float64, additionalData map[string]interface{})

Logs detailed arbitrage opportunities with security filtering:

• Transaction hash
• From/to addresses
• Method and protocol
• Amounts and profit
• Additional data (filtered for security)

OpportunitySimple(v ...interface{})

Logs simple opportunity messages for backwards compatibility.

Performance(component, operation string, duration time.Duration, metadata map[string]interface{})

Logs performance metrics for optimization analysis:

• Component name
• Operation name
• Duration
• Additional metadata

Metrics(name string, value float64, unit string, tags map[string]string)

Logs business metrics for analysis:

• Metric name
• Value
• Unit
• Tags

Transaction(txHash, from, to, method, protocol string, gasUsed, gasPrice uint64, value float64, success bool, metadata map[string]interface{})

Logs detailed transaction information for MEV analysis:

• Transaction hash
• From/to addresses
• Method and protocol
• Gas information
• Value and success status
• Additional metadata (filtered for security)

BlockProcessing(blockNumber uint64, txCount, dexTxCount int, processingTime time.Duration)

Logs block processing metrics for sequencer monitoring:

• Block number
• Transaction counts
• Processing time

ArbitrageAnalysis(poolA, poolB, tokenPair string, priceA, priceB, priceDiff, estimatedProfit float64, feasible bool)

Logs arbitrage opportunity analysis results:

• Pool addresses
• Token pair
• Prices and differences
• Profit estimation
• Feasibility status

RPC(endpoint, method string, duration time.Duration, success bool, errorMsg string)

Logs RPC call metrics for endpoint optimization:

• Endpoint URL
• Method name
• Duration
• Success status
• Error message (if any)

SwapAnalysis(tokenIn, tokenOut string, amountIn, amountOut float64, protocol, poolAddr string, metadata map[string]interface{})

Logs swap event analysis with security filtering:

• Token addresses
• Amounts
• Protocol and pool
• Additional metadata

Security Features

Security Filtering

The logger includes sophisticated security filtering to prevent sensitive data leakage:

NewSecureFilter(level SecurityLevel) *SecureFilter

Creates a new secure filter with the specified security level.

FilterMessage(message string) string

Filters sensitive data from log messages based on security level:

• Debug Mode - No filtering
• Info Mode - Filter amounts and values
• Production Mode - Filter addresses, amounts, and values

SanitizeForProduction(data map[string]interface{}) map[string]interface{}

Removes all sensitive data for production logging:

• Filters sensitive keys (amount, value, profit, etc.)
• Shortens addresses for privacy
• Preserves non-sensitive data

Sensitive Data Protection

The logger automatically protects:

• Private keys and secrets
• Wallet addresses (in production)
• Transaction amounts (in production)
• Profit values (in production)
• Gas prices and limits
• Contract addresses (filtered in production)

Configuration

Log Levels

The logger supports standard log levels:

• debug - Most verbose, detailed debugging
• info - General information
• warn - Warning conditions
• error - Error conditions only

Security Levels

Security filtering is based on environment and log level:

• Development - Debug level, no filtering
• Testing - Info level, basic filtering
• Production - Production level, maximum filtering

File Configuration

The logger supports:

• Single file logging (stdout)
• Multiple file logging (separated by concern)
• Custom file paths and naming
• Automatic directory creation

Performance Considerations

Efficient Logging

The logger is optimized for performance:

• Minimal overhead for filtered messages
• Efficient string formatting
• Separate file handles for different concerns
• Buffered I/O operations

Memory Management

The logger manages memory efficiently:

• Reuses log message buffers
• Minimizes string allocations
• Efficient regex pattern matching
• Proper resource cleanup

Best Practices

Log Organization

1. Use appropriate log levels for different messages
2. Separate concerns with specialized loggers
3. Include relevant context in log messages
4. Use structured logging for complex data
5. Implement security filtering for sensitive data

Security

1. Never log private keys or secrets
2. Filter sensitive data in production
3. Use appropriate security levels for environments
4. Regularly audit log content
5. Monitor log file access and permissions

Performance

1. Use appropriate log levels for production
2. Avoid expensive operations in log messages
3. Monitor log file sizes
4. Implement log rotation
5. Use asynchronous logging for high-frequency operations

Error Handling

Logging Failures

The logger handles logging failures gracefully:

• Falls back to stdout when file creation fails
• Continues operation despite individual logging failures
• Logs errors to error logger when possible
• Maintains application stability

Security Failures

The logger prevents security issues:

• Filters sensitive data automatically
• Prevents data leakage in production
• Handles malformed data gracefully
• Maintains security even when logging fails

Testing

Unit Tests

The logger package includes comprehensive tests:

• Log level parsing
• Security filtering
• Message formatting
• File creation and management

Integration Tests

Integration testing covers:

• End-to-end logging workflows
• Security filtering in different environments
• Performance under load
• Error handling scenarios

Future Improvements

Enhanced Features

1. Log rotation and archiving
2. Remote logging destinations
3. Log compression
4. Alerting integration
5. Log search and analysis tools

Security Enhancements

1. Advanced pattern matching for sensitive data
2. Machine learning-based filtering
3. Encrypted log storage
4. Audit trails for log access
5. Compliance reporting