mev-beta/CLAUDE.md

# MEV Bot Project - Claude Code Configuration

This file contains comprehensive Claude Code configuration and context information for the MEV Bot project.

**Note:** For Claude-specific CLI configuration, commands, and development guidelines, please refer to the [.claude/](.claude/) directory which contains:
- [.claude/CLAUDE.md](.claude/CLAUDE.md) - Complete Claude Code CLI configuration
- [.claude/commands/](.claude/commands/) - Slash commands for Claude Code
- [.claude/scripts/](.claude/scripts/) - Scripts for Claude Code development workflow

## 🚀 Quick Start Commands

### Essential Build & Test Commands
```bash
# Build the MEV bot binary
make build

# Run tests
make test

# Run deterministic math audit and profitability replay
./scripts/run_audit_suite.sh
make simulate-profit

# Start development server with hot reload
./scripts/run.sh

# Build and run with logging
./scripts/build.sh && ./mev-bot start

# Check for Go modules issues
go mod tidy && go mod verify

# Run linter
golangci-lint run

# Run security analysis
gosec ./...
```

### Production Log Management Commands
```bash
# Production Log Manager - Comprehensive System
./scripts/log-manager.sh full                # Complete log management cycle
./scripts/log-manager.sh analyze             # Real-time analysis (Health Score: 97.97/100)
./scripts/log-manager.sh health              # Corruption detection & integrity checks
./scripts/log-manager.sh monitor             # Performance tracking with MEV metrics
./scripts/log-manager.sh archive             # Advanced archiving with metadata
./scripts/log-manager.sh start-daemon        # Background monitoring daemon
./scripts/log-manager.sh dashboard           # Generate operations dashboard
./scripts/log-manager.sh status              # System status overview

# Basic Archive Commands (Legacy Support)
./scripts/archive-logs.sh                    # Basic archiving
./scripts/quick-archive.sh                   # Quick archive and clear
./scripts/view-latest-archive.sh             # Browse archives

# Production Monitoring & Alerting
./scripts/log-manager.sh start-daemon        # Start real-time monitoring
./scripts/log-manager.sh stop-daemon         # Stop monitoring daemon
./scripts/demo-production-logs.sh            # Full system demonstration
```

### Development Workflow Commands
```bash
# Setup development environment
export ARBITRUM_RPC_ENDPOINT="wss://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57"
export ARBITRUM_WS_ENDPOINT="wss://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57"
export METRICS_ENABLED="false"

# Run with timeout for testing
timeout 30 ./mev-bot start

# Debug with verbose logging
LOG_LEVEL=debug ./mev-bot start

# Profile performance
go tool pprof http://localhost:6060/debug/pprof/profile
```

## Project Overview
This is a production-ready MEV (Maximal Extractable Value) bot written in Go 1.24+ that monitors the Arbitrum network for profitable arbitrage opportunities across multiple DEX protocols. The bot has been extensively optimized with critical fixes applied for transaction processing, ABI decoding, connection stability, and arbitrage detection.

### 🚀 Recent Major Improvements
- **Transaction Pipeline**: Fixed bottleneck causing 26,750+ dropped transactions (99.5% improvement)
- **Multicall Support**: Enhanced ABI decoding for complex multicall transactions
- **Connection Resilience**: Automatic RPC failover and health monitoring
- **Detection Sensitivity**: Lowered arbitrage threshold from 0.5% to 0.1% (5x improvement)
- **Mathematical Precision**: Corrected TPS calculations and improved decimal handling

## Project Structure
- `cmd/` - Main applications (specifically `cmd/mev-bot/main.go`)
- `internal/` - Private application and library code
  - `internal/config` - Configuration management
  - `internal/logger` - Logging functionality
  - `internal/ratelimit` - Rate limiting implementations
  - `internal/utils` - Utility functions
- `pkg/` - Library code that can be used by external projects
  - `pkg/events` - Event processing system
  - `pkg/market` - Market data handling
  - `pkg/monitor` - Arbitrum sequencer monitoring
  - `pkg/scanner` - Market scanning functionality
  - `pkg/test` - Test utilities and helpers
  - `pkg/uniswap` - Uniswap V3 specific implementations
- `config/` - Configuration files
- `@prompts/` - AI prompts for development assistance
- `docs/` - Documentation
- `scripts/` - Scripts for building, testing, and deployment
- `.claude/` - Claude Code specific configuration and tools

## Key Integration Points
- Follow the modular architecture with independent components
- Use the transaction pipeline for high-throughput processing
- Implement proper error handling and recovery mechanisms
- Adhere to the production-ready architecture patterns defined in PROJECT_SPECIFICATION.md

## 📋 Development Guidelines & Code Style

### Go Best Practices
- **Error Handling**: Always wrap errors with context using `fmt.Errorf("operation failed: %w", err)`
- **Concurrency**: Use worker pools for processing large datasets (see `pkg/market/pipeline.go`)
- **Interfaces**: Keep interfaces small and focused (1-3 methods maximum)
- **Testing**: Aim for >90% test coverage with table-driven tests
- **Logging**: Use structured logging with `slog` package
- **Performance**: Profile regularly with `go tool pprof`

### Code Organization Rules
- **File Size**: Keep files under 500 lines (split larger files into logical components)
- **Package Structure**: Follow Go standard layout (cmd/, internal/, pkg/)
- **Naming**: Use Go naming conventions (PascalCase for exports, camelCase for private)
- **Documentation**: Document all exported functions with examples
- **Constants**: Group related constants in blocks with `iota` when appropriate

### Required Checks Before Commit
```bash
# Run all checks before committing
make test && make lint && go mod tidy

# Security scan
gosec ./...

# Dependency vulnerability check
go list -json -m all | nancy sleuth
```

## System Architecture
The MEV Bot follows a modular architecture with clearly defined components that communicate through well-defined interfaces.

### Core Components and Their Responsibilities

1. **Arbitrum Monitor (`pkg/monitor/concurrent.go`)**
   - Real-time Arbitrum sequencer monitoring with health checks
   - High-throughput transaction processing (50,000 transaction buffer)
   - Automatic RPC connection management with failover
   - ConnectionManager integration for stability

2. **ABI Decoder (`pkg/arbitrum/abi_decoder.go`)**
   - Advanced multicall transaction parsing
   - Multi-protocol ABI decoding (Uniswap V2/V3, SushiSwap, 1inch, etc.)
   - Enhanced token address extraction with validation
   - Supports complex transaction patterns and function signatures

3. **Arbitrage Detection Engine (`pkg/arbitrage/detection_engine.go`)**
   - Configurable opportunity detection (0.1% minimum threshold)
   - Multi-exchange price comparison and analysis
   - Worker pool-based concurrent processing
   - Real-time profit calculation and ranking

4. **Market Scanner (`pkg/scanner/concurrent.go`)**
   - Event-driven arbitrage opportunity analysis
   - Concurrent worker pools for high-throughput processing
   - Advanced swap analysis with price impact calculations
   - Integration with detection engine for opportunity identification

5. **Connection Management (`pkg/arbitrum/connection.go`)**
   - Automatic RPC failover and health monitoring
   - Rate limiting and circuit breaker patterns
   - Connection pooling and retry mechanisms
   - Multi-endpoint redundancy for reliability

### Communication Patterns
- Pipeline pattern for multi-stage processing
- Worker pool pattern for concurrent execution
- Message passing through Go channels
- Shared memory access with proper synchronization

## Claude's Primary Focus Areas
As Claude, you're particularly skilled at:

1. **Code Architecture and Design Patterns**
   - Implementing clean, maintainable architectures
   - Applying appropriate design patterns (pipeline, worker pool, etc.)
   - Creating well-structured interfaces between components
   - Ensuring loose coupling and high cohesion

2. **System Integration and APIs**
   - Designing clear APIs between components
   - Implementing proper data flow between modules
   - Creating robust configuration management
   - Building error handling and recovery mechanisms

3. **Writing Clear Documentation**
   - Documenting complex algorithms and mathematical calculations
   - Creating clear API documentation
   - Writing architectural decision records
   - Producing user guides and examples

4. **Implementing Robust Error Handling**
   - Using Go's error wrapping with context
   - Implementing retry mechanisms with exponential backoff
   - Handling timeouts appropriately
   - Creating comprehensive logging strategies

5. **Creating Maintainable and Scalable Code Structures**
   - Organizing code for easy testing and maintenance
   - Implementing performance monitoring and metrics
   - Designing for horizontal scalability
   - Ensuring code follows established patterns and conventions

When working on this project, please focus on these areas where your strengths will be most beneficial.

## 🛠 Claude Code Optimization Settings

### Workflow Preferences
- **Always commit changes**: Use `git commit -am "descriptive message"` after significant changes
- **Branch naming**: Use hyphens (`feat-add-new-parser`, `fix-memory-leak`)
- **Context management**: Use `/compact` to manage long conversations
- **Parallel processing**: Leverage Go's concurrency patterns extensively

### File Organization Preferences
- **Never save temporary files to root**: Use `/tmp/` or `internal/temp/`
- **Log files**: Always save to `logs/` directory
- **Test files**: Place alongside source files with `_test.go` suffix
- **Documentation**: Keep in `docs/` with clear naming

### Performance Monitoring
```bash
# Enable metrics endpoint
export METRICS_ENABLED="true"
export METRICS_PORT="9090"

# Monitor memory usage
go tool pprof http://localhost:9090/debug/pprof/heap

# Monitor CPU usage
go tool pprof http://localhost:9090/debug/pprof/profile?seconds=30

# Monitor goroutines
go tool pprof http://localhost:9090/debug/pprof/goroutine
```

## 🔧 Environment Configuration

### Required Environment Variables
```bash
# Arbitrum RPC Configuration
export ARBITRUM_RPC_ENDPOINT="wss://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57"
export ARBITRUM_WS_ENDPOINT="wss://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57"

# Application Configuration
export LOG_LEVEL="info"
export METRICS_ENABLED="false"
export METRICS_PORT="9090"

# Development Configuration
export GO_ENV="development"
export DEBUG="true"
```

### Optional Environment Variables
```bash
# Performance Tuning
export GOMAXPROCS=4
export GOGC=100

# Logging Configuration
export LOG_FORMAT="json"
export LOG_OUTPUT="logs/mev-bot.log"

# Rate Limiting
export MAX_RPS=100
export RATE_LIMIT_BURST=200
```

## 📝 Commit Message Conventions

### Format
```
type(scope): brief description

- Detailed explanation of changes
- Why the change was needed
- Any breaking changes or migration notes

🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
```

### Types
- `feat`: New feature implementation
- `fix`: Bug fix
- `perf`: Performance improvement
- `refactor`: Code restructuring without feature changes
- `test`: Adding or updating tests
- `docs`: Documentation updates
- `build`: Build system or dependency changes
- `ci`: CI/CD pipeline changes

## 🚨 Security Guidelines

### Never Commit
- Private keys or wallet seeds
- API keys or secrets
- RPC endpoints with authentication
- Personal configuration files

### Always Validate
- Input parameters for all functions
- RPC responses before processing
- Mathematical calculations for overflow
- Memory allocations for bounds

### Security Commands
```bash
# Scan for secrets
git-secrets --scan

# Security audit
gosec ./...

# Dependency vulnerabilities
go list -json -m all | nancy sleuth

# Check for hardcoded credentials
grep -r "password\|secret\|key" --exclude-dir=.git .
```

## 📁 Production Log Management & Operations System

### Production Architecture
The MEV bot uses a comprehensive production-grade log management system with real-time monitoring, analytics, and alerting:

```
logs/
├── archives/                             # Compressed archives with metadata
│   ├── mev_logs_YYYYMMDD_HHMMSS.tar.gz  # Timestamped archives
│   ├── latest_archive.tar.gz             # Symlink to newest archive
│   └── archive_report_YYYYMMDD_HHMMSS.txt # Detailed reports
├── analytics/                            # Real-time analysis & metrics
│   ├── analysis_YYYYMMDD_HHMMSS.json     # Comprehensive log analysis
│   ├── performance_YYYYMMDD_HHMMSS.json  # Performance metrics
│   └── dashboard_YYYYMMDD_HHMMSS.html    # Operations dashboard
├── health/                               # Health monitoring & corruption detection
│   └── health_YYYYMMDD_HHMMSS.json       # Health reports
├── alerts/                               # Alert management
│   └── alert_YYYYMMDD_HHMMSS.json        # Alert records
├── rotated/                              # Rotated log files
│   └── *.log.gz                          # Compressed rotated logs
├── mev_bot.log                           # Main application log
├── mev_bot_errors.log                    # Error-specific logs
├── mev_bot_performance.log               # Performance metrics
└── diagnostics/                          # Diagnostic data and corruption logs
```

### Production Features
- **Real-time Analysis**: Continuous log analysis with health scoring (97.97/100)
- **Performance Monitoring**: System and MEV-specific metrics tracking
- **Corruption Detection**: Automated health checks and integrity validation
- **Multi-channel Alerting**: Email and Slack notifications with thresholds
- **Background Daemon**: Continuous monitoring with configurable intervals
- **Operations Dashboard**: HTML dashboard with live metrics and charts
- **Intelligent Rotation**: Size and time-based log rotation with compression
- **Advanced Archiving**: Metadata-rich archives with system snapshots

### Operational Metrics
Current system status provides:
- **Health Score**: 97.97/100 (Excellent)
- **Error Rate**: 2.03% (Low)
- **Success Rate**: 0.03% (Normal for MEV detection)
- **MEV Opportunities**: 12 detected
- **Events Rejected**: 9,888 (due to parsing fixes)
- **System Load**: 0.84 (Normal)
- **Memory Usage**: 55.4% (Optimal)

### Alert Thresholds
Automated alerts trigger on:
- Error rate > 10%
- Health score < 80
- Parsing failures > 50
- Zero address issues > 100
- CPU usage > 80%
- Memory usage > 85%
- Disk usage > 90%

### Configuration
Customize behavior via `config/log-manager.conf`:
- Retention policies and size limits
- Alert thresholds and notification channels
- Monitoring intervals and daemon settings
- Compression levels and archive policies

- make sure we keep `TODO_AUDIT_FIX.md` updated at all times