refactor: move all remaining files to orig/ directory
Completed clean root directory structure: - Root now contains only: .git, .env, docs/, orig/ - Moved all remaining files and directories to orig/: - Config files (.claude, .dockerignore, .drone.yml, etc.) - All .env variants (except active .env) - Git config (.gitconfig, .github, .gitignore, etc.) - Tool configs (.golangci.yml, .revive.toml, etc.) - Documentation (*.md files, @prompts) - Build files (Dockerfiles, Makefile, go.mod, go.sum) - Docker compose files - All source directories (scripts, tests, tools, etc.) - Runtime directories (logs, monitoring, reports) - Dependency files (node_modules, lib, cache) - Special files (--delete) - Removed empty runtime directories (bin/, data/) V2 structure is now clean: - docs/planning/ - V2 planning documents - orig/ - Complete V1 codebase preserved - .env - Active environment config (not in git) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Administrator
320
orig/.qwen/FILE_REORGANIZATION_PLAN.md
Normal file
320
orig/.qwen/FILE_REORGANIZATION_PLAN.md
Normal file
@@ -0,0 +1,320 @@
|
||||
# 📁 MEV Bot File Reorganization Plan
|
||||
|
||||
## Current Problem: Chaotic File Structure
|
||||
|
||||
The current arbitrum package has scattered functionality with confusing prefixes:
|
||||
- `enhanced_parser.go`, `enhanced_example.go`, `enhanced_types.go`
|
||||
- `optimized_*.go` files with duplicated functionality
|
||||
- Broken files moved to `.broken` extensions
|
||||
- Type conflicts between packages
|
||||
- Missing interfaces and poor separation of concerns
|
||||
|
||||
## 🎯 **PROPOSED NEW STRUCTURE**
|
||||
|
||||
### **Phase 1: Core Package Reorganization**
|
||||
|
||||
```
|
||||
pkg/arbitrum/
|
||||
├── core/
|
||||
│ ├── types.go # Core types (Protocol, PoolInfo, etc.)
|
||||
│ ├── interfaces.go # All interfaces in one place
|
||||
│ └── constants.go # Protocol constants and addresses
|
||||
├── parsing/
|
||||
│ ├── parser.go # Main parser interface
|
||||
│ ├── dex_parser.go # DEX-specific parsing logic
|
||||
│ ├── event_parser.go # Event parsing (merge enhanced_parser)
|
||||
│ └── transaction_parser.go # Transaction parsing
|
||||
├── discovery/
|
||||
│ ├── market_discovery.go # Market discovery (existing)
|
||||
│ ├── pool_discovery.go # Pool discovery logic
|
||||
│ └── factory_discovery.go # Factory contract discovery
|
||||
├── monitoring/
|
||||
│ ├── event_monitor.go # Event monitoring (existing)
|
||||
│ ├── sequencer_monitor.go # Sequencer-specific monitoring
|
||||
│ └── metrics.go # Monitoring metrics
|
||||
├── pipeline/
|
||||
│ ├── swap_pipeline.go # Swap processing pipeline (existing)
|
||||
│ ├── mev_pipeline.go # MEV opportunity pipeline
|
||||
│ └── arbitrage_pipeline.go # Arbitrage detection pipeline
|
||||
├── cache/
|
||||
│ ├── pool_cache.go # Pool caching (existing)
|
||||
│ ├── token_cache.go # Token metadata caching
|
||||
│ └── price_cache.go # Price data caching
|
||||
└── integration/
|
||||
├── examples.go # Integration examples (merge enhanced_example)
|
||||
├── guides.go # Integration guides
|
||||
└── testing.go # Integration testing helpers
|
||||
```
|
||||
|
||||
### **Phase 2: Merge Enhanced/Optimized Functionality**
|
||||
|
||||
#### **2.1 Enhanced Parser → parsing/event_parser.go**
|
||||
- Merge `enhanced_parser.go` functionality into `parsing/event_parser.go`
|
||||
- Keep advanced features: comprehensive DEX support, price enrichment
|
||||
- Maintain backwards compatibility with existing parser interface
|
||||
|
||||
#### **2.2 Protocol Parsers → parsing/dex_parser.go**
|
||||
- Consolidate `protocol_parsers.go` into `parsing/dex_parser.go`
|
||||
- Factory pattern for different DEX protocols
|
||||
- Clean interface for adding new protocols
|
||||
|
||||
#### **2.3 Type Definitions → core/types.go**
|
||||
- Single source of truth for all types
|
||||
- Remove duplicate definitions from `enhanced_types.go`
|
||||
- Proper package organization for types
|
||||
|
||||
#### **2.4 Integration Examples → integration/examples.go**
|
||||
- Merge `enhanced_example.go` and `integration_guide.go`
|
||||
- Comprehensive examples for different use cases
|
||||
- Clear documentation and best practices
|
||||
|
||||
### **Phase 3: Interface Standardization**
|
||||
|
||||
#### **3.1 Create core/interfaces.go**
|
||||
```go
|
||||
package core
|
||||
|
||||
// ParserInterface - unified parser interface
|
||||
type ParserInterface interface {
|
||||
ParseTransaction(tx *types.Transaction) (*ParseResult, error)
|
||||
ParseEvent(log types.Log) (*EventResult, error)
|
||||
GetSupportedProtocols() []Protocol
|
||||
}
|
||||
|
||||
// DiscoveryInterface - market discovery interface
|
||||
type DiscoveryInterface interface {
|
||||
DiscoverPools(tokenA, tokenB common.Address) ([]*PoolInfo, error)
|
||||
GetPoolInfo(address common.Address) (*PoolInfo, error)
|
||||
UpdatePoolState(update *PoolStateUpdate) error
|
||||
}
|
||||
|
||||
// CacheInterface - caching interface
|
||||
type CacheInterface interface {
|
||||
Get(key string) (interface{}, bool)
|
||||
Set(key string, value interface{}, ttl time.Duration)
|
||||
Invalidate(key string)
|
||||
}
|
||||
```
|
||||
|
||||
#### **3.2 Backwards Compatibility**
|
||||
- Keep existing public APIs working
|
||||
- Add deprecation warnings for old interfaces
|
||||
- Gradual migration path for consumers
|
||||
|
||||
### **Phase 4: File Naming Convention**
|
||||
|
||||
#### **4.1 Clear Naming Rules**
|
||||
```
|
||||
✅ GOOD:
|
||||
- parser.go (main functionality)
|
||||
- event_monitor.go (specific purpose)
|
||||
- pool_cache.go (clear responsibility)
|
||||
|
||||
❌ BAD:
|
||||
- enhanced_parser.go (ambiguous prefix)
|
||||
- optimized_scanner.go (unclear what's optimized)
|
||||
- new_types.go (temporary-sounding name)
|
||||
```
|
||||
|
||||
#### **4.2 Package-Level Organization**
|
||||
- Each directory has a clear, single responsibility
|
||||
- Interfaces defined at package level
|
||||
- Implementation details internal to package
|
||||
|
||||
## 🔧 **IMPLEMENTATION STRATEGY**
|
||||
|
||||
### **Step 1: Create New Structure (Non-Breaking)**
|
||||
```bash
|
||||
# Create new directory structure
|
||||
mkdir -p pkg/arbitrum/{core,parsing,discovery,monitoring,pipeline,cache,integration}
|
||||
|
||||
# Move and rename files systematically
|
||||
mv pkg/arbitrum/event_monitor.go pkg/arbitrum/monitoring/
|
||||
mv pkg/arbitrum/pool_cache.go pkg/arbitrum/cache/
|
||||
mv pkg/arbitrum/swap_pipeline.go pkg/arbitrum/pipeline/
|
||||
```
|
||||
|
||||
### **Step 2: Merge Enhanced Functionality**
|
||||
```bash
|
||||
# Merge enhanced files into appropriate locations
|
||||
# enhanced_parser.go → parsing/event_parser.go
|
||||
# enhanced_example.go → integration/examples.go
|
||||
# enhanced_types.go → core/types.go (merge with existing)
|
||||
```
|
||||
|
||||
### **Step 3: Fix Imports and Dependencies**
|
||||
```bash
|
||||
# Update all import statements
|
||||
# Fix interface implementations
|
||||
# Resolve type conflicts
|
||||
```
|
||||
|
||||
### **Step 4: Update Consumers**
|
||||
```bash
|
||||
# Update pkg/monitor/ to use new structure
|
||||
# Update pkg/market/ to use new interfaces
|
||||
# Fix any external dependencies
|
||||
```
|
||||
|
||||
## 📋 **SPECIFIC MERGE OPERATIONS**
|
||||
|
||||
### **Enhanced Parser Merge**
|
||||
```go
|
||||
// FROM: enhanced_parser.go (broken)
|
||||
// TO: parsing/event_parser.go
|
||||
|
||||
type EventParser struct {
|
||||
// Merge: EnhancedDEXParser functionality
|
||||
client *rpc.Client
|
||||
logger *logger.Logger
|
||||
oracle *oracle.PriceOracle
|
||||
protocolParsers map[core.Protocol]ParserInterface
|
||||
|
||||
// Keep: Enhanced features
|
||||
enrichmentService *EventEnrichmentService
|
||||
tokenMetadata *TokenMetadataService
|
||||
metrics *core.ParserMetrics
|
||||
}
|
||||
```
|
||||
|
||||
### **Protocol Parsers Merge**
|
||||
```go
|
||||
// FROM: protocol_parsers.go
|
||||
// TO: parsing/dex_parser.go
|
||||
|
||||
type DEXParserFactory struct {
|
||||
client *rpc.Client
|
||||
logger *logger.Logger
|
||||
}
|
||||
|
||||
func (f *DEXParserFactory) CreateParser(protocol core.Protocol) core.ParserInterface {
|
||||
// Factory method for all supported DEX protocols
|
||||
}
|
||||
```
|
||||
|
||||
### **Types Consolidation**
|
||||
```go
|
||||
// FROM: enhanced_types.go, existing types
|
||||
// TO: core/types.go
|
||||
|
||||
package core
|
||||
|
||||
// Single source of truth for all types
|
||||
type Protocol string
|
||||
type PoolInfo struct { /* unified definition */ }
|
||||
type ParseResult struct { /* unified definition */ }
|
||||
// etc.
|
||||
```
|
||||
|
||||
## 🎯 **BENEFITS OF REORGANIZATION**
|
||||
|
||||
### **1. Developer Experience**
|
||||
- **Clear file discovery**: Find functionality by logical grouping
|
||||
- **Reduced confusion**: No more `enhanced_` vs `optimized_` guessing
|
||||
- **Better IDE support**: Proper package structure for navigation
|
||||
|
||||
### **2. Maintainability**
|
||||
- **Single responsibility**: Each file has one clear purpose
|
||||
- **Easier testing**: Package-level testing strategies
|
||||
- **Simpler debugging**: Clear call paths through organized structure
|
||||
|
||||
### **3. Backwards Compatibility**
|
||||
- **Gradual migration**: Old imports still work during transition
|
||||
- **Deprecation warnings**: Clear upgrade path for consumers
|
||||
- **Version compatibility**: Maintain API contracts
|
||||
|
||||
### **4. Performance**
|
||||
- **Reduced import cycles**: Clear dependency hierarchy
|
||||
- **Better compilation**: Smaller packages compile faster
|
||||
- **Optimized loading**: Import only what you need
|
||||
|
||||
## 🚀 **IMPLEMENTATION TIMELINE**
|
||||
|
||||
### **Phase 1: Structure Setup (1-2 hours)**
|
||||
- Create new directory structure
|
||||
- Move existing files to appropriate locations
|
||||
- Basic import fixes
|
||||
|
||||
### **Phase 2: Enhanced Merging (2-3 hours)**
|
||||
- Merge enhanced_parser.go into parsing/event_parser.go
|
||||
- Consolidate types into core/types.go
|
||||
- Merge examples into integration/examples.go
|
||||
|
||||
### **Phase 3: Interface Cleanup (1-2 hours)**
|
||||
- Create unified interfaces in core/interfaces.go
|
||||
- Fix all import statements
|
||||
- Resolve remaining type conflicts
|
||||
|
||||
### **Phase 4: Testing & Validation (1 hour)**
|
||||
- Ensure all packages compile
|
||||
- Run existing tests
|
||||
- Validate functionality preservation
|
||||
|
||||
## ⚠️ **CRITICAL SUCCESS FACTORS**
|
||||
|
||||
### **1. Preserve Functionality**
|
||||
- All existing features must continue working
|
||||
- No performance regressions
|
||||
- Maintain API compatibility
|
||||
|
||||
### **2. Fix Build Issues First**
|
||||
- Complete current build error fixes
|
||||
- Establish baseline working state
|
||||
- Then reorganize incrementally
|
||||
|
||||
### **3. Documentation Updates**
|
||||
- Update CLAUDE.md with new structure
|
||||
- Create migration guide for developers
|
||||
- Update import examples
|
||||
|
||||
## 🎯 **FINAL STRUCTURE PREVIEW**
|
||||
|
||||
```
|
||||
pkg/arbitrum/
|
||||
├── arbitrum.go # Main package interface (backwards compat)
|
||||
├── core/ # Core types and interfaces
|
||||
├── parsing/ # All parsing functionality
|
||||
├── discovery/ # Market and pool discovery
|
||||
├── monitoring/ # Event and sequencer monitoring
|
||||
├── pipeline/ # Processing pipelines
|
||||
├── cache/ # Caching implementations
|
||||
└── integration/ # Examples and guides
|
||||
```
|
||||
|
||||
This organization provides:
|
||||
- ✅ **Clear separation of concerns**
|
||||
- ✅ **Backwards compatibility**
|
||||
- ✅ **Easy feature discovery**
|
||||
- ✅ **Better testing structure**
|
||||
- ✅ **Reduced type conflicts**
|
||||
- ✅ **Simplified maintenance**
|
||||
|
||||
---
|
||||
|
||||
## 📊 **ADDITIONAL OPTIMIZATIONS COMPLETED**
|
||||
|
||||
### Mathematical Function Optimizations
|
||||
While working on the arbitrum package reorganization, additional mathematical optimizations have been completed in the `pkg/uniswap/` and `pkg/math/` packages:
|
||||
|
||||
- **SqrtPriceX96ToPriceCached**: 24% faster than original (1406 ns/op → 1060 ns/op)
|
||||
- **PriceToSqrtPriceX96Cached**: 19% faster than original (1324 ns/op → 1072 ns/op)
|
||||
- **Memory Allocations**: Reduced by 20-33% across all optimized functions
|
||||
- **Caching Strategy**: Precomputing expensive constants (`2^96`, `2^192`) for improved performance
|
||||
|
||||
These optimizations will significantly improve the performance of the MEV bot, especially during high-frequency arbitrage detection where these mathematical functions are called repeatedly.
|
||||
|
||||
### Integration with Reorganization Plan
|
||||
The mathematical optimizations are independent of the arbitrum package reorganization and can be implemented in parallel. The new structure should accommodate these optimizations by potentially adding:
|
||||
```
|
||||
pkg/uniswap/
|
||||
├── cached.go # Cached mathematical functions
|
||||
├── pricing.go # Original pricing functions
|
||||
├── pricing_test.go # Pricing function tests
|
||||
├── cached_test.go # Cached function tests
|
||||
├── pricing_bench_test.go # Pricing benchmarks
|
||||
└── cached_bench_test.go # Cached benchmarks
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Ready to implement?** This reorganization will transform the chaotic file structure into a clean, maintainable architecture while preserving all functionality and fixing the current build issues.
|
||||
Reference in New Issue
Block a user