copper-tone-tech/mev-beta
1
0
Fork 0
Code Issues Pull Requests 7 Actions Packages Projects Releases Wiki Activity

feat: create v2-prep branch with comprehensive planning

Browse Source 
Restructured project for V2 refactor:

**Structure Changes:**
- Moved all V1 code to orig/ folder (preserved with git mv)
- Created docs/planning/ directory
- Added orig/README_V1.md explaining V1 preservation

**Planning Documents:**
- 00_V2_MASTER_PLAN.md: Complete architecture overview
  - Executive summary of critical V1 issues
  - High-level component architecture diagrams
  - 5-phase implementation roadmap
  - Success metrics and risk mitigation

- 07_TASK_BREAKDOWN.md: Atomic task breakdown
  - 99+ hours of detailed tasks
  - Every task < 2 hours (atomic)
  - Clear dependencies and success criteria
  - Organized by implementation phase

**V2 Key Improvements:**
- Per-exchange parsers (factory pattern)
- Multi-layer strict validation
- Multi-index pool cache
- Background validation pipeline
- Comprehensive observability

**Critical Issues Addressed:**
- Zero address tokens (strict validation + cache enrichment)
- Parsing accuracy (protocol-specific parsers)
- No audit trail (background validation channel)
- Inefficient lookups (multi-index cache)
- Stats disconnection (event-driven metrics)

Next Steps:
1. Review planning documents
2. Begin Phase 1: Foundation (P1-001 through P1-010)
3. Implement parsers in Phase 2
4. Build cache system in Phase 3
5. Add validation pipeline in Phase 4
6. Migrate and test in Phase 5

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Administrator
2025-11-10 10:14:26 +01:00
parent 1773daffe7
commit 803de231ba
411 changed files with 20390 additions and 8680 deletions
Download Patch File Download Diff File Expand all files Collapse all files

431
orig/config/providers_runtime.yaml.backup Normal file
View File

@@ -0,0 +1,431 @@
# RPC Provider Configuration for MEV Bot
# Supports separate provider pools for read-only, execution, and testing operations
# Provider Pool Configuration
provider_pools:
# Read-Only Pool: Optimized for high-frequency data fetching and real-time events
read_only:
strategy: "websocket_preferred" # Prefer WebSocket for real-time data
max_concurrent_connections: 5
health_check_interval: "30s"
failover_enabled: true
providers: ["chainstack_arbitrum", "quicknode_arbitrum", "alchemy_arbitrum"]
# Execution Pool: Dedicated for transaction submission with higher security
execution:
strategy: "reliability_first" # Prefer most reliable providers
max_concurrent_connections: 2
health_check_interval: "15s"
failover_enabled: true
providers: ["chainstack_arbitrum", "quicknode_arbitrum"]
# Testing Pool: Anvil forked instances for safe dry-run testing
testing:
strategy: "anvil_preferred" # Use Anvil instances first
max_concurrent_connections: 3
health_check_interval: "60s"
failover_enabled: true
providers: ["anvil_local_fork", "anvil_staging_fork"]
# Individual Provider Definitions
providers:
# Chainstack (current primary)
- name: "chainstack_arbitrum"
type: "arbitrum"
http_endpoint: "https://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57"
ws_endpoint: "wss://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57"
priority: 1
rate_limit:
requests_per_second: 50 # Free plan limit
burst: 100
timeout: "30s"
retry_delay: "1s"
max_retries: 3
features:
- "archive_data"
- "websocket"
- "trace_api"
health_check:
enabled: true
interval: "30s"
timeout: "10s"
# Alchemy (backup)
- name: "alchemy_arbitrum"
type: "arbitrum"
http_endpoint: "https://arb-mainnet.g.alchemy.com/v2/YOUR_API_KEY"
ws_endpoint: "wss://arb-mainnet.g.alchemy.com/v2/YOUR_API_KEY"
priority: 2
rate_limit:
requests_per_second: 100 # Free tier: 100 RPS
burst: 200
timeout: "30s"
retry_delay: "2s"
max_retries: 3
features:
- "archive_data"
- "websocket"
- "enhanced_apis"
health_check:
enabled: true
interval: "45s"
timeout: "15s"
# Infura (tertiary)
- name: "infura_arbitrum"
type: "arbitrum"
http_endpoint: "https://arbitrum-mainnet.infura.io/v3/YOUR_PROJECT_ID"
ws_endpoint: "wss://arbitrum-mainnet.infura.io/ws/v3/YOUR_PROJECT_ID"
priority: 3
rate_limit:
requests_per_second: 10 # Free tier: 100k requests/day (~1.15 RPS)
burst: 20
timeout: "30s"
retry_delay: "3s"
max_retries: 2
features:
- "archive_data"
- "websocket"
health_check:
enabled: true
interval: "60s"
timeout: "20s"
# QuickNode (backup)
- name: "quicknode_arbitrum"
type: "arbitrum"
http_endpoint: "https://YOUR_ENDPOINT.arbitrum-mainnet.quiknode.pro/YOUR_TOKEN/"
ws_endpoint: "wss://YOUR_ENDPOINT.arbitrum-mainnet.quiknode.pro/YOUR_TOKEN/"
priority: 4
rate_limit:
requests_per_second: 25 # Free tier: varies
burst: 50
timeout: "30s"
retry_delay: "2s"
max_retries: 3
features:
- "archive_data"
- "websocket"
- "debug_api"
health_check:
enabled: true
interval: "45s"
timeout: "15s"
# Ankr (public endpoint - lowest priority)
- name: "ankr_arbitrum"
type: "arbitrum"
http_endpoint: "https://rpc.ankr.com/arbitrum"
ws_endpoint: "" # No WS support on free tier
priority: 5
rate_limit:
requests_per_second: 5 # Public endpoint - very conservative
burst: 10
timeout: "45s"
retry_delay: "5s"
max_retries: 2
features:
- "basic_rpc"
health_check:
enabled: true
interval: "120s"
timeout: "30s"
# Anvil Local Fork (for testing)
- name: "anvil_local_fork"
type: "anvil_fork"
http_endpoint: "http://127.0.0.1:8545"
ws_endpoint: "ws://127.0.0.1:8545"
priority: 1
rate_limit:
requests_per_second: 1000 # Local instance - very high limits
burst: 2000
timeout: "5s"
retry_delay: "100ms"
max_retries: 2
features:
- "fork_testing"
- "state_snapshots"
- "debug_api"
- "trace_api"
health_check:
enabled: true
interval: "30s"
timeout: "5s"
anvil_config:
fork_url: "https://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57"
chain_id: 31337
port: 8545
block_time: 1
auto_impersonate: true
state_interval: 1000 # Save state every 1000 blocks
# Anvil Staging Fork (for staging tests)
- name: "anvil_staging_fork"
type: "anvil_fork"
http_endpoint: "http://127.0.0.1:8546"
ws_endpoint: "ws://127.0.0.1:8546"
priority: 2
rate_limit:
requests_per_second: 1000
burst: 2000
timeout: "5s"
retry_delay: "100ms"
max_retries: 2
features:
- "fork_testing"
- "state_snapshots"
- "debug_api"
health_check:
enabled: true
interval: "45s"
timeout: "10s"
anvil_config:
fork_url: "https://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57"
chain_id: 31338
port: 8546
block_time: 2
auto_impersonate: true
state_interval: 2000
# Provider rotation and failover settings
rotation:
strategy: "round_robin" # Options: round_robin, weighted, priority_based
health_check_required: true
fallback_enabled: true
retry_failed_after: "300s" # 5 minutes
# Global rate limiting settings
global_limits:
max_concurrent_connections: 10
connection_timeout: "30s"
read_timeout: "45s"
write_timeout: "30s"
idle_timeout: "120s"
# Monitoring and metrics
monitoring:
enabled: true
metrics_interval: "60s"
log_slow_requests: true
slow_request_threshold: "5s"
track_provider_performance: true
# Provider-specific rate limits by plan type
# This allows for easy upgrade when moving to paid plans
rate_limit_profiles:
free_tier:
requests_per_second: 10
burst: 20
daily_limit: 100000
basic_paid:
requests_per_second: 100
burst: 200
daily_limit: 10000000
premium:
requests_per_second: 500
burst: 1000
daily_limit: 100000000
enterprise:
requests_per_second: 2000
burst: 4000
daily_limit: 1000000000
# Provider Comparison Chart - Ranked by MEV Bot Profitability
# Factors: Rate limits, latency, reliability, cost, archive data, MEV-specific features
# Rating: 1-10 (10 = best for MEV operations)
provider_comparison:
# TIER 1: Premium MEV-Optimized Providers
chainstack:
overall_rating: 9.5
mev_suitability: 10 # Excellent for MEV
pros:
- "Highest free tier rate limits (50 RPS)"
- "Low latency infrastructure"
- "Archive data included"
- "WebSocket support"
- "No daily request limits on paid plans"
- "MEV-friendly ToS"
cons:
- "Higher cost for premium plans"
free_tier: { rps: 50, daily: 3000000, cost: "$0" }
paid_plans:
developer: { rps: 100, daily: 30000000, cost: "$49/month" }
startup: { rps: 200, daily: 300000000, cost: "$299/month" }
business: { rps: 500, daily: 1000000000, cost: "$999/month" }
best_for: "High-frequency MEV bots, professional trading"
quicknode:
overall_rating: 9.0
mev_suitability: 9
pros:
- "Excellent performance and reliability"
- "Advanced APIs (trace, debug)"
- "Global edge infrastructure"
- "MEV-specific features"
- "High free tier requests"
cons:
- "Premium pricing"
- "Complex pricing tiers"
free_tier: { rps: 25, daily: 50000000, cost: "$0" }
paid_plans:
discover: { rps: 100, daily: 500000000, cost: "$20/month" }
build: { rps: 200, daily: 2000000000, cost: "$99/month" }
scale: { rps: 500, daily: 10000000000, cost: "$399/month" }
best_for: "Production MEV bots, enterprise applications"
# TIER 2: Mainstream Providers
alchemy:
overall_rating: 8.5
mev_suitability: 8
pros:
- "Excellent documentation and tools"
- "Enhanced APIs and webhooks"
- "Good reliability"
- "Strong developer ecosystem"
- "Archive data access"
cons:
- "Lower free tier rate limits"
- "Can be expensive for high-volume"
free_tier: { rps: 5, daily: 100000, cost: "$0" }
paid_plans:
basic: { rps: 25, daily: 40000000, cost: "$49/month" }
growth: { rps: 100, daily: 150000000, cost: "$199/month" }
scale: { rps: 330, daily: 300000000, cost: "$499/month" }
best_for: "Development and testing, medium-scale operations"
infura:
overall_rating: 7.5
mev_suitability: 7
pros:
- "Reliable and stable"
- "Good free tier daily limits"
- "Wide network support"
- "Enterprise-grade infrastructure"
cons:
- "Lower rate limits"
- "Basic feature set"
- "Higher latency for some regions"
free_tier: { rps: 10, daily: 100000, cost: "$0" }
paid_plans:
developer: { rps: 100, daily: 100000000, cost: "$50/month" }
team: { rps: 200, daily: 400000000, cost: "$225/month" }
growth: { rps: 700, daily: 1500000000, cost: "$1000/month" }
best_for: "Backup provider, basic MEV operations"
# TIER 3: Budget/Backup Options
ankr:
overall_rating: 6.5
mev_suitability: 5
pros:
- "Public endpoints available"
- "No registration required for basic use"
- "Decent reliability"
- "Multi-chain support"
cons:
- "Very low rate limits"
- "No WebSocket on free tier"
- "Limited features"
- "Higher latency"
free_tier: { rps: 5, daily: "unlimited", cost: "$0" }
paid_plans:
premium: { rps: 1500, daily: "unlimited", cost: "$250/month" }
best_for: "Emergency backup, development testing"
llamarpc:
overall_rating: 6.0
mev_suitability: 4
pros:
- "Free public endpoint"
- "No registration required"
- "Open source friendly"
cons:
- "Very low rate limits"
- "Unreliable performance"
- "No SLA or support"
- "Not suitable for production"
free_tier: { rps: 3, daily: "unlimited", cost: "$0" }
best_for: "Development only, emergency fallback"
# MEV-Specific Performance Metrics (based on testing)
mev_performance_rankings:
latency_ranking: # Lower latency = better for MEV
1: "chainstack" # ~15ms average
2: "quicknode" # ~20ms average
3: "alchemy" # ~25ms average
4: "infura" # ~35ms average
5: "ankr" # ~50ms average
reliability_ranking: # Uptime and consistency
1: "quicknode" # 99.9% uptime
2: "chainstack" # 99.8% uptime
3: "alchemy" # 99.7% uptime
4: "infura" # 99.5% uptime
5: "ankr" # 98.5% uptime
cost_efficiency_ranking: # Best value for money
1: "chainstack" # Best free tier
2: "quicknode" # Good paid plans
3: "infura" # Reasonable pricing
4: "alchemy" # Premium pricing
5: "ankr" # Good for basic use
mev_features_ranking: # MEV-specific capabilities
1: "quicknode" # Trace API, debug tools
2: "chainstack" # Archive data, low latency
3: "alchemy" # Enhanced APIs, webhooks
4: "infura" # Basic features
5: "ankr" # Limited features
# Recommended Configuration by Use Case
recommended_configs:
development:
primary: "chainstack" # Good free tier
backup: "alchemy" # Good docs and tools
emergency: "ankr" # Public endpoint
production_low_volume:
primary: "chainstack" # Best free tier
backup: "quicknode" # Reliable paid backup
emergency: "infura" # Stable fallback
production_high_volume:
primary: "quicknode" # Best performance
backup: "chainstack" # Good secondary
emergency: "alchemy" # Enterprise backup
enterprise:
primary: "quicknode" # Premium features
backup: "chainstack" # High performance backup
tertiary: "alchemy" # Additional redundancy
# Known provider rate limits for reference
# Updated as of 2024 - check provider docs for current limits
provider_defaults:
alchemy:
free: { rps: 5, daily: 100000 }
basic: { rps: 25, daily: 40000000 }
growth: { rps: 100, daily: 150000000 }
scale: { rps: 330, daily: 300000000 }
infura:
free: { rps: 10, daily: 100000 }
developer: { rps: 100, daily: 100000000 }
team: { rps: 200, daily: 400000000 }
growth: { rps: 700, daily: 1500000000 }
quicknode:
free: { rps: 25, daily: 50000000 }
discover: { rps: 100, daily: 500000000 }
build: { rps: 200, daily: 2000000000 }
scale: { rps: 500, daily: 10000000000 }
chainstack:
free: { rps: 50, daily: 3000000 }
developer: { rps: 100, daily: 30000000 }
startup: { rps: 200, daily: 300000000 }
business: { rps: 500, daily: 1000000000 }