feat(optimization): add pool detection, price impact validation, and production infrastructure
This commit adds critical production-ready optimizations and infrastructure: New Features: 1. Pool Version Detector - Detects pool versions before calling slot0() - Eliminates ABI unpacking errors from V2 pools - Caches detection results for performance 2. Price Impact Validation System - Comprehensive risk categorization - Three threshold profiles (Conservative, Default, Aggressive) - Automatic trade splitting recommendations - All tests passing (10/10) 3. Flash Loan Execution Architecture - Complete execution flow design - Multi-provider support (Aave, Balancer, Uniswap) - Safety and risk management systems - Transaction signing and dispatch strategies 4. 24-Hour Validation Test Infrastructure - Production testing framework - Comprehensive monitoring with real-time metrics - Automatic report generation - System health tracking 5. Production Deployment Runbook - Complete deployment procedures - Pre-deployment checklist - Configuration templates - Monitoring and rollback procedures Files Added: - pkg/uniswap/pool_detector.go (273 lines) - pkg/validation/price_impact_validator.go (265 lines) - pkg/validation/price_impact_validator_test.go (242 lines) - docs/architecture/flash_loan_execution_architecture.md (808 lines) - docs/PRODUCTION_DEPLOYMENT_RUNBOOK.md (615 lines) - scripts/24h-validation-test.sh (352 lines) Testing: Core functionality tests passing. Stress test showing 867 TPS (below 1000 TPS target - to be investigated) Impact: Ready for 24-hour validation test and production deployment 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Krypto Kajun
615
docs/PRODUCTION_DEPLOYMENT_RUNBOOK.md
Normal file
615
docs/PRODUCTION_DEPLOYMENT_RUNBOOK.md
Normal file
@@ -0,0 +1,615 @@
|
||||
# MEV Bot - Production Deployment Runbook
|
||||
**Version:** 1.0
|
||||
**Last Updated:** October 28, 2025
|
||||
**Audience:** DevOps, Production Engineers
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Pre-Deployment Checklist](#pre-deployment-checklist)
|
||||
2. [Environment Setup](#environment-setup)
|
||||
3. [Configuration](#configuration)
|
||||
4. [Deployment Steps](#deployment-steps)
|
||||
5. [Post-Deployment Validation](#post-deployment-validation)
|
||||
6. [Monitoring & Alerting](#monitoring--alerting)
|
||||
7. [Rollback Procedures](#rollback-procedures)
|
||||
8. [Troubleshooting](#troubleshooting)
|
||||
|
||||
---
|
||||
|
||||
## Pre-Deployment Checklist
|
||||
|
||||
### Code Readiness
|
||||
- [ ] All tests passing (`make test`)
|
||||
- [ ] Security audit completed and issues addressed
|
||||
- [ ] Code review approved
|
||||
- [ ] 24-hour validation test completed successfully
|
||||
- [ ] Performance benchmarks meet targets
|
||||
- [ ] No critical TODOs in codebase
|
||||
|
||||
### Infrastructure Readiness
|
||||
- [ ] RPC endpoints configured and tested
|
||||
- [ ] Private key/wallet funded with gas (minimum 0.1 ETH)
|
||||
- [ ] Monitoring systems operational
|
||||
- [ ] Alert channels configured (Slack, email, PagerDuty)
|
||||
- [ ] Backup RPC endpoints ready
|
||||
- [ ] Database/storage systems ready
|
||||
|
||||
### Team Readiness
|
||||
- [ ] On-call engineer assigned
|
||||
- [ ] Runbook reviewed by team
|
||||
- [ ] Communication channels established
|
||||
- [ ] Rollback plan understood
|
||||
- [ ] Emergency contacts documented
|
||||
|
||||
---
|
||||
|
||||
## Environment Setup
|
||||
|
||||
### System Requirements
|
||||
|
||||
**Minimum:**
|
||||
- CPU: 4 cores
|
||||
- RAM: 8 GB
|
||||
- Disk: 50 GB SSD
|
||||
- Network: 100 Mbps, low latency
|
||||
|
||||
**Recommended (Production):**
|
||||
- CPU: 8 cores
|
||||
- RAM: 16 GB
|
||||
- Disk: 100 GB NVMe SSD
|
||||
- Network: 1 Gbps, < 20ms latency to Arbitrum RPC
|
||||
|
||||
### Dependencies
|
||||
|
||||
```bash
|
||||
# Install Go 1.24+
|
||||
wget https://go.dev/dl/go1.24.linux-amd64.tar.gz
|
||||
sudo tar -C /usr/local -xzf go1.24.linux-amd64.tar.gz
|
||||
export PATH=$PATH:/usr/local/go/bin
|
||||
|
||||
# Verify installation
|
||||
go version # Should show go1.24 or later
|
||||
|
||||
# Install build tools
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y build-essential git curl
|
||||
```
|
||||
|
||||
### Repository Setup
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/your-org/mev-beta.git
|
||||
cd mev-beta
|
||||
|
||||
# Checkout production branch
|
||||
git checkout feature/production-profit-optimization
|
||||
|
||||
# Verify correct branch
|
||||
git log -1 --oneline
|
||||
|
||||
# Install dependencies
|
||||
go mod download
|
||||
go mod verify
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### 1. Environment Variables
|
||||
|
||||
Create `/etc/systemd/system/mev-bot.env`:
|
||||
|
||||
```bash
|
||||
# RPC Configuration
|
||||
ARBITRUM_RPC_ENDPOINT=https://arbitrum-mainnet.core.chainstack.com/YOUR_KEY
|
||||
ARBITRUM_WS_ENDPOINT=wss://arbitrum-mainnet.core.chainstack.com/YOUR_KEY
|
||||
|
||||
# Backup RPC (fallback)
|
||||
BACKUP_RPC_ENDPOINT=https://arb1.arbitrum.io/rpc
|
||||
|
||||
# Application Configuration
|
||||
LOG_LEVEL=info
|
||||
LOG_FORMAT=json
|
||||
LOG_OUTPUT=/var/log/mev-bot/mev_bot.log
|
||||
|
||||
# Metrics & Monitoring
|
||||
METRICS_ENABLED=true
|
||||
METRICS_PORT=9090
|
||||
|
||||
# Security
|
||||
MEV_BOT_ENCRYPTION_KEY=your-32-char-encryption-key-here-minimum-length-required
|
||||
|
||||
# Execution Configuration (IMPORTANT: Set to false for detection-only mode)
|
||||
EXECUTION_ENABLED=false
|
||||
MAX_POSITION_SIZE=1000000000000000000 # 1 ETH in wei
|
||||
MIN_PROFIT_THRESHOLD=50000000000000000 # 0.05 ETH in wei
|
||||
|
||||
# Provider Configuration
|
||||
PROVIDER_CONFIG_PATH=/opt/mev-bot/config/providers_runtime.yaml
|
||||
```
|
||||
|
||||
**CRITICAL:** Never commit `.env` files with real credentials to version control!
|
||||
|
||||
### 2. Provider Configuration
|
||||
|
||||
Edit `config/providers_runtime.yaml`:
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- name: "chainstack-primary"
|
||||
endpoint: "${ARBITRUM_RPC_ENDPOINT}"
|
||||
type: "https"
|
||||
weight: 100
|
||||
timeout: 30s
|
||||
rateLimit: 100
|
||||
|
||||
- name: "chainstack-websocket"
|
||||
endpoint: "${ARBITRUM_WS_ENDPOINT}"
|
||||
type: "wss"
|
||||
weight: 90
|
||||
timeout: 30s
|
||||
rateLimit: 100
|
||||
|
||||
- name: "public-fallback"
|
||||
endpoint: "https://arb1.arbitrum.io/rpc"
|
||||
type: "https"
|
||||
weight: 50
|
||||
timeout: 30s
|
||||
rateLimit: 50
|
||||
|
||||
pooling:
|
||||
maxIdleConnections: 10
|
||||
maxOpenConnections: 50
|
||||
connectionTimeout: 30s
|
||||
idleTimeout: 300s
|
||||
|
||||
retry:
|
||||
maxRetries: 3
|
||||
retryDelay: 1s
|
||||
backoffMultiplier: 2
|
||||
maxBackoff: 8s
|
||||
```
|
||||
|
||||
### 3. Systemd Service Configuration
|
||||
|
||||
Create `/etc/systemd/system/mev-bot.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=MEV Arbitrage Bot
|
||||
After=network.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=mev-bot
|
||||
Group=mev-bot
|
||||
WorkingDirectory=/opt/mev-bot
|
||||
EnvironmentFile=/etc/systemd/system/mev-bot.env
|
||||
|
||||
ExecStart=/opt/mev-bot/bin/mev-bot start
|
||||
ExecReload=/bin/kill -HUP $MAINPID
|
||||
KillMode=process
|
||||
Restart=on-failure
|
||||
RestartSec=10s
|
||||
|
||||
# Resource limits
|
||||
LimitNOFILE=65536
|
||||
MemoryMax=4G
|
||||
CPUQuota=400%
|
||||
|
||||
# Security hardening
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/log/mev-bot /opt/mev-bot/data
|
||||
|
||||
# Logging
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
SyslogIdentifier=mev-bot
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deployment Steps
|
||||
|
||||
### Phase 1: Build & Prepare (10-15 minutes)
|
||||
|
||||
```bash
|
||||
# 1. Build binary
|
||||
cd /opt/mev-bot
|
||||
make build
|
||||
|
||||
# Verify binary
|
||||
./bin/mev-bot --version
|
||||
# Expected: MEV Bot v1.0.0 (or similar)
|
||||
|
||||
# 2. Run tests
|
||||
make test
|
||||
# Ensure all tests pass
|
||||
|
||||
# 3. Check binary size and dependencies
|
||||
ls -lh bin/mev-bot
|
||||
ldd bin/mev-bot # Should show minimal dependencies
|
||||
|
||||
# 4. Create necessary directories
|
||||
sudo mkdir -p /var/log/mev-bot
|
||||
sudo mkdir -p /opt/mev-bot/data
|
||||
sudo chown -R mev-bot:mev-bot /var/log/mev-bot /opt/mev-bot/data
|
||||
|
||||
# 5. Set permissions
|
||||
chmod +x bin/mev-bot
|
||||
chmod 600 /etc/systemd/system/mev-bot.env # Protect sensitive config
|
||||
```
|
||||
|
||||
### Phase 2: Dry Run (5-10 minutes)
|
||||
|
||||
```bash
|
||||
# Run bot in foreground to verify configuration
|
||||
sudo -u mev-bot /opt/mev-bot/bin/mev-bot start &
|
||||
BOT_PID=$!
|
||||
|
||||
# Wait 2 minutes for initialization
|
||||
sleep 120
|
||||
|
||||
# Check if running
|
||||
ps aux | grep mev-bot
|
||||
|
||||
# Check logs for errors
|
||||
tail -100 /var/log/mev-bot/mev_bot.log | grep -i error
|
||||
|
||||
# Verify RPC connection
|
||||
tail -100 /var/log/mev-bot/mev_bot.log | grep -i "connected"
|
||||
|
||||
# Stop dry run
|
||||
kill $BOT_PID
|
||||
```
|
||||
|
||||
### Phase 3: Production Start (5 minutes)
|
||||
|
||||
```bash
|
||||
# 1. Reload systemd
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
# 2. Enable service (start on boot)
|
||||
sudo systemctl enable mev-bot
|
||||
|
||||
# 3. Start service
|
||||
sudo systemctl start mev-bot
|
||||
|
||||
# 4. Verify status
|
||||
sudo systemctl status mev-bot
|
||||
# Expected: active (running)
|
||||
|
||||
# 5. Check logs
|
||||
sudo journalctl -u mev-bot -f --lines=50
|
||||
|
||||
# 6. Wait for initialization (30-60 seconds)
|
||||
sleep 60
|
||||
|
||||
# 7. Verify healthy operation
|
||||
curl -s http://localhost:9090/health/live | jq .
|
||||
# Expected: {"status": "healthy"}
|
||||
```
|
||||
|
||||
### Phase 4: Validation (15-30 minutes)
|
||||
|
||||
```bash
|
||||
# 1. Monitor for opportunities
|
||||
tail -f /var/log/mev-bot/mev_bot.log | grep "ARBITRAGE OPPORTUNITY"
|
||||
|
||||
# 2. Check metrics endpoint
|
||||
curl -s http://localhost:9090/metrics | grep mev_
|
||||
|
||||
# 3. Verify cache performance
|
||||
tail -100 /var/log/mev-bot/mev_bot.log | grep "cache metrics"
|
||||
# Look for hit rate 75-85%
|
||||
|
||||
# 4. Check for errors
|
||||
sudo journalctl -u mev-bot --since "10 minutes ago" | grep ERROR
|
||||
# Should have minimal errors
|
||||
|
||||
# 5. Monitor resource usage
|
||||
htop # Check CPU and memory
|
||||
# CPU should be 50-80%, Memory < 2GB
|
||||
|
||||
# 6. Test failover (optional)
|
||||
# Temporarily block primary RPC, verify fallback works
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Post-Deployment Validation
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Liveness probe (should return 200)
|
||||
curl -f http://localhost:9090/health/live || echo "LIVENESS FAILED"
|
||||
|
||||
# Readiness probe (should return 200)
|
||||
curl -f http://localhost:9090/health/ready || echo "READINESS FAILED"
|
||||
|
||||
# Startup probe (should return 200 after initialization)
|
||||
curl -f http://localhost:9090/health/startup || echo "STARTUP FAILED"
|
||||
```
|
||||
|
||||
### Performance Metrics
|
||||
|
||||
```bash
|
||||
# Check Prometheus metrics
|
||||
curl -s http://localhost:9090/metrics | grep -E "mev_(opportunities|executions|profit)"
|
||||
|
||||
# Expected metrics:
|
||||
# - mev_opportunities_detected{} <number>
|
||||
# - mev_opportunities_profitable{} <number>
|
||||
# - mev_cache_hit_rate{} 0.75-0.85
|
||||
# - mev_rpc_calls_total{} <number>
|
||||
```
|
||||
|
||||
### Log Analysis
|
||||
|
||||
```bash
|
||||
# Analyze last hour of logs
|
||||
./scripts/log-manager.sh analyze
|
||||
|
||||
# Check health score (target: > 90)
|
||||
./scripts/log-manager.sh health
|
||||
|
||||
# Expected output:
|
||||
# Health Score: 95.5/100 (Excellent)
|
||||
# Error Rate: < 5%
|
||||
# Cache Hit Rate: 75-85%
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitoring & Alerting
|
||||
|
||||
### Key Metrics to Monitor
|
||||
|
||||
| Metric | Threshold | Action |
|
||||
|--------|-----------|--------|
|
||||
| CPU Usage | > 90% | Scale up or investigate |
|
||||
| Memory Usage | > 85% | Potential memory leak |
|
||||
| Error Rate | > 10% | Check logs, may need rollback |
|
||||
| RPC Failures | > 5/min | Check RPC provider |
|
||||
| Opportunities/hour | < 1 | May indicate detection issue |
|
||||
| Cache Hit Rate | < 70% | Review cache configuration |
|
||||
|
||||
### Alert Configuration
|
||||
|
||||
**Slack Webhook** (edit in `config/alerts.yaml`):
|
||||
```yaml
|
||||
alerts:
|
||||
slack:
|
||||
enabled: true
|
||||
webhook_url: "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
|
||||
channel: "#mev-bot-alerts"
|
||||
|
||||
thresholds:
|
||||
error_rate: 0.10 # 10%
|
||||
cpu_usage: 0.90 # 90%
|
||||
memory_usage: 0.85 # 85%
|
||||
min_opportunities_per_hour: 1
|
||||
```
|
||||
|
||||
### Monitoring Commands
|
||||
|
||||
```bash
|
||||
# Real-time monitoring
|
||||
watch -n 5 'systemctl status mev-bot && curl -s http://localhost:9090/metrics | grep mev_'
|
||||
|
||||
# Start monitoring daemon (background)
|
||||
./scripts/log-manager.sh start-daemon
|
||||
|
||||
# View operations dashboard
|
||||
./scripts/log-manager.sh dashboard
|
||||
# Opens HTML dashboard in browser
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rollback Procedures
|
||||
|
||||
### Quick Rollback (< 5 minutes)
|
||||
|
||||
```bash
|
||||
# 1. Stop current version
|
||||
sudo systemctl stop mev-bot
|
||||
|
||||
# 2. Restore previous binary
|
||||
sudo cp /opt/mev-bot/bin/mev-bot.backup /opt/mev-bot/bin/mev-bot
|
||||
|
||||
# 3. Restart service
|
||||
sudo systemctl start mev-bot
|
||||
|
||||
# 4. Verify rollback
|
||||
sudo systemctl status mev-bot
|
||||
tail -100 /var/log/mev-bot/mev_bot.log
|
||||
```
|
||||
|
||||
### Full Rollback (< 15 minutes)
|
||||
|
||||
```bash
|
||||
# 1. Stop service
|
||||
sudo systemctl stop mev-bot
|
||||
|
||||
# 2. Checkout previous version
|
||||
cd /opt/mev-bot
|
||||
git fetch
|
||||
git checkout <previous-commit-hash>
|
||||
|
||||
# 3. Rebuild
|
||||
make build
|
||||
|
||||
# 4. Restart service
|
||||
sudo systemctl start mev-bot
|
||||
|
||||
# 5. Validate
|
||||
curl http://localhost:9090/health/live
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Issue: Bot fails to start
|
||||
|
||||
**Symptoms:**
|
||||
```
|
||||
systemctl status mev-bot
|
||||
● mev-bot.service - MEV Arbitrage Bot
|
||||
Loaded: loaded
|
||||
Active: failed (Result: exit-code)
|
||||
```
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Check logs
|
||||
sudo journalctl -u mev-bot -n 100 --no-pager
|
||||
|
||||
# Common causes:
|
||||
# 1. Missing environment variables
|
||||
# 2. Invalid RPC endpoint
|
||||
# 3. Permission issues
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Verify environment file
|
||||
cat /etc/systemd/system/mev-bot.env
|
||||
|
||||
# Test RPC connection manually
|
||||
curl -X POST -H "Content-Type: application/json" \
|
||||
--data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
|
||||
$ARBITRUM_RPC_ENDPOINT
|
||||
|
||||
# Fix permissions
|
||||
sudo chown -R mev-bot:mev-bot /opt/mev-bot
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Issue: High error rate
|
||||
|
||||
**Symptoms:**
|
||||
```
|
||||
[ERROR] Failed to fetch pool state
|
||||
[ERROR] RPC call failed
|
||||
[ERROR] 429 Too Many Requests
|
||||
```
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Check error rate
|
||||
./scripts/log-manager.sh analyze | grep "Error Rate"
|
||||
|
||||
# Check RPC provider status
|
||||
curl -s $ARBITRUM_RPC_ENDPOINT
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# 1. Enable backup RPC endpoint in config
|
||||
# 2. Reduce rate limits
|
||||
# 3. Contact RPC provider
|
||||
# 4. Switch to different provider
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Issue: No opportunities detected
|
||||
|
||||
**Symptoms:**
|
||||
```
|
||||
Blocks processed: 10000
|
||||
Opportunities detected: 0
|
||||
```
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Check if events are being detected
|
||||
tail -100 /var/log/mev-bot/mev_bot.log | grep "processing.*event"
|
||||
|
||||
# Check profit thresholds
|
||||
grep MIN_PROFIT_THRESHOLD /etc/systemd/system/mev-bot.env
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# 1. Lower MIN_PROFIT_THRESHOLD (carefully!)
|
||||
# 2. Check market conditions (volatility)
|
||||
# 3. Verify DEX integrations working
|
||||
# 4. Review price impact thresholds
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Issue: Memory leak
|
||||
|
||||
**Symptoms:**
|
||||
```
|
||||
Memory usage increasing over time
|
||||
OOM killer may terminate process
|
||||
```
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Monitor memory over time
|
||||
watch -n 10 'ps aux | grep mev-bot | grep -v grep'
|
||||
|
||||
# Generate heap profile
|
||||
curl http://localhost:9090/debug/pprof/heap > heap.prof
|
||||
go tool pprof heap.prof
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# 1. Restart service (temporary fix)
|
||||
sudo systemctl restart mev-bot
|
||||
|
||||
# 2. Investigate with profiler
|
||||
# 3. Check for goroutine leaks
|
||||
curl http://localhost:9090/debug/pprof/goroutine?debug=1
|
||||
|
||||
# 4. May need code fix and redeploy
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Emergency Contacts
|
||||
|
||||
| Role | Name | Contact | Availability |
|
||||
|------|------|---------|--------------|
|
||||
| On-Call Engineer | TBD | +1-XXX-XXX-XXXX | 24/7 |
|
||||
| DevOps Lead | TBD | Slack: @devops | Business hours |
|
||||
| Product Owner | TBD | Email: product@company.com | Business hours |
|
||||
|
||||
## Change Log
|
||||
|
||||
| Date | Version | Changes | Author |
|
||||
|------|---------|---------|--------|
|
||||
| 2025-10-28 | 1.0 | Initial runbook | Claude Code |
|
||||
|
||||
---
|
||||
|
||||
**END OF RUNBOOK**
|
||||
|
||||
**Remember:**
|
||||
1. Always test in staging first
|
||||
2. Have rollback plan ready
|
||||
3. Monitor closely after deployment
|
||||
4. Document any issues encountered
|
||||
5. Keep this runbook updated
|
||||
808
docs/architecture/flash_loan_execution_architecture.md
Normal file
808
docs/architecture/flash_loan_execution_architecture.md
Normal file
@@ -0,0 +1,808 @@
|
||||
# Flash Loan Execution Architecture
|
||||
**Version:** 1.0
|
||||
**Date:** October 28, 2025
|
||||
**Status:** Design Document
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document outlines the comprehensive architecture for flash loan-based arbitrage execution in the MEV bot. The system supports multiple flash loan providers (Aave, Balancer, Uniswap), implements robust safety checks, and handles the complete lifecycle from opportunity detection to profit realization.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [System Overview](#system-overview)
|
||||
2. [Architecture Components](#architecture-components)
|
||||
3. [Execution Flow](#execution-flow)
|
||||
4. [Provider Implementations](#provider-implementations)
|
||||
5. [Safety & Risk Management](#safety--risk-management)
|
||||
6. [Transaction Signing & Dispatch](#transaction-signing--dispatch)
|
||||
7. [Error Handling & Recovery](#error-handling--recovery)
|
||||
8. [Monitoring & Analytics](#monitoring--analytics)
|
||||
|
||||
---
|
||||
|
||||
## System Overview
|
||||
|
||||
### Goals
|
||||
|
||||
- **Capital Efficiency**: Execute arbitrage with zero upfront capital using flash loans
|
||||
- **Safety First**: Comprehensive validation and risk management at every step
|
||||
- **Multi-Provider Support**: Use the best flash loan provider for each opportunity
|
||||
- **Production Ready**: Handle real-world edge cases, errors, and race conditions
|
||||
|
||||
### High-Level Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Opportunity Detection Layer │
|
||||
│ (Market Scanner, Price Feed, Arbitrage Detector) │
|
||||
└──────────────────┬──────────────────────────────────────┘
|
||||
│ Opportunities
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Opportunity Validation & Ranking │
|
||||
│ (Profit Calculator, Risk Assessor, Price Impact) │
|
||||
└──────────────────┬──────────────────────────────────────┘
|
||||
│ Validated Opportunities
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Flash Loan Provider Selection │
|
||||
│ (Aave, Balancer, Uniswap Flash Swap Selector) │
|
||||
└──────────────────┬──────────────────────────────────────┘
|
||||
│ Provider + Execution Plan
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Transaction Builder & Signer │
|
||||
│ (Calldata Encoder, Gas Estimator, Nonce Manager) │
|
||||
└──────────────────┬──────────────────────────────────────┘
|
||||
│ Signed Transaction
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Transaction Dispatcher │
|
||||
│ (Mempool Broadcaster, Flashbots Relay, Private RPC) │
|
||||
└──────────────────┬──────────────────────────────────────┘
|
||||
│ Transaction Hash
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Execution Monitor & Confirmation │
|
||||
│ (Receipt Waiter, Event Parser, Profit Calculator) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture Components
|
||||
|
||||
### 1. Flash Loan Provider Interface
|
||||
|
||||
All flash loan providers implement this common interface:
|
||||
|
||||
```go
|
||||
type FlashLoanProvider interface {
|
||||
// Execute flash loan with given opportunity
|
||||
ExecuteFlashLoan(ctx context.Context, opp *ArbitrageOpportunity, config *ExecutionConfig) (*ExecutionResult, error)
|
||||
|
||||
// Get maximum borrowable amount for token
|
||||
GetMaxLoanAmount(ctx context.Context, token common.Address) (*big.Int, error)
|
||||
|
||||
// Calculate flash loan fee
|
||||
GetFee(ctx context.Context, amount *big.Int) (*big.Int, error)
|
||||
|
||||
// Check if provider supports token
|
||||
SupportsToken(token common.Address) bool
|
||||
|
||||
// Get provider name
|
||||
Name() string
|
||||
|
||||
// Get provider priority (lower = higher priority)
|
||||
Priority() int
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Flash Loan Orchestrator
|
||||
|
||||
Central coordinator that:
|
||||
- Receives validated arbitrage opportunities
|
||||
- Selects optimal flash loan provider
|
||||
- Manages execution queue and priority
|
||||
- Handles concurrent execution limits
|
||||
- Tracks execution state and history
|
||||
|
||||
```go
|
||||
type FlashLoanOrchestrator struct {
|
||||
providers []FlashLoanProvider
|
||||
executionQueue *PriorityQueue
|
||||
executionLimiter *ConcurrencyLimiter
|
||||
stateTracker *ExecutionStateTracker
|
||||
metricsCollector *MetricsCollector
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Transaction Builder
|
||||
|
||||
Constructs and signs transactions for flash loan execution:
|
||||
|
||||
```go
|
||||
type TransactionBuilder struct {
|
||||
client *ethclient.Client
|
||||
keyManager *security.KeyManager
|
||||
nonceManager *arbitrage.NonceManager
|
||||
gasEstimator *arbitrum.L2GasEstimator
|
||||
|
||||
// Build transaction calldata
|
||||
BuildCalldata(opp *ArbitrageOpportunity, provider FlashLoanProvider) ([]byte, error)
|
||||
|
||||
// Estimate gas for transaction
|
||||
EstimateGas(tx *types.Transaction) (uint64, error)
|
||||
|
||||
// Sign transaction
|
||||
SignTransaction(tx *types.Transaction) (*types.Transaction, error)
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Transaction Dispatcher
|
||||
|
||||
Sends signed transactions to the network:
|
||||
|
||||
```go
|
||||
type TransactionDispatcher struct {
|
||||
client *ethclient.Client
|
||||
logger *logger.Logger
|
||||
|
||||
// Dispatch modes
|
||||
useFlashbots bool
|
||||
flashbotsRelay string
|
||||
usePrivateRPC bool
|
||||
privateRPCURL string
|
||||
|
||||
// Dispatch transaction
|
||||
Dispatch(ctx context.Context, tx *types.Transaction, mode DispatchMode) (common.Hash, error)
|
||||
|
||||
// Wait for confirmation
|
||||
WaitForConfirmation(ctx context.Context, txHash common.Hash, confirmations uint64) (*types.Receipt, error)
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Execution Monitor
|
||||
|
||||
Monitors transaction execution and parses results:
|
||||
|
||||
```go
|
||||
type ExecutionMonitor struct {
|
||||
client *ethclient.Client
|
||||
eventParser *events.Parser
|
||||
|
||||
// Monitor execution
|
||||
MonitorExecution(ctx context.Context, txHash common.Hash) (*ExecutionResult, error)
|
||||
|
||||
// Parse profit from receipt
|
||||
ParseProfit(receipt *types.Receipt) (*big.Int, error)
|
||||
|
||||
// Handle reverts
|
||||
ParseRevertReason(receipt *types.Receipt) string
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Execution Flow
|
||||
|
||||
### Step-by-Step Execution Process
|
||||
|
||||
#### Phase 1: Pre-Execution Validation (500ms max)
|
||||
|
||||
```
|
||||
1. Opportunity Received
|
||||
├─ Validate opportunity structure
|
||||
├─ Check price impact thresholds
|
||||
├─ Verify tokens are not blacklisted
|
||||
└─ Calculate expected profit
|
||||
|
||||
2. Provider Selection
|
||||
├─ Check token support across providers
|
||||
├─ Calculate fees for each provider
|
||||
├─ Select provider with lowest cost
|
||||
└─ Verify provider has sufficient liquidity
|
||||
|
||||
3. Risk Assessment
|
||||
├─ Check current gas prices
|
||||
├─ Validate slippage limits
|
||||
├─ Verify position size limits
|
||||
└─ Check daily volume limits
|
||||
|
||||
4. Final Profitability Check
|
||||
├─ Net Profit = Gross Profit - Gas Costs - Flash Loan Fees
|
||||
├─ Reject if Net Profit < MinProfitThreshold
|
||||
└─ Continue if profitable
|
||||
```
|
||||
|
||||
#### Phase 2: Transaction Construction (200ms max)
|
||||
|
||||
```
|
||||
1. Build Flash Loan Calldata
|
||||
├─ Encode arbitrage path
|
||||
├─ Calculate minimum output amounts
|
||||
├─ Set recipient address
|
||||
└─ Add safety parameters
|
||||
|
||||
2. Estimate Gas
|
||||
├─ Call estimateGas on RPC
|
||||
├─ Apply safety multiplier (1.2x)
|
||||
├─ Calculate gas cost in ETH
|
||||
└─ Re-validate profitability with gas cost
|
||||
|
||||
3. Get Nonce
|
||||
├─ Query pending nonce from network
|
||||
├─ Check nonce manager for next available
|
||||
├─ Handle nonce collisions
|
||||
└─ Reserve nonce for this transaction
|
||||
|
||||
4. Build Transaction Object
|
||||
├─ Set to: Flash Loan Provider address
|
||||
├─ Set data: Encoded calldata
|
||||
├─ Set gas: Estimated gas limit
|
||||
├─ Set gasPrice: Current gas price + priority fee
|
||||
├─ Set nonce: Reserved nonce
|
||||
└─ Set value: 0 (flash loans don't require upfront payment)
|
||||
|
||||
5. Sign Transaction
|
||||
├─ Load private key from KeyManager
|
||||
├─ Sign with EIP-155 (ChainID: 42161 for Arbitrum)
|
||||
├─ Verify signature
|
||||
└─ Serialize to RLP
|
||||
```
|
||||
|
||||
#### Phase 3: Transaction Dispatch (1-2s max)
|
||||
|
||||
```
|
||||
1. Choose Dispatch Method
|
||||
├─ If MEV Protection Enabled → Use Flashbots/Private RPC
|
||||
├─ If High Competition → Use Private RPC
|
||||
└─ Default → Public Mempool
|
||||
|
||||
2. Send Transaction
|
||||
├─ Dispatch via chosen method
|
||||
├─ Receive transaction hash
|
||||
├─ Log submission
|
||||
└─ Start monitoring
|
||||
|
||||
3. Handle Errors
|
||||
├─ If "nonce too low" → Get new nonce and retry
|
||||
├─ If "gas too low" → Increase gas and retry
|
||||
├─ If "insufficient funds" → Abort (critical error)
|
||||
├─ If "already known" → Wait for existing tx
|
||||
└─ If network error → Retry with exponential backoff
|
||||
```
|
||||
|
||||
#### Phase 4: Execution Monitoring (5-30s)
|
||||
|
||||
```
|
||||
1. Wait for Inclusion
|
||||
├─ Poll for transaction receipt
|
||||
├─ Timeout after 30 seconds
|
||||
├─ Check if transaction replaced
|
||||
└─ Handle dropped transactions
|
||||
|
||||
2. Verify Execution
|
||||
├─ Check receipt status (1 = success, 0 = revert)
|
||||
├─ If reverted → Parse revert reason
|
||||
├─ If succeeded → Continue
|
||||
└─ If dropped → Handle re-submission
|
||||
|
||||
3. Parse Events
|
||||
├─ Extract ArbitrageExecuted event
|
||||
├─ Parse actual profit
|
||||
├─ Parse gas used
|
||||
└─ Calculate ROI
|
||||
|
||||
4. Update State
|
||||
├─ Mark nonce as confirmed
|
||||
├─ Update profit tracking
|
||||
├─ Log execution result
|
||||
└─ Emit metrics
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Provider Implementations
|
||||
|
||||
### 1. Aave Flash Loan Provider
|
||||
|
||||
**Advantages:**
|
||||
- Large liquidity pools
|
||||
- Supports many tokens
|
||||
- Fixed fee (0.09%)
|
||||
- Very reliable
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```go
|
||||
func (a *AaveFlashLoanProvider) ExecuteFlashLoan(
|
||||
ctx context.Context,
|
||||
opp *ArbitrageOpportunity,
|
||||
config *ExecutionConfig,
|
||||
) (*ExecutionResult, error) {
|
||||
// 1. Build flash loan parameters
|
||||
assets := []common.Address{opp.TokenIn}
|
||||
amounts := []*big.Int{opp.AmountIn}
|
||||
modes := []*big.Int{big.NewInt(0)} // 0 = no debt, must repay in same transaction
|
||||
|
||||
// 2. Encode arbitrage path as userData
|
||||
userData := encodeArbitragePath(opp)
|
||||
|
||||
// 3. Build flashLoan() calldata
|
||||
aaveABI := getAavePoolABI()
|
||||
calldata, err := aaveABI.Pack(
|
||||
"flashLoan",
|
||||
a.receiverContract, // Receiver contract
|
||||
assets, // Assets to borrow
|
||||
amounts, // Amounts to borrow
|
||||
modes, // Interest rate modes (0 for none)
|
||||
a.onBehalfOf, // On behalf of address
|
||||
userData, // Encoded arbitrage data
|
||||
uint16(0), // Referral code
|
||||
)
|
||||
|
||||
// 4. Build and sign transaction
|
||||
tx := buildTransaction(a.poolAddress, calldata, config)
|
||||
signedTx, err := signTransaction(tx, keyManager)
|
||||
|
||||
// 5. Dispatch transaction
|
||||
txHash, err := dispatcher.Dispatch(ctx, signedTx, DispatchModeMEV)
|
||||
|
||||
// 6. Monitor execution
|
||||
receipt, err := monitor.WaitForConfirmation(ctx, txHash, 1)
|
||||
|
||||
// 7. Parse result
|
||||
result := parseExecutionResult(receipt, opp)
|
||||
|
||||
return result, nil
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Balancer Flash Loan Provider
|
||||
|
||||
**Advantages:**
|
||||
- Zero fees (!)
|
||||
- Large liquidity
|
||||
- Multi-token flash loans supported
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```go
|
||||
func (b *BalancerFlashLoanProvider) ExecuteFlashLoan(
|
||||
ctx context.Context,
|
||||
opp *ArbitrageOpportunity,
|
||||
config *ExecutionConfig,
|
||||
) (*ExecutionResult, error) {
|
||||
// 1. Build flash loan parameters
|
||||
tokens := []common.Address{opp.TokenIn}
|
||||
amounts := []*big.Int{opp.AmountIn}
|
||||
|
||||
// 2. Encode arbitrage path
|
||||
userData := encodeArbitragePath(opp)
|
||||
|
||||
// 3. Build flashLoan() calldata for Balancer Vault
|
||||
vaultABI := getBalancerVaultABI()
|
||||
calldata, err := vaultABI.Pack(
|
||||
"flashLoan",
|
||||
b.receiverContract, // IFlashLoanReceiver
|
||||
tokens, // Tokens to borrow
|
||||
amounts, // Amounts to borrow
|
||||
userData, // Encoded arbitrage path
|
||||
)
|
||||
|
||||
// 4-7. Same as Aave (build, sign, dispatch, monitor)
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Uniswap Flash Swap Provider
|
||||
|
||||
**Advantages:**
|
||||
- Available on all token pairs
|
||||
- No separate flash loan contract needed
|
||||
- Fee is same as swap fee (0.3%)
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```go
|
||||
func (u *UniswapFlashSwapProvider) ExecuteFlashLoan(
|
||||
ctx context.Context,
|
||||
opp *ArbitrageOpportunity,
|
||||
config *ExecutionConfig,
|
||||
) (*ExecutionResult, error) {
|
||||
// 1. Find optimal pool for flash swap
|
||||
pool := findBestPoolForFlashSwap(opp.TokenIn, opp.AmountIn)
|
||||
|
||||
// 2. Determine amount0Out and amount1Out
|
||||
amount0Out, amount1Out := calculateSwapAmounts(pool, opp)
|
||||
|
||||
// 3. Encode arbitrage path
|
||||
userData := encodeArbitragePath(opp)
|
||||
|
||||
// 4. Build swap() calldata for Uniswap V2 pair
|
||||
pairABI := getUniswapV2PairABI()
|
||||
calldata, err := pairABI.Pack(
|
||||
"swap",
|
||||
amount0Out, // Amount of token0 to receive
|
||||
amount1Out, // Amount of token1 to receive
|
||||
u.receiverContract, // Recipient (our contract)
|
||||
userData, // Triggers callback
|
||||
)
|
||||
|
||||
// 5-8. Same as others
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Safety & Risk Management
|
||||
|
||||
### Pre-Execution Checks
|
||||
|
||||
```go
|
||||
type SafetyValidator struct {
|
||||
priceImpactValidator *validation.PriceImpactValidator
|
||||
blacklistChecker *security.BlacklistChecker
|
||||
positionLimiter *risk.PositionLimiter
|
||||
}
|
||||
|
||||
func (sv *SafetyValidator) ValidateExecution(opp *ArbitrageOpportunity) error {
|
||||
// 1. Price Impact
|
||||
if result := sv.priceImpactValidator.ValidatePriceImpact(opp.PriceImpact); !result.IsAcceptable {
|
||||
return fmt.Errorf("price impact too high: %s", result.Recommendation)
|
||||
}
|
||||
|
||||
// 2. Blacklist Check
|
||||
if sv.blacklistChecker.IsBlacklisted(opp.TokenIn) || sv.blacklistChecker.IsBlacklisted(opp.TokenOut) {
|
||||
return fmt.Errorf("token is blacklisted")
|
||||
}
|
||||
|
||||
// 3. Position Size
|
||||
if opp.AmountIn.Cmp(sv.positionLimiter.MaxPositionSize) > 0 {
|
||||
return fmt.Errorf("position size exceeds limit")
|
||||
}
|
||||
|
||||
// 4. Slippage Protection
|
||||
if opp.Slippage > sv.maxSlippage {
|
||||
return fmt.Errorf("slippage %f%% exceeds max %f%%", opp.Slippage, sv.maxSlippage)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Circuit Breakers
|
||||
|
||||
```go
|
||||
type CircuitBreaker struct {
|
||||
consecutiveFailures int
|
||||
maxFailures int
|
||||
resetTimeout time.Duration
|
||||
lastFailure time.Time
|
||||
state CircuitState
|
||||
}
|
||||
|
||||
func (cb *CircuitBreaker) ShouldExecute() bool {
|
||||
if cb.state == CircuitStateOpen {
|
||||
// Check if we should try half-open
|
||||
if time.Since(cb.lastFailure) > cb.resetTimeout {
|
||||
cb.state = CircuitStateHalfOpen
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (cb *CircuitBreaker) RecordSuccess() {
|
||||
cb.consecutiveFailures = 0
|
||||
cb.state = CircuitStateClosed
|
||||
}
|
||||
|
||||
func (cb *CircuitBreaker) RecordFailure() {
|
||||
cb.consecutiveFailures++
|
||||
cb.lastFailure = time.Now()
|
||||
|
||||
if cb.consecutiveFailures >= cb.maxFailures {
|
||||
cb.state = CircuitStateOpen
|
||||
// Trigger alerts
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Transaction Signing & Dispatch
|
||||
|
||||
### Transaction Signing Flow
|
||||
|
||||
```go
|
||||
func SignFlashLoanTransaction(
|
||||
opp *ArbitrageOpportunity,
|
||||
provider FlashLoanProvider,
|
||||
keyManager *security.KeyManager,
|
||||
nonceManager *NonceManager,
|
||||
gasEstimator *GasEstimator,
|
||||
) (*types.Transaction, error) {
|
||||
|
||||
// 1. Build calldata
|
||||
calldata, err := provider.BuildCalldata(opp)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to build calldata: %w", err)
|
||||
}
|
||||
|
||||
// 2. Estimate gas
|
||||
gasLimit, err := gasEstimator.EstimateGas(provider.Address(), calldata)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to estimate gas: %w", err)
|
||||
}
|
||||
|
||||
// 3. Get gas price
|
||||
gasPrice, priorityFee, err := gasEstimator.GetGasPrice(context.Background())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get gas price: %w", err)
|
||||
}
|
||||
|
||||
// 4. Get nonce
|
||||
nonce, err := nonceManager.GetNextNonce(context.Background())
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get nonce: %w", err)
|
||||
}
|
||||
|
||||
// 5. Build transaction
|
||||
tx := types.NewTx(&types.DynamicFeeTx{
|
||||
ChainID: big.NewInt(42161), // Arbitrum
|
||||
Nonce: nonce,
|
||||
GasTipCap: priorityFee,
|
||||
GasFeeCap: gasPrice,
|
||||
Gas: gasLimit,
|
||||
To: &provider.Address(),
|
||||
Value: big.NewInt(0),
|
||||
Data: calldata,
|
||||
})
|
||||
|
||||
// 6. Sign transaction
|
||||
privateKey, err := keyManager.GetPrivateKey()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get private key: %w", err)
|
||||
}
|
||||
|
||||
signer := types.LatestSignerForChainID(big.NewInt(42161))
|
||||
signedTx, err := types.SignTx(tx, signer, privateKey)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to sign transaction: %w", err)
|
||||
}
|
||||
|
||||
return signedTx, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Dispatch Strategies
|
||||
|
||||
**1. Public Mempool (Default)**
|
||||
```go
|
||||
func (d *TransactionDispatcher) DispatchPublic(ctx context.Context, tx *types.Transaction) (common.Hash, error) {
|
||||
err := d.client.SendTransaction(ctx, tx)
|
||||
if err != nil {
|
||||
return common.Hash{}, err
|
||||
}
|
||||
return tx.Hash(), nil
|
||||
}
|
||||
```
|
||||
|
||||
**2. Flashbots Relay (MEV Protection)**
|
||||
```go
|
||||
func (d *TransactionDispatcher) DispatchFlashbots(ctx context.Context, tx *types.Transaction) (common.Hash, error) {
|
||||
bundle := types.MevBundle{
|
||||
Txs: types.Transactions{tx},
|
||||
BlockNumber: currentBlock + 1,
|
||||
}
|
||||
|
||||
bundleHash, err := d.flashbotsClient.SendBundle(ctx, bundle)
|
||||
if err != nil {
|
||||
return common.Hash{}, err
|
||||
}
|
||||
|
||||
return bundleHash, nil
|
||||
}
|
||||
```
|
||||
|
||||
**3. Private RPC (Low Latency)**
|
||||
```go
|
||||
func (d *TransactionDispatcher) DispatchPrivate(ctx context.Context, tx *types.Transaction) (common.Hash, error) {
|
||||
err := d.privateClient.SendTransaction(ctx, tx)
|
||||
if err != nil {
|
||||
return common.Hash{}, err
|
||||
}
|
||||
return tx.Hash(), nil
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Handling & Recovery
|
||||
|
||||
### Common Errors and Responses
|
||||
|
||||
| Error | Cause | Response |
|
||||
|-------|-------|----------|
|
||||
| `nonce too low` | Transaction already mined | Get new nonce, retry |
|
||||
| `nonce too high` | Nonce gap exists | Reset nonce manager, retry |
|
||||
| `insufficient funds` | Not enough ETH for gas | Abort, alert operator |
|
||||
| `gas price too low` | Network congestion | Increase gas price, retry |
|
||||
| `execution reverted` | Smart contract revert | Parse reason, log, abort |
|
||||
| `transaction underpriced` | Gas price below network minimum | Get current gas price, retry |
|
||||
| `already known` | Duplicate transaction | Wait for confirmation |
|
||||
| `replacement transaction underpriced` | Replacement needs higher gas | Increase gas by 10%, retry |
|
||||
|
||||
### Retry Strategy
|
||||
|
||||
```go
|
||||
func (executor *FlashLoanExecutor) executeWithRetry(
|
||||
ctx context.Context,
|
||||
opp *ArbitrageOpportunity,
|
||||
) (*ExecutionResult, error) {
|
||||
|
||||
var lastErr error
|
||||
for attempt := 0; attempt < executor.config.RetryAttempts; attempt++ {
|
||||
result, err := executor.attemptExecution(ctx, opp)
|
||||
|
||||
if err == nil {
|
||||
return result, nil
|
||||
}
|
||||
|
||||
lastErr = err
|
||||
|
||||
// Check if error is retryable
|
||||
if !isRetryable(err) {
|
||||
return nil, fmt.Errorf("non-retryable error: %w", err)
|
||||
}
|
||||
|
||||
// Handle specific errors
|
||||
if strings.Contains(err.Error(), "nonce too low") {
|
||||
executor.nonceManager.Reset()
|
||||
} else if strings.Contains(err.Error(), "gas price too low") {
|
||||
executor.increaseGasPrice()
|
||||
}
|
||||
|
||||
// Exponential backoff
|
||||
backoff := time.Duration(stdmath.Pow(2, float64(attempt))) * executor.config.RetryDelay
|
||||
time.Sleep(backoff)
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("max retries exceeded: %w", lastErr)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitoring & Analytics
|
||||
|
||||
### Metrics to Track
|
||||
|
||||
1. **Execution Metrics**
|
||||
- Total executions (successful / failed / reverted)
|
||||
- Average execution time
|
||||
- Gas used per execution
|
||||
- Nonce collision rate
|
||||
|
||||
2. **Profit Metrics**
|
||||
- Total profit (gross / net)
|
||||
- Average profit per execution
|
||||
- Profit by provider
|
||||
- ROI by token pair
|
||||
|
||||
3. **Performance Metrics**
|
||||
- Latency from opportunity detection to execution
|
||||
- Transaction confirmation time
|
||||
- Success rate by provider
|
||||
- Revert rate by reason
|
||||
|
||||
4. **Risk Metrics**
|
||||
- Largest position size executed
|
||||
- Highest price impact accepted
|
||||
- Slippage encountered
|
||||
- Failed transactions by reason
|
||||
|
||||
### Logging Format
|
||||
|
||||
```go
|
||||
type ExecutionLog struct {
|
||||
Timestamp time.Time
|
||||
OpportunityID string
|
||||
Provider string
|
||||
TokenIn string
|
||||
TokenOut string
|
||||
AmountIn string
|
||||
EstimatedProfit string
|
||||
ActualProfit string
|
||||
GasUsed uint64
|
||||
GasCost string
|
||||
TransactionHash string
|
||||
Status string
|
||||
Error string
|
||||
ExecutionTime time.Duration
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Checklist
|
||||
|
||||
### Phase 1: Core Infrastructure (Week 1)
|
||||
- [ ] Implement TransactionBuilder
|
||||
- [ ] Implement NonceManager improvements
|
||||
- [ ] Implement TransactionDispatcher
|
||||
- [ ] Add comprehensive error handling
|
||||
- [ ] Create execution state tracking
|
||||
|
||||
### Phase 2: Provider Implementation (Week 2)
|
||||
- [ ] Complete Balancer flash loan provider
|
||||
- [ ] Complete Aave flash loan provider
|
||||
- [ ] Complete Uniswap flash swap provider
|
||||
- [ ] Add provider selection logic
|
||||
- [ ] Implement fee comparison
|
||||
|
||||
### Phase 3: Safety & Testing (Week 3)
|
||||
- [ ] Implement circuit breakers
|
||||
- [ ] Add position size limits
|
||||
- [ ] Create simulation/dry-run mode
|
||||
- [ ] Comprehensive unit tests
|
||||
- [ ] Integration tests with testnet
|
||||
|
||||
### Phase 4: Production Deployment (Week 4)
|
||||
- [ ] Deploy flash loan receiver contracts
|
||||
- [ ] Configure private RPC/Flashbots
|
||||
- [ ] Set up monitoring dashboards
|
||||
- [ ] Production smoke tests
|
||||
- [ ] Gradual rollout with small positions
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Private Key Management
|
||||
|
||||
1. **Never log private keys**
|
||||
2. **Use hardware security modules (HSM) in production**
|
||||
3. **Implement key rotation**
|
||||
4. **Encrypt keys at rest**
|
||||
5. **Limit key access to execution process only**
|
||||
|
||||
### Smart Contract Security
|
||||
|
||||
1. **Audit all receiver contracts before deployment**
|
||||
2. **Use access control (Ownable)**
|
||||
3. **Implement reentrancy guards**
|
||||
4. **Set maximum borrow limits**
|
||||
5. **Add emergency pause functionality**
|
||||
|
||||
### Transaction Security
|
||||
|
||||
1. **Validate all inputs before signing**
|
||||
2. **Use EIP-155 replay protection**
|
||||
3. **Verify transaction before dispatch**
|
||||
4. **Monitor for front-running**
|
||||
5. **Use private mempools when needed**
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
This flash loan execution architecture provides a robust, production-ready system for executing MEV arbitrage opportunities. Key features include:
|
||||
|
||||
- **Multi-provider support** for optimal cost and availability
|
||||
- **Comprehensive safety checks** at every stage
|
||||
- **Robust error handling** with intelligent retry logic
|
||||
- **Detailed monitoring** for operations and debugging
|
||||
- **Production hardened** design for real-world usage
|
||||
|
||||
The modular design allows for easy extension, testing, and maintenance while ensuring safety and profitability.
|
||||
|
||||
---
|
||||
|
||||
**Next Steps**: Proceed with implementation following the phased checklist above.
|
||||
Reference in New Issue
Block a user