- Sequencer integration strategy (core competitive advantage) - High-performance parsing architecture (< 5ms target) - Multi-hop arbitrage detection algorithms - Execution strategies (front-running, batching) - Dynamic gas optimization for maximum profit - Comprehensive risk management (slippage, circuit breaker, position sizing) - Profitability metrics and tracking - Complete system architecture with performance targets Key Features: - Sequencer front-running (100-500ms advantage) - Multi-protocol coverage (13+ DEXs) - Sub-50ms end-to-end latency target - 85%+ success rate target - Batch execution for gas savings - Flashbots integration option Profitability Targets: - Conservative: 18 ETH/month (180% monthly ROI) - Moderate: 141 ETH/month (705% monthly ROI) - Optimistic: 456 ETH/month (912% monthly ROI) Implementation: 8-week roadmap with clear phases 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1739 lines
51 KiB
Markdown
1739 lines
51 KiB
Markdown
# V2 Profitability Plan
|
|
|
|
## Executive Summary
|
|
|
|
This document outlines the complete architecture for a profitable MEV bot that leverages **Arbitrum sequencer access** to execute arbitrage trades ahead of the chain. The sequencer reader/parser is the **core competitive advantage**, providing visibility into pending transactions before they're confirmed on-chain.
|
|
|
|
**Key Profitability Drivers:**
|
|
1. **Sequencer Front-Running**: Execute before price-moving transactions hit the chain
|
|
2. **Multi-Protocol Coverage**: Capture arbitrage across 13+ DEX protocols
|
|
3. **Fast Parsing**: Sub-millisecond transaction parsing and validation
|
|
4. **Batch Execution**: Combine multiple opportunities to save gas
|
|
5. **Dynamic Gas Optimization**: Maximize net profit through intelligent gas pricing
|
|
6. **Risk Management**: Protect capital with slippage limits and circuit breakers
|
|
|
|
**Target Metrics:**
|
|
- **Latency**: < 50ms from sequencer event to execution decision
|
|
- **Success Rate**: > 85% of executed trades profitable
|
|
- **Average Profit**: > 0.05 ETH per trade (after gas)
|
|
- **Daily Volume**: 50-200 trades per day
|
|
- **ROI**: > 20% monthly on deployed capital
|
|
|
|
---
|
|
|
|
## 1. Sequencer Integration Strategy
|
|
|
|
### 1.1 Core Advantage: Reading Pending Transactions
|
|
|
|
**Arbitrum Sequencer Access:**
|
|
```
|
|
Sequencer Feed (WebSocket)
|
|
↓
|
|
Pending Transaction Stream
|
|
↓
|
|
Parse ALL Swap Events (before on-chain)
|
|
↓
|
|
Calculate Price Impact
|
|
↓
|
|
Detect Arbitrage Opportunity
|
|
↓
|
|
Execute Front-Run Transaction
|
|
↓
|
|
Capture Profit
|
|
```
|
|
|
|
**Why This Matters:**
|
|
- Traditional bots read from the mempool AFTER transactions are broadcast
|
|
- We read from the sequencer BEFORE transactions are finalized
|
|
- This gives us a **time advantage** of 100-500ms
|
|
- In MEV, milliseconds = millions
|
|
|
|
### 1.2 Sequencer Reader Architecture
|
|
|
|
```go
|
|
// pkg/sequencer/reader.go
|
|
type SequencerReader struct {
|
|
wsClient *websocket.Conn
|
|
parsers *parsers.Factory
|
|
validator *validation.Validator
|
|
cache *cache.PoolCache
|
|
arbDetector *arbitrage.Detector
|
|
executor *execution.Executor
|
|
metrics *metrics.Collector
|
|
|
|
// Performance critical
|
|
parseLatency prometheus.Histogram
|
|
detectLatency prometheus.Histogram
|
|
executeLatency prometheus.Histogram
|
|
}
|
|
|
|
func (r *SequencerReader) Start(ctx context.Context) error {
|
|
// Subscribe to Arbitrum sequencer WebSocket feed
|
|
err := r.wsClient.Subscribe(ctx, "newPendingTransactions")
|
|
if err != nil {
|
|
return fmt.Errorf("sequencer subscription failed: %w", err)
|
|
}
|
|
|
|
// Process pending transactions in real-time
|
|
for {
|
|
select {
|
|
case tx := <-r.wsClient.Transactions():
|
|
// CRITICAL PATH: Every nanosecond counts
|
|
go r.ProcessPendingTx(ctx, tx)
|
|
|
|
case <-ctx.Done():
|
|
return ctx.Err()
|
|
}
|
|
}
|
|
}
|
|
|
|
func (r *SequencerReader) ProcessPendingTx(ctx context.Context, tx *types.Transaction) {
|
|
startTime := time.Now()
|
|
|
|
// Step 1: Parse transaction (< 5ms target)
|
|
events, err := r.parsers.ParseTransaction(tx)
|
|
if err != nil {
|
|
r.metrics.ParseErrors.Inc()
|
|
return
|
|
}
|
|
r.parseLatency.Observe(time.Since(startTime).Seconds())
|
|
|
|
// Step 2: Validate events (< 2ms target)
|
|
validEvents := r.validator.FilterValid(events)
|
|
if len(validEvents) == 0 {
|
|
return
|
|
}
|
|
|
|
// Step 3: Calculate price impact (< 3ms target)
|
|
priceImpacts := r.calculatePriceImpacts(validEvents)
|
|
|
|
// Step 4: Detect arbitrage (< 10ms target)
|
|
opportunities := r.arbDetector.FindOpportunities(priceImpacts)
|
|
r.detectLatency.Observe(time.Since(startTime).Seconds())
|
|
|
|
// Step 5: Execute profitable trades (< 30ms target)
|
|
for _, opp := range opportunities {
|
|
if opp.NetProfit.Cmp(r.minProfitThreshold) > 0 {
|
|
r.executor.ExecuteFrontRun(ctx, opp, tx)
|
|
}
|
|
}
|
|
r.executeLatency.Observe(time.Since(startTime).Seconds())
|
|
|
|
// TOTAL TARGET: < 50ms from sequencer event to execution
|
|
}
|
|
```
|
|
|
|
### 1.3 Sequencer Connection Management
|
|
|
|
```go
|
|
// pkg/sequencer/connection.go
|
|
type ConnectionManager struct {
|
|
primaryURL string
|
|
backupURLs []string
|
|
currentConn *websocket.Conn
|
|
reconnectBackoff time.Duration
|
|
maxReconnectDelay time.Duration
|
|
}
|
|
|
|
func (cm *ConnectionManager) MaintainConnection(ctx context.Context) error {
|
|
for {
|
|
conn, err := cm.connectWithBackoff(ctx)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
cm.currentConn = conn
|
|
|
|
// Monitor connection health
|
|
if err := cm.monitorConnection(ctx, conn); err != nil {
|
|
log.Error().Err(err).Msg("Connection lost, reconnecting...")
|
|
continue
|
|
}
|
|
}
|
|
}
|
|
|
|
func (cm *ConnectionManager) connectWithBackoff(ctx context.Context) (*websocket.Conn, error) {
|
|
backoff := cm.reconnectBackoff
|
|
|
|
for {
|
|
// Try primary endpoint
|
|
conn, err := websocket.Dial(cm.primaryURL)
|
|
if err == nil {
|
|
log.Info().Msg("Connected to primary sequencer endpoint")
|
|
return conn, nil
|
|
}
|
|
|
|
// Try backup endpoints
|
|
for _, backupURL := range cm.backupURLs {
|
|
conn, err := websocket.Dial(backupURL)
|
|
if err == nil {
|
|
log.Info().Str("endpoint", backupURL).Msg("Connected to backup sequencer endpoint")
|
|
return conn, nil
|
|
}
|
|
}
|
|
|
|
// Exponential backoff
|
|
select {
|
|
case <-time.After(backoff):
|
|
backoff = min(backoff*2, cm.maxReconnectDelay)
|
|
case <-ctx.Done():
|
|
return nil, ctx.Err()
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### 1.4 Sequencer Data Validation
|
|
|
|
**CRITICAL**: Sequencer data is unconfirmed and may be reorganized. We MUST validate:
|
|
|
|
```go
|
|
// pkg/sequencer/validator.go
|
|
type SequencerDataValidator struct {
|
|
cache *cache.PoolCache
|
|
}
|
|
|
|
func (v *SequencerDataValidator) ValidatePendingTx(tx *types.Transaction) error {
|
|
// 1. Validate transaction structure
|
|
if tx.To() == nil {
|
|
return errors.New("transaction has no recipient")
|
|
}
|
|
|
|
// 2. Validate gas price is reasonable
|
|
if tx.GasPrice().Cmp(maxReasonableGas) > 0 {
|
|
return errors.New("gas price unreasonably high")
|
|
}
|
|
|
|
// 3. Validate nonce progression
|
|
if !v.isNonceValid(tx) {
|
|
return errors.New("invalid nonce")
|
|
}
|
|
|
|
// 4. Validate signature
|
|
if !v.isSignatureValid(tx) {
|
|
return errors.New("invalid signature")
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (v *SequencerDataValidator) ValidateSwapEvent(event *SwapEvent) error {
|
|
// 1. Validate pool exists in cache
|
|
pool := v.cache.GetByAddress(event.PoolAddress)
|
|
if pool == nil {
|
|
return errors.New("unknown pool")
|
|
}
|
|
|
|
// 2. Validate tokens match pool
|
|
if pool.Token0 != event.Token0 || pool.Token1 != event.Token1 {
|
|
return errors.New("token mismatch")
|
|
}
|
|
|
|
// 3. Validate amounts are non-zero
|
|
if event.Amount0.Cmp(big.NewInt(0)) == 0 && event.Amount1.Cmp(big.NewInt(0)) == 0 {
|
|
return errors.New("zero amounts")
|
|
}
|
|
|
|
// 4. Validate amounts are reasonable (not dust, not astronomical)
|
|
if !v.isAmountReasonable(event.Amount0, pool.Token0Decimals) {
|
|
return errors.New("unreasonable amount0")
|
|
}
|
|
|
|
if !v.isAmountReasonable(event.Amount1, pool.Token1Decimals) {
|
|
return errors.New("unreasonable amount1")
|
|
}
|
|
|
|
return nil
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 2. Fast Parsing for Profitability
|
|
|
|
### 2.1 Parsing Speed Requirements
|
|
|
|
**Why Speed Matters:**
|
|
- If we take 100ms to parse, another bot executes first
|
|
- If we take 50ms, we capture the profit
|
|
- **Target: < 5ms per transaction parse**
|
|
|
|
### 2.2 Optimized Parser Architecture
|
|
|
|
```go
|
|
// pkg/parsers/optimized_factory.go
|
|
type OptimizedFactory struct {
|
|
// Pre-compiled ABI parsers (cached at startup)
|
|
uniswapV2ABI abi.ABI
|
|
uniswapV3ABI abi.ABI
|
|
curveABI abi.ABI
|
|
balancerV2ABI abi.ABI
|
|
kyberABI abi.ABI
|
|
camelotABI abi.ABI
|
|
|
|
// Protocol router cache (address -> protocol type)
|
|
routerCache map[common.Address]ProtocolType
|
|
routerCacheMutex sync.RWMutex
|
|
|
|
// Pool cache for fast lookups
|
|
poolCache *cache.PoolCache
|
|
}
|
|
|
|
func (f *OptimizedFactory) ParseTransaction(tx *types.Transaction) ([]*SwapEvent, error) {
|
|
// Fast path: Check if this is a known router
|
|
f.routerCacheMutex.RLock()
|
|
protocol, exists := f.routerCache[*tx.To()]
|
|
f.routerCacheMutex.RUnlock()
|
|
|
|
if exists {
|
|
// Use cached protocol parser (FAST)
|
|
return f.parseWithProtocol(tx, protocol)
|
|
}
|
|
|
|
// Slow path: Identify protocol from transaction data
|
|
protocol = f.identifyProtocol(tx)
|
|
if protocol == ProtocolUnknown {
|
|
return nil, ErrUnknownProtocol
|
|
}
|
|
|
|
// Cache for next time
|
|
f.routerCacheMutex.Lock()
|
|
f.routerCache[*tx.To()] = protocol
|
|
f.routerCacheMutex.Unlock()
|
|
|
|
return f.parseWithProtocol(tx, protocol)
|
|
}
|
|
|
|
func (f *OptimizedFactory) parseWithProtocol(tx *types.Transaction, protocol ProtocolType) ([]*SwapEvent, error) {
|
|
// Route to optimized protocol-specific parser
|
|
switch protocol {
|
|
case ProtocolUniswapV2:
|
|
return f.parseUniswapV2(tx) // Optimized for UniV2
|
|
case ProtocolUniswapV3:
|
|
return f.parseUniswapV3(tx) // Optimized for UniV3
|
|
case ProtocolCurve:
|
|
return f.parseCurve(tx) // Optimized for Curve
|
|
// ... other protocols
|
|
default:
|
|
return nil, ErrUnknownProtocol
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2.3 Parallel Parsing for Batches
|
|
|
|
```go
|
|
// pkg/parsers/parallel.go
|
|
type ParallelParser struct {
|
|
factory *OptimizedFactory
|
|
workerPool *WorkerPool
|
|
}
|
|
|
|
func (p *ParallelParser) ParseBatch(txs []*types.Transaction) ([]*SwapEvent, error) {
|
|
results := make(chan *ParseResult, len(txs))
|
|
|
|
// Fan-out: Distribute transactions to worker pool
|
|
for _, tx := range txs {
|
|
tx := tx // Capture for goroutine
|
|
p.workerPool.Submit(func() {
|
|
events, err := p.factory.ParseTransaction(tx)
|
|
results <- &ParseResult{Events: events, Err: err}
|
|
})
|
|
}
|
|
|
|
// Fan-in: Collect results
|
|
var allEvents []*SwapEvent
|
|
for i := 0; i < len(txs); i++ {
|
|
result := <-results
|
|
if result.Err == nil {
|
|
allEvents = append(allEvents, result.Events...)
|
|
}
|
|
}
|
|
|
|
return allEvents, nil
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 3. Arbitrage Detection Algorithms
|
|
|
|
### 3.1 Multi-Hop Path Finding
|
|
|
|
```go
|
|
// pkg/arbitrage/pathfinder.go
|
|
type PathFinder struct {
|
|
marketGraph *MarketGraph
|
|
maxHops int // Typically 2-4 hops
|
|
minProfit *big.Float
|
|
}
|
|
|
|
type ArbitragePath struct {
|
|
Hops []*Hop
|
|
InputToken common.Address
|
|
OutputToken common.Address
|
|
InputAmount *big.Int
|
|
OutputAmount *big.Int
|
|
GrossProfit *big.Float
|
|
GasCost *big.Int
|
|
NetProfit *big.Float
|
|
}
|
|
|
|
type Hop struct {
|
|
Pool *PoolState
|
|
Protocol ProtocolType
|
|
TokenIn common.Address
|
|
TokenOut common.Address
|
|
AmountIn *big.Int
|
|
AmountOut *big.Int
|
|
}
|
|
|
|
func (pf *PathFinder) FindArbitragePaths(priceImpact *PriceImpact) []*ArbitragePath {
|
|
// Starting token and amount from price impact
|
|
startToken := priceImpact.TokenOut
|
|
startAmount := priceImpact.AmountOut
|
|
|
|
var paths []*ArbitragePath
|
|
|
|
// BFS/DFS to find circular paths back to start token
|
|
pf.searchPaths(startToken, startToken, startAmount, []*Hop{}, &paths)
|
|
|
|
// Filter paths by profitability
|
|
var profitablePaths []*ArbitragePath
|
|
for _, path := range paths {
|
|
if path.NetProfit.Cmp(pf.minProfit) > 0 {
|
|
profitablePaths = append(profitablePaths, path)
|
|
}
|
|
}
|
|
|
|
// Sort by net profit (highest first)
|
|
sort.Slice(profitablePaths, func(i, j int) bool {
|
|
return profitablePaths[i].NetProfit.Cmp(profitablePaths[j].NetProfit) > 0
|
|
})
|
|
|
|
return profitablePaths
|
|
}
|
|
|
|
func (pf *PathFinder) searchPaths(
|
|
currentToken common.Address,
|
|
targetToken common.Address,
|
|
currentAmount *big.Int,
|
|
currentPath []*Hop,
|
|
results *[]*ArbitragePath,
|
|
) {
|
|
// Max depth reached
|
|
if len(currentPath) >= pf.maxHops {
|
|
return
|
|
}
|
|
|
|
// Get all pools with currentToken
|
|
pools := pf.marketGraph.GetPoolsWithToken(currentToken)
|
|
|
|
for _, pool := range pools {
|
|
// Determine output token
|
|
var outputToken common.Address
|
|
if pool.Token0 == currentToken {
|
|
outputToken = pool.Token1
|
|
} else {
|
|
outputToken = pool.Token0
|
|
}
|
|
|
|
// Calculate output amount
|
|
outputAmount := pf.calculateSwapOutput(pool, currentToken, outputToken, currentAmount)
|
|
if outputAmount.Cmp(big.NewInt(0)) == 0 {
|
|
continue // Insufficient liquidity
|
|
}
|
|
|
|
// Create hop
|
|
hop := &Hop{
|
|
Pool: pool,
|
|
Protocol: pool.Protocol,
|
|
TokenIn: currentToken,
|
|
TokenOut: outputToken,
|
|
AmountIn: currentAmount,
|
|
AmountOut: outputAmount,
|
|
}
|
|
|
|
// Add to path
|
|
newPath := append(currentPath, hop)
|
|
|
|
// Check if we've returned to target token (arbitrage loop complete)
|
|
if outputToken == targetToken && len(newPath) >= 2 {
|
|
// Calculate profitability
|
|
path := pf.buildArbitragePath(newPath)
|
|
if path != nil {
|
|
*results = append(*results, path)
|
|
}
|
|
continue // Don't recurse further
|
|
}
|
|
|
|
// Recurse to find more hops
|
|
pf.searchPaths(outputToken, targetToken, outputAmount, newPath, results)
|
|
}
|
|
}
|
|
```
|
|
|
|
### 3.2 Profitability Calculation
|
|
|
|
```go
|
|
// pkg/arbitrage/profitability.go
|
|
type ProfitCalculator struct {
|
|
gasOracle *GasOracle
|
|
ethPrice *big.Float // ETH price in USD
|
|
}
|
|
|
|
func (pc *ProfitCalculator) CalculateNetProfit(path *ArbitragePath) *big.Float {
|
|
// 1. Calculate gross profit (output - input, in same token)
|
|
inputAmountFloat := new(big.Float).SetInt(path.InputAmount)
|
|
outputAmountFloat := new(big.Float).SetInt(path.OutputAmount)
|
|
|
|
// Scale to 18 decimals for comparison
|
|
inputScaled := pc.scaleToDecimals(inputAmountFloat, 18)
|
|
outputScaled := pc.scaleToDecimals(outputAmountFloat, 18)
|
|
|
|
grossProfit := new(big.Float).Sub(outputScaled, inputScaled)
|
|
|
|
// 2. Estimate gas cost
|
|
gasCost := pc.estimateGasCost(path)
|
|
gasCostFloat := new(big.Float).SetInt(gasCost)
|
|
|
|
// 3. Calculate net profit
|
|
netProfit := new(big.Float).Sub(grossProfit, gasCostFloat)
|
|
|
|
return netProfit
|
|
}
|
|
|
|
func (pc *ProfitCalculator) estimateGasCost(path *ArbitragePath) *big.Int {
|
|
// Base transaction cost
|
|
baseCost := big.NewInt(21000)
|
|
|
|
// Per-hop cost (varies by protocol)
|
|
var totalGas = big.NewInt(0)
|
|
totalGas.Add(totalGas, baseCost)
|
|
|
|
for _, hop := range path.Hops {
|
|
var hopGas *big.Int
|
|
switch hop.Protocol {
|
|
case ProtocolUniswapV2:
|
|
hopGas = big.NewInt(100000) // ~100k gas per UniV2 swap
|
|
case ProtocolUniswapV3:
|
|
hopGas = big.NewInt(150000) // ~150k gas per UniV3 swap
|
|
case ProtocolCurve:
|
|
hopGas = big.NewInt(120000) // ~120k gas per Curve swap
|
|
case ProtocolBalancerV2:
|
|
hopGas = big.NewInt(130000) // ~130k gas per Balancer swap
|
|
default:
|
|
hopGas = big.NewInt(120000) // Default estimate
|
|
}
|
|
totalGas.Add(totalGas, hopGas)
|
|
}
|
|
|
|
// Get current gas price from oracle
|
|
gasPrice := pc.gasOracle.GetCurrentGasPrice()
|
|
|
|
// Calculate total cost
|
|
gasCost := new(big.Int).Mul(totalGas, gasPrice)
|
|
|
|
return gasCost
|
|
}
|
|
```
|
|
|
|
### 3.3 Real-Time Arbitrage Detection
|
|
|
|
```go
|
|
// pkg/arbitrage/detector.go
|
|
type Detector struct {
|
|
pathFinder *PathFinder
|
|
profitCalc *ProfitCalculator
|
|
marketGraph *MarketGraph
|
|
minProfitUSD float64 // Minimum profit in USD
|
|
|
|
// Performance metrics
|
|
detectLatency prometheus.Histogram
|
|
opportunitiesFound prometheus.Counter
|
|
}
|
|
|
|
func (d *Detector) FindOpportunities(priceImpacts []*PriceImpact) []*ArbitragePath {
|
|
startTime := time.Now()
|
|
defer func() {
|
|
d.detectLatency.Observe(time.Since(startTime).Seconds())
|
|
}()
|
|
|
|
var allPaths []*ArbitragePath
|
|
|
|
// Check each price impact for arbitrage opportunities
|
|
for _, impact := range priceImpacts {
|
|
// Find all possible arbitrage paths
|
|
paths := d.pathFinder.FindArbitragePaths(impact)
|
|
|
|
// Calculate profitability for each path
|
|
for _, path := range paths {
|
|
netProfit := d.profitCalc.CalculateNetProfit(path)
|
|
path.NetProfit = netProfit
|
|
|
|
// Filter by minimum profit
|
|
minProfitFloat := big.NewFloat(d.minProfitUSD)
|
|
if netProfit.Cmp(minProfitFloat) > 0 {
|
|
allPaths = append(allPaths, path)
|
|
d.opportunitiesFound.Inc()
|
|
}
|
|
}
|
|
}
|
|
|
|
return allPaths
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Execution Strategies
|
|
|
|
### 4.1 Front-Running Execution
|
|
|
|
```go
|
|
// pkg/execution/frontrun.go
|
|
type FrontRunner struct {
|
|
client *ethclient.Client
|
|
signer types.Signer
|
|
privateKey *ecdsa.PrivateKey
|
|
flashbots *FlashbotsClient // Optional: MEV protection
|
|
|
|
maxGasPrice *big.Int
|
|
gasPremium float64 // Multiplier above target tx gas price
|
|
}
|
|
|
|
func (fr *FrontRunner) ExecuteFrontRun(
|
|
ctx context.Context,
|
|
opportunity *ArbitragePath,
|
|
targetTx *types.Transaction,
|
|
) (*types.Transaction, error) {
|
|
// 1. Build arbitrage transaction
|
|
arbTx, err := fr.buildArbitrageTx(opportunity)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to build arbitrage tx: %w", err)
|
|
}
|
|
|
|
// 2. Set gas price higher than target transaction
|
|
targetGasPrice := targetTx.GasPrice()
|
|
frontRunGasPrice := fr.calculateFrontRunGasPrice(targetGasPrice)
|
|
|
|
if frontRunGasPrice.Cmp(fr.maxGasPrice) > 0 {
|
|
return nil, errors.New("required gas price exceeds maximum")
|
|
}
|
|
|
|
arbTx.GasPrice = frontRunGasPrice
|
|
|
|
// 3. Sign transaction
|
|
signedTx, err := types.SignTx(arbTx, fr.signer, fr.privateKey)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to sign tx: %w", err)
|
|
}
|
|
|
|
// 4. Submit to network (choose path based on configuration)
|
|
if fr.flashbots != nil {
|
|
// Use Flashbots to avoid frontrunning ourselves
|
|
return fr.flashbots.SendBundle(ctx, []*types.Transaction{signedTx, targetTx})
|
|
} else {
|
|
// Direct submission to mempool
|
|
err = fr.client.SendTransaction(ctx, signedTx)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to send tx: %w", err)
|
|
}
|
|
return signedTx, nil
|
|
}
|
|
}
|
|
|
|
func (fr *FrontRunner) calculateFrontRunGasPrice(targetGasPrice *big.Int) *big.Int {
|
|
// Add premium to ensure we're ahead
|
|
premium := new(big.Float).SetFloat64(fr.gasPremium)
|
|
targetFloat := new(big.Float).SetInt(targetGasPrice)
|
|
|
|
frontRunFloat := new(big.Float).Mul(targetFloat, premium)
|
|
|
|
frontRunGasPrice, _ := frontRunFloat.Int(nil)
|
|
return frontRunGasPrice
|
|
}
|
|
```
|
|
|
|
### 4.2 Batch Execution
|
|
|
|
```go
|
|
// pkg/execution/batch.go
|
|
type BatchExecutor struct {
|
|
multicallContract common.Address
|
|
client *ethclient.Client
|
|
signer types.Signer
|
|
privateKey *ecdsa.PrivateKey
|
|
}
|
|
|
|
// Multicall contract interface
|
|
type Multicall interface {
|
|
Aggregate(calls []Call) (*big.Int, []byte, error)
|
|
}
|
|
|
|
type Call struct {
|
|
Target common.Address
|
|
CallData []byte
|
|
}
|
|
|
|
func (be *BatchExecutor) ExecuteBatch(
|
|
ctx context.Context,
|
|
opportunities []*ArbitragePath,
|
|
) (*types.Transaction, error) {
|
|
// 1. Build multicall data
|
|
calls := make([]Call, 0, len(opportunities))
|
|
|
|
for _, opp := range opportunities {
|
|
callData, err := be.buildSwapCallData(opp)
|
|
if err != nil {
|
|
continue // Skip this opportunity
|
|
}
|
|
|
|
calls = append(calls, Call{
|
|
Target: opp.Hops[0].Pool.Address,
|
|
CallData: callData,
|
|
})
|
|
}
|
|
|
|
if len(calls) == 0 {
|
|
return nil, errors.New("no valid calls to batch")
|
|
}
|
|
|
|
// 2. Encode multicall transaction
|
|
multicallABI, _ := abi.JSON(strings.NewReader(MulticallABI))
|
|
multicallData, err := multicallABI.Pack("aggregate", calls)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to pack multicall: %w", err)
|
|
}
|
|
|
|
// 3. Build transaction
|
|
nonce, err := be.client.PendingNonceAt(ctx, crypto.PubkeyToAddress(be.privateKey.PublicKey))
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
gasPrice, err := be.client.SuggestGasPrice(ctx)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
tx := types.NewTransaction(
|
|
nonce,
|
|
be.multicallContract,
|
|
big.NewInt(0), // No ETH sent
|
|
2000000, // Gas limit (adjust based on batch size)
|
|
gasPrice,
|
|
multicallData,
|
|
)
|
|
|
|
// 4. Sign and send
|
|
signedTx, err := types.SignTx(tx, be.signer, be.privateKey)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
err = be.client.SendTransaction(ctx, signedTx)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
return signedTx, nil
|
|
}
|
|
```
|
|
|
|
### 4.3 Smart Execution Router
|
|
|
|
```go
|
|
// pkg/execution/router.go
|
|
type ExecutionRouter struct {
|
|
frontRunner *FrontRunner
|
|
batchExecutor *BatchExecutor
|
|
|
|
batchThreshold int // Min opportunities to batch
|
|
batchWindow time.Duration // Time to wait for batching
|
|
}
|
|
|
|
func (er *ExecutionRouter) Execute(
|
|
ctx context.Context,
|
|
opportunities []*ArbitragePath,
|
|
targetTx *types.Transaction,
|
|
) error {
|
|
// Decision: Single vs Batch execution
|
|
|
|
if len(opportunities) == 1 {
|
|
// Single opportunity: Front-run immediately
|
|
_, err := er.frontRunner.ExecuteFrontRun(ctx, opportunities[0], targetTx)
|
|
return err
|
|
}
|
|
|
|
if len(opportunities) >= er.batchThreshold {
|
|
// Multiple opportunities: Batch for gas savings
|
|
_, err := er.batchExecutor.ExecuteBatch(ctx, opportunities)
|
|
return err
|
|
}
|
|
|
|
// Between 1 and threshold: Wait for batch window
|
|
select {
|
|
case <-time.After(er.batchWindow):
|
|
// Execute what we have
|
|
if len(opportunities) > 1 {
|
|
_, err := er.batchExecutor.ExecuteBatch(ctx, opportunities)
|
|
return err
|
|
}
|
|
_, err := er.frontRunner.ExecuteFrontRun(ctx, opportunities[0], targetTx)
|
|
return err
|
|
|
|
case <-ctx.Done():
|
|
return ctx.Err()
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 5. Gas Optimization for Maximum Profit
|
|
|
|
### 5.1 Dynamic Gas Pricing
|
|
|
|
```go
|
|
// pkg/gas/oracle.go
|
|
type GasOracle struct {
|
|
client *ethclient.Client
|
|
historicalPrices []HistoricalGasPrice
|
|
|
|
// Gas price strategy
|
|
strategy GasStrategy
|
|
}
|
|
|
|
type GasStrategy int
|
|
|
|
const (
|
|
StrategyFast GasStrategy = iota // Pay more to execute faster
|
|
StrategyOptimal // Balance speed vs cost
|
|
StrategyEconomy // Minimize gas cost
|
|
)
|
|
|
|
type HistoricalGasPrice struct {
|
|
Timestamp time.Time
|
|
GasPrice *big.Int
|
|
BaseFee *big.Int
|
|
}
|
|
|
|
func (go *GasOracle) GetOptimalGasPrice(opportunity *ArbitragePath) *big.Int {
|
|
// Get current network gas prices
|
|
currentGasPrice, _ := go.client.SuggestGasPrice(context.Background())
|
|
|
|
// Calculate maximum gas price we can pay while remaining profitable
|
|
maxProfitableGasPrice := go.calculateMaxProfitableGasPrice(opportunity)
|
|
|
|
// Apply strategy
|
|
switch go.strategy {
|
|
case StrategyFast:
|
|
// Use higher percentile (90th) to ensure quick inclusion
|
|
return go.calculatePercentileGasPrice(90)
|
|
|
|
case StrategyOptimal:
|
|
// Use median gas price, capped at max profitable
|
|
medianGas := go.calculatePercentileGasPrice(50)
|
|
if medianGas.Cmp(maxProfitableGasPrice) > 0 {
|
|
return maxProfitableGasPrice
|
|
}
|
|
return medianGas
|
|
|
|
case StrategyEconomy:
|
|
// Use 25th percentile, capped at max profitable
|
|
lowGas := go.calculatePercentileGasPrice(25)
|
|
if lowGas.Cmp(maxProfitableGasPrice) > 0 {
|
|
return maxProfitableGasPrice
|
|
}
|
|
return lowGas
|
|
|
|
default:
|
|
return currentGasPrice
|
|
}
|
|
}
|
|
|
|
func (go *GasOracle) calculateMaxProfitableGasPrice(opportunity *ArbitragePath) *big.Int {
|
|
// Given net profit and gas usage, what's the max gas price?
|
|
// netProfit = grossProfit - (gasUsed * gasPrice)
|
|
// gasPrice = (grossProfit - netProfit) / gasUsed
|
|
|
|
grossProfit := opportunity.GrossProfit
|
|
minAcceptableProfit := big.NewFloat(0.01) // 0.01 ETH minimum
|
|
|
|
profitForGas := new(big.Float).Sub(grossProfit, minAcceptableProfit)
|
|
gasUsed := new(big.Float).SetInt64(int64(opportunity.EstimatedGas))
|
|
|
|
maxGasPrice := new(big.Float).Quo(profitForGas, gasUsed)
|
|
maxGasPriceInt, _ := maxGasPrice.Int(nil)
|
|
|
|
return maxGasPriceInt
|
|
}
|
|
```
|
|
|
|
### 5.2 Gas Estimation
|
|
|
|
```go
|
|
// pkg/gas/estimator.go
|
|
type GasEstimator struct {
|
|
client *ethclient.Client
|
|
|
|
// Cached estimates by protocol and hop count
|
|
estimateCache map[string]uint64
|
|
cacheMutex sync.RWMutex
|
|
}
|
|
|
|
func (ge *GasEstimator) EstimateArbitrageGas(path *ArbitragePath) (uint64, error) {
|
|
// Generate cache key
|
|
cacheKey := ge.generateCacheKey(path)
|
|
|
|
// Check cache
|
|
ge.cacheMutex.RLock()
|
|
if cached, exists := ge.estimateCache[cacheKey]; exists {
|
|
ge.cacheMutex.RUnlock()
|
|
return cached, nil
|
|
}
|
|
ge.cacheMutex.RUnlock()
|
|
|
|
// Build transaction
|
|
tx, err := ge.buildArbitrageTx(path)
|
|
if err != nil {
|
|
return 0, err
|
|
}
|
|
|
|
// Estimate gas
|
|
estimatedGas, err := ge.client.EstimateGas(context.Background(), ethereum.CallMsg{
|
|
From: crypto.PubkeyToAddress(ge.privateKey.PublicKey),
|
|
To: tx.To(),
|
|
Gas: 0,
|
|
GasPrice: big.NewInt(0),
|
|
Value: big.NewInt(0),
|
|
Data: tx.Data(),
|
|
})
|
|
|
|
if err != nil {
|
|
return 0, err
|
|
}
|
|
|
|
// Add safety margin (10%)
|
|
estimatedGas = estimatedGas * 110 / 100
|
|
|
|
// Cache for future use
|
|
ge.cacheMutex.Lock()
|
|
ge.estimateCache[cacheKey] = estimatedGas
|
|
ge.cacheMutex.Unlock()
|
|
|
|
return estimatedGas, nil
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 6. Risk Management
|
|
|
|
### 6.1 Slippage Protection
|
|
|
|
```go
|
|
// pkg/execution/slippage.go
|
|
type SlippageProtector struct {
|
|
maxSlippage float64 // e.g., 0.005 = 0.5%
|
|
}
|
|
|
|
func (sp *SlippageProtector) CalculateMinOutput(
|
|
expectedOutput *big.Int,
|
|
) *big.Int {
|
|
// Calculate minimum acceptable output with slippage
|
|
expectedFloat := new(big.Float).SetInt(expectedOutput)
|
|
slippageFactor := 1.0 - sp.maxSlippage
|
|
|
|
minOutputFloat := new(big.Float).Mul(
|
|
expectedFloat,
|
|
big.NewFloat(slippageFactor),
|
|
)
|
|
|
|
minOutput, _ := minOutputFloat.Int(nil)
|
|
return minOutput
|
|
}
|
|
|
|
func (sp *SlippageProtector) ValidateExecution(
|
|
expectedProfit *big.Float,
|
|
actualProfit *big.Float,
|
|
) error {
|
|
// Check if actual profit is within acceptable slippage
|
|
expectedFloat := expectedProfit
|
|
slippageFactor := big.NewFloat(1.0 - sp.maxSlippage)
|
|
|
|
minAcceptableProfit := new(big.Float).Mul(expectedFloat, slippageFactor)
|
|
|
|
if actualProfit.Cmp(minAcceptableProfit) < 0 {
|
|
return fmt.Errorf(
|
|
"slippage too high: expected %s, got %s",
|
|
expectedProfit.String(),
|
|
actualProfit.String(),
|
|
)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
```
|
|
|
|
### 6.2 Circuit Breaker
|
|
|
|
```go
|
|
// pkg/safety/circuit_breaker.go
|
|
type CircuitBreaker struct {
|
|
maxLossPerHour *big.Float
|
|
maxFailuresPerMin int
|
|
|
|
// State tracking
|
|
hourlyLoss *big.Float
|
|
recentFailures []time.Time
|
|
|
|
// Circuit state
|
|
state CircuitState
|
|
stateMutex sync.RWMutex
|
|
|
|
resetAfter time.Duration
|
|
}
|
|
|
|
type CircuitState int
|
|
|
|
const (
|
|
StateClosed CircuitState = iota // Normal operation
|
|
StateOpen // Circuit tripped, blocking trades
|
|
StateHalfOpen // Testing if safe to resume
|
|
)
|
|
|
|
func (cb *CircuitBreaker) AllowExecution() bool {
|
|
cb.stateMutex.RLock()
|
|
defer cb.stateMutex.RUnlock()
|
|
|
|
return cb.state == StateClosed || cb.state == StateHalfOpen
|
|
}
|
|
|
|
func (cb *CircuitBreaker) RecordFailure(loss *big.Float) {
|
|
cb.stateMutex.Lock()
|
|
defer cb.stateMutex.Unlock()
|
|
|
|
// Track recent failures
|
|
cb.recentFailures = append(cb.recentFailures, time.Now())
|
|
|
|
// Add to hourly loss
|
|
cb.hourlyLoss.Add(cb.hourlyLoss, loss)
|
|
|
|
// Check if we should trip the circuit
|
|
if cb.shouldTrip() {
|
|
cb.state = StateOpen
|
|
log.Error().Msg("Circuit breaker TRIPPED - halting trading")
|
|
|
|
// Schedule automatic reset
|
|
time.AfterFunc(cb.resetAfter, cb.attemptReset)
|
|
}
|
|
}
|
|
|
|
func (cb *CircuitBreaker) shouldTrip() bool {
|
|
// Check hourly loss threshold
|
|
if cb.hourlyLoss.Cmp(cb.maxLossPerHour) > 0 {
|
|
return true
|
|
}
|
|
|
|
// Check failure rate
|
|
oneMinAgo := time.Now().Add(-1 * time.Minute)
|
|
recentFailureCount := 0
|
|
for _, t := range cb.recentFailures {
|
|
if t.After(oneMinAgo) {
|
|
recentFailureCount++
|
|
}
|
|
}
|
|
|
|
if recentFailureCount >= cb.maxFailuresPerMin {
|
|
return true
|
|
}
|
|
|
|
return false
|
|
}
|
|
|
|
func (cb *CircuitBreaker) attemptReset() {
|
|
cb.stateMutex.Lock()
|
|
defer cb.stateMutex.Unlock()
|
|
|
|
// Move to half-open state (allow limited testing)
|
|
cb.state = StateHalfOpen
|
|
log.Info().Msg("Circuit breaker entering HALF-OPEN state")
|
|
|
|
// Reset counters
|
|
cb.hourlyLoss = big.NewFloat(0)
|
|
cb.recentFailures = []time.Time{}
|
|
}
|
|
```
|
|
|
|
### 6.3 Position Sizing
|
|
|
|
```go
|
|
// pkg/safety/position_sizer.go
|
|
type PositionSizer struct {
|
|
totalCapital *big.Float
|
|
maxPositionSize float64 // e.g., 0.1 = 10% of capital
|
|
maxConcurrentTrades int
|
|
|
|
currentPositions []*Position
|
|
positionsMutex sync.RWMutex
|
|
}
|
|
|
|
type Position struct {
|
|
ID string
|
|
Size *big.Float
|
|
EntryTime time.Time
|
|
ExitTime *time.Time
|
|
Profit *big.Float
|
|
}
|
|
|
|
func (ps *PositionSizer) CalculatePositionSize(opportunity *ArbitragePath) *big.Int {
|
|
ps.positionsMutex.RLock()
|
|
defer ps.positionsMutex.RUnlock()
|
|
|
|
// Calculate maximum allowed position size
|
|
maxSize := new(big.Float).Mul(
|
|
ps.totalCapital,
|
|
big.NewFloat(ps.maxPositionSize),
|
|
)
|
|
|
|
// Check if we have room for another position
|
|
if len(ps.currentPositions) >= ps.maxConcurrentTrades {
|
|
return big.NewInt(0) // No room for more trades
|
|
}
|
|
|
|
// Use the smaller of: max position size or opportunity input amount
|
|
opportunitySize := new(big.Float).SetInt(opportunity.InputAmount)
|
|
|
|
var positionSize *big.Float
|
|
if opportunitySize.Cmp(maxSize) < 0 {
|
|
positionSize = opportunitySize
|
|
} else {
|
|
positionSize = maxSize
|
|
}
|
|
|
|
positionSizeInt, _ := positionSize.Int(nil)
|
|
return positionSizeInt
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 7. Profitability Metrics and Tracking
|
|
|
|
### 7.1 Metrics Collection
|
|
|
|
```go
|
|
// pkg/metrics/profitability.go
|
|
type ProfitabilityMetrics struct {
|
|
// Prometheus metrics
|
|
totalProfit prometheus.Counter
|
|
totalLoss prometheus.Counter
|
|
tradesExecuted prometheus.Counter
|
|
tradesProfitable prometheus.Counter
|
|
tradesUnprofitable prometheus.Counter
|
|
|
|
averageProfit prometheus.Gauge
|
|
averageLoss prometheus.Gauge
|
|
successRate prometheus.Gauge
|
|
|
|
gasCostTotal prometheus.Counter
|
|
netProfitTotal prometheus.Counter
|
|
|
|
// Profit by protocol
|
|
profitByProtocol map[ProtocolType]prometheus.Counter
|
|
|
|
// Database for detailed tracking
|
|
db *sql.DB
|
|
}
|
|
|
|
func (pm *ProfitabilityMetrics) RecordTrade(trade *CompletedTrade) error {
|
|
// Update Prometheus metrics
|
|
if trade.NetProfit.Cmp(big.NewFloat(0)) > 0 {
|
|
pm.totalProfit.Add(trade.NetProfit.Float64())
|
|
pm.tradesProfitable.Inc()
|
|
} else {
|
|
loss := new(big.Float).Abs(trade.NetProfit)
|
|
pm.totalLoss.Add(loss.Float64())
|
|
pm.tradesUnprofitable.Inc()
|
|
}
|
|
|
|
pm.tradesExecuted.Inc()
|
|
pm.gasCostTotal.Add(trade.GasCost.Float64())
|
|
pm.netProfitTotal.Add(trade.NetProfit.Float64())
|
|
|
|
// Calculate success rate
|
|
total := pm.tradesProfitable.Value() + pm.tradesUnprofitable.Value()
|
|
if total > 0 {
|
|
successRate := pm.tradesProfitable.Value() / total
|
|
pm.successRate.Set(successRate)
|
|
}
|
|
|
|
// Update per-protocol metrics
|
|
for _, hop := range trade.Path.Hops {
|
|
pm.profitByProtocol[hop.Protocol].Add(trade.NetProfit.Float64())
|
|
}
|
|
|
|
// Store in database for historical analysis
|
|
return pm.storeTrade(trade)
|
|
}
|
|
|
|
func (pm *ProfitabilityMetrics) storeTrade(trade *CompletedTrade) error {
|
|
_, err := pm.db.Exec(`
|
|
INSERT INTO trades (
|
|
tx_hash,
|
|
timestamp,
|
|
path,
|
|
gross_profit,
|
|
gas_cost,
|
|
net_profit,
|
|
success
|
|
) VALUES (?, ?, ?, ?, ?, ?, ?)
|
|
`,
|
|
trade.TxHash.Hex(),
|
|
trade.Timestamp,
|
|
trade.Path.String(),
|
|
trade.GrossProfit.String(),
|
|
trade.GasCost.String(),
|
|
trade.NetProfit.String(),
|
|
trade.Success,
|
|
)
|
|
|
|
return err
|
|
}
|
|
```
|
|
|
|
### 7.2 Real-Time Profitability Dashboard
|
|
|
|
```go
|
|
// pkg/metrics/dashboard.go
|
|
type Dashboard struct {
|
|
metrics *ProfitabilityMetrics
|
|
}
|
|
|
|
func (d *Dashboard) GetCurrentStats() *DashboardStats {
|
|
return &DashboardStats{
|
|
TotalTrades: d.metrics.tradesExecuted.Value(),
|
|
ProfitableTrades: d.metrics.tradesProfitable.Value(),
|
|
FailedTrades: d.metrics.tradesUnprofitable.Value(),
|
|
SuccessRate: d.metrics.successRate.Value(),
|
|
|
|
TotalProfit: d.metrics.totalProfit.Value(),
|
|
TotalLoss: d.metrics.totalLoss.Value(),
|
|
TotalGasCost: d.metrics.gasCostTotal.Value(),
|
|
NetProfit: d.metrics.netProfitTotal.Value(),
|
|
|
|
AverageProfit: d.metrics.averageProfit.Value(),
|
|
AverageLoss: d.metrics.averageLoss.Value(),
|
|
|
|
ProfitByProtocol: d.getProfitByProtocol(),
|
|
}
|
|
}
|
|
|
|
type DashboardStats struct {
|
|
TotalTrades float64
|
|
ProfitableTrades float64
|
|
FailedTrades float64
|
|
SuccessRate float64
|
|
|
|
TotalProfit float64
|
|
TotalLoss float64
|
|
TotalGasCost float64
|
|
NetProfit float64
|
|
|
|
AverageProfit float64
|
|
AverageLoss float64
|
|
|
|
ProfitByProtocol map[ProtocolType]float64
|
|
}
|
|
```
|
|
|
|
### 7.3 Performance Analysis
|
|
|
|
```go
|
|
// pkg/analytics/performance.go
|
|
type PerformanceAnalyzer struct {
|
|
db *sql.DB
|
|
}
|
|
|
|
func (pa *PerformanceAnalyzer) AnalyzeDaily() (*DailyReport, error) {
|
|
// Query database for today's trades
|
|
rows, err := pa.db.Query(`
|
|
SELECT
|
|
COUNT(*) as total_trades,
|
|
SUM(CASE WHEN net_profit > 0 THEN 1 ELSE 0 END) as profitable,
|
|
SUM(gross_profit) as gross_profit,
|
|
SUM(gas_cost) as gas_cost,
|
|
SUM(net_profit) as net_profit,
|
|
AVG(net_profit) as avg_profit,
|
|
MAX(net_profit) as max_profit,
|
|
MIN(net_profit) as min_profit
|
|
FROM trades
|
|
WHERE DATE(timestamp) = CURDATE()
|
|
`)
|
|
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
defer rows.Close()
|
|
|
|
var report DailyReport
|
|
rows.Next()
|
|
rows.Scan(
|
|
&report.TotalTrades,
|
|
&report.ProfitableTrades,
|
|
&report.GrossProfit,
|
|
&report.GasCost,
|
|
&report.NetProfit,
|
|
&report.AvgProfit,
|
|
&report.MaxProfit,
|
|
&report.MinProfit,
|
|
)
|
|
|
|
report.SuccessRate = float64(report.ProfitableTrades) / float64(report.TotalTrades)
|
|
report.ROI = report.NetProfit / report.TotalCapitalDeployed
|
|
|
|
return &report, nil
|
|
}
|
|
|
|
type DailyReport struct {
|
|
TotalTrades int
|
|
ProfitableTrades int
|
|
SuccessRate float64
|
|
|
|
GrossProfit float64
|
|
GasCost float64
|
|
NetProfit float64
|
|
|
|
AvgProfit float64
|
|
MaxProfit float64
|
|
MinProfit float64
|
|
|
|
TotalCapitalDeployed float64
|
|
ROI float64
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 8. Complete System Architecture
|
|
|
|
### 8.1 High-Level Flow
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ ARBITRUM SEQUENCER │
|
|
│ (Pending Transactions) │
|
|
└────────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ SEQUENCER READER │
|
|
│ • WebSocket connection │
|
|
│ • Connection health monitoring │
|
|
│ • Automatic reconnection │
|
|
└────────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ TRANSACTION PARSER │
|
|
│ • Protocol identification (< 1ms) │
|
|
│ • Protocol-specific parsing (< 5ms) │
|
|
│ • Parallel batch processing │
|
|
│ • ABI caching │
|
|
└────────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ VALIDATOR │
|
|
│ • Validate swap events │
|
|
│ • Check pool cache │
|
|
│ • Verify amounts (non-zero, reasonable) │
|
|
│ • Validate tokens (no zero addresses) │
|
|
└────────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ PRICE IMPACT CALCULATOR │
|
|
│ • Calculate price change from swap │
|
|
│ • Update pool reserves │
|
|
│ • Proper decimal handling │
|
|
└────────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ ARBITRAGE DETECTOR │
|
|
│ • Multi-hop path finding (BFS/DFS) │
|
|
│ • Calculate gross profit │
|
|
│ • Estimate gas cost │
|
|
│ • Calculate net profit │
|
|
│ • Filter by minimum profit │
|
|
└────────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ RISK MANAGEMENT │
|
|
│ • Position sizing │
|
|
│ • Slippage protection │
|
|
│ • Circuit breaker check │
|
|
│ • Max concurrent positions │
|
|
└────────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ EXECUTION ROUTER │
|
|
│ • Single vs batch decision │
|
|
│ • Gas optimization │
|
|
│ • Transaction building │
|
|
│ • Signing │
|
|
└────────────────────────────┬────────────────────────────────────┘
|
|
│
|
|
┌──────────┴──────────┐
|
|
│ │
|
|
▼ ▼
|
|
┌────────────────────┐ ┌────────────────────┐
|
|
│ FRONT-RUNNER │ │ BATCH EXECUTOR │
|
|
│ │ │ │
|
|
│ • Higher gas price │ │ • Multicall │
|
|
│ • Single trade │ │ • Multiple trades │
|
|
│ • Flashbots option │ │ • Gas savings │
|
|
└──────────┬─────────┘ └─────────┬──────────┘
|
|
│ │
|
|
└──────────┬───────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────┐
|
|
│ ETHEREUM NETWORK │
|
|
│ (Transaction Submitted) │
|
|
└─────────────┬────────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────┐
|
|
│ METRICS COLLECTOR │
|
|
│ │
|
|
│ • Record trade │
|
|
│ • Calculate profit │
|
|
│ • Update dashboard │
|
|
│ • Store in database │
|
|
└──────────────────────────┘
|
|
```
|
|
|
|
### 8.2 Performance Targets
|
|
|
|
```yaml
|
|
Latency Targets:
|
|
Sequencer to Parse: < 5ms
|
|
Parse to Validate: < 2ms
|
|
Validate to Detect: < 10ms
|
|
Detect to Execute: < 30ms
|
|
Total (End-to-End): < 50ms
|
|
|
|
Throughput Targets:
|
|
Transactions per second: 1000+
|
|
Concurrent parsing: 100+ goroutines
|
|
Batch size: 10-50 trades
|
|
|
|
Profitability Targets:
|
|
Success rate: > 85%
|
|
Minimum profit per trade: > 0.05 ETH
|
|
Daily trades: 50-200
|
|
Monthly ROI: > 20%
|
|
Max drawdown: < 5%
|
|
```
|
|
|
|
### 8.3 Configuration
|
|
|
|
```yaml
|
|
# config/production.yaml
|
|
|
|
sequencer:
|
|
primary_url: "wss://arb1.arbitrum.io/feed"
|
|
backup_urls:
|
|
- "wss://arbitrum-one.publicnode.com"
|
|
- "wss://arb-mainnet.g.alchemy.com/v2/YOUR_KEY"
|
|
reconnect_backoff: 1s
|
|
max_reconnect_delay: 60s
|
|
|
|
parsing:
|
|
worker_pool_size: 100
|
|
batch_size: 50
|
|
timeout: 5ms
|
|
|
|
arbitrage:
|
|
min_profit_usd: 50.0
|
|
max_hops: 4
|
|
path_finding_timeout: 10ms
|
|
|
|
execution:
|
|
strategy: "smart" # frontrun, batch, or smart
|
|
batch_threshold: 3
|
|
batch_window: 100ms
|
|
max_gas_price: "500000000000" # 500 gwei
|
|
gas_premium: 1.1 # 10% above target tx
|
|
|
|
gas:
|
|
strategy: "optimal" # fast, optimal, or economy
|
|
safety_margin: 1.1 # 10% buffer on estimates
|
|
|
|
risk:
|
|
max_slippage: 0.005 # 0.5%
|
|
max_position_pct: 0.1 # 10% of capital
|
|
max_concurrent_trades: 5
|
|
max_loss_per_hour: 1.0 # 1 ETH
|
|
max_failures_per_min: 10
|
|
circuit_breaker_reset: 1h
|
|
|
|
monitoring:
|
|
metrics_port: 9090
|
|
dashboard_port: 8080
|
|
log_level: "info"
|
|
enable_profiling: true
|
|
```
|
|
|
|
---
|
|
|
|
## 9. Implementation Roadmap
|
|
|
|
### Phase 1: Sequencer Integration (Week 1-2)
|
|
**Tasks:**
|
|
- [ ] Set up Arbitrum sequencer WebSocket connection
|
|
- [ ] Implement connection health monitoring
|
|
- [ ] Build automatic reconnection logic
|
|
- [ ] Create pending transaction stream processor
|
|
- [ ] Add latency monitoring (sequencer → parser)
|
|
- [ ] Test with live sequencer feed
|
|
- [ ] Validate transaction ordering
|
|
|
|
**Success Criteria:**
|
|
- Stable connection for 24+ hours
|
|
- < 1ms latency from sequencer event to parser
|
|
- Automatic recovery from disconnections
|
|
- Zero missed transactions
|
|
|
|
### Phase 2: High-Performance Parsing (Week 2-3)
|
|
**Tasks:**
|
|
- [ ] Implement optimized parser factory
|
|
- [ ] Build protocol identification cache
|
|
- [ ] Create parallel parsing worker pool
|
|
- [ ] Add per-protocol optimized parsers
|
|
- [ ] Implement decimal handling (critical)
|
|
- [ ] Add parsing validation
|
|
- [ ] Benchmark parsing performance
|
|
|
|
**Success Criteria:**
|
|
- < 5ms per transaction parse
|
|
- 100% accuracy on all supported protocols
|
|
- Proper decimal scaling
|
|
- No zero addresses
|
|
- Handle 1000+ tx/sec
|
|
|
|
### Phase 3: Arbitrage Detection (Week 3-4)
|
|
**Tasks:**
|
|
- [ ] Build market graph from pool cache
|
|
- [ ] Implement BFS/DFS path finding
|
|
- [ ] Create profitability calculator
|
|
- [ ] Add gas cost estimator
|
|
- [ ] Implement profit filtering
|
|
- [ ] Optimize path finding performance
|
|
- [ ] Test with historical data
|
|
|
|
**Success Criteria:**
|
|
- < 10ms path finding
|
|
- Find all 2-4 hop opportunities
|
|
- Accurate profit calculations
|
|
- Gas estimates within 10% of actual
|
|
|
|
### Phase 4: Execution Engine (Week 4-5)
|
|
**Tasks:**
|
|
- [ ] Build front-run executor
|
|
- [ ] Implement batch executor
|
|
- [ ] Create smart execution router
|
|
- [ ] Add gas optimization
|
|
- [ ] Integrate Flashbots (optional)
|
|
- [ ] Build transaction monitoring
|
|
- [ ] Add execution metrics
|
|
|
|
**Success Criteria:**
|
|
- < 30ms execution decision
|
|
- Successful front-running
|
|
- Gas optimization working
|
|
- Transaction monitoring
|
|
- Flashbots integration (if used)
|
|
|
|
### Phase 5: Risk Management (Week 5-6)
|
|
**Tasks:**
|
|
- [ ] Implement slippage protection
|
|
- [ ] Build circuit breaker
|
|
- [ ] Add position sizing
|
|
- [ ] Create risk monitoring dashboard
|
|
- [ ] Test failure scenarios
|
|
- [ ] Add automated alerts
|
|
|
|
**Success Criteria:**
|
|
- Slippage protection working
|
|
- Circuit breaker trips on losses
|
|
- Position limits enforced
|
|
- Alert system functional
|
|
|
|
### Phase 6: Metrics & Monitoring (Week 6-7)
|
|
**Tasks:**
|
|
- [ ] Set up Prometheus metrics
|
|
- [ ] Build profitability tracker
|
|
- [ ] Create real-time dashboard
|
|
- [ ] Add database for trade history
|
|
- [ ] Implement performance analyzer
|
|
- [ ] Build daily reports
|
|
|
|
**Success Criteria:**
|
|
- All metrics collected
|
|
- Dashboard shows real-time data
|
|
- Trade history stored
|
|
- Daily reports generated
|
|
|
|
### Phase 7: Testing & Optimization (Week 7-8)
|
|
**Tasks:**
|
|
- [ ] End-to-end testing with testnet
|
|
- [ ] Performance optimization
|
|
- [ ] Load testing (1000+ tx/sec)
|
|
- [ ] Chaos testing (failure scenarios)
|
|
- [ ] Production deployment preparation
|
|
- [ ] Documentation
|
|
|
|
**Success Criteria:**
|
|
- < 50ms end-to-end latency
|
|
- 85%+ success rate in testing
|
|
- Passes all load tests
|
|
- Handles failure scenarios
|
|
- Ready for production
|
|
|
|
---
|
|
|
|
## 10. Profitability Projections
|
|
|
|
### Conservative Scenario
|
|
```
|
|
Assumptions:
|
|
- 50 trades per day
|
|
- 60% success rate
|
|
- Average profit per successful trade: 0.05 ETH
|
|
- Average loss per failed trade: 0.02 ETH
|
|
- Average gas cost: 0.01 ETH per trade
|
|
|
|
Daily Calculation:
|
|
- Successful trades: 30 * 0.05 ETH = 1.5 ETH
|
|
- Failed trades: 20 * -0.02 ETH = -0.4 ETH
|
|
- Gas costs: 50 * 0.01 ETH = -0.5 ETH
|
|
- Net daily profit: 0.6 ETH
|
|
|
|
Monthly: 0.6 ETH * 30 = 18 ETH
|
|
Yearly: 18 ETH * 12 = 216 ETH
|
|
|
|
With 10 ETH capital deployed:
|
|
Monthly ROI: 180%
|
|
Yearly ROI: 2160%
|
|
```
|
|
|
|
### Moderate Scenario
|
|
```
|
|
Assumptions:
|
|
- 100 trades per day
|
|
- 75% success rate
|
|
- Average profit per successful trade: 0.08 ETH
|
|
- Average loss per failed trade: 0.02 ETH
|
|
- Average gas cost: 0.008 ETH per trade (batch optimization)
|
|
|
|
Daily Calculation:
|
|
- Successful trades: 75 * 0.08 ETH = 6.0 ETH
|
|
- Failed trades: 25 * -0.02 ETH = -0.5 ETH
|
|
- Gas costs: 100 * 0.008 ETH = -0.8 ETH
|
|
- Net daily profit: 4.7 ETH
|
|
|
|
Monthly: 4.7 ETH * 30 = 141 ETH
|
|
Yearly: 141 ETH * 12 = 1692 ETH
|
|
|
|
With 20 ETH capital deployed:
|
|
Monthly ROI: 705%
|
|
Yearly ROI: 8460%
|
|
```
|
|
|
|
### Optimistic Scenario
|
|
```
|
|
Assumptions:
|
|
- 200 trades per day
|
|
- 85% success rate
|
|
- Average profit per successful trade: 0.1 ETH
|
|
- Average loss per failed trade: 0.02 ETH
|
|
- Average gas cost: 0.006 ETH per trade (aggressive batching)
|
|
|
|
Daily Calculation:
|
|
- Successful trades: 170 * 0.1 ETH = 17.0 ETH
|
|
- Failed trades: 30 * -0.02 ETH = -0.6 ETH
|
|
- Gas costs: 200 * 0.006 ETH = -1.2 ETH
|
|
- Net daily profit: 15.2 ETH
|
|
|
|
Monthly: 15.2 ETH * 30 = 456 ETH
|
|
Yearly: 456 ETH * 12 = 5472 ETH
|
|
|
|
With 50 ETH capital deployed:
|
|
Monthly ROI: 912%
|
|
Yearly ROI: 10944%
|
|
```
|
|
|
|
**Note**: These are theoretical projections. Actual results will vary based on market conditions, competition, execution quality, and unforeseen factors.
|
|
|
|
---
|
|
|
|
## 11. Key Success Factors
|
|
|
|
### 1. Speed is Everything
|
|
- **50ms end-to-end latency** is the target
|
|
- Every millisecond counts in MEV
|
|
- Optimize at every layer (network, parsing, detection, execution)
|
|
|
|
### 2. Sequencer Access is the Moat
|
|
- Direct sequencer feed = competitive advantage
|
|
- Reading pending transactions before they're on-chain
|
|
- 100-500ms time advantage over traditional bots
|
|
|
|
### 3. Accurate Parsing is Critical
|
|
- Wrong parsing = wrong profit calculations = losses
|
|
- Protocol-specific parsers for accuracy
|
|
- Proper decimal handling (non-negotiable)
|
|
- Validate everything
|
|
|
|
### 4. Gas Optimization Directly Impacts Profit
|
|
- High gas prices eat into profits
|
|
- Batching saves gas
|
|
- Smart gas pricing (don't overpay)
|
|
- Flashbots for MEV protection
|
|
|
|
### 5. Risk Management Protects Capital
|
|
- Slippage protection prevents bad trades
|
|
- Circuit breaker prevents cascading losses
|
|
- Position sizing limits exposure
|
|
- Monitor everything
|
|
|
|
### 6. Continuous Optimization
|
|
- Monitor metrics constantly
|
|
- Analyze what works and what doesn't
|
|
- Optimize slow paths
|
|
- Add new protocols as they launch
|
|
|
|
---
|
|
|
|
## 12. Next Steps
|
|
|
|
1. **Review and approve this plan**
|
|
2. **Set up development environment**
|
|
3. **Begin Phase 1: Sequencer Integration**
|
|
4. **Follow task breakdown from 07_TASK_BREAKDOWN.md**
|
|
5. **Create feature branches: `feature/v2/sequencer/<task-id>`**
|
|
6. **Implement with 100% test coverage**
|
|
7. **Deploy to testnet for validation**
|
|
8. **Deploy to production with monitoring**
|
|
|
|
---
|
|
|
|
**This is the complete profitability plan. The sequencer reader/parser is the foundation that enables everything else. Let's build it.**
|