Files
mev-beta/docs/planning/04_PROFITABILITY_PLAN.md
Administrator 9a92f43edf docs: add comprehensive profitability plan with sequencer integration
- Sequencer integration strategy (core competitive advantage)
- High-performance parsing architecture (< 5ms target)
- Multi-hop arbitrage detection algorithms
- Execution strategies (front-running, batching)
- Dynamic gas optimization for maximum profit
- Comprehensive risk management (slippage, circuit breaker, position sizing)
- Profitability metrics and tracking
- Complete system architecture with performance targets

Key Features:
- Sequencer front-running (100-500ms advantage)
- Multi-protocol coverage (13+ DEXs)
- Sub-50ms end-to-end latency target
- 85%+ success rate target
- Batch execution for gas savings
- Flashbots integration option

Profitability Targets:
- Conservative: 18 ETH/month (180% monthly ROI)
- Moderate: 141 ETH/month (705% monthly ROI)
- Optimistic: 456 ETH/month (912% monthly ROI)

Implementation: 8-week roadmap with clear phases

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-10 14:34:19 +01:00

1739 lines
51 KiB
Markdown

# V2 Profitability Plan
## Executive Summary
This document outlines the complete architecture for a profitable MEV bot that leverages **Arbitrum sequencer access** to execute arbitrage trades ahead of the chain. The sequencer reader/parser is the **core competitive advantage**, providing visibility into pending transactions before they're confirmed on-chain.
**Key Profitability Drivers:**
1. **Sequencer Front-Running**: Execute before price-moving transactions hit the chain
2. **Multi-Protocol Coverage**: Capture arbitrage across 13+ DEX protocols
3. **Fast Parsing**: Sub-millisecond transaction parsing and validation
4. **Batch Execution**: Combine multiple opportunities to save gas
5. **Dynamic Gas Optimization**: Maximize net profit through intelligent gas pricing
6. **Risk Management**: Protect capital with slippage limits and circuit breakers
**Target Metrics:**
- **Latency**: < 50ms from sequencer event to execution decision
- **Success Rate**: > 85% of executed trades profitable
- **Average Profit**: > 0.05 ETH per trade (after gas)
- **Daily Volume**: 50-200 trades per day
- **ROI**: > 20% monthly on deployed capital
---
## 1. Sequencer Integration Strategy
### 1.1 Core Advantage: Reading Pending Transactions
**Arbitrum Sequencer Access:**
```
Sequencer Feed (WebSocket)
Pending Transaction Stream
Parse ALL Swap Events (before on-chain)
Calculate Price Impact
Detect Arbitrage Opportunity
Execute Front-Run Transaction
Capture Profit
```
**Why This Matters:**
- Traditional bots read from the mempool AFTER transactions are broadcast
- We read from the sequencer BEFORE transactions are finalized
- This gives us a **time advantage** of 100-500ms
- In MEV, milliseconds = millions
### 1.2 Sequencer Reader Architecture
```go
// pkg/sequencer/reader.go
type SequencerReader struct {
wsClient *websocket.Conn
parsers *parsers.Factory
validator *validation.Validator
cache *cache.PoolCache
arbDetector *arbitrage.Detector
executor *execution.Executor
metrics *metrics.Collector
// Performance critical
parseLatency prometheus.Histogram
detectLatency prometheus.Histogram
executeLatency prometheus.Histogram
}
func (r *SequencerReader) Start(ctx context.Context) error {
// Subscribe to Arbitrum sequencer WebSocket feed
err := r.wsClient.Subscribe(ctx, "newPendingTransactions")
if err != nil {
return fmt.Errorf("sequencer subscription failed: %w", err)
}
// Process pending transactions in real-time
for {
select {
case tx := <-r.wsClient.Transactions():
// CRITICAL PATH: Every nanosecond counts
go r.ProcessPendingTx(ctx, tx)
case <-ctx.Done():
return ctx.Err()
}
}
}
func (r *SequencerReader) ProcessPendingTx(ctx context.Context, tx *types.Transaction) {
startTime := time.Now()
// Step 1: Parse transaction (< 5ms target)
events, err := r.parsers.ParseTransaction(tx)
if err != nil {
r.metrics.ParseErrors.Inc()
return
}
r.parseLatency.Observe(time.Since(startTime).Seconds())
// Step 2: Validate events (< 2ms target)
validEvents := r.validator.FilterValid(events)
if len(validEvents) == 0 {
return
}
// Step 3: Calculate price impact (< 3ms target)
priceImpacts := r.calculatePriceImpacts(validEvents)
// Step 4: Detect arbitrage (< 10ms target)
opportunities := r.arbDetector.FindOpportunities(priceImpacts)
r.detectLatency.Observe(time.Since(startTime).Seconds())
// Step 5: Execute profitable trades (< 30ms target)
for _, opp := range opportunities {
if opp.NetProfit.Cmp(r.minProfitThreshold) > 0 {
r.executor.ExecuteFrontRun(ctx, opp, tx)
}
}
r.executeLatency.Observe(time.Since(startTime).Seconds())
// TOTAL TARGET: < 50ms from sequencer event to execution
}
```
### 1.3 Sequencer Connection Management
```go
// pkg/sequencer/connection.go
type ConnectionManager struct {
primaryURL string
backupURLs []string
currentConn *websocket.Conn
reconnectBackoff time.Duration
maxReconnectDelay time.Duration
}
func (cm *ConnectionManager) MaintainConnection(ctx context.Context) error {
for {
conn, err := cm.connectWithBackoff(ctx)
if err != nil {
return err
}
cm.currentConn = conn
// Monitor connection health
if err := cm.monitorConnection(ctx, conn); err != nil {
log.Error().Err(err).Msg("Connection lost, reconnecting...")
continue
}
}
}
func (cm *ConnectionManager) connectWithBackoff(ctx context.Context) (*websocket.Conn, error) {
backoff := cm.reconnectBackoff
for {
// Try primary endpoint
conn, err := websocket.Dial(cm.primaryURL)
if err == nil {
log.Info().Msg("Connected to primary sequencer endpoint")
return conn, nil
}
// Try backup endpoints
for _, backupURL := range cm.backupURLs {
conn, err := websocket.Dial(backupURL)
if err == nil {
log.Info().Str("endpoint", backupURL).Msg("Connected to backup sequencer endpoint")
return conn, nil
}
}
// Exponential backoff
select {
case <-time.After(backoff):
backoff = min(backoff*2, cm.maxReconnectDelay)
case <-ctx.Done():
return nil, ctx.Err()
}
}
}
```
### 1.4 Sequencer Data Validation
**CRITICAL**: Sequencer data is unconfirmed and may be reorganized. We MUST validate:
```go
// pkg/sequencer/validator.go
type SequencerDataValidator struct {
cache *cache.PoolCache
}
func (v *SequencerDataValidator) ValidatePendingTx(tx *types.Transaction) error {
// 1. Validate transaction structure
if tx.To() == nil {
return errors.New("transaction has no recipient")
}
// 2. Validate gas price is reasonable
if tx.GasPrice().Cmp(maxReasonableGas) > 0 {
return errors.New("gas price unreasonably high")
}
// 3. Validate nonce progression
if !v.isNonceValid(tx) {
return errors.New("invalid nonce")
}
// 4. Validate signature
if !v.isSignatureValid(tx) {
return errors.New("invalid signature")
}
return nil
}
func (v *SequencerDataValidator) ValidateSwapEvent(event *SwapEvent) error {
// 1. Validate pool exists in cache
pool := v.cache.GetByAddress(event.PoolAddress)
if pool == nil {
return errors.New("unknown pool")
}
// 2. Validate tokens match pool
if pool.Token0 != event.Token0 || pool.Token1 != event.Token1 {
return errors.New("token mismatch")
}
// 3. Validate amounts are non-zero
if event.Amount0.Cmp(big.NewInt(0)) == 0 && event.Amount1.Cmp(big.NewInt(0)) == 0 {
return errors.New("zero amounts")
}
// 4. Validate amounts are reasonable (not dust, not astronomical)
if !v.isAmountReasonable(event.Amount0, pool.Token0Decimals) {
return errors.New("unreasonable amount0")
}
if !v.isAmountReasonable(event.Amount1, pool.Token1Decimals) {
return errors.New("unreasonable amount1")
}
return nil
}
```
---
## 2. Fast Parsing for Profitability
### 2.1 Parsing Speed Requirements
**Why Speed Matters:**
- If we take 100ms to parse, another bot executes first
- If we take 50ms, we capture the profit
- **Target: < 5ms per transaction parse**
### 2.2 Optimized Parser Architecture
```go
// pkg/parsers/optimized_factory.go
type OptimizedFactory struct {
// Pre-compiled ABI parsers (cached at startup)
uniswapV2ABI abi.ABI
uniswapV3ABI abi.ABI
curveABI abi.ABI
balancerV2ABI abi.ABI
kyberABI abi.ABI
camelotABI abi.ABI
// Protocol router cache (address -> protocol type)
routerCache map[common.Address]ProtocolType
routerCacheMutex sync.RWMutex
// Pool cache for fast lookups
poolCache *cache.PoolCache
}
func (f *OptimizedFactory) ParseTransaction(tx *types.Transaction) ([]*SwapEvent, error) {
// Fast path: Check if this is a known router
f.routerCacheMutex.RLock()
protocol, exists := f.routerCache[*tx.To()]
f.routerCacheMutex.RUnlock()
if exists {
// Use cached protocol parser (FAST)
return f.parseWithProtocol(tx, protocol)
}
// Slow path: Identify protocol from transaction data
protocol = f.identifyProtocol(tx)
if protocol == ProtocolUnknown {
return nil, ErrUnknownProtocol
}
// Cache for next time
f.routerCacheMutex.Lock()
f.routerCache[*tx.To()] = protocol
f.routerCacheMutex.Unlock()
return f.parseWithProtocol(tx, protocol)
}
func (f *OptimizedFactory) parseWithProtocol(tx *types.Transaction, protocol ProtocolType) ([]*SwapEvent, error) {
// Route to optimized protocol-specific parser
switch protocol {
case ProtocolUniswapV2:
return f.parseUniswapV2(tx) // Optimized for UniV2
case ProtocolUniswapV3:
return f.parseUniswapV3(tx) // Optimized for UniV3
case ProtocolCurve:
return f.parseCurve(tx) // Optimized for Curve
// ... other protocols
default:
return nil, ErrUnknownProtocol
}
}
```
### 2.3 Parallel Parsing for Batches
```go
// pkg/parsers/parallel.go
type ParallelParser struct {
factory *OptimizedFactory
workerPool *WorkerPool
}
func (p *ParallelParser) ParseBatch(txs []*types.Transaction) ([]*SwapEvent, error) {
results := make(chan *ParseResult, len(txs))
// Fan-out: Distribute transactions to worker pool
for _, tx := range txs {
tx := tx // Capture for goroutine
p.workerPool.Submit(func() {
events, err := p.factory.ParseTransaction(tx)
results <- &ParseResult{Events: events, Err: err}
})
}
// Fan-in: Collect results
var allEvents []*SwapEvent
for i := 0; i < len(txs); i++ {
result := <-results
if result.Err == nil {
allEvents = append(allEvents, result.Events...)
}
}
return allEvents, nil
}
```
---
## 3. Arbitrage Detection Algorithms
### 3.1 Multi-Hop Path Finding
```go
// pkg/arbitrage/pathfinder.go
type PathFinder struct {
marketGraph *MarketGraph
maxHops int // Typically 2-4 hops
minProfit *big.Float
}
type ArbitragePath struct {
Hops []*Hop
InputToken common.Address
OutputToken common.Address
InputAmount *big.Int
OutputAmount *big.Int
GrossProfit *big.Float
GasCost *big.Int
NetProfit *big.Float
}
type Hop struct {
Pool *PoolState
Protocol ProtocolType
TokenIn common.Address
TokenOut common.Address
AmountIn *big.Int
AmountOut *big.Int
}
func (pf *PathFinder) FindArbitragePaths(priceImpact *PriceImpact) []*ArbitragePath {
// Starting token and amount from price impact
startToken := priceImpact.TokenOut
startAmount := priceImpact.AmountOut
var paths []*ArbitragePath
// BFS/DFS to find circular paths back to start token
pf.searchPaths(startToken, startToken, startAmount, []*Hop{}, &paths)
// Filter paths by profitability
var profitablePaths []*ArbitragePath
for _, path := range paths {
if path.NetProfit.Cmp(pf.minProfit) > 0 {
profitablePaths = append(profitablePaths, path)
}
}
// Sort by net profit (highest first)
sort.Slice(profitablePaths, func(i, j int) bool {
return profitablePaths[i].NetProfit.Cmp(profitablePaths[j].NetProfit) > 0
})
return profitablePaths
}
func (pf *PathFinder) searchPaths(
currentToken common.Address,
targetToken common.Address,
currentAmount *big.Int,
currentPath []*Hop,
results *[]*ArbitragePath,
) {
// Max depth reached
if len(currentPath) >= pf.maxHops {
return
}
// Get all pools with currentToken
pools := pf.marketGraph.GetPoolsWithToken(currentToken)
for _, pool := range pools {
// Determine output token
var outputToken common.Address
if pool.Token0 == currentToken {
outputToken = pool.Token1
} else {
outputToken = pool.Token0
}
// Calculate output amount
outputAmount := pf.calculateSwapOutput(pool, currentToken, outputToken, currentAmount)
if outputAmount.Cmp(big.NewInt(0)) == 0 {
continue // Insufficient liquidity
}
// Create hop
hop := &Hop{
Pool: pool,
Protocol: pool.Protocol,
TokenIn: currentToken,
TokenOut: outputToken,
AmountIn: currentAmount,
AmountOut: outputAmount,
}
// Add to path
newPath := append(currentPath, hop)
// Check if we've returned to target token (arbitrage loop complete)
if outputToken == targetToken && len(newPath) >= 2 {
// Calculate profitability
path := pf.buildArbitragePath(newPath)
if path != nil {
*results = append(*results, path)
}
continue // Don't recurse further
}
// Recurse to find more hops
pf.searchPaths(outputToken, targetToken, outputAmount, newPath, results)
}
}
```
### 3.2 Profitability Calculation
```go
// pkg/arbitrage/profitability.go
type ProfitCalculator struct {
gasOracle *GasOracle
ethPrice *big.Float // ETH price in USD
}
func (pc *ProfitCalculator) CalculateNetProfit(path *ArbitragePath) *big.Float {
// 1. Calculate gross profit (output - input, in same token)
inputAmountFloat := new(big.Float).SetInt(path.InputAmount)
outputAmountFloat := new(big.Float).SetInt(path.OutputAmount)
// Scale to 18 decimals for comparison
inputScaled := pc.scaleToDecimals(inputAmountFloat, 18)
outputScaled := pc.scaleToDecimals(outputAmountFloat, 18)
grossProfit := new(big.Float).Sub(outputScaled, inputScaled)
// 2. Estimate gas cost
gasCost := pc.estimateGasCost(path)
gasCostFloat := new(big.Float).SetInt(gasCost)
// 3. Calculate net profit
netProfit := new(big.Float).Sub(grossProfit, gasCostFloat)
return netProfit
}
func (pc *ProfitCalculator) estimateGasCost(path *ArbitragePath) *big.Int {
// Base transaction cost
baseCost := big.NewInt(21000)
// Per-hop cost (varies by protocol)
var totalGas = big.NewInt(0)
totalGas.Add(totalGas, baseCost)
for _, hop := range path.Hops {
var hopGas *big.Int
switch hop.Protocol {
case ProtocolUniswapV2:
hopGas = big.NewInt(100000) // ~100k gas per UniV2 swap
case ProtocolUniswapV3:
hopGas = big.NewInt(150000) // ~150k gas per UniV3 swap
case ProtocolCurve:
hopGas = big.NewInt(120000) // ~120k gas per Curve swap
case ProtocolBalancerV2:
hopGas = big.NewInt(130000) // ~130k gas per Balancer swap
default:
hopGas = big.NewInt(120000) // Default estimate
}
totalGas.Add(totalGas, hopGas)
}
// Get current gas price from oracle
gasPrice := pc.gasOracle.GetCurrentGasPrice()
// Calculate total cost
gasCost := new(big.Int).Mul(totalGas, gasPrice)
return gasCost
}
```
### 3.3 Real-Time Arbitrage Detection
```go
// pkg/arbitrage/detector.go
type Detector struct {
pathFinder *PathFinder
profitCalc *ProfitCalculator
marketGraph *MarketGraph
minProfitUSD float64 // Minimum profit in USD
// Performance metrics
detectLatency prometheus.Histogram
opportunitiesFound prometheus.Counter
}
func (d *Detector) FindOpportunities(priceImpacts []*PriceImpact) []*ArbitragePath {
startTime := time.Now()
defer func() {
d.detectLatency.Observe(time.Since(startTime).Seconds())
}()
var allPaths []*ArbitragePath
// Check each price impact for arbitrage opportunities
for _, impact := range priceImpacts {
// Find all possible arbitrage paths
paths := d.pathFinder.FindArbitragePaths(impact)
// Calculate profitability for each path
for _, path := range paths {
netProfit := d.profitCalc.CalculateNetProfit(path)
path.NetProfit = netProfit
// Filter by minimum profit
minProfitFloat := big.NewFloat(d.minProfitUSD)
if netProfit.Cmp(minProfitFloat) > 0 {
allPaths = append(allPaths, path)
d.opportunitiesFound.Inc()
}
}
}
return allPaths
}
```
---
## 4. Execution Strategies
### 4.1 Front-Running Execution
```go
// pkg/execution/frontrun.go
type FrontRunner struct {
client *ethclient.Client
signer types.Signer
privateKey *ecdsa.PrivateKey
flashbots *FlashbotsClient // Optional: MEV protection
maxGasPrice *big.Int
gasPremium float64 // Multiplier above target tx gas price
}
func (fr *FrontRunner) ExecuteFrontRun(
ctx context.Context,
opportunity *ArbitragePath,
targetTx *types.Transaction,
) (*types.Transaction, error) {
// 1. Build arbitrage transaction
arbTx, err := fr.buildArbitrageTx(opportunity)
if err != nil {
return nil, fmt.Errorf("failed to build arbitrage tx: %w", err)
}
// 2. Set gas price higher than target transaction
targetGasPrice := targetTx.GasPrice()
frontRunGasPrice := fr.calculateFrontRunGasPrice(targetGasPrice)
if frontRunGasPrice.Cmp(fr.maxGasPrice) > 0 {
return nil, errors.New("required gas price exceeds maximum")
}
arbTx.GasPrice = frontRunGasPrice
// 3. Sign transaction
signedTx, err := types.SignTx(arbTx, fr.signer, fr.privateKey)
if err != nil {
return nil, fmt.Errorf("failed to sign tx: %w", err)
}
// 4. Submit to network (choose path based on configuration)
if fr.flashbots != nil {
// Use Flashbots to avoid frontrunning ourselves
return fr.flashbots.SendBundle(ctx, []*types.Transaction{signedTx, targetTx})
} else {
// Direct submission to mempool
err = fr.client.SendTransaction(ctx, signedTx)
if err != nil {
return nil, fmt.Errorf("failed to send tx: %w", err)
}
return signedTx, nil
}
}
func (fr *FrontRunner) calculateFrontRunGasPrice(targetGasPrice *big.Int) *big.Int {
// Add premium to ensure we're ahead
premium := new(big.Float).SetFloat64(fr.gasPremium)
targetFloat := new(big.Float).SetInt(targetGasPrice)
frontRunFloat := new(big.Float).Mul(targetFloat, premium)
frontRunGasPrice, _ := frontRunFloat.Int(nil)
return frontRunGasPrice
}
```
### 4.2 Batch Execution
```go
// pkg/execution/batch.go
type BatchExecutor struct {
multicallContract common.Address
client *ethclient.Client
signer types.Signer
privateKey *ecdsa.PrivateKey
}
// Multicall contract interface
type Multicall interface {
Aggregate(calls []Call) (*big.Int, []byte, error)
}
type Call struct {
Target common.Address
CallData []byte
}
func (be *BatchExecutor) ExecuteBatch(
ctx context.Context,
opportunities []*ArbitragePath,
) (*types.Transaction, error) {
// 1. Build multicall data
calls := make([]Call, 0, len(opportunities))
for _, opp := range opportunities {
callData, err := be.buildSwapCallData(opp)
if err != nil {
continue // Skip this opportunity
}
calls = append(calls, Call{
Target: opp.Hops[0].Pool.Address,
CallData: callData,
})
}
if len(calls) == 0 {
return nil, errors.New("no valid calls to batch")
}
// 2. Encode multicall transaction
multicallABI, _ := abi.JSON(strings.NewReader(MulticallABI))
multicallData, err := multicallABI.Pack("aggregate", calls)
if err != nil {
return nil, fmt.Errorf("failed to pack multicall: %w", err)
}
// 3. Build transaction
nonce, err := be.client.PendingNonceAt(ctx, crypto.PubkeyToAddress(be.privateKey.PublicKey))
if err != nil {
return nil, err
}
gasPrice, err := be.client.SuggestGasPrice(ctx)
if err != nil {
return nil, err
}
tx := types.NewTransaction(
nonce,
be.multicallContract,
big.NewInt(0), // No ETH sent
2000000, // Gas limit (adjust based on batch size)
gasPrice,
multicallData,
)
// 4. Sign and send
signedTx, err := types.SignTx(tx, be.signer, be.privateKey)
if err != nil {
return nil, err
}
err = be.client.SendTransaction(ctx, signedTx)
if err != nil {
return nil, err
}
return signedTx, nil
}
```
### 4.3 Smart Execution Router
```go
// pkg/execution/router.go
type ExecutionRouter struct {
frontRunner *FrontRunner
batchExecutor *BatchExecutor
batchThreshold int // Min opportunities to batch
batchWindow time.Duration // Time to wait for batching
}
func (er *ExecutionRouter) Execute(
ctx context.Context,
opportunities []*ArbitragePath,
targetTx *types.Transaction,
) error {
// Decision: Single vs Batch execution
if len(opportunities) == 1 {
// Single opportunity: Front-run immediately
_, err := er.frontRunner.ExecuteFrontRun(ctx, opportunities[0], targetTx)
return err
}
if len(opportunities) >= er.batchThreshold {
// Multiple opportunities: Batch for gas savings
_, err := er.batchExecutor.ExecuteBatch(ctx, opportunities)
return err
}
// Between 1 and threshold: Wait for batch window
select {
case <-time.After(er.batchWindow):
// Execute what we have
if len(opportunities) > 1 {
_, err := er.batchExecutor.ExecuteBatch(ctx, opportunities)
return err
}
_, err := er.frontRunner.ExecuteFrontRun(ctx, opportunities[0], targetTx)
return err
case <-ctx.Done():
return ctx.Err()
}
}
```
---
## 5. Gas Optimization for Maximum Profit
### 5.1 Dynamic Gas Pricing
```go
// pkg/gas/oracle.go
type GasOracle struct {
client *ethclient.Client
historicalPrices []HistoricalGasPrice
// Gas price strategy
strategy GasStrategy
}
type GasStrategy int
const (
StrategyFast GasStrategy = iota // Pay more to execute faster
StrategyOptimal // Balance speed vs cost
StrategyEconomy // Minimize gas cost
)
type HistoricalGasPrice struct {
Timestamp time.Time
GasPrice *big.Int
BaseFee *big.Int
}
func (go *GasOracle) GetOptimalGasPrice(opportunity *ArbitragePath) *big.Int {
// Get current network gas prices
currentGasPrice, _ := go.client.SuggestGasPrice(context.Background())
// Calculate maximum gas price we can pay while remaining profitable
maxProfitableGasPrice := go.calculateMaxProfitableGasPrice(opportunity)
// Apply strategy
switch go.strategy {
case StrategyFast:
// Use higher percentile (90th) to ensure quick inclusion
return go.calculatePercentileGasPrice(90)
case StrategyOptimal:
// Use median gas price, capped at max profitable
medianGas := go.calculatePercentileGasPrice(50)
if medianGas.Cmp(maxProfitableGasPrice) > 0 {
return maxProfitableGasPrice
}
return medianGas
case StrategyEconomy:
// Use 25th percentile, capped at max profitable
lowGas := go.calculatePercentileGasPrice(25)
if lowGas.Cmp(maxProfitableGasPrice) > 0 {
return maxProfitableGasPrice
}
return lowGas
default:
return currentGasPrice
}
}
func (go *GasOracle) calculateMaxProfitableGasPrice(opportunity *ArbitragePath) *big.Int {
// Given net profit and gas usage, what's the max gas price?
// netProfit = grossProfit - (gasUsed * gasPrice)
// gasPrice = (grossProfit - netProfit) / gasUsed
grossProfit := opportunity.GrossProfit
minAcceptableProfit := big.NewFloat(0.01) // 0.01 ETH minimum
profitForGas := new(big.Float).Sub(grossProfit, minAcceptableProfit)
gasUsed := new(big.Float).SetInt64(int64(opportunity.EstimatedGas))
maxGasPrice := new(big.Float).Quo(profitForGas, gasUsed)
maxGasPriceInt, _ := maxGasPrice.Int(nil)
return maxGasPriceInt
}
```
### 5.2 Gas Estimation
```go
// pkg/gas/estimator.go
type GasEstimator struct {
client *ethclient.Client
// Cached estimates by protocol and hop count
estimateCache map[string]uint64
cacheMutex sync.RWMutex
}
func (ge *GasEstimator) EstimateArbitrageGas(path *ArbitragePath) (uint64, error) {
// Generate cache key
cacheKey := ge.generateCacheKey(path)
// Check cache
ge.cacheMutex.RLock()
if cached, exists := ge.estimateCache[cacheKey]; exists {
ge.cacheMutex.RUnlock()
return cached, nil
}
ge.cacheMutex.RUnlock()
// Build transaction
tx, err := ge.buildArbitrageTx(path)
if err != nil {
return 0, err
}
// Estimate gas
estimatedGas, err := ge.client.EstimateGas(context.Background(), ethereum.CallMsg{
From: crypto.PubkeyToAddress(ge.privateKey.PublicKey),
To: tx.To(),
Gas: 0,
GasPrice: big.NewInt(0),
Value: big.NewInt(0),
Data: tx.Data(),
})
if err != nil {
return 0, err
}
// Add safety margin (10%)
estimatedGas = estimatedGas * 110 / 100
// Cache for future use
ge.cacheMutex.Lock()
ge.estimateCache[cacheKey] = estimatedGas
ge.cacheMutex.Unlock()
return estimatedGas, nil
}
```
---
## 6. Risk Management
### 6.1 Slippage Protection
```go
// pkg/execution/slippage.go
type SlippageProtector struct {
maxSlippage float64 // e.g., 0.005 = 0.5%
}
func (sp *SlippageProtector) CalculateMinOutput(
expectedOutput *big.Int,
) *big.Int {
// Calculate minimum acceptable output with slippage
expectedFloat := new(big.Float).SetInt(expectedOutput)
slippageFactor := 1.0 - sp.maxSlippage
minOutputFloat := new(big.Float).Mul(
expectedFloat,
big.NewFloat(slippageFactor),
)
minOutput, _ := minOutputFloat.Int(nil)
return minOutput
}
func (sp *SlippageProtector) ValidateExecution(
expectedProfit *big.Float,
actualProfit *big.Float,
) error {
// Check if actual profit is within acceptable slippage
expectedFloat := expectedProfit
slippageFactor := big.NewFloat(1.0 - sp.maxSlippage)
minAcceptableProfit := new(big.Float).Mul(expectedFloat, slippageFactor)
if actualProfit.Cmp(minAcceptableProfit) < 0 {
return fmt.Errorf(
"slippage too high: expected %s, got %s",
expectedProfit.String(),
actualProfit.String(),
)
}
return nil
}
```
### 6.2 Circuit Breaker
```go
// pkg/safety/circuit_breaker.go
type CircuitBreaker struct {
maxLossPerHour *big.Float
maxFailuresPerMin int
// State tracking
hourlyLoss *big.Float
recentFailures []time.Time
// Circuit state
state CircuitState
stateMutex sync.RWMutex
resetAfter time.Duration
}
type CircuitState int
const (
StateClosed CircuitState = iota // Normal operation
StateOpen // Circuit tripped, blocking trades
StateHalfOpen // Testing if safe to resume
)
func (cb *CircuitBreaker) AllowExecution() bool {
cb.stateMutex.RLock()
defer cb.stateMutex.RUnlock()
return cb.state == StateClosed || cb.state == StateHalfOpen
}
func (cb *CircuitBreaker) RecordFailure(loss *big.Float) {
cb.stateMutex.Lock()
defer cb.stateMutex.Unlock()
// Track recent failures
cb.recentFailures = append(cb.recentFailures, time.Now())
// Add to hourly loss
cb.hourlyLoss.Add(cb.hourlyLoss, loss)
// Check if we should trip the circuit
if cb.shouldTrip() {
cb.state = StateOpen
log.Error().Msg("Circuit breaker TRIPPED - halting trading")
// Schedule automatic reset
time.AfterFunc(cb.resetAfter, cb.attemptReset)
}
}
func (cb *CircuitBreaker) shouldTrip() bool {
// Check hourly loss threshold
if cb.hourlyLoss.Cmp(cb.maxLossPerHour) > 0 {
return true
}
// Check failure rate
oneMinAgo := time.Now().Add(-1 * time.Minute)
recentFailureCount := 0
for _, t := range cb.recentFailures {
if t.After(oneMinAgo) {
recentFailureCount++
}
}
if recentFailureCount >= cb.maxFailuresPerMin {
return true
}
return false
}
func (cb *CircuitBreaker) attemptReset() {
cb.stateMutex.Lock()
defer cb.stateMutex.Unlock()
// Move to half-open state (allow limited testing)
cb.state = StateHalfOpen
log.Info().Msg("Circuit breaker entering HALF-OPEN state")
// Reset counters
cb.hourlyLoss = big.NewFloat(0)
cb.recentFailures = []time.Time{}
}
```
### 6.3 Position Sizing
```go
// pkg/safety/position_sizer.go
type PositionSizer struct {
totalCapital *big.Float
maxPositionSize float64 // e.g., 0.1 = 10% of capital
maxConcurrentTrades int
currentPositions []*Position
positionsMutex sync.RWMutex
}
type Position struct {
ID string
Size *big.Float
EntryTime time.Time
ExitTime *time.Time
Profit *big.Float
}
func (ps *PositionSizer) CalculatePositionSize(opportunity *ArbitragePath) *big.Int {
ps.positionsMutex.RLock()
defer ps.positionsMutex.RUnlock()
// Calculate maximum allowed position size
maxSize := new(big.Float).Mul(
ps.totalCapital,
big.NewFloat(ps.maxPositionSize),
)
// Check if we have room for another position
if len(ps.currentPositions) >= ps.maxConcurrentTrades {
return big.NewInt(0) // No room for more trades
}
// Use the smaller of: max position size or opportunity input amount
opportunitySize := new(big.Float).SetInt(opportunity.InputAmount)
var positionSize *big.Float
if opportunitySize.Cmp(maxSize) < 0 {
positionSize = opportunitySize
} else {
positionSize = maxSize
}
positionSizeInt, _ := positionSize.Int(nil)
return positionSizeInt
}
```
---
## 7. Profitability Metrics and Tracking
### 7.1 Metrics Collection
```go
// pkg/metrics/profitability.go
type ProfitabilityMetrics struct {
// Prometheus metrics
totalProfit prometheus.Counter
totalLoss prometheus.Counter
tradesExecuted prometheus.Counter
tradesProfitable prometheus.Counter
tradesUnprofitable prometheus.Counter
averageProfit prometheus.Gauge
averageLoss prometheus.Gauge
successRate prometheus.Gauge
gasCostTotal prometheus.Counter
netProfitTotal prometheus.Counter
// Profit by protocol
profitByProtocol map[ProtocolType]prometheus.Counter
// Database for detailed tracking
db *sql.DB
}
func (pm *ProfitabilityMetrics) RecordTrade(trade *CompletedTrade) error {
// Update Prometheus metrics
if trade.NetProfit.Cmp(big.NewFloat(0)) > 0 {
pm.totalProfit.Add(trade.NetProfit.Float64())
pm.tradesProfitable.Inc()
} else {
loss := new(big.Float).Abs(trade.NetProfit)
pm.totalLoss.Add(loss.Float64())
pm.tradesUnprofitable.Inc()
}
pm.tradesExecuted.Inc()
pm.gasCostTotal.Add(trade.GasCost.Float64())
pm.netProfitTotal.Add(trade.NetProfit.Float64())
// Calculate success rate
total := pm.tradesProfitable.Value() + pm.tradesUnprofitable.Value()
if total > 0 {
successRate := pm.tradesProfitable.Value() / total
pm.successRate.Set(successRate)
}
// Update per-protocol metrics
for _, hop := range trade.Path.Hops {
pm.profitByProtocol[hop.Protocol].Add(trade.NetProfit.Float64())
}
// Store in database for historical analysis
return pm.storeTrade(trade)
}
func (pm *ProfitabilityMetrics) storeTrade(trade *CompletedTrade) error {
_, err := pm.db.Exec(`
INSERT INTO trades (
tx_hash,
timestamp,
path,
gross_profit,
gas_cost,
net_profit,
success
) VALUES (?, ?, ?, ?, ?, ?, ?)
`,
trade.TxHash.Hex(),
trade.Timestamp,
trade.Path.String(),
trade.GrossProfit.String(),
trade.GasCost.String(),
trade.NetProfit.String(),
trade.Success,
)
return err
}
```
### 7.2 Real-Time Profitability Dashboard
```go
// pkg/metrics/dashboard.go
type Dashboard struct {
metrics *ProfitabilityMetrics
}
func (d *Dashboard) GetCurrentStats() *DashboardStats {
return &DashboardStats{
TotalTrades: d.metrics.tradesExecuted.Value(),
ProfitableTrades: d.metrics.tradesProfitable.Value(),
FailedTrades: d.metrics.tradesUnprofitable.Value(),
SuccessRate: d.metrics.successRate.Value(),
TotalProfit: d.metrics.totalProfit.Value(),
TotalLoss: d.metrics.totalLoss.Value(),
TotalGasCost: d.metrics.gasCostTotal.Value(),
NetProfit: d.metrics.netProfitTotal.Value(),
AverageProfit: d.metrics.averageProfit.Value(),
AverageLoss: d.metrics.averageLoss.Value(),
ProfitByProtocol: d.getProfitByProtocol(),
}
}
type DashboardStats struct {
TotalTrades float64
ProfitableTrades float64
FailedTrades float64
SuccessRate float64
TotalProfit float64
TotalLoss float64
TotalGasCost float64
NetProfit float64
AverageProfit float64
AverageLoss float64
ProfitByProtocol map[ProtocolType]float64
}
```
### 7.3 Performance Analysis
```go
// pkg/analytics/performance.go
type PerformanceAnalyzer struct {
db *sql.DB
}
func (pa *PerformanceAnalyzer) AnalyzeDaily() (*DailyReport, error) {
// Query database for today's trades
rows, err := pa.db.Query(`
SELECT
COUNT(*) as total_trades,
SUM(CASE WHEN net_profit > 0 THEN 1 ELSE 0 END) as profitable,
SUM(gross_profit) as gross_profit,
SUM(gas_cost) as gas_cost,
SUM(net_profit) as net_profit,
AVG(net_profit) as avg_profit,
MAX(net_profit) as max_profit,
MIN(net_profit) as min_profit
FROM trades
WHERE DATE(timestamp) = CURDATE()
`)
if err != nil {
return nil, err
}
defer rows.Close()
var report DailyReport
rows.Next()
rows.Scan(
&report.TotalTrades,
&report.ProfitableTrades,
&report.GrossProfit,
&report.GasCost,
&report.NetProfit,
&report.AvgProfit,
&report.MaxProfit,
&report.MinProfit,
)
report.SuccessRate = float64(report.ProfitableTrades) / float64(report.TotalTrades)
report.ROI = report.NetProfit / report.TotalCapitalDeployed
return &report, nil
}
type DailyReport struct {
TotalTrades int
ProfitableTrades int
SuccessRate float64
GrossProfit float64
GasCost float64
NetProfit float64
AvgProfit float64
MaxProfit float64
MinProfit float64
TotalCapitalDeployed float64
ROI float64
}
```
---
## 8. Complete System Architecture
### 8.1 High-Level Flow
```
┌─────────────────────────────────────────────────────────────────┐
│ ARBITRUM SEQUENCER │
│ (Pending Transactions) │
└────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ SEQUENCER READER │
│ • WebSocket connection │
│ • Connection health monitoring │
│ • Automatic reconnection │
└────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ TRANSACTION PARSER │
│ • Protocol identification (< 1ms) │
│ • Protocol-specific parsing (< 5ms) │
│ • Parallel batch processing │
│ • ABI caching │
└────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ VALIDATOR │
│ • Validate swap events │
│ • Check pool cache │
│ • Verify amounts (non-zero, reasonable) │
│ • Validate tokens (no zero addresses) │
└────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ PRICE IMPACT CALCULATOR │
│ • Calculate price change from swap │
│ • Update pool reserves │
│ • Proper decimal handling │
└────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ ARBITRAGE DETECTOR │
│ • Multi-hop path finding (BFS/DFS) │
│ • Calculate gross profit │
│ • Estimate gas cost │
│ • Calculate net profit │
│ • Filter by minimum profit │
└────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ RISK MANAGEMENT │
│ • Position sizing │
│ • Slippage protection │
│ • Circuit breaker check │
│ • Max concurrent positions │
└────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ EXECUTION ROUTER │
│ • Single vs batch decision │
│ • Gas optimization │
│ • Transaction building │
│ • Signing │
└────────────────────────────┬────────────────────────────────────┘
┌──────────┴──────────┐
│ │
▼ ▼
┌────────────────────┐ ┌────────────────────┐
│ FRONT-RUNNER │ │ BATCH EXECUTOR │
│ │ │ │
│ • Higher gas price │ │ • Multicall │
│ • Single trade │ │ • Multiple trades │
│ • Flashbots option │ │ • Gas savings │
└──────────┬─────────┘ └─────────┬──────────┘
│ │
└──────────┬───────────┘
┌──────────────────────────┐
│ ETHEREUM NETWORK │
│ (Transaction Submitted) │
└─────────────┬────────────┘
┌──────────────────────────┐
│ METRICS COLLECTOR │
│ │
│ • Record trade │
│ • Calculate profit │
│ • Update dashboard │
│ • Store in database │
└──────────────────────────┘
```
### 8.2 Performance Targets
```yaml
Latency Targets:
Sequencer to Parse: < 5ms
Parse to Validate: < 2ms
Validate to Detect: < 10ms
Detect to Execute: < 30ms
Total (End-to-End): < 50ms
Throughput Targets:
Transactions per second: 1000+
Concurrent parsing: 100+ goroutines
Batch size: 10-50 trades
Profitability Targets:
Success rate: > 85%
Minimum profit per trade: > 0.05 ETH
Daily trades: 50-200
Monthly ROI: > 20%
Max drawdown: < 5%
```
### 8.3 Configuration
```yaml
# config/production.yaml
sequencer:
primary_url: "wss://arb1.arbitrum.io/feed"
backup_urls:
- "wss://arbitrum-one.publicnode.com"
- "wss://arb-mainnet.g.alchemy.com/v2/YOUR_KEY"
reconnect_backoff: 1s
max_reconnect_delay: 60s
parsing:
worker_pool_size: 100
batch_size: 50
timeout: 5ms
arbitrage:
min_profit_usd: 50.0
max_hops: 4
path_finding_timeout: 10ms
execution:
strategy: "smart" # frontrun, batch, or smart
batch_threshold: 3
batch_window: 100ms
max_gas_price: "500000000000" # 500 gwei
gas_premium: 1.1 # 10% above target tx
gas:
strategy: "optimal" # fast, optimal, or economy
safety_margin: 1.1 # 10% buffer on estimates
risk:
max_slippage: 0.005 # 0.5%
max_position_pct: 0.1 # 10% of capital
max_concurrent_trades: 5
max_loss_per_hour: 1.0 # 1 ETH
max_failures_per_min: 10
circuit_breaker_reset: 1h
monitoring:
metrics_port: 9090
dashboard_port: 8080
log_level: "info"
enable_profiling: true
```
---
## 9. Implementation Roadmap
### Phase 1: Sequencer Integration (Week 1-2)
**Tasks:**
- [ ] Set up Arbitrum sequencer WebSocket connection
- [ ] Implement connection health monitoring
- [ ] Build automatic reconnection logic
- [ ] Create pending transaction stream processor
- [ ] Add latency monitoring (sequencer → parser)
- [ ] Test with live sequencer feed
- [ ] Validate transaction ordering
**Success Criteria:**
- Stable connection for 24+ hours
- < 1ms latency from sequencer event to parser
- Automatic recovery from disconnections
- Zero missed transactions
### Phase 2: High-Performance Parsing (Week 2-3)
**Tasks:**
- [ ] Implement optimized parser factory
- [ ] Build protocol identification cache
- [ ] Create parallel parsing worker pool
- [ ] Add per-protocol optimized parsers
- [ ] Implement decimal handling (critical)
- [ ] Add parsing validation
- [ ] Benchmark parsing performance
**Success Criteria:**
- < 5ms per transaction parse
- 100% accuracy on all supported protocols
- Proper decimal scaling
- No zero addresses
- Handle 1000+ tx/sec
### Phase 3: Arbitrage Detection (Week 3-4)
**Tasks:**
- [ ] Build market graph from pool cache
- [ ] Implement BFS/DFS path finding
- [ ] Create profitability calculator
- [ ] Add gas cost estimator
- [ ] Implement profit filtering
- [ ] Optimize path finding performance
- [ ] Test with historical data
**Success Criteria:**
- < 10ms path finding
- Find all 2-4 hop opportunities
- Accurate profit calculations
- Gas estimates within 10% of actual
### Phase 4: Execution Engine (Week 4-5)
**Tasks:**
- [ ] Build front-run executor
- [ ] Implement batch executor
- [ ] Create smart execution router
- [ ] Add gas optimization
- [ ] Integrate Flashbots (optional)
- [ ] Build transaction monitoring
- [ ] Add execution metrics
**Success Criteria:**
- < 30ms execution decision
- Successful front-running
- Gas optimization working
- Transaction monitoring
- Flashbots integration (if used)
### Phase 5: Risk Management (Week 5-6)
**Tasks:**
- [ ] Implement slippage protection
- [ ] Build circuit breaker
- [ ] Add position sizing
- [ ] Create risk monitoring dashboard
- [ ] Test failure scenarios
- [ ] Add automated alerts
**Success Criteria:**
- Slippage protection working
- Circuit breaker trips on losses
- Position limits enforced
- Alert system functional
### Phase 6: Metrics & Monitoring (Week 6-7)
**Tasks:**
- [ ] Set up Prometheus metrics
- [ ] Build profitability tracker
- [ ] Create real-time dashboard
- [ ] Add database for trade history
- [ ] Implement performance analyzer
- [ ] Build daily reports
**Success Criteria:**
- All metrics collected
- Dashboard shows real-time data
- Trade history stored
- Daily reports generated
### Phase 7: Testing & Optimization (Week 7-8)
**Tasks:**
- [ ] End-to-end testing with testnet
- [ ] Performance optimization
- [ ] Load testing (1000+ tx/sec)
- [ ] Chaos testing (failure scenarios)
- [ ] Production deployment preparation
- [ ] Documentation
**Success Criteria:**
- < 50ms end-to-end latency
- 85%+ success rate in testing
- Passes all load tests
- Handles failure scenarios
- Ready for production
---
## 10. Profitability Projections
### Conservative Scenario
```
Assumptions:
- 50 trades per day
- 60% success rate
- Average profit per successful trade: 0.05 ETH
- Average loss per failed trade: 0.02 ETH
- Average gas cost: 0.01 ETH per trade
Daily Calculation:
- Successful trades: 30 * 0.05 ETH = 1.5 ETH
- Failed trades: 20 * -0.02 ETH = -0.4 ETH
- Gas costs: 50 * 0.01 ETH = -0.5 ETH
- Net daily profit: 0.6 ETH
Monthly: 0.6 ETH * 30 = 18 ETH
Yearly: 18 ETH * 12 = 216 ETH
With 10 ETH capital deployed:
Monthly ROI: 180%
Yearly ROI: 2160%
```
### Moderate Scenario
```
Assumptions:
- 100 trades per day
- 75% success rate
- Average profit per successful trade: 0.08 ETH
- Average loss per failed trade: 0.02 ETH
- Average gas cost: 0.008 ETH per trade (batch optimization)
Daily Calculation:
- Successful trades: 75 * 0.08 ETH = 6.0 ETH
- Failed trades: 25 * -0.02 ETH = -0.5 ETH
- Gas costs: 100 * 0.008 ETH = -0.8 ETH
- Net daily profit: 4.7 ETH
Monthly: 4.7 ETH * 30 = 141 ETH
Yearly: 141 ETH * 12 = 1692 ETH
With 20 ETH capital deployed:
Monthly ROI: 705%
Yearly ROI: 8460%
```
### Optimistic Scenario
```
Assumptions:
- 200 trades per day
- 85% success rate
- Average profit per successful trade: 0.1 ETH
- Average loss per failed trade: 0.02 ETH
- Average gas cost: 0.006 ETH per trade (aggressive batching)
Daily Calculation:
- Successful trades: 170 * 0.1 ETH = 17.0 ETH
- Failed trades: 30 * -0.02 ETH = -0.6 ETH
- Gas costs: 200 * 0.006 ETH = -1.2 ETH
- Net daily profit: 15.2 ETH
Monthly: 15.2 ETH * 30 = 456 ETH
Yearly: 456 ETH * 12 = 5472 ETH
With 50 ETH capital deployed:
Monthly ROI: 912%
Yearly ROI: 10944%
```
**Note**: These are theoretical projections. Actual results will vary based on market conditions, competition, execution quality, and unforeseen factors.
---
## 11. Key Success Factors
### 1. Speed is Everything
- **50ms end-to-end latency** is the target
- Every millisecond counts in MEV
- Optimize at every layer (network, parsing, detection, execution)
### 2. Sequencer Access is the Moat
- Direct sequencer feed = competitive advantage
- Reading pending transactions before they're on-chain
- 100-500ms time advantage over traditional bots
### 3. Accurate Parsing is Critical
- Wrong parsing = wrong profit calculations = losses
- Protocol-specific parsers for accuracy
- Proper decimal handling (non-negotiable)
- Validate everything
### 4. Gas Optimization Directly Impacts Profit
- High gas prices eat into profits
- Batching saves gas
- Smart gas pricing (don't overpay)
- Flashbots for MEV protection
### 5. Risk Management Protects Capital
- Slippage protection prevents bad trades
- Circuit breaker prevents cascading losses
- Position sizing limits exposure
- Monitor everything
### 6. Continuous Optimization
- Monitor metrics constantly
- Analyze what works and what doesn't
- Optimize slow paths
- Add new protocols as they launch
---
## 12. Next Steps
1. **Review and approve this plan**
2. **Set up development environment**
3. **Begin Phase 1: Sequencer Integration**
4. **Follow task breakdown from 07_TASK_BREAKDOWN.md**
5. **Create feature branches: `feature/v2/sequencer/<task-id>`**
6. **Implement with 100% test coverage**
7. **Deploy to testnet for validation**
8. **Deploy to production with monitoring**
---
**This is the complete profitability plan. The sequencer reader/parser is the foundation that enables everything else. Let's build it.**