mev-beta/docs/planning/05_CI_CD_SETUP.md

V2 CI/CD Pipeline Setup

Overview

This document describes the comprehensive CI/CD pipeline for MEV Bot V2, designed to enforce code quality, test coverage, and deployment readiness.

Pipeline Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    DEVELOPER WORKSTATION                         │
└────────────────────────┬────────────────────────────────────────┘
                         │
                         ▼
                  ┌─────────────┐
                  │ Git Hooks   │
                  │ (Local)     │
                  └──────┬──────┘
                         │
                         ├─► Pre-commit Hook
                         │   • Format check
                         │   • Quick tests
                         │   • go vet
                         │   • Secret detection
                         │
                         └─► Commit-msg Hook
                             • Message format validation
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                      GITHUB REPOSITORY                           │
└────────────────────────┬────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────────┐
│                    GITHUB ACTIONS CI/CD                          │
│                                                                  │
│  ┌────────────────┐  ┌────────────────┐  ┌────────────────┐   │
│  │ Pre-Flight     │  │ Build &        │  │ Code Quality   │   │
│  │ Checks         │─►│ Dependencies   │─►│ & Linting      │   │
│  └────────────────┘  └────────────────┘  └────────┬───────┘   │
│                                                     │            │
│  ┌────────────────┐  ┌────────────────┐           │            │
│  │ Unit Tests     │◄─┤ Modularity     │◄──────────┘            │
│  │ (100% Coverage)│  │ Validation     │                        │
│  └────────┬───────┘  └────────────────┘                        │
│           │                                                      │
│           ├─► Integration Tests                                 │
│           ├─► Performance Benchmarks                            │
│           ├─► Decimal Precision Tests                           │
│           │                                                      │
│           ▼                                                      │
│  ┌────────────────┐                                             │
│  │ Final          │                                             │
│  │ Validation     │                                             │
│  └────────┬───────┘                                             │
└───────────┼─────────────────────────────────────────────────────┘
            │
            ▼
     ┌─────────────┐
     │ Deployment  │
     │ Ready       │
     └─────────────┘

Local Development Workflow

Git Hooks

Pre-Commit Hook

Location: .git-hooks/pre-commit

Checks performed:

1. Branch name validation
2. Merge conflict detection
3. Secret and forbidden pattern detection
4. go.mod/go.sum tidiness
5. Code formatting (auto-fix)
6. Quick tests on changed packages
7. go vet static analysis
8. File size warnings

Installation:

./scripts/install-git-hooks.sh

Manual bypass (emergency only):

git commit --no-verify

Commit-msg Hook

Location: .git-hooks/commit-msg

Validates:

• Commit message format: type(scope): description
• Valid types: feat, fix, perf, refactor, test, docs, build, ci
• Minimum description length: 10 characters
• Maximum first line: 72 characters (warning)

Example valid commit:

feat(parsers): add UniswapV2 parser with event validation

- Implements ParseLog() for Swap events
- Adds token extraction from pool cache
- Includes comprehensive validation rules
- Achieves 100% test coverage

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>

Local Testing Commands

# Quick pre-commit check
make pre-commit

# Full local CI validation (mimics GitHub Actions)
make validate

# Individual checks
make fmt           # Format code
make test          # Run all tests
make test-coverage # Run tests with 100% coverage enforcement
make lint          # Run linters
make security      # Security scan

# Build
make build

# Dependencies
make deps-tidy
make deps-check

GitHub Actions Pipeline

Workflow: v2-ci.yml

Triggers:

• Push to feature/v2-** or feature/v2/** branches
• Pull requests to feature/v2-prep or master
• Manual workflow dispatch

Environment:

• Go version: 1.25
• Minimum coverage: 100%
• golangci-lint version: v1.61.0

Pipeline Jobs

1. Pre-Flight Checks

Purpose: Validate branch naming and commit messages

Checks:

• Branch name follows convention: feature/v2/<component>/<TASK-ID>-<description>
• Commit message follows format: type(scope): description

Example branch names:

feature/v2/parsers/P2-002-uniswap-v2-base
feature/v2/cache/P3-001-address-index
feature/v2/validation/P4-001-validation-rules

Failure behavior: Block pipeline if validation fails

2. Build & Dependencies

Purpose: Ensure code compiles and dependencies are correct

Steps:

1. Set up Go 1.25
2. Download and cache dependencies
3. Verify dependencies
4. Check go.mod tidiness
5. Build all packages
6. Build main binary (if exists)

Caching:

• Go modules: ~/go/pkg/mod
• Build cache: ~/.cache/go-build
• Cache key: ${{ runner.os }}-go-${{ GO_VERSION }}-${{ hashFiles('**/go.sum') }}

Failure behavior: Pipeline stops if build fails

3. Code Quality & Linting

Purpose: Enforce code quality standards

Checks:

1. gofmt - Code formatting
2. go vet - Static analysis
3. golangci-lint - Comprehensive linting
4. gosec - Security scanning
5. TODO/FIXME detection - Warning only

Configuration: .golangci.yml

Enabled linters (40+):

• errcheck, gosimple, govet, ineffassign, staticcheck, typecheck, unused
• bodyclose, cyclop, dupl, errname, exhaustive, exportloopref
• gocognit, goconst, gocritic, gocyclo, godot, goimports
• gomnd, gosec, lll, misspell, nakedret, nestif
• And many more...

Failure behavior: Pipeline stops if linting fails

4. Unit Tests (100% Coverage Enforcement)

Purpose: Run all tests with mandatory 100% coverage

Steps:

1. Run tests with race detector
2. Generate coverage report
3. Calculate coverage percentage
4. ENFORCE 100% COVERAGE REQUIREMENT
5. Upload coverage to Codecov
6. Generate HTML coverage report

Coverage calculation:

COVERAGE=$(go tool cover -func=coverage.out | grep total | awk '{print $3}' | sed 's/%//')

if (( $(echo "$COVERAGE < 100" | bc -l) )); then
    echo "❌ COVERAGE FAILURE"
    exit 1
fi

Failure behavior: Pipeline STOPS if coverage < 100%

Artifacts uploaded:

• coverage.out - Coverage profile
• coverage.html - HTML coverage report

5. Integration Tests

Purpose: Test component interactions

Triggers:

• Commit message contains [integration]
• Pull request events

Tests:

• Integration tests (tag: integration)
• End-to-end tests (if exists)

Timeout: 30 minutes

6. Performance Benchmarks

Purpose: Ensure performance targets are met

Triggers:

• Manual workflow dispatch with run_benchmarks: true
• Commit message contains [bench]

Benchmarks:

• Parser performance: < 5ms per transaction
• Arbitrage detection: < 10ms
• End-to-end: < 50ms

Command:

go test -bench=. -benchmem -benchtime=10s ./...

Artifacts uploaded:

• benchmark.txt - Detailed benchmark results (retained 90 days)

7. Decimal Precision Tests

Purpose: Validate critical decimal handling

Tests:

• Decimal scaling accuracy
• Rounding error accumulation
• Cross-protocol decimal handling
• Edge case decimal values

Failure behavior: Pipeline stops if decimal tests fail

8. Modularity Validation

Purpose: Ensure component independence

Checks:

1. Each pkg/* compiles independently
2. No circular dependencies
3. Standalone executability

Command:

for dir in pkg/*/; do
    (cd "$dir" && go build .) || exit 1
done

Dependency check:

godepgraph ./... | grep -i cycle && exit 1

Failure behavior: Pipeline stops if modularity fails

9. Final Validation Summary

Purpose: Aggregate all results and determine deployment readiness

Checks all jobs:

• Pre-flight ✅
• Build ✅
• Code quality ✅
• Unit tests (100% coverage) ✅
• Modularity ✅
• Decimal precision ✅

Generates:

• CI/CD summary markdown
• Deployment readiness status

PR Comments:

• Automatically comments on PRs with summary

Failure behavior:

• Exit 0 if all checks pass
• Exit 1 if any check fails

Performance Optimizations

Caching Strategy

Go Modules Cache:

uses: actions/cache@v4
with:
  path: |
    ~/.cache/go-build
    ~/go/pkg/mod
  key: ${{ runner.os }}-go-${{ GO_VERSION }}-${{ hashFiles('**/go.sum') }}
  restore-keys: |
    ${{ runner.os }}-go-${{ GO_VERSION }}-

Benefits:

• Faster dependency downloads
• Reduced bandwidth usage
• Faster builds (2-5x speedup)

Parallel Execution

Jobs run in parallel:

• Code quality & Unit tests (independent)
• Modularity & Decimal tests (after unit tests)
• Integration & Benchmarks (after unit tests)

Estimated pipeline time:

• Pre-flight: 30 seconds
• Build: 1-2 minutes
• Code quality: 2-3 minutes
• Unit tests: 3-5 minutes
• Integration tests: 5-10 minutes
• Total: 10-15 minutes (with caching)

Failure Handling

Pipeline Stops On:

1. Pre-flight failures:

   • Invalid branch name
   • Invalid commit message

2. Build failures:

   • Compilation errors
   • Untidy dependencies

3. Code quality failures:

   • Formatting issues
   • Linting errors
   • Security vulnerabilities

4. Test failures:

   • Unit test failures
   • Coverage < 100%
   • Integration test failures
   • Decimal precision errors

5. Modularity failures:

   • Components don't compile independently
   • Circular dependencies detected

Debugging Pipeline Failures

View detailed logs:

1. Go to GitHub Actions tab
2. Click on failed workflow run
3. Expand failed job
4. Review error messages

Download artifacts:

gh run download <run-id>

Common fixes:

# Fix formatting
make fmt

# Fix coverage
make test-coverage

# Fix linting
make lint

# Fix dependencies
make deps-tidy

Branch Protection Rules

Protected Branches:

• master
• feature/v2-prep

Required Status Checks:

• ✅ Pre-Flight Checks
• ✅ Build & Dependencies
• ✅ Code Quality & Linting
• ✅ Unit Tests (100% Coverage)
• ✅ Modularity Validation
• ✅ Decimal Precision Tests

Additional Rules:

• Require pull request reviews: 1
• Require conversation resolution
• Require linear history
• Do not allow bypassing the above settings

Deployment

Deployment Ready Criteria:

All of the following must be true:

1. All CI/CD checks pass ✅
2. Code coverage = 100% ✅
3. No security vulnerabilities ✅
4. All components compile independently ✅
5. Performance benchmarks meet targets ✅
6. Pull request approved ✅

Deployment Process:

# 1. Merge to v2-prep
git checkout feature/v2-prep
git merge feature/v2/parsers/P2-002-uniswap-v2-base

# 2. Verify CI passes
# (GitHub Actions runs automatically)

# 3. Create release branch (when ready)
git checkout -b release/v2.0.0-alpha

# 4. Tag release
git tag -a v2.0.0-alpha -m "V2 Alpha Release"

# 5. Deploy
# (Deployment automation TBD)

Metrics and Monitoring

Pipeline Metrics Tracked:

1. Success rate

   • Target: > 95%

2. Pipeline duration

   • Target: < 15 minutes
   • Current average: ~12 minutes

3. Coverage trends

   • Target: 100% (enforced)

4. Security vulnerabilities

   • Target: 0 (enforced)

Monitoring Tools:

• GitHub Actions Insights - Pipeline statistics
• Codecov - Coverage trends and reporting
• Dependabot - Dependency vulnerability scanning

Best Practices

For Developers:

1. Always run make validate before pushing
2. Never bypass git hooks (except emergencies)
3. Write tests first (TDD approach)
4. Keep commits small and focused
5. Follow branch naming convention
6. Write descriptive commit messages

For Code Reviewers:

1. Check CI/CD status before reviewing code
2. Verify test coverage (should always be 100%)
3. Review security scan results
4. Ensure modularity is maintained
5. Verify performance benchmarks (if applicable)

For Maintainers:

1. Monitor pipeline success rate
2. Update dependencies regularly
3. Review and update linter rules
4. Optimize caching strategy
5. Keep documentation up to date

Troubleshooting

Common Issues:

1. Coverage not 100%:

# View uncovered lines
make test-coverage

# Check specific package
go test -coverprofile=coverage.out ./pkg/parsers
go tool cover -func=coverage.out | grep -v "100.0%"

2. Linting failures:

# Run locally
make lint

# Auto-fix some issues
golangci-lint run --fix

3. Test failures:

# Run with verbose output
go test -v ./...

# Run specific test
go test -v -run TestSpecificTest ./pkg/parsers

4. Build failures:

# Clean and rebuild
make clean
make build

# Check dependencies
make deps-verify
make deps-tidy

Future Enhancements

Planned Improvements:

1. Automated Performance Regression Detection

   • Compare benchmarks against baseline
   • Fail if performance degrades > 10%

2. Automated Dependency Updates

   • Dependabot integration
   • Auto-merge minor/patch updates

3. Deployment Automation

   • Automated staging deployments
   • Blue-green deployment strategy
   • Automated rollback on failures

4. Advanced Security Scanning

   • Container image scanning
   • SAST/DAST integration
   • Supply chain security

5. Performance Monitoring

   • Real-time performance dashboards
   • Historical performance trends
   • Automated alerting on regressions

References

• GitHub Actions Documentation
• golangci-lint Documentation
• Go Testing Best Practices
• Conventional Commits

---

Last Updated: 2025-11-10 Version: 1.0 Maintainer: MEV Bot Team