# V2 CI/CD Pipeline Setup

## Overview

This document describes the comprehensive CI/CD pipeline for MEV Bot V2, designed to enforce code quality, test coverage, and deployment readiness.

## Pipeline Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                    DEVELOPER WORKSTATION                         │
└────────────────────────┬────────────────────────────────────────┘
                         │
                         ▼
                  ┌─────────────┐
                  │ Git Hooks   │
                  │ (Local)     │
                  └──────┬──────┘
                         │
                         ├─► Pre-commit Hook
                         │   • Format check
                         │   • Quick tests
                         │   • go vet
                         │   • Secret detection
                         │
                         └─► Commit-msg Hook
                             • Message format validation
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                      GITHUB REPOSITORY                           │
└────────────────────────┬────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────────┐
│                    GITHUB ACTIONS CI/CD                          │
│                                                                  │
│  ┌────────────────┐  ┌────────────────┐  ┌────────────────┐   │
│  │ Pre-Flight     │  │ Build &        │  │ Code Quality   │   │
│  │ Checks         │─►│ Dependencies   │─►│ & Linting      │   │
│  └────────────────┘  └────────────────┘  └────────┬───────┘   │
│                                                     │            │
│  ┌────────────────┐  ┌────────────────┐           │            │
│  │ Unit Tests     │◄─┤ Modularity     │◄──────────┘            │
│  │ (100% Coverage)│  │ Validation     │                        │
│  └────────┬───────┘  └────────────────┘                        │
│           │                                                      │
│           ├─► Integration Tests                                 │
│           ├─► Performance Benchmarks                            │
│           ├─► Decimal Precision Tests                           │
│           │                                                      │
│           ▼                                                      │
│  ┌────────────────┐                                             │
│  │ Final          │                                             │
│  │ Validation     │                                             │
│  └────────┬───────┘                                             │
└───────────┼─────────────────────────────────────────────────────┘
            │
            ▼
     ┌─────────────┐
     │ Deployment  │
     │ Ready       │
     └─────────────┘
```

## Local Development Workflow

### Git Hooks

#### Pre-Commit Hook

Location: `.git-hooks/pre-commit`

**Checks performed:**
1. Branch name validation
2. Merge conflict detection
3. Secret and forbidden pattern detection
4. go.mod/go.sum tidiness
5. Code formatting (auto-fix)
6. Quick tests on changed packages
7. go vet static analysis
8. File size warnings

**Installation:**
```bash
./scripts/install-git-hooks.sh
```

**Manual bypass (emergency only):**
```bash
git commit --no-verify
```

#### Commit-msg Hook

Location: `.git-hooks/commit-msg`

**Validates:**
- Commit message format: `type(scope): description`
- Valid types: feat, fix, perf, refactor, test, docs, build, ci
- Minimum description length: 10 characters
- Maximum first line: 72 characters (warning)

**Example valid commit:**
```
feat(parsers): add UniswapV2 parser with event validation

- Implements ParseLog() for Swap events
- Adds token extraction from pool cache
- Includes comprehensive validation rules
- Achieves 100% test coverage

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
```

### Local Testing Commands

```bash
# Quick pre-commit check
make pre-commit

# Full local CI validation (mimics GitHub Actions)
make validate

# Individual checks
make fmt           # Format code
make test          # Run all tests
make test-coverage # Run tests with 100% coverage enforcement
make lint          # Run linters
make security      # Security scan

# Build
make build

# Dependencies
make deps-tidy
make deps-check
```

## GitHub Actions Pipeline

### Workflow: v2-ci.yml

**Triggers:**
- Push to `feature/v2-**` or `feature/v2/**` branches
- Pull requests to `feature/v2-prep` or `master`
- Manual workflow dispatch

**Environment:**
- Go version: 1.25
- Minimum coverage: 100%
- golangci-lint version: v1.61.0

### Pipeline Jobs

#### 1. Pre-Flight Checks

**Purpose:** Validate branch naming and commit messages

**Checks:**
- Branch name follows convention: `feature/v2/<component>/<TASK-ID>-<description>`
- Commit message follows format: `type(scope): description`

**Example branch names:**
```
feature/v2/parsers/P2-002-uniswap-v2-base
feature/v2/cache/P3-001-address-index
feature/v2/validation/P4-001-validation-rules
```

**Failure behavior:** Block pipeline if validation fails

#### 2. Build & Dependencies

**Purpose:** Ensure code compiles and dependencies are correct

**Steps:**
1. Set up Go 1.25
2. Download and cache dependencies
3. Verify dependencies
4. Check go.mod tidiness
5. Build all packages
6. Build main binary (if exists)

**Caching:**
- Go modules: `~/go/pkg/mod`
- Build cache: `~/.cache/go-build`
- Cache key: `${{ runner.os }}-go-${{ GO_VERSION }}-${{ hashFiles('**/go.sum') }}`

**Failure behavior:** Pipeline stops if build fails

#### 3. Code Quality & Linting

**Purpose:** Enforce code quality standards

**Checks:**
1. **gofmt** - Code formatting
2. **go vet** - Static analysis
3. **golangci-lint** - Comprehensive linting
4. **gosec** - Security scanning
5. **TODO/FIXME detection** - Warning only

**Configuration:** `.golangci.yml`

**Enabled linters (40+):**
- errcheck, gosimple, govet, ineffassign, staticcheck, typecheck, unused
- bodyclose, cyclop, dupl, errname, exhaustive, exportloopref
- gocognit, goconst, gocritic, gocyclo, godot, goimports
- gomnd, gosec, lll, misspell, nakedret, nestif
- And many more...

**Failure behavior:** Pipeline stops if linting fails

#### 4. Unit Tests (100% Coverage Enforcement)

**Purpose:** Run all tests with mandatory 100% coverage

**Steps:**
1. Run tests with race detector
2. Generate coverage report
3. Calculate coverage percentage
4. **ENFORCE 100% COVERAGE REQUIREMENT**
5. Upload coverage to Codecov
6. Generate HTML coverage report

**Coverage calculation:**
```bash
COVERAGE=$(go tool cover -func=coverage.out | grep total | awk '{print $3}' | sed 's/%//')

if (( $(echo "$COVERAGE < 100" | bc -l) )); then
    echo "❌ COVERAGE FAILURE"
    exit 1
fi
```

**Failure behavior:** Pipeline STOPS if coverage < 100%

**Artifacts uploaded:**
- `coverage.out` - Coverage profile
- `coverage.html` - HTML coverage report

#### 5. Integration Tests

**Purpose:** Test component interactions

**Triggers:**
- Commit message contains `[integration]`
- Pull request events

**Tests:**
- Integration tests (tag: `integration`)
- End-to-end tests (if exists)

**Timeout:** 30 minutes

#### 6. Performance Benchmarks

**Purpose:** Ensure performance targets are met

**Triggers:**
- Manual workflow dispatch with `run_benchmarks: true`
- Commit message contains `[bench]`

**Benchmarks:**
- Parser performance: < 5ms per transaction
- Arbitrage detection: < 10ms
- End-to-end: < 50ms

**Command:**
```bash
go test -bench=. -benchmem -benchtime=10s ./...
```

**Artifacts uploaded:**
- `benchmark.txt` - Detailed benchmark results (retained 90 days)

#### 7. Decimal Precision Tests

**Purpose:** Validate critical decimal handling

**Tests:**
- Decimal scaling accuracy
- Rounding error accumulation
- Cross-protocol decimal handling
- Edge case decimal values

**Failure behavior:** Pipeline stops if decimal tests fail

#### 8. Modularity Validation

**Purpose:** Ensure component independence

**Checks:**
1. Each `pkg/*` compiles independently
2. No circular dependencies
3. Standalone executability

**Command:**
```bash
for dir in pkg/*/; do
    (cd "$dir" && go build .) || exit 1
done
```

**Dependency check:**
```bash
godepgraph ./... | grep -i cycle && exit 1
```

**Failure behavior:** Pipeline stops if modularity fails

#### 9. Final Validation Summary

**Purpose:** Aggregate all results and determine deployment readiness

**Checks all jobs:**
- Pre-flight ✅
- Build ✅
- Code quality ✅
- Unit tests (100% coverage) ✅
- Modularity ✅
- Decimal precision ✅

**Generates:**
- CI/CD summary markdown
- Deployment readiness status

**PR Comments:**
- Automatically comments on PRs with summary

**Failure behavior:**
- Exit 0 if all checks pass
- Exit 1 if any check fails

## Performance Optimizations

### Caching Strategy

**Go Modules Cache:**
```yaml
uses: actions/cache@v4
with:
  path: |
    ~/.cache/go-build
    ~/go/pkg/mod
  key: ${{ runner.os }}-go-${{ GO_VERSION }}-${{ hashFiles('**/go.sum') }}
  restore-keys: |
    ${{ runner.os }}-go-${{ GO_VERSION }}-
```

**Benefits:**
- Faster dependency downloads
- Reduced bandwidth usage
- Faster builds (2-5x speedup)

### Parallel Execution

**Jobs run in parallel:**
- Code quality & Unit tests (independent)
- Modularity & Decimal tests (after unit tests)
- Integration & Benchmarks (after unit tests)

**Estimated pipeline time:**
- Pre-flight: 30 seconds
- Build: 1-2 minutes
- Code quality: 2-3 minutes
- Unit tests: 3-5 minutes
- Integration tests: 5-10 minutes
- Total: 10-15 minutes (with caching)

## Failure Handling

### Pipeline Stops On:

1. **Pre-flight failures:**
   - Invalid branch name
   - Invalid commit message

2. **Build failures:**
   - Compilation errors
   - Untidy dependencies

3. **Code quality failures:**
   - Formatting issues
   - Linting errors
   - Security vulnerabilities

4. **Test failures:**
   - Unit test failures
   - Coverage < 100%
   - Integration test failures
   - Decimal precision errors

5. **Modularity failures:**
   - Components don't compile independently
   - Circular dependencies detected

### Debugging Pipeline Failures

**View detailed logs:**
1. Go to GitHub Actions tab
2. Click on failed workflow run
3. Expand failed job
4. Review error messages

**Download artifacts:**
```bash
gh run download <run-id>
```

**Common fixes:**
```bash
# Fix formatting
make fmt

# Fix coverage
make test-coverage

# Fix linting
make lint

# Fix dependencies
make deps-tidy
```

## Branch Protection Rules

### Protected Branches:
- `master`
- `feature/v2-prep`

### Required Status Checks:
- ✅ Pre-Flight Checks
- ✅ Build & Dependencies
- ✅ Code Quality & Linting
- ✅ Unit Tests (100% Coverage)
- ✅ Modularity Validation
- ✅ Decimal Precision Tests

### Additional Rules:
- Require pull request reviews: 1
- Require conversation resolution
- Require linear history
- Do not allow bypassing the above settings

## Deployment

### Deployment Ready Criteria:

**All of the following must be true:**
1. All CI/CD checks pass ✅
2. Code coverage = 100% ✅
3. No security vulnerabilities ✅
4. All components compile independently ✅
5. Performance benchmarks meet targets ✅
6. Pull request approved ✅

### Deployment Process:

```bash
# 1. Merge to v2-prep
git checkout feature/v2-prep
git merge feature/v2/parsers/P2-002-uniswap-v2-base

# 2. Verify CI passes
# (GitHub Actions runs automatically)

# 3. Create release branch (when ready)
git checkout -b release/v2.0.0-alpha

# 4. Tag release
git tag -a v2.0.0-alpha -m "V2 Alpha Release"

# 5. Deploy
# (Deployment automation TBD)
```

## Metrics and Monitoring

### Pipeline Metrics Tracked:

1. **Success rate**
   - Target: > 95%

2. **Pipeline duration**
   - Target: < 15 minutes
   - Current average: ~12 minutes

3. **Coverage trends**
   - Target: 100% (enforced)

4. **Security vulnerabilities**
   - Target: 0 (enforced)

### Monitoring Tools:

- **GitHub Actions Insights** - Pipeline statistics
- **Codecov** - Coverage trends and reporting
- **Dependabot** - Dependency vulnerability scanning

## Best Practices

### For Developers:

1. **Always run `make validate` before pushing**
2. **Never bypass git hooks** (except emergencies)
3. **Write tests first** (TDD approach)
4. **Keep commits small and focused**
5. **Follow branch naming convention**
6. **Write descriptive commit messages**

### For Code Reviewers:

1. **Check CI/CD status** before reviewing code
2. **Verify test coverage** (should always be 100%)
3. **Review security scan results**
4. **Ensure modularity is maintained**
5. **Verify performance benchmarks** (if applicable)

### For Maintainers:

1. **Monitor pipeline success rate**
2. **Update dependencies regularly**
3. **Review and update linter rules**
4. **Optimize caching strategy**
5. **Keep documentation up to date**

## Troubleshooting

### Common Issues:

**1. Coverage not 100%:**
```bash
# View uncovered lines
make test-coverage

# Check specific package
go test -coverprofile=coverage.out ./pkg/parsers
go tool cover -func=coverage.out | grep -v "100.0%"
```

**2. Linting failures:**
```bash
# Run locally
make lint

# Auto-fix some issues
golangci-lint run --fix
```

**3. Test failures:**
```bash
# Run with verbose output
go test -v ./...

# Run specific test
go test -v -run TestSpecificTest ./pkg/parsers
```

**4. Build failures:**
```bash
# Clean and rebuild
make clean
make build

# Check dependencies
make deps-verify
make deps-tidy
```

## Future Enhancements

### Planned Improvements:

1. **Automated Performance Regression Detection**
   - Compare benchmarks against baseline
   - Fail if performance degrades > 10%

2. **Automated Dependency Updates**
   - Dependabot integration
   - Auto-merge minor/patch updates

3. **Deployment Automation**
   - Automated staging deployments
   - Blue-green deployment strategy
   - Automated rollback on failures

4. **Advanced Security Scanning**
   - Container image scanning
   - SAST/DAST integration
   - Supply chain security

5. **Performance Monitoring**
   - Real-time performance dashboards
   - Historical performance trends
   - Automated alerting on regressions

## References

- [GitHub Actions Documentation](https://docs.github.com/en/actions)
- [golangci-lint Documentation](https://golangci-lint.run/)
- [Go Testing Best Practices](https://go.dev/doc/tutorial/add-a-test)
- [Conventional Commits](https://www.conventionalcommits.org/)

---

**Last Updated:** 2025-11-10
**Version:** 1.0
**Maintainer:** MEV Bot Team