ci: add comprehensive CI/CD pipeline with 100% coverage enforcement
Some checks failed
V2 CI/CD Pipeline / Pre-Flight Checks (push) Has been cancelled
V2 CI/CD Pipeline / Build & Dependencies (push) Has been cancelled
V2 CI/CD Pipeline / Code Quality & Linting (push) Has been cancelled
V2 CI/CD Pipeline / Unit Tests (100% Coverage Required) (push) Has been cancelled
V2 CI/CD Pipeline / Integration Tests (push) Has been cancelled
V2 CI/CD Pipeline / Performance Benchmarks (push) Has been cancelled
V2 CI/CD Pipeline / Decimal Precision Validation (push) Has been cancelled
V2 CI/CD Pipeline / Modularity Validation (push) Has been cancelled
V2 CI/CD Pipeline / Final Validation Summary (push) Has been cancelled

Created complete CI/CD infrastructure for V2 development:

GitHub Actions Pipeline (.github/workflows/v2-ci.yml):
- Pre-flight checks (branch naming, commit messages)
- Build & dependency validation
- Code quality with 40+ linters (golangci-lint)
- Unit tests with MANDATORY 100% coverage enforcement
- Integration tests with timeout management
- Performance benchmarks (parser < 5ms, detection < 10ms, e2e < 50ms)
- Decimal precision validation
- Modularity validation (component independence)
- Final validation summary with PR comments

Code Quality (.golangci.yml):
- 40+ enabled linters for comprehensive checks
- Cyclomatic complexity limits (max 15)
- Magic number detection
- Security scanning (gosec)
- Style checking with MEV/DEX terminology
- Test file exclusions for appropriate linters

Build Automation (Makefile):
- build, test, test-coverage with 100% enforcement
- lint, fmt, vet, security targets
- deps-download, deps-verify, deps-tidy, deps-check
- validate (full CI/CD locally)
- bench (performance benchmarks)
- check-modularity, check-circular
- Color-coded output for better UX

Git Optimization (.gitattributes):
- LF normalization for cross-platform consistency
- Binary file handling
- Diff settings for Go files
- Merge strategies
- Export-ignore for archives

Git Hooks (.git-hooks/):
- pre-commit: format, tests, vet, secret detection, go.mod tidy
- commit-msg: message format validation
- README with installation instructions
- install-git-hooks.sh script for easy setup

Documentation (docs/planning/05_CI_CD_SETUP.md):
- Complete pipeline architecture diagram
- Local development workflow
- GitHub Actions job descriptions
- Performance optimizations (caching, parallel execution)
- Failure handling and debugging
- Branch protection rules
- Deployment process
- Best practices and troubleshooting

Performance Targets:
- Pipeline duration: < 15 minutes
- Test coverage: 100% (enforced, non-negotiable)
- Parser latency: < 5ms
- Arbitrage detection: < 10ms
- End-to-end: < 50ms

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Administrator
2025-11-10 14:40:08 +01:00
parent 9a92f43edf
commit 24b4d90e98
9 changed files with 2209 additions and 0 deletions

250
Makefile Normal file
View File

@@ -0,0 +1,250 @@
.PHONY: help build test test-coverage lint fmt vet security clean install-tools bench test-integration test-unit
# Default target
.DEFAULT_GOAL := help
# Go parameters
GOCMD=go
GOBUILD=$(GOCMD) build
GOCLEAN=$(GOCMD) clean
GOTEST=$(GOCMD) test
GOGET=$(GOCMD) get
GOMOD=$(GOCMD) mod
GOFMT=gofmt
# Binary names
BINARY_NAME=mev-bot
BINARY_PATH=bin/$(BINARY_NAME)
# Directories
PKG_DIR=./pkg/...
CMD_DIR=./cmd/...
INTERNAL_DIR=./internal/...
ALL_DIRS=$(PKG_DIR) $(CMD_DIR) $(INTERNAL_DIR)
# Coverage requirements
MIN_COVERAGE=100
# Color output
RED=\033[0;31m
GREEN=\033[0;32m
YELLOW=\033[1;33m
NC=\033[0m # No Color
help: ## Display this help message
@echo "$(GREEN)MEV Bot V2 - Development Commands$(NC)"
@echo ""
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf " $(YELLOW)%-20s$(NC) %s\n", $$1, $$2}'
@echo ""
# ==============================================================================
# DEVELOPMENT
# ==============================================================================
build: ## Build the application
@echo "$(GREEN)Building $(BINARY_NAME)...$(NC)"
@mkdir -p bin
@if [ -d "cmd/mev-bot" ]; then \
$(GOBUILD) -v -o $(BINARY_PATH) ./cmd/mev-bot; \
echo "$(GREEN)✅ Build successful: $(BINARY_PATH)$(NC)"; \
else \
echo "$(YELLOW)⚠️ No cmd/mev-bot yet (planning phase)$(NC)"; \
fi
clean: ## Clean build artifacts
@echo "$(YELLOW)Cleaning...$(NC)"
@$(GOCLEAN)
@rm -rf bin/
@rm -f coverage.out coverage.html
@echo "$(GREEN)✅ Cleaned$(NC)"
install-tools: ## Install development tools
@echo "$(GREEN)Installing development tools...$(NC)"
@$(GOCMD) install github.com/golangci/golangci-lint/cmd/golangci-lint@latest
@$(GOCMD) install github.com/securecodewarrior/gosec/v2/cmd/gosec@latest
@$(GOCMD) install golang.org/x/tools/cmd/godepgraph@latest
@$(GOCMD) install github.com/axw/gocov/gocov@latest
@$(GOCMD) install github.com/AlekSi/gocov-xml@latest
@echo "$(GREEN)✅ Tools installed$(NC)"
# ==============================================================================
# TESTING
# ==============================================================================
test: ## Run all tests
@echo "$(GREEN)Running all tests...$(NC)"
@$(GOTEST) -v -race -timeout=30m $(ALL_DIRS)
test-unit: ## Run unit tests only
@echo "$(GREEN)Running unit tests...$(NC)"
@$(GOTEST) -v -race -short -timeout=10m $(ALL_DIRS)
test-integration: ## Run integration tests
@echo "$(GREEN)Running integration tests...$(NC)"
@$(GOTEST) -v -race -run Integration -timeout=30m $(ALL_DIRS)
test-coverage: ## Run tests with coverage and enforce 100% requirement
@echo "$(GREEN)Running tests with coverage...$(NC)"
@$(GOTEST) -v -race -coverprofile=coverage.out -covermode=atomic $(ALL_DIRS)
@$(GOCMD) tool cover -html=coverage.out -o coverage.html
@echo ""
@echo "$(YELLOW)Coverage Report:$(NC)"
@$(GOCMD) tool cover -func=coverage.out
@echo ""
@COVERAGE=$$($(GOCMD) tool cover -func=coverage.out | grep total | awk '{print $$3}' | sed 's/%//'); \
echo "Total Coverage: $$COVERAGE%"; \
echo "Required: $(MIN_COVERAGE)%"; \
if [ "$$(echo "$$COVERAGE < $(MIN_COVERAGE)" | bc -l)" -eq 1 ]; then \
echo "$(RED)❌ Coverage $$COVERAGE% is below required $(MIN_COVERAGE)%$(NC)"; \
echo ""; \
echo "$(YELLOW)Uncovered lines:$(NC)"; \
$(GOCMD) tool cover -func=coverage.out | grep -v "100.0%"; \
exit 1; \
else \
echo "$(GREEN)✅ Coverage requirement met: $$COVERAGE%$(NC)"; \
fi
@echo ""
@echo "HTML report: file://$(PWD)/coverage.html"
bench: ## Run benchmarks
@echo "$(GREEN)Running benchmarks...$(NC)"
@$(GOTEST) -bench=. -benchmem -benchtime=10s $(ALL_DIRS) | tee benchmark.txt
@echo "$(GREEN)✅ Benchmarks complete: benchmark.txt$(NC)"
# ==============================================================================
# CODE QUALITY
# ==============================================================================
fmt: ## Format code
@echo "$(GREEN)Formatting code...$(NC)"
@$(GOFMT) -w -s .
@echo "$(GREEN)✅ Code formatted$(NC)"
fmt-check: ## Check if code is formatted
@echo "$(GREEN)Checking code formatting...$(NC)"
@UNFORMATTED=$$($(GOFMT) -l .); \
if [ -n "$$UNFORMATTED" ]; then \
echo "$(RED)❌ Code is not formatted:$(NC)"; \
echo "$$UNFORMATTED"; \
echo ""; \
echo "Run: make fmt"; \
exit 1; \
else \
echo "$(GREEN)✅ Code formatting passed$(NC)"; \
fi
vet: ## Run go vet
@echo "$(GREEN)Running go vet...$(NC)"
@$(GOCMD) vet $(ALL_DIRS)
@echo "$(GREEN)✅ go vet passed$(NC)"
lint: ## Run golangci-lint
@echo "$(GREEN)Running golangci-lint...$(NC)"
@golangci-lint run --config=.golangci.yml --timeout=10m
@echo "$(GREEN)✅ Linting passed$(NC)"
security: ## Run security scans
@echo "$(GREEN)Running security scans...$(NC)"
@gosec -fmt=text ./...
@echo "$(GREEN)✅ Security scan complete$(NC)"
# ==============================================================================
# DEPENDENCY MANAGEMENT
# ==============================================================================
deps-download: ## Download dependencies
@echo "$(GREEN)Downloading dependencies...$(NC)"
@$(GOMOD) download
@echo "$(GREEN)✅ Dependencies downloaded$(NC)"
deps-verify: ## Verify dependencies
@echo "$(GREEN)Verifying dependencies...$(NC)"
@$(GOMOD) verify
@echo "$(GREEN)✅ Dependencies verified$(NC)"
deps-tidy: ## Tidy dependencies
@echo "$(GREEN)Tidying dependencies...$(NC)"
@$(GOMOD) tidy
@echo "$(GREEN)✅ Dependencies tidied$(NC)"
deps-check: ## Check if go.mod is tidy
@echo "$(GREEN)Checking if dependencies are tidy...$(NC)"
@$(GOMOD) tidy
@if [ -n "$$(git status --porcelain go.mod go.sum)" ]; then \
echo "$(RED)❌ go.mod or go.sum is not tidy$(NC)"; \
git diff go.mod go.sum; \
exit 1; \
else \
echo "$(GREEN)✅ Dependencies are tidy$(NC)"; \
fi
# ==============================================================================
# VALIDATION (Full CI/CD locally)
# ==============================================================================
validate: deps-check fmt-check vet lint test-coverage security ## Run all validation checks (CI/CD locally)
@echo ""
@echo "$(GREEN)╔══════════════════════════════════════╗$(NC)"
@echo "$(GREEN)║ ✅ ALL VALIDATION CHECKS PASSED ✅ ║$(NC)"
@echo "$(GREEN)╔══════════════════════════════════════╗$(NC)"
@echo ""
@echo "Ready to commit and push!"
pre-commit: fmt-check test-coverage ## Run pre-commit checks
@echo "$(GREEN)✅ Pre-commit checks passed$(NC)"
ci: validate ## Alias for validate (run full CI locally)
# ==============================================================================
# MODULARITY CHECKS
# ==============================================================================
check-modularity: ## Verify component independence
@echo "$(GREEN)Checking component modularity...$(NC)"
@for dir in pkg/*/; do \
if [ -d "$$dir" ]; then \
echo "Testing $$dir..."; \
(cd "$$dir" && go build .) || exit 1; \
fi \
done
@echo "$(GREEN)✅ All components compile independently$(NC)"
check-circular: ## Check for circular dependencies
@echo "$(GREEN)Checking for circular dependencies...$(NC)"
@godepgraph ./... | grep -i cycle && exit 1 || echo "$(GREEN)✅ No circular dependencies$(NC)"
# ==============================================================================
# DOCUMENTATION
# ==============================================================================
docs: ## Generate documentation
@echo "$(GREEN)Generating documentation...$(NC)"
@$(GOCMD) doc -all ./... > docs/api.txt
@echo "$(GREEN)✅ Documentation generated: docs/api.txt$(NC)"
# ==============================================================================
# DOCKER
# ==============================================================================
docker-build: ## Build Docker image
@echo "$(GREEN)Building Docker image...$(NC)"
@docker build -t mev-bot:v2-dev .
@echo "$(GREEN)✅ Docker image built: mev-bot:v2-dev$(NC)"
docker-run: ## Run Docker container
@echo "$(GREEN)Running Docker container...$(NC)"
@docker run --rm -it mev-bot:v2-dev
# ==============================================================================
# QUICK COMMANDS
# ==============================================================================
quick-test: ## Quick test (no race, no coverage)
@$(GOTEST) -short $(ALL_DIRS)
watch: ## Watch for changes and run tests
@echo "$(GREEN)Watching for changes...$(NC)"
@while true; do \
inotifywait -e modify -r pkg/ cmd/ internal/; \
make quick-test; \
done