3505921207
This commit includes: ## Audit & Testing Infrastructure - scripts/audit.sh: 12-section comprehensive codebase audit - scripts/test.sh: 7 test types (unit, integration, race, bench, coverage, contracts, pkg) - scripts/check-compliance.sh: SPEC.md compliance validation - scripts/check-docs.sh: Documentation coverage checker - scripts/dev.sh: Unified development script with all commands ## Documentation - SPEC.md: Authoritative technical specification - docs/AUDIT_AND_TESTING.md: Complete testing guide (600+ lines) - docs/SCRIPTS_REFERENCE.md: All scripts documented (700+ lines) - docs/README.md: Documentation index and navigation - docs/DEVELOPMENT_SETUP.md: Environment setup guide - docs/REFACTORING_PLAN.md: Systematic refactoring plan ## Phase 1 Refactoring (Critical Fixes) - pkg/validation/helpers.go: Validation functions for addresses/amounts - pkg/sequencer/selector_registry.go: Thread-safe selector registry - pkg/sequencer/reader.go: Fixed race conditions with atomic metrics - pkg/sequencer/swap_filter.go: Fixed race conditions, added error logging - pkg/sequencer/decoder.go: Added address validation ## Changes Summary - Fixed race conditions on 13 metric counters (atomic operations) - Added validation at all ingress points - Eliminated silent error handling - Created selector registry for future ABI migration - Reduced SPEC.md violations from 7 to 5 Build Status: ✅ All packages compile Compliance: ✅ No race conditions, no silent failures Documentation: ✅ 1,700+ lines across 5 comprehensive guides 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
297 lines
7.2 KiB
Markdown
297 lines
7.2 KiB
Markdown
# Development Environment Setup
|
|
|
|
## Overview
|
|
|
|
This project enforces **containerized development** for consistency and adherence to SPEC.md requirements.
|
|
|
|
## Key Documents
|
|
|
|
- **SPEC.md** - Technical specification and architecture requirements
|
|
- **CLAUDE.md** - Development guidelines and workflow
|
|
- **bindings/README.md** - Contract bindings usage and generation
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# 1. Start development containers
|
|
./scripts/dev.sh up
|
|
|
|
# 2. Build contracts
|
|
./scripts/dev.sh forge-build
|
|
|
|
# 3. Generate Go bindings
|
|
./scripts/dev.sh bindings
|
|
|
|
# 4. Build Go application
|
|
./scripts/dev.sh build
|
|
|
|
# 5. Run tests
|
|
./scripts/dev.sh test
|
|
```
|
|
|
|
## Development Workflow
|
|
|
|
### 1. Container-Based Development
|
|
|
|
**NEVER run builds or tests outside of containers.**
|
|
|
|
All commands use `./scripts/dev.sh` which ensures:
|
|
- Consistent build environment
|
|
- Proper dependency management
|
|
- Compliance with SPEC.md
|
|
- Reproducible builds
|
|
|
|
### 2. Contract Development
|
|
|
|
```bash
|
|
# Build all Solidity contracts
|
|
./scripts/dev.sh forge-build
|
|
|
|
# Run contract tests
|
|
./scripts/dev.sh forge-test
|
|
|
|
# Generate Go bindings from ABIs
|
|
./scripts/dev.sh bindings
|
|
```
|
|
|
|
**Contract Structure:**
|
|
```
|
|
contracts/
|
|
├── lib/ # Foundry dependencies (official DEX contracts)
|
|
│ ├── v2-core/ # Uniswap V2
|
|
│ ├── v3-core/ # Uniswap V3
|
|
│ ├── v3-periphery/ # Uniswap V3 Router
|
|
│ └── openzeppelin/ # OpenZeppelin contracts
|
|
├── src/ # Custom contracts and interfaces
|
|
│ ├── interfaces/ # Interface definitions
|
|
│ ├── libraries/ # Utility libraries
|
|
│ └── utils/ # Helper contracts
|
|
├── out/ # Build artifacts (ABIs, bytecode)
|
|
└── foundry.toml # Foundry configuration
|
|
```
|
|
|
|
### 3. Go Development
|
|
|
|
```bash
|
|
# Build application
|
|
./scripts/dev.sh build
|
|
|
|
# Run tests
|
|
./scripts/dev.sh test
|
|
|
|
# Enter Go container for interactive development
|
|
./scripts/dev.sh go
|
|
# Inside container:
|
|
# go test ./pkg/sequencer/... -v
|
|
# go run ./cmd/mev-bot/main.go
|
|
```
|
|
|
|
### 4. Binding Generation
|
|
|
|
**IMPORTANT**: Use official contract ABIs from Foundry builds, NOT manual JSON files.
|
|
|
|
```bash
|
|
# 1. Build contracts first
|
|
./scripts/dev.sh forge-build
|
|
|
|
# 2. Generate bindings
|
|
./scripts/dev.sh bindings
|
|
|
|
# Generated bindings appear in:
|
|
bindings/
|
|
├── uniswap_v2/
|
|
│ ├── router.go # UniswapV2Router02
|
|
│ └── pair.go # UniswapV2Pair
|
|
├── uniswap_v3/
|
|
│ ├── router.go # SwapRouter
|
|
│ └── pool.go # UniswapV3Pool
|
|
└── README.md # Usage documentation
|
|
```
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
/docker/mev-beta/
|
|
├── SPEC.md # Technical specification
|
|
├── CLAUDE.md # Development guidelines
|
|
│
|
|
├── scripts/
|
|
│ ├── dev.sh # Main development script ⭐
|
|
│ ├── dev-up.sh # Start containers
|
|
│ ├── dev-down.sh # Stop containers
|
|
│ ├── generate-bindings.sh # Legacy bindings script
|
|
│ └── generate-bindings-in-container.sh # Container-based bindings
|
|
│
|
|
├── contracts/ # Foundry project
|
|
│ ├── lib/ # Dependencies (git submodules)
|
|
│ ├── src/ # Contract sources
|
|
│ ├── out/ # Build artifacts
|
|
│ └── foundry.toml # Configuration
|
|
│
|
|
├── bindings/ # Generated Go bindings
|
|
│ ├── uniswap_v2/
|
|
│ ├── uniswap_v3/
|
|
│ └── README.md
|
|
│
|
|
├── pkg/ # Go packages
|
|
│ ├── sequencer/ # Arbitrum sequencer feed
|
|
│ ├── pools/ # Pool cache and discovery
|
|
│ └── ...
|
|
│
|
|
├── cmd/
|
|
│ └── mev-bot/ # Application entry point
|
|
│
|
|
├── docker-compose.yml # Container definitions
|
|
└── docs/
|
|
├── planning/ # V2 architecture plans
|
|
└── DEVELOPMENT_SETUP.md # This file
|
|
```
|
|
|
|
## Container Reference
|
|
|
|
### mev-go-dev
|
|
- **Image**: `golang:1.21-alpine`
|
|
- **Purpose**: Go development, testing, building
|
|
- **Tools**: go, abigen, gcc, git
|
|
- **Working Dir**: `/workspace`
|
|
- **Access**: `./scripts/dev.sh go`
|
|
|
|
### mev-foundry
|
|
- **Image**: `ghcr.io/foundry-rs/foundry:latest`
|
|
- **Purpose**: Solidity contract development
|
|
- **Tools**: forge, cast, anvil
|
|
- **Working Dir**: `/workspace`
|
|
- **Access**: `./scripts/dev.sh foundry`
|
|
|
|
### mev-python-dev
|
|
- **Image**: `python:3.11-slim`
|
|
- **Purpose**: Analysis scripts, data processing
|
|
- **Tools**: python, pip
|
|
- **Working Dir**: `/workspace`
|
|
- **Access**: `./scripts/dev.sh python`
|
|
|
|
## Common Tasks
|
|
|
|
### Adding New DEX Contract
|
|
|
|
```bash
|
|
# 1. Enter Foundry container
|
|
./scripts/dev.sh foundry
|
|
|
|
# 2. Install official contract
|
|
cd /workspace/contracts
|
|
forge install <org>/<repo>
|
|
|
|
# 3. Exit container and rebuild
|
|
exit
|
|
./scripts/dev.sh forge-build
|
|
|
|
# 4. Generate Go bindings
|
|
./scripts/dev.sh bindings
|
|
```
|
|
|
|
### Debugging Build Issues
|
|
|
|
```bash
|
|
# Check container status
|
|
./scripts/dev.sh ps
|
|
|
|
# View logs
|
|
./scripts/dev.sh logs go-dev
|
|
|
|
# Clean and rebuild
|
|
./scripts/dev.sh clean
|
|
./scripts/dev.sh forge-build
|
|
./scripts/dev.sh build
|
|
```
|
|
|
|
### Running Specific Tests
|
|
|
|
```bash
|
|
# Enter Go container
|
|
./scripts/dev.sh go
|
|
|
|
# Inside container, run specific tests
|
|
go test ./pkg/sequencer/... -v -run TestSwapFilter
|
|
go test ./pkg/pools/... -v -run TestPoolCache
|
|
```
|
|
|
|
## Development Rules (from SPEC.md)
|
|
|
|
### MUST DO ✅
|
|
- Use `./scripts/dev.sh` for all operations
|
|
- Use Arbitrum sequencer feed as primary data source
|
|
- Derive contract ABIs from official sources via Foundry
|
|
- Generate Go bindings with `abigen`
|
|
- Use channels for ALL inter-component communication
|
|
- Validate ALL parsed data before propagation
|
|
- Emit comprehensive metrics and structured logs
|
|
|
|
### MUST NOT DO ❌
|
|
- Run builds outside of containers
|
|
- Use HTTP RPC as primary data source
|
|
- Write manual ABI JSON files
|
|
- Hardcode function selectors
|
|
- Allow zero addresses or zero amounts to propagate
|
|
- Use blocking operations in hot paths
|
|
|
|
## Troubleshooting
|
|
|
|
### Container Won't Start
|
|
|
|
```bash
|
|
# Check Podman status
|
|
podman ps -a
|
|
|
|
# Check logs
|
|
podman logs mev-go-dev
|
|
|
|
# Force restart
|
|
./scripts/dev.sh down
|
|
./scripts/dev.sh up
|
|
```
|
|
|
|
### Build Failures
|
|
|
|
```bash
|
|
# Clean artifacts
|
|
./scripts/dev.sh clean
|
|
|
|
# Restart containers
|
|
./scripts/dev.sh restart
|
|
|
|
# Rebuild
|
|
./scripts/dev.sh forge-build
|
|
./scripts/dev.sh build
|
|
```
|
|
|
|
### Binding Generation Fails
|
|
|
|
```bash
|
|
# Ensure contracts are built first
|
|
./scripts/dev.sh forge-build
|
|
|
|
# Check for artifacts
|
|
ls -la contracts/out/
|
|
|
|
# Manually generate if needed
|
|
./scripts/dev.sh go
|
|
# Inside container:
|
|
./scripts/generate-bindings-in-container.sh
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
1. **Add More DEX Contracts**: Install Camelot, Balancer, Curve, Kyber
|
|
2. **Fix Contract Compilation**: Resolve Solidity errors in src/
|
|
3. **Complete Swap Detection**: Implement ABI-based swap parsing
|
|
4. **Test Pool Discovery**: Verify pool cache functionality
|
|
5. **Deploy Phase 1**: Monitor sequencer feed in production
|
|
|
|
## References
|
|
|
|
- SPEC.md - Complete technical specification
|
|
- CLAUDE.md - Project guidelines and status
|
|
- bindings/README.md - Contract binding usage
|
|
- docs/planning/00_V2_MASTER_PLAN.md - Architecture overview
|