269 lines
8.4 KiB
Markdown
269 lines
8.4 KiB
Markdown
# Development Tools & Commands Documentation
|
|
|
|
This document provides comprehensive documentation for all development tools and commands available in this project, as well as reusable tools for other projects.
|
|
|
|
## Makefile Commands
|
|
|
|
### Build & Run Commands
|
|
- `make build` - Build the application (`bin/mev-beta` for production)
|
|
- `make run` - Build and run the application
|
|
- `make build-race` - Build with race detection
|
|
|
|
### Production Commands
|
|
- `./bin/mev-beta start` - Run production binary (requires PROVIDER_CONFIG_PATH env var)
|
|
- `./scripts/log-manager.sh analyze` - Production log analysis with health scoring
|
|
- `./scripts/log-manager.sh dashboard` - Generate operations dashboard
|
|
- `./scripts/log-manager.sh monitor` - Real-time performance monitoring
|
|
- `./scripts/archive-logs.sh` - Archive and compress log files
|
|
|
|
### Multi-Level Testing System
|
|
|
|
#### Basic Testing
|
|
- `make test` - Run all tests
|
|
- `make test-basic` - Run basic tests (fast, with `-short` flag)
|
|
|
|
#### Unit Testing
|
|
- `make test-unit` - Run unit tests (from `./test/unit/...` and `./pkg/...`)
|
|
|
|
#### Integration Testing
|
|
- `make test-integration` - Run integration tests (from `./test/integration/...`)
|
|
|
|
#### End-to-End Testing
|
|
- `make test-e2e` - Run end-to-end tests (from `./test/e2e/...`)
|
|
|
|
#### Advanced Testing
|
|
- `make test-property` - Run property tests (from `./test/property/...`)
|
|
- `make test-fuzzing` - Run fuzzing tests (from `./test/fuzzing/...`)
|
|
- `make test-stress` - Run stress tests (from `./test/stress/...`)
|
|
- `make test-security` - Run security tests (from `./test/security/...`)
|
|
- `make test-bench` - Run benchmark tests
|
|
|
|
#### Comprehensive Testing
|
|
- `make test-comprehensive` - Run unit + integration + e2e tests
|
|
- `make test-audit` - Run comprehensive + security + stress + benchmark tests
|
|
|
|
#### Test Coverage
|
|
- `make test-coverage` - Run tests with coverage report
|
|
- `make test-coverage-pkg PKG=./pkg/my-package/` - Coverage for specific package
|
|
|
|
### Math-Specific Testing
|
|
- `make test-math` - Run math tests
|
|
- `make test-math-bench` - Run math benchmarks
|
|
- `make test-math-property` - Run math property tests
|
|
- `make test-math-all` - Run all math tests
|
|
|
|
### Code Quality & Auditing
|
|
- `make fmt` - Format code with `go fmt`
|
|
- `make vet` - Vet code with `go vet`
|
|
- `make lint` - Lint code with `golangci-lint`
|
|
- `make audit-security` - Run security audit with `gosec` and `govulncheck`
|
|
- `make audit-deps` - Run dependency audit
|
|
- `make audit-quality` - Run code quality audit (vet + lint)
|
|
- `make audit-full` - Run comprehensive audit (all checks)
|
|
- `make math-audit` - Run deterministic math audit
|
|
|
|
### Development Workflow
|
|
- `make dev-setup` - Setup development environment
|
|
- `make dev-deps` - Install development dependencies
|
|
- `make dev-workflow` - Run development workflow (fmt + vet + lint + basic test)
|
|
- `make dev-workflow-full` - Run development workflow with coverage
|
|
- `make debug` - Run application in debug mode with `dlv`
|
|
- `make watch-tests` - Watch for changes and run basic tests (requires `entr`)
|
|
- `make watch-dev` - Watch for changes and run dev workflow (requires `entr`)
|
|
|
|
### Documentation Generation
|
|
- `make docs` - Generate general code documentation
|
|
- `make docs-api` - Generate API documentation
|
|
- `make docs-all` - Generate all documentation
|
|
|
|
### Maintenance Commands
|
|
- `make clean` - Clean build artifacts
|
|
- `make deps` - Install dependencies
|
|
- `make update` - Update dependencies
|
|
- `make help` - Show help
|
|
|
|
### Simulation Commands
|
|
- `make simulate-profit` - Run profitability simulation
|
|
|
|
## Reusable Scripts
|
|
|
|
### Universal Test Runner
|
|
- **Script**: `scripts/test-runner.sh`
|
|
- **Purpose**: Run tests with different levels of coverage, reusable across projects
|
|
- **Usage**:
|
|
```bash
|
|
# Run basic tests
|
|
./scripts/test-runner.sh -l basic
|
|
|
|
# Run comprehensive tests with coverage
|
|
./scripts/test-runner.sh -l comprehensive -c
|
|
|
|
# Run tests for specific packages
|
|
./scripts/test-runner.sh -p ./pkg/my-package/...
|
|
```
|
|
|
|
### Universal Build Tool
|
|
- **Script**: `scripts/build.sh`
|
|
- **Purpose**: Build Go applications with configurable options, reusable across projects
|
|
- **Usage**:
|
|
```bash
|
|
# Build with default settings
|
|
./scripts/build.sh
|
|
|
|
# Build with specific name and output
|
|
./scripts/build.sh -n myapp -o bin/myapp
|
|
|
|
# Cross-compile for different OS/arch
|
|
./scripts/build.sh --goos linux --goarch amd64
|
|
```
|
|
|
|
### Development Setup Script
|
|
- **Script**: `scripts/setup-dev.sh`
|
|
- **Purpose**: Setup development environment with required tools and directories
|
|
- **Usage**: `./scripts/setup-dev.sh`
|
|
|
|
### Performance Profiling Script
|
|
- **Script**: `scripts/performance-profile.sh`
|
|
- **Purpose**: Profile application performance and run benchmarks
|
|
- **Usage**: `./scripts/performance-profile.sh`
|
|
|
|
### Project Template Creator
|
|
- **Script**: `scripts/create-project-template.sh`
|
|
- **Purpose**: Create a new Go project with standardized structure and tooling
|
|
- **Usage**: `./scripts/create-project-template.sh my-new-project`
|
|
|
|
### Global Tools Manager
|
|
- **Script**: `scripts/mev-tools.sh`
|
|
- **Purpose**: Install and manage global tools across multiple projects
|
|
- **Usage**:
|
|
```bash
|
|
# Install tools to $HOME/bin
|
|
./scripts/mev-tools.sh install
|
|
|
|
# Setup a new project
|
|
./scripts/mev-tools.sh setup-project my-project
|
|
|
|
# Check environment
|
|
./scripts/mev-tools.sh check-env
|
|
```
|
|
|
|
## Global Tools Installation
|
|
|
|
To install tools globally for use across all projects:
|
|
|
|
1. Run the installation command:
|
|
```bash
|
|
./scripts/mev-tools.sh install
|
|
```
|
|
|
|
2. Add `$HOME/bin` to your PATH (if not automatically added):
|
|
```bash
|
|
echo 'export PATH="$HOME/bin:$PATH"' >> ~/.bashrc
|
|
source ~/.bashrc
|
|
```
|
|
|
|
3. After installation, you can use tools like:
|
|
```bash
|
|
test-runner -l comprehensive
|
|
build-tool -n myapp
|
|
```
|
|
|
|
## Configuration
|
|
|
|
### Environment Variables
|
|
|
|
The scripts support several environment variables for customizing behavior:
|
|
|
|
- `TEST_LEVEL`: Level of tests to run (default: "basic")
|
|
- `COVERAGE`: Enable test coverage (default: "false")
|
|
- `OUTPUT_DIR`: Output directory for reports (default: "test-results")
|
|
- `PACKAGE_FILTER`: Go package filter (default: "./...")
|
|
- `TIMEOUT`: Test timeout duration (default: "10m")
|
|
- `JUNIT_OUTPUT`: Generate JUnit output (default: "false")
|
|
- `BINARY_NAME`: Name of the binary (default: current directory name)
|
|
- `BINARY_DIR`: Directory for binaries (default: "bin")
|
|
- `BUILD_TAGS`: Build tags for Go
|
|
- `LDFLAGS`: Ldflags for Go build
|
|
- `PROFILE_TYPES`: Profile types (default: "cpu,mem,block,mutex")
|
|
- `REPORT_DIR`: Directory for reports (default: "reports/performance")
|
|
|
|
### Configuration File
|
|
|
|
You can export these variables in a `.envrc` file that can be sourced by scripts:
|
|
|
|
```bash
|
|
export TEST_LEVEL="comprehensive"
|
|
export COVERAGE="true"
|
|
export OUTPUT_DIR="reports/test-results"
|
|
```
|
|
|
|
Then source it before running commands:
|
|
```bash
|
|
source .envrc
|
|
make test-comprehensive
|
|
```
|
|
|
|
## Recommended Development Workflow
|
|
|
|
1. **Setup Environment**:
|
|
```bash
|
|
make dev-setup
|
|
```
|
|
|
|
2. **Daily Development**:
|
|
```bash
|
|
make dev-workflow
|
|
```
|
|
|
|
3. **Before Commit**:
|
|
```bash
|
|
make dev-workflow-full
|
|
```
|
|
|
|
4. **Periodic Comprehensive Testing**:
|
|
```bash
|
|
make test-audit
|
|
make audit-full
|
|
```
|
|
|
|
5. **Production Deployment** (Validated Oct 24, 2025):
|
|
```bash
|
|
# Set required environment variables
|
|
export MEV_BOT_ENCRYPTION_KEY="production_ready_encryption_key_32_chars_minimum_length_required"
|
|
export PROVIDER_CONFIG_PATH=$PWD/config/providers_runtime.yaml
|
|
|
|
# Start bot
|
|
./bin/mev-beta start
|
|
|
|
# Monitor in separate terminal
|
|
./scripts/log-manager.sh analyze
|
|
tail -f logs/mev_bot.log | grep -E "Arbitrage|PROFIT|DEX Transaction"
|
|
```
|
|
|
|
## CI/CD Integration
|
|
|
|
All commands are designed to work well in CI/CD pipelines. You can use the following patterns:
|
|
|
|
- **Pull Request Validation**: `make dev-workflow`
|
|
- **Nightly Builds**: `make test-audit`
|
|
- **Release Validation**: `make audit-full`
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **Entr Command Not Found**:
|
|
- Install with `apt-get install entr` (Ubuntu/Debian) or `brew install entr` (macOS)
|
|
- Required for `watch-tests` and `watch-dev` targets
|
|
|
|
2. **Missing Dependencies**:
|
|
- Run `make dev-deps` to install all development dependencies
|
|
- Or run `make dev-setup` for complete setup
|
|
|
|
3. **Permission Issues**:
|
|
- Ensure scripts are executable: `chmod +x scripts/*.sh`
|
|
- Make sure you have write permissions to the project directory
|
|
|
|
4. **Go Version Compatibility**:
|
|
- Check `go.mod` for required Go version
|
|
- Update with `go mod tidy` if needed |