803de231ba
Restructured project for V2 refactor: **Structure Changes:** - Moved all V1 code to orig/ folder (preserved with git mv) - Created docs/planning/ directory - Added orig/README_V1.md explaining V1 preservation **Planning Documents:** - 00_V2_MASTER_PLAN.md: Complete architecture overview - Executive summary of critical V1 issues - High-level component architecture diagrams - 5-phase implementation roadmap - Success metrics and risk mitigation - 07_TASK_BREAKDOWN.md: Atomic task breakdown - 99+ hours of detailed tasks - Every task < 2 hours (atomic) - Clear dependencies and success criteria - Organized by implementation phase **V2 Key Improvements:** - Per-exchange parsers (factory pattern) - Multi-layer strict validation - Multi-index pool cache - Background validation pipeline - Comprehensive observability **Critical Issues Addressed:** - Zero address tokens (strict validation + cache enrichment) - Parsing accuracy (protocol-specific parsers) - No audit trail (background validation channel) - Inefficient lookups (multi-index cache) - Stats disconnection (event-driven metrics) Next Steps: 1. Review planning documents 2. Begin Phase 1: Foundation (P1-001 through P1-010) 3. Implement parsers in Phase 2 4. Build cache system in Phase 3 5. Add validation pipeline in Phase 4 6. Migrate and test in Phase 5 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
296 lines
6.3 KiB
Markdown
296 lines
6.3 KiB
Markdown
# MEV Bot Development Environment
|
|
|
|
This guide explains how to use the development environment with branch selection and podman-in-podman support.
|
|
|
|
## Quick Start
|
|
|
|
### Start Development Environment
|
|
|
|
```bash
|
|
# Start with default branch (master-dev)
|
|
./scripts/dev-env.sh start
|
|
|
|
# Start with specific branch
|
|
./scripts/dev-env.sh start fix-critical-arbitrage-bugs
|
|
|
|
# Start with master branch
|
|
./scripts/dev-env.sh start master
|
|
```
|
|
|
|
### Switch Branches
|
|
|
|
```bash
|
|
# Switch to a different branch (stops current, checks out, rebuilds)
|
|
./scripts/dev-env.sh switch feat-new-feature
|
|
|
|
# List available branches
|
|
./scripts/dev-env.sh branches
|
|
```
|
|
|
|
### View Logs
|
|
|
|
```bash
|
|
# Follow logs
|
|
./scripts/dev-env.sh logs -f
|
|
|
|
# Show last 100 lines
|
|
./scripts/dev-env.sh logs --tail 100
|
|
```
|
|
|
|
### Access Container Shell
|
|
|
|
```bash
|
|
# Open interactive shell in running container
|
|
./scripts/dev-env.sh shell
|
|
```
|
|
|
|
### Stop/Restart
|
|
|
|
```bash
|
|
# Stop development environment
|
|
./scripts/dev-env.sh stop
|
|
|
|
# Restart with same branch
|
|
./scripts/dev-env.sh restart
|
|
|
|
# Restart with different branch
|
|
./scripts/dev-env.sh restart master
|
|
```
|
|
|
|
### Rebuild
|
|
|
|
```bash
|
|
# Full rebuild (no cache) with current branch
|
|
./scripts/dev-env.sh rebuild
|
|
|
|
# Full rebuild with specific branch
|
|
./scripts/dev-env.sh rebuild master-dev
|
|
```
|
|
|
|
## Architecture
|
|
|
|
### Files
|
|
|
|
- **Dockerfile.dev** - Development dockerfile with branch selection support
|
|
- **docker-compose.dev.yml** - Development compose configuration
|
|
- **scripts/dev-env.sh** - Development environment management script
|
|
|
|
### Features
|
|
|
|
1. **Branch Selection**: Build and run from any git branch
|
|
2. **Podman-in-Podman**: Container can run podman commands (privileged mode)
|
|
3. **Live Logs**: Persistent log directory mounted from host
|
|
4. **Hot Reload**: Source code mounted for development (optional)
|
|
5. **Resource Management**: Configurable CPU and memory limits
|
|
|
|
## Environment Variables
|
|
|
|
Set these in `.env` or export before running:
|
|
|
|
```bash
|
|
# Branch to build from
|
|
export GIT_BRANCH=master-dev
|
|
|
|
# Logging level
|
|
export LOG_LEVEL=debug
|
|
|
|
# Enable metrics
|
|
export METRICS_ENABLED=true
|
|
|
|
# RPC endpoint
|
|
export ARBITRUM_RPC_ENDPOINT=https://arb1.arbitrum.io/rpc
|
|
```
|
|
|
|
## Podman-in-Podman Support
|
|
|
|
The development environment supports running podman inside the container:
|
|
|
|
```bash
|
|
# Access container shell
|
|
./scripts/dev-env.sh shell
|
|
|
|
# Inside container, you can use podman
|
|
podman ps
|
|
podman images
|
|
podman run ...
|
|
```
|
|
|
|
### Requirements
|
|
|
|
- Container runs in privileged mode
|
|
- Podman socket mounted from host: `/run/podman/podman.sock`
|
|
- Persistent storage volume: `podman-storage`
|
|
|
|
## Development Workflow
|
|
|
|
### Feature Development
|
|
|
|
```bash
|
|
# 1. Create feature branch
|
|
git checkout -b feat-my-feature
|
|
|
|
# 2. Start development environment with your branch
|
|
./scripts/dev-env.sh start feat-my-feature
|
|
|
|
# 3. Make changes in your local files
|
|
|
|
# 4. Rebuild to test changes
|
|
./scripts/dev-env.sh rebuild feat-my-feature
|
|
|
|
# 5. View logs to verify
|
|
./scripts/dev-env.sh logs -f
|
|
|
|
# 6. When done, stop environment
|
|
./scripts/dev-env.sh stop
|
|
```
|
|
|
|
### Testing Different Branches
|
|
|
|
```bash
|
|
# Test fix branch
|
|
./scripts/dev-env.sh switch fix-critical-arbitrage-bugs
|
|
|
|
# Test feature branch
|
|
./scripts/dev-env.sh switch feat-podman-compose-support
|
|
|
|
# Go back to master-dev
|
|
./scripts/dev-env.sh switch master-dev
|
|
```
|
|
|
|
### Debugging
|
|
|
|
```bash
|
|
# Start with debug logging
|
|
export LOG_LEVEL=debug
|
|
./scripts/dev-env.sh start
|
|
|
|
# Access container for inspection
|
|
./scripts/dev-env.sh shell
|
|
|
|
# Inside container:
|
|
ps aux # Check processes
|
|
cat /app/config/config.yaml # View config
|
|
ls -la /app # List files
|
|
```
|
|
|
|
## Comparison: Development vs Production
|
|
|
|
| Feature | Development (dev-env.sh) | Production (deploy-production-docker.sh) |
|
|
|---------|-------------------------|------------------------------------------|
|
|
| Branch Selection | ✅ Yes (any branch) | ✅ Yes (defaults to master) |
|
|
| Dockerfile | Dockerfile.dev | Dockerfile |
|
|
| Compose File | docker-compose.dev.yml | docker-compose.yml |
|
|
| Privileged Mode | ✅ Yes (for podman-in-podman) | ❌ No |
|
|
| Resource Limits | Higher (4 CPU, 4GB RAM) | Lower (2 CPU, 2GB RAM) |
|
|
| Restart Policy | unless-stopped | always |
|
|
| Container Name | mev-bot-dev-{branch} | mev-bot-production |
|
|
| Source Mount | ✅ Yes (optional) | ❌ No |
|
|
| Auto-start on Boot | ❌ No | ✅ Yes (via systemd) |
|
|
|
|
## Troubleshooting
|
|
|
|
### Container Won't Start
|
|
|
|
```bash
|
|
# Check status
|
|
./scripts/dev-env.sh status
|
|
|
|
# View logs
|
|
./scripts/dev-env.sh logs
|
|
|
|
# Rebuild from scratch
|
|
./scripts/dev-env.sh clean
|
|
./scripts/dev-env.sh rebuild
|
|
```
|
|
|
|
### Branch Doesn't Exist
|
|
|
|
```bash
|
|
# List available branches
|
|
./scripts/dev-env.sh branches
|
|
|
|
# Update branch list
|
|
git fetch --all
|
|
./scripts/dev-env.sh branches
|
|
```
|
|
|
|
### Podman Socket Not Available
|
|
|
|
```bash
|
|
# Check if podman socket is running
|
|
systemctl --user status podman.socket
|
|
|
|
# Start podman socket
|
|
systemctl --user start podman.socket
|
|
```
|
|
|
|
### Out of Disk Space
|
|
|
|
```bash
|
|
# Clean old images and containers
|
|
./scripts/dev-env.sh clean
|
|
|
|
# Prune podman system
|
|
podman system prune -a
|
|
```
|
|
|
|
## Advanced Usage
|
|
|
|
### Custom Configuration
|
|
|
|
Create a `.env.dev` file for development-specific settings:
|
|
|
|
```bash
|
|
# .env.dev
|
|
GIT_BRANCH=master-dev
|
|
LOG_LEVEL=debug
|
|
METRICS_ENABLED=true
|
|
METRICS_PORT=9090
|
|
PORT=8080
|
|
```
|
|
|
|
Load it before starting:
|
|
|
|
```bash
|
|
set -a && source .env.dev && set +a
|
|
./scripts/dev-env.sh start
|
|
```
|
|
|
|
### Manual Podman Commands
|
|
|
|
```bash
|
|
# Use docker-compose directly
|
|
export GIT_BRANCH=feat-my-branch
|
|
podman-compose -f docker-compose.dev.yml build
|
|
podman-compose -f docker-compose.dev.yml up -d
|
|
|
|
# Or use single command
|
|
GIT_BRANCH=feat-my-branch podman-compose -f docker-compose.dev.yml up -d --build
|
|
```
|
|
|
|
### Performance Monitoring
|
|
|
|
```bash
|
|
# Access metrics endpoint (if enabled)
|
|
curl http://localhost:9090/metrics
|
|
|
|
# View resource usage
|
|
podman stats
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
1. **Use feature branches** for development, not master
|
|
2. **Test in dev environment** before merging to master-dev
|
|
3. **Rebuild after git pull** to ensure latest code
|
|
4. **Monitor logs** during development with `-f` flag
|
|
5. **Clean regularly** to free up disk space
|
|
6. **Use specific branches** instead of relying on defaults
|
|
|
|
## Related Documentation
|
|
|
|
- [DEPLOYMENT.md](DEPLOYMENT.md) - Production deployment guide
|
|
- [CLAUDE.md](CLAUDE.md) - Project overview and guidelines
|
|
- [docker-compose.dev.yml](docker-compose.dev.yml) - Development compose configuration
|
|
- [Dockerfile.dev](Dockerfile.dev) - Development dockerfile
|