feat: create v2-prep branch with comprehensive planning
Restructured project for V2 refactor: **Structure Changes:** - Moved all V1 code to orig/ folder (preserved with git mv) - Created docs/planning/ directory - Added orig/README_V1.md explaining V1 preservation **Planning Documents:** - 00_V2_MASTER_PLAN.md: Complete architecture overview - Executive summary of critical V1 issues - High-level component architecture diagrams - 5-phase implementation roadmap - Success metrics and risk mitigation - 07_TASK_BREAKDOWN.md: Atomic task breakdown - 99+ hours of detailed tasks - Every task < 2 hours (atomic) - Clear dependencies and success criteria - Organized by implementation phase **V2 Key Improvements:** - Per-exchange parsers (factory pattern) - Multi-layer strict validation - Multi-index pool cache - Background validation pipeline - Comprehensive observability **Critical Issues Addressed:** - Zero address tokens (strict validation + cache enrichment) - Parsing accuracy (protocol-specific parsers) - No audit trail (background validation channel) - Inefficient lookups (multi-index cache) - Stats disconnection (event-driven metrics) Next Steps: 1. Review planning documents 2. Begin Phase 1: Foundation (P1-001 through P1-010) 3. Implement parsers in Phase 2 4. Build cache system in Phase 3 5. Add validation pipeline in Phase 4 6. Migrate and test in Phase 5 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Administrator
450
AUTO_UPDATE_GUIDE.md
Normal file
450
AUTO_UPDATE_GUIDE.md
Normal file
@@ -0,0 +1,450 @@
|
||||
# MEV Bot - Auto-Update Guide
|
||||
|
||||
This guide explains how to set up automatic updates for the MEV Bot. When the master branch is updated, your production bot will automatically pull changes, rebuild, and restart.
|
||||
|
||||
## Quick Setup
|
||||
|
||||
### Automated Setup (Recommended)
|
||||
|
||||
```bash
|
||||
# For auto-updates with systemd timer (checks every 5 minutes)
|
||||
sudo ./scripts/setup-auto-update.sh
|
||||
|
||||
# Or without systemd timer (manual updates only)
|
||||
./scripts/setup-auto-update.sh
|
||||
```
|
||||
|
||||
This installs:
|
||||
- Git hooks for auto-rebuild after `git pull`
|
||||
- Systemd timer for periodic update checks (if run with sudo)
|
||||
- Logging infrastructure
|
||||
|
||||
## Update Methods
|
||||
|
||||
The auto-update system supports three methods:
|
||||
|
||||
### 1. Systemd Timer (Recommended for Production)
|
||||
|
||||
Automatically checks for updates every 5 minutes.
|
||||
|
||||
**Setup:**
|
||||
```bash
|
||||
sudo ./scripts/setup-auto-update.sh
|
||||
```
|
||||
|
||||
**Manage the timer:**
|
||||
```bash
|
||||
# Check status
|
||||
sudo systemctl status mev-bot-auto-update.timer
|
||||
|
||||
# View logs
|
||||
sudo journalctl -u mev-bot-auto-update -f
|
||||
|
||||
# Stop timer
|
||||
sudo systemctl stop mev-bot-auto-update.timer
|
||||
|
||||
# Start timer
|
||||
sudo systemctl start mev-bot-auto-update.timer
|
||||
|
||||
# Disable auto-updates
|
||||
sudo systemctl disable mev-bot-auto-update.timer
|
||||
|
||||
# See when next check will run
|
||||
systemctl list-timers mev-bot-auto-update.timer
|
||||
```
|
||||
|
||||
**Change update frequency:**
|
||||
|
||||
Edit `/etc/systemd/system/mev-bot-auto-update.timer`:
|
||||
```ini
|
||||
[Timer]
|
||||
# Check every 10 minutes instead of 5
|
||||
OnUnitActiveSec=10min
|
||||
```
|
||||
|
||||
Then reload:
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart mev-bot-auto-update.timer
|
||||
```
|
||||
|
||||
### 2. Manual Updates with Auto-Rebuild
|
||||
|
||||
Git hooks automatically rebuild and restart after pulling updates.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Just pull - hooks handle the rest
|
||||
git pull
|
||||
|
||||
# Or use the auto-update script
|
||||
./scripts/auto-update.sh
|
||||
```
|
||||
|
||||
The post-merge hook will:
|
||||
- Rebuild the Docker image
|
||||
- Restart the container
|
||||
- Show you the logs
|
||||
- Log everything to `logs/auto-update.log`
|
||||
|
||||
### 3. Webhook-Triggered Updates
|
||||
|
||||
Get instant updates when you push to GitHub/GitLab.
|
||||
|
||||
**Setup webhook receiver:**
|
||||
```bash
|
||||
# Start the webhook receiver
|
||||
./scripts/webhook-receiver.sh
|
||||
|
||||
# Or run in background
|
||||
nohup ./scripts/webhook-receiver.sh > logs/webhook.log 2>&1 &
|
||||
```
|
||||
|
||||
**Configure GitHub webhook:**
|
||||
1. Go to your repository → Settings → Webhooks → Add webhook
|
||||
2. Set Payload URL: `http://your-server:9000/webhook`
|
||||
3. Content type: `application/json`
|
||||
4. Secret: (optional, configure in script)
|
||||
5. Events: Just the push event
|
||||
6. Branch filter: `master`
|
||||
|
||||
**Configure GitLab webhook:**
|
||||
1. Go to your repository → Settings → Webhooks
|
||||
2. Set URL: `http://your-server:9000/webhook`
|
||||
3. Secret token: (optional, configure in script)
|
||||
4. Trigger: Push events
|
||||
5. Branch filter: `master`
|
||||
|
||||
## How It Works
|
||||
|
||||
### Update Flow
|
||||
|
||||
```
|
||||
Remote Update Detected
|
||||
↓
|
||||
git fetch
|
||||
↓
|
||||
Compare local vs remote commits
|
||||
↓
|
||||
git pull
|
||||
↓
|
||||
Post-merge hook triggers
|
||||
↓
|
||||
Docker rebuild
|
||||
↓
|
||||
Container restart
|
||||
↓
|
||||
Verification
|
||||
↓
|
||||
Logs & notification
|
||||
```
|
||||
|
||||
### What Gets Updated
|
||||
|
||||
- ✅ Application code
|
||||
- ✅ Dependencies (go.mod)
|
||||
- ✅ Docker image
|
||||
- ✅ Configuration (if changed)
|
||||
- ❌ .env file (preserved)
|
||||
- ❌ Runtime data
|
||||
- ❌ Logs
|
||||
|
||||
### Safety Features
|
||||
|
||||
The auto-update system includes safety checks:
|
||||
|
||||
1. **Uncommitted changes**: Won't update if you have local changes
|
||||
2. **Branch check**: Only updates on master branch
|
||||
3. **Build verification**: Ensures Docker build succeeds
|
||||
4. **Container health**: Verifies container starts properly
|
||||
5. **Rollback capability**: Previous image remains available
|
||||
6. **Detailed logging**: All actions logged to `logs/auto-update.log`
|
||||
|
||||
## Monitoring Auto-Updates
|
||||
|
||||
### View Update Logs
|
||||
|
||||
```bash
|
||||
# Live tail
|
||||
tail -f logs/auto-update.log
|
||||
|
||||
# Last 50 lines
|
||||
tail -50 logs/auto-update.log
|
||||
|
||||
# Search for specific updates
|
||||
grep "Update Complete" logs/auto-update.log
|
||||
|
||||
# See what commits were deployed
|
||||
grep "Updated from" logs/auto-update.log
|
||||
```
|
||||
|
||||
### Check Last Update
|
||||
|
||||
```bash
|
||||
# Using git
|
||||
git log -1 --pretty=format:"%h - %an, %ar : %s"
|
||||
|
||||
# Using logs
|
||||
tail -20 logs/auto-update.log | grep "Updated to"
|
||||
|
||||
# Container restart time
|
||||
docker inspect mev-bot-production | grep StartedAt
|
||||
```
|
||||
|
||||
### Systemd Status
|
||||
|
||||
```bash
|
||||
# Timer status
|
||||
systemctl status mev-bot-auto-update.timer
|
||||
|
||||
# Service status
|
||||
systemctl status mev-bot-auto-update.service
|
||||
|
||||
# Recent logs
|
||||
journalctl -u mev-bot-auto-update -n 50
|
||||
|
||||
# Follow logs
|
||||
journalctl -u mev-bot-auto-update -f
|
||||
```
|
||||
|
||||
## Notifications
|
||||
|
||||
### Slack Notifications
|
||||
|
||||
Add to your `.env`:
|
||||
```bash
|
||||
WEBHOOK_URL="https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"
|
||||
```
|
||||
|
||||
### Discord Notifications
|
||||
|
||||
Add to your `.env`:
|
||||
```bash
|
||||
WEBHOOK_URL="https://discord.com/api/webhooks/YOUR/DISCORD/WEBHOOK"
|
||||
```
|
||||
|
||||
### Custom Notifications
|
||||
|
||||
Edit `scripts/auto-update.sh` and modify the notification section:
|
||||
```bash
|
||||
if command -v curl &> /dev/null && [ -n "$WEBHOOK_URL" ]; then
|
||||
# Custom notification logic here
|
||||
curl -X POST "$WEBHOOK_URL" ...
|
||||
fi
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Updates Not Happening
|
||||
|
||||
**Check timer is running:**
|
||||
```bash
|
||||
systemctl status mev-bot-auto-update.timer
|
||||
systemctl list-timers | grep mev-bot
|
||||
```
|
||||
|
||||
**Check for errors:**
|
||||
```bash
|
||||
journalctl -u mev-bot-auto-update -n 50
|
||||
tail -50 logs/auto-update.log
|
||||
```
|
||||
|
||||
**Verify git access:**
|
||||
```bash
|
||||
git fetch origin
|
||||
git status
|
||||
```
|
||||
|
||||
### Build Failures
|
||||
|
||||
**Check build logs:**
|
||||
```bash
|
||||
tail -100 logs/auto-update.log | grep -A 20 "ERROR"
|
||||
```
|
||||
|
||||
**Try manual build:**
|
||||
```bash
|
||||
docker compose build
|
||||
```
|
||||
|
||||
**Check disk space:**
|
||||
```bash
|
||||
df -h
|
||||
docker system df
|
||||
```
|
||||
|
||||
### Uncommitted Changes Blocking Updates
|
||||
|
||||
```bash
|
||||
# See what's uncommitted
|
||||
git status
|
||||
|
||||
# Stash changes
|
||||
git stash save "Local changes before auto-update"
|
||||
|
||||
# Or commit them
|
||||
git add .
|
||||
git commit -m "Local production changes"
|
||||
|
||||
# Or reset (careful!)
|
||||
git reset --hard origin/master
|
||||
```
|
||||
|
||||
### Rollback to Previous Version
|
||||
|
||||
```bash
|
||||
# See available images
|
||||
docker images | grep mev-bot
|
||||
|
||||
# Stop current container
|
||||
docker compose down
|
||||
|
||||
# Use previous image (if available)
|
||||
docker tag mev-bot:latest mev-bot:backup
|
||||
# Then restore from backup or rebuild specific commit
|
||||
|
||||
# Or rollback via git
|
||||
git log --oneline -10 # Find commit to rollback to
|
||||
git reset --hard <commit-hash>
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Test Updates in Staging First
|
||||
|
||||
```bash
|
||||
# On staging server
|
||||
git pull
|
||||
docker compose up -d --build
|
||||
# Run tests, verify functionality
|
||||
# Then push to production (auto-updates will handle it)
|
||||
```
|
||||
|
||||
### 2. Monitor After Updates
|
||||
|
||||
```bash
|
||||
# Watch logs for 5 minutes after update
|
||||
docker compose logs -f mev-bot
|
||||
|
||||
# Check metrics
|
||||
curl http://localhost:9090/metrics
|
||||
|
||||
# Verify trading activity
|
||||
grep "opportunity detected" logs/auto-update.log
|
||||
```
|
||||
|
||||
### 3. Backup Before Major Updates
|
||||
|
||||
```bash
|
||||
# Backup database/state
|
||||
cp -r data/ backup/data-$(date +%Y%m%d-%H%M%S)
|
||||
|
||||
# Tag current version
|
||||
git tag -a v$(date +%Y%m%d-%H%M%S) -m "Pre-update backup"
|
||||
```
|
||||
|
||||
### 4. Use Feature Branches
|
||||
|
||||
```bash
|
||||
# Develop on feature branches
|
||||
git checkout -b feature/new-strategy
|
||||
|
||||
# Test thoroughly
|
||||
# Merge to master only when ready
|
||||
git checkout master
|
||||
git merge feature/new-strategy
|
||||
git push origin master
|
||||
|
||||
# Production auto-updates within 5 minutes
|
||||
```
|
||||
|
||||
### 5. Schedule Maintenance Windows
|
||||
|
||||
For major updates, temporarily disable auto-updates:
|
||||
```bash
|
||||
# Disable timer
|
||||
sudo systemctl stop mev-bot-auto-update.timer
|
||||
|
||||
# Perform manual update with monitoring
|
||||
./scripts/auto-update.sh
|
||||
|
||||
# Re-enable timer
|
||||
sudo systemctl start mev-bot-auto-update.timer
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Custom Update Frequency
|
||||
|
||||
Create a custom timer:
|
||||
```bash
|
||||
# Edit /etc/systemd/system/mev-bot-auto-update.timer
|
||||
[Timer]
|
||||
OnBootSec=5min
|
||||
OnUnitActiveSec=1min # Check every minute (aggressive)
|
||||
# or
|
||||
OnCalendar=*:0/15 # Check every 15 minutes
|
||||
# or
|
||||
OnCalendar=hourly # Check every hour
|
||||
```
|
||||
|
||||
### Update Specific Branches
|
||||
|
||||
Edit `/etc/systemd/system/mev-bot-auto-update.service`:
|
||||
```ini
|
||||
[Service]
|
||||
Environment="GIT_BRANCH=production"
|
||||
Environment="GIT_REMOTE=origin"
|
||||
```
|
||||
|
||||
### Pre/Post Update Hooks
|
||||
|
||||
Edit `scripts/auto-update.sh` to add custom logic:
|
||||
```bash
|
||||
# Before update
|
||||
pre_update_hook() {
|
||||
# Custom logic (e.g., backup, notifications)
|
||||
}
|
||||
|
||||
# After update
|
||||
post_update_hook() {
|
||||
# Custom logic (e.g., warmup, health checks)
|
||||
}
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Use SSH keys for git** - Avoid storing credentials
|
||||
2. **Validate webhook signatures** - Prevent unauthorized updates
|
||||
3. **Limit network access** - Firewall webhook receiver port
|
||||
4. **Monitor update logs** - Detect suspicious activity
|
||||
5. **Use signed commits** - Verify update authenticity
|
||||
|
||||
## Uninstall Auto-Updates
|
||||
|
||||
```bash
|
||||
# Remove systemd timer
|
||||
sudo systemctl stop mev-bot-auto-update.timer
|
||||
sudo systemctl disable mev-bot-auto-update.timer
|
||||
sudo rm /etc/systemd/system/mev-bot-auto-update.*
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
# Remove git hooks
|
||||
rm .git/hooks/post-merge
|
||||
|
||||
# Stop webhook receiver
|
||||
pkill -f webhook-receiver.sh
|
||||
```
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues:
|
||||
1. Check logs: `tail -100 logs/auto-update.log`
|
||||
2. Verify git status: `git status`
|
||||
3. Test manual update: `./scripts/auto-update.sh`
|
||||
4. Check systemd: `journalctl -u mev-bot-auto-update -n 50`
|
||||
|
||||
---
|
||||
|
||||
**Your MEV bot now auto-updates whenever master branch changes!** 🎉
|
||||
Reference in New Issue
Block a user