mev-beta/docs/security/CREDENTIAL_ROTATION.md

# Credential Rotation Procedure

## Overview
This document describes the procedure for rotating leaked or compromised credentials in the MEV Bot system.

## IMMEDIATE ACTION REQUIRED

**CRITICAL SECURITY ISSUE**: The current `config/providers.yaml` and `.env` files contain a leaked Chainstack API token that is exposed in version control.

### Token Information
- **Service**: Chainstack Arbitrum RPC
- **Exposed Locations**:
  - config/providers.yaml (lines 46, 54)
  - .env (lines 5-7)
  - docker-compose.production.yaml (if exists)
- **Git History**: Token appears in multiple commits

### Leaked Token (MUST BE ROTATED IMMEDIATELY)
```
53c30e7a941160679fdcc396c894fc57
```

## Step 1: Rotate Chainstack Credentials

### 1.1 Generate New API Token

1. Log in to Chainstack dashboard: https://console.chainstack.com
2. Navigate to your Arbitrum node
3. Click "Access and Credentials"
4. Generate new API endpoint (this will create a new token)
5. Copy the new endpoint URLs (HTTP and WebSocket)

### 1.2 Update Local Configuration

1. Copy template file:
```bash
cp config/providers.yaml.template config/providers.yaml
cp .env.example .env
```

2. Edit `config/providers.yaml`:
```yaml
providers:
    - ws_endpoint: wss://arbitrum-mainnet.core.chainstack.com/YOUR_NEW_TOKEN
    - http_endpoint: https://arbitrum-mainnet.core.chainstack.com/YOUR_NEW_TOKEN
```

3. Edit `.env`:
```bash
ARBITRUM_RPC_ENDPOINT=https://arbitrum-mainnet.core.chainstack.com/YOUR_NEW_TOKEN
ARBITRUM_WS_ENDPOINT=wss://arbitrum-mainnet.core.chainstack.com/YOUR_NEW_TOKEN
```

### 1.3 Revoke Old Token

1. In Chainstack dashboard, delete or disable the old endpoint
2. Verify old token no longer works:
```bash
curl https://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57 \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}'
```
Expected result: 401 Unauthorized or connection refused

## Step 2: Clean Git History

**WARNING**: This operation rewrites git history and affects all collaborators.

### Option A: BFG Repo-Cleaner (Recommended)

```bash
# Install BFG Repo-Cleaner
brew install bfg  # macOS
# or download from: https://rtyley.github.io/bfg-repo-cleaner/

# Clone a fresh copy of the repo
cd ..
git clone --mirror git@github.com:your-org/mev-beta.git mev-beta-clean.git
cd mev-beta-clean.git

# Replace leaked token in all history
echo '53c30e7a941160679fdcc396c894fc57' > ../token-to-remove.txt
bfg --replace-text ../token-to-remove.txt

# Clean up and force push
git reflog expire --expire=now --all
git gc --prune=now --aggressive

# Force push (WARNING: Coordinate with team first!)
git push --force
```

### Option B: git filter-repo

```bash
# Install git-filter-repo
pip3 install git-filter-repo

# Clone fresh copy
cd ..
git clone git@github.com:your-org/mev-beta.git mev-beta-clean
cd mev-beta-clean

# Create replacement file
cat > replacements.txt << 'EOF'
53c30e7a941160679fdcc396c894fc57==>YOUR_NEW_TOKEN
wss://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57==>wss://arbitrum-mainnet.core.chainstack.com/YOUR_NEW_TOKEN
https://arbitrum-mainnet.core.chainstack.com/53c30e7a941160679fdcc396c894fc57==>https://arbitrum-mainnet.core.chainstack.com/YOUR_NEW_TOKEN
EOF

# Run filter
git filter-repo --replace-text replacements.txt

# Force push
git push --force --all
```

### Option C: New Repository (If history can't be cleaned)

If the repository is small or history is not critical:

```bash
# Create new repo without history
cd /path/to/mev-beta
rm -rf .git
git init
git add .
git commit -m "Initial commit with cleaned credentials"

# Push to new remote
git remote add origin git@github.com:your-org/mev-beta-new.git
git push -u origin main
```

## Step 3: Update .gitignore

Already completed in Phase 1 fixes. Verify:

```bash
cat .gitignore | grep -E "(providers.yaml|.env|.salt)"
```

Expected output:
```
config/providers.yaml
.env
.env.local
.env.production
.env.staging
keystore/.salt
```

## Step 4: Verify Security

### 4.1 Check No Credentials in Git

```bash
# Search for any remaining tokens
git log -p | grep "53c30e7a941160679fdcc396c894fc57"
# Should return nothing after history cleaning

# Search for API patterns
git log -p | grep -E "chainstack\.com/[a-f0-9]{32}"
# Should only show template placeholders
```

### 4.2 Test New Credentials

```bash
# Test RPC endpoint
curl $ARBITRUM_RPC_ENDPOINT \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}'

# Should return current block number

# Test WebSocket endpoint
wscat -c $ARBITRUM_WS_ENDPOINT
> {"jsonrpc":"2.0","id":1,"method":"eth_blockNumber","params":[]}
```

### 4.3 Verify Bot Starts Successfully

```bash
# Load new credentials
source .env

# Test build
make build

# Test startup (30 second timeout)
timeout 30 ./mev-bot start
```

## Step 5: Notify Team

Send notification to all team members:

```
SECURITY ALERT: Credential Rotation Required

We have rotated the Chainstack API credentials due to a leak in version control.

ACTION REQUIRED:
1. Pull latest changes: git pull --force
2. Copy configuration templates:
   - cp config/providers.yaml.template config/providers.yaml
   - cp .env.example .env
3. Request new credentials from [lead developer]
4. Update your local .env and providers.yaml files
5. DO NOT commit .env or providers.yaml files
6. Verify .gitignore excludes these files

Timeline: Complete by [DATE]
Contact: [SECURITY CONTACT]
```

## Step 6: Implement Monitoring

Add monitoring for credential usage:

```bash
# Chainstack dashboard - check for unusual activity
# Look for:
# - Requests from unknown IPs
# - Spike in request volume
# - Failed authentication attempts

# Set up alerts for:
# - RPC rate limit errors
# - Authentication failures
# - Unusual geographic access patterns
```

## Prevention Checklist

- [x] Created .env.example template
- [x] Created providers.yaml.template template
- [x] Updated .gitignore to exclude sensitive files
- [x] Added validation for missing config files
- [ ] Rotate Chainstack credentials
- [ ] Clean git history
- [ ] Test new credentials
- [ ] Notify team members
- [ ] Set up credential monitoring
- [ ] Schedule next credential rotation (90 days)

## Future Improvements

1. **Secret Management Service**: Migrate to HashiCorp Vault or AWS Secrets Manager
2. **Automated Rotation**: Implement automated credential rotation
3. **Pre-commit Hooks**: Add git hooks to prevent credential commits
4. **Secret Scanning**: Set up GitHub secret scanning
5. **Audit Logging**: Log all credential access attempts

## Contact

For questions or issues with credential rotation:
- Security Team: security@yourcompany.com
- On-call: +1-xxx-xxx-xxxx
- Slack: #security-incidents