# Secrets Management Strategy

This guide defines how secrets move through every environment for the MEV Bot. It is now the source of truth for operators, CI, and fellow agents when handling API keys, private keys, TLS material, and configuration values.

## Summary

| Environment | Storage Backend | Access Method | Rotation Cadence | Notes |
|-------------|-----------------|---------------|------------------|-------|
| Local development | `.env.local` / `.env.devenv` (gitignored) generated from `env/templates/*.env` | `make init-env` or manual copy | On demand | Never commit generated files. Default values are placeholders only. |
| CI (Drone, Harness) | AWS SSM Parameter Store (`/mev-beta/<env>/<name>`) | Pipeline IAM role + `aws ssm get-parameter` wrapper in CI scripts | 30 days | Parameters are encrypted with KMS key `alias/mev-beta-secrets`. |
| Staging | AWS SSM Parameter Store (same hierarchy) | Harness service account role, read‑only | 30 days | Harness pipelines sync secrets into staging pods via `scripts/render-config.sh`. |
| Production | HashiCorp Vault (namespace `mev/prod`) | Vault Agent sidecar + AppRole login (`role_id` vaulted in SSM) | 14 days for API keys, 7 days for signing keys | Vault is the source of truth. SSM only stores bootstrap AppRole credentials. |

## Vault Layout (Production)

- **Mount:** `secret/mev/prod/`
- **Paths:**
  - `secret/mev/prod/rpc/{provider}` – HTTPS RPC credentials and rotation metadata.
  - `secret/mev/prod/keystore/main` – Encrypted MEV bot keystore bundle (GCM sealed).
  - `secret/mev/prod/webhooks/{service}` – Alerting/webhook tokens.
  - `secret/mev/prod/infra/harness` – Harness deploy tokens (read-only by Deploy role).
- **Policies:**
  - `mev-bot-runtime` – read `secret/mev/prod/*` (excluding `/infra`).
  - `mev-deployer` – read all + update deploy tokens.
  - `mev-operations` – read/write all but requires multi-factor.

### Runtime Retrieval Flow

1. Harness deploy job injects Vault Agent sidecar with AppRole credentials (stored in SSM).
2. Sidecar renders secrets into `/run/secrets/*` using template file `config/vault/runtime.hcl`.
3. Bot reads `MEV_BOT_KEYSTORE_PATH` and other environment variables from rendered files.
4. Agent renews leases automatically; restarts occur if lease renewal fails.

## AWS SSM Hierarchy (CI & Staging)

```
/mev-beta/
  staging/
    arbitrum/rpc_url
    arbitrum/ws_url
    mev/security_webhook
  ci/
    drone/github_token
    drone/harness_token
```

- Values are encrypted with customer-managed KMS key (`alias/mev-beta-secrets`).
- Drone CLI wrapper `scripts/ci/export_ssm.sh` exports required values at runtime.
- Harness pipeline uses the same helper via `scripts/ci/harness_env.sh`.

## Local Development Flow

1. Copy template: `cp env/templates/.env.devenv env/.env.local`.
2. Fill required fields manually (Infura API key, test private key, etc.).
3. `cmd/mev-bot` loads `.env.local` automatically when `GO_ENV=development`.
4. Never commit generated files—`.gitignore` already covers everything under `env/`.

## Rotation Playbooks

### Vault API Key Rotation
1. Operator authenticates with Vault MFA (`vault login -method=oidc`).
2. Run `scripts/secrets/rotate_vault_key.sh --path secret/mev/prod/rpc/primary`.
3. Script writes new value, updates rotation metadata, and notifies Slack via webhook.
4. Confirm bot picked up new credentials via `kubectl logs` (look for `Vault secret refresh`).

### SSM Parameter Rotation
1. Use `scripts/secrets/rotate_ssm.sh staging arbitrum/rpc_url`.
2. Harness staging pipeline automatically pulls new value on next deployment.

### Keystore Rotation
1. Generate new key with `scripts/keys/create_keystore.sh`.
2. Upload encrypted archive to Vault path `secret/mev/prod/keystore/main`.
3. Update `MEV_BOT_KEYSTORE_PATH` pointer in Vault template.
4. Trigger rolling restart (`kubectl rollout restart deployment/mev-bot`).

## Auditing & Compliance

- All secret reads are logged:
  - Vault audit backend (`file` + `syslog`).
  - AWS CloudTrail for SSM + KMS.
- Weekly report generated via `scripts/secrets/audit_report.sh`, stored in `reports/security/<date>_secrets_audit.json`.
- Any ad-hoc access must be recorded in `docs/8_reports/security_incident_log.md`.

## Operational Guardrails

- **Never** embed secrets in Kubernetes manifests, Compose files, or config YAML checked into git.
- Use `config/providers.yaml` placeholders only; runtime values always come from Vault/SSM.
- Secrets destined for CI must go through the helper scripts—manual `aws ssm put-parameter` calls are forbidden to keep rotation metadata consistent.
- Keep `MEV_BOT_ENCRYPTION_KEY` out of local shells when screen-sharing. Use `aws ssm get-parameter --with-decryption` piped directly into files.

## Next Steps

- [ ] Automate Vault Agent deployment manifest for production (tracked separately).
- [ ] Add staging smoke test that verifies SSM sync prior to deployment.