mev-beta/docs/5_development/LOCAL_CICD.md

Local CI/CD with Drone & Harness

This project now treats GitHub Actions configs as legacy. The authoritative automation lives in the Drone and Harness definitions checked into the repository.

Running the Drone pipelines locally

Prerequisites:

• Docker Engine available (for the Drone runner images).
• drone CLI installed (brew install drone-cli or go install github.com/harness/drone-cli/drone@latest).

Common commands:

# Execute the primary test suite locally
DRONE_GIT_BRANCH=$(git rev-parse --abbrev-ref HEAD) \
DRONE_COMMIT_SHA=$(git rev-parse HEAD) \
drone exec --pipeline test-suite

# Run the security pipeline (same environment variables as above)
drone exec --pipeline security-suite

# Kick off the optional integration run (requires RPC endpoints/mocks)
drone exec --pipeline integration-opt-in

Optional tags:

• legacy enables the archived integration suites (RPC-heavy).
• forked enables the fork/anvil smoke tests (e.g., flash swap executor).

Pipeline summary (mirrors historical GitHub jobs):

Drone Stage	What it Does
setup-go-cache	Warm caches, verify modules.
lint	golangci-lint with 10m timeout.
unit-tests	Full go test -race -cover ./....
build-binary	Compiles ./cmd/mev-bot into bin/mev-bot.
smoke-start	Boots the binary for 5s using a test encryption key (expected to fail without keystore).
math-audit	Runs tools/math-audit and validates artifacts.
simulate-profit	Executes ./scripts/run_profit_simulation.sh.
docker-build	Dry-run Docker build via plugins/docker.
security-suite	Gosec, govulncheck, Nancy, and fuzz tests for pkg/security.
integration-opt-in	Executes go test -tags=integration ./... when explicitly triggered.

Harness pipeline hand-off

Harness orchestration focuses on promotion to staging/production. See harness/pipelines/staging.yaml for the canonical workflow. Use Harness CLI (harness pipeline execute ...) or the UI to run the same stages locally.

Running the staging workflow without Harness

For offline validation you can mirror the Harness stages with the helper script scripts/staging-pipeline-local.sh. Every stage runs inside a container using either Podman or Docker (auto-detected, or honour LOCAL_STAGING_RUNTIME). It executes the same lint, test, audit, simulation, image build, and Helm deployment steps that the Harness staging_promotion pipeline performs.

# end-to-end local staging run (writes logs under reports/ci/local-staging)
./scripts/staging-pipeline-local.sh

# example with custom image tag and real Helm upgrade instead of dry-run
LOCAL_STAGING_IMAGE_TAG=$(git rev-parse --short HEAD) \
LOCAL_STAGING_HELM_DRY_RUN=false \
./scripts/staging-pipeline-local.sh

# skip Docker and deploy stages (lint/tests/audit/simulation only)
LOCAL_STAGING_SKIP_DOCKER=true \
LOCAL_STAGING_SKIP_DEPLOY=true \
./scripts/staging-pipeline-local.sh

Key environment toggles:

• LOCAL_STAGING_BRANCH – branch recorded in logs (defaults to git rev-parse --abbrev-ref HEAD).
• LOCAL_STAGING_RUNTIME – force docker or podman (defaults to auto-detect).
• LOCAL_STAGING_IMAGE_NAME, LOCAL_STAGING_IMAGE_TAG, LOCAL_STAGING_IMAGE_TAR – Docker image reference and saved tarball path.
• LOCAL_STAGING_SKIP_DOCKER – skip the Docker build/save stage when true.
• LOCAL_STAGING_HELM_DRY_RUN – set to false to perform a real Helm upgrade; defaults to true (safe dry-run).
• LOCAL_STAGING_SKIP_DEPLOY – skip the Helm/Kubernetes stage when true.
• LOCAL_STAGING_KUBECONFIG – path to the kubeconfig file mounted inside the Helm/Kubectl containers (defaults to ~/.kube/config).

The script only needs a container runtime and will pull the required tool images (golang:1.24, golangci-lint, helm, kubectl). Logs and artifacts are saved in reports/ci/local-staging, mirroring the Harness pipeline output layout.

Migrating from GitHub Actions

• .github/workflows/ remains for reference only (manual dispatch only). New checks must be added to Drone or Harness.
• Update the Agent checklist (AGENTS.md) when a GitHub workflow is fully retired.
• Security and compliance reporting artifacts still upload to reports/ for archival.

Troubleshooting

• Drone steps run inside containers; ensure required host folders (e.g., reports/) are writable.
• Some integration tests require RPC endpoints. Set environment variables (ARBITRUM_RPC_ENDPOINT, etc.) or skip the pipeline.
• For Harness, secrets mount from the Harness secret manager—you will need the CLI logged in to your Harness account before executing pipelines locally.