Files
mev-beta/docs/SCRIPTS_REFERENCE.md
Administrator 3505921207 feat: comprehensive audit infrastructure and Phase 1 refactoring
This commit includes:

## Audit & Testing Infrastructure
- scripts/audit.sh: 12-section comprehensive codebase audit
- scripts/test.sh: 7 test types (unit, integration, race, bench, coverage, contracts, pkg)
- scripts/check-compliance.sh: SPEC.md compliance validation
- scripts/check-docs.sh: Documentation coverage checker
- scripts/dev.sh: Unified development script with all commands

## Documentation
- SPEC.md: Authoritative technical specification
- docs/AUDIT_AND_TESTING.md: Complete testing guide (600+ lines)
- docs/SCRIPTS_REFERENCE.md: All scripts documented (700+ lines)
- docs/README.md: Documentation index and navigation
- docs/DEVELOPMENT_SETUP.md: Environment setup guide
- docs/REFACTORING_PLAN.md: Systematic refactoring plan

## Phase 1 Refactoring (Critical Fixes)
- pkg/validation/helpers.go: Validation functions for addresses/amounts
- pkg/sequencer/selector_registry.go: Thread-safe selector registry
- pkg/sequencer/reader.go: Fixed race conditions with atomic metrics
- pkg/sequencer/swap_filter.go: Fixed race conditions, added error logging
- pkg/sequencer/decoder.go: Added address validation

## Changes Summary
- Fixed race conditions on 13 metric counters (atomic operations)
- Added validation at all ingress points
- Eliminated silent error handling
- Created selector registry for future ABI migration
- Reduced SPEC.md violations from 7 to 5

Build Status:  All packages compile
Compliance:  No race conditions, no silent failures
Documentation:  1,700+ lines across 5 comprehensive guides

🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-11 07:17:13 +01:00

10 KiB
Raw Blame History

Scripts Reference Guide

Complete reference for all development, testing, and audit scripts.

Table of Contents

Main Development Script: dev.sh

Location: scripts/dev.sh

Purpose: Unified interface for all development operations. Enforces containerized development workflow per SPEC.md.

Usage

./scripts/dev.sh <command> [args]

Commands

Container Management

./scripts/dev.sh up          # Start all dev containers
./scripts/dev.sh down        # Stop all dev containers
./scripts/dev.sh restart     # Restart dev containers
./scripts/dev.sh ps          # Show container status
./scripts/dev.sh logs <svc>  # View logs for service

Container Access

./scripts/dev.sh go          # Enter Go container (interactive shell)
./scripts/dev.sh python      # Enter Python container
./scripts/dev.sh foundry     # Enter Foundry container

Build & Test

./scripts/dev.sh build       # Build Go application
./scripts/dev.sh test <type> # Run tests (see Testing section)

Audit & Quality

./scripts/dev.sh audit            # Comprehensive code audit
./scripts/dev.sh check-docs       # Documentation coverage
./scripts/dev.sh check-compliance # SPEC.md compliance

Contract Operations

./scripts/dev.sh forge-build # Build Solidity contracts
./scripts/dev.sh forge-test  # Run contract tests
./scripts/dev.sh bindings    # Generate Go bindings

Cleanup

./scripts/dev.sh clean   # Remove build artifacts
./scripts/dev.sh reset   # Stop containers + clean

Testing Scripts

test.sh - Comprehensive Test Suite

Location: scripts/test.sh

Purpose: Run all types of tests with proper containerization.

Usage

./scripts/test.sh [type] [verbose]

Test Types

# All tests (unit + integration + race + contracts)
./scripts/test.sh all

# Unit tests only
./scripts/test.sh unit

# Integration tests
./scripts/test.sh integration

# Race detection
./scripts/test.sh race

# Benchmarks
./scripts/test.sh bench

# Coverage report
./scripts/test.sh coverage

# Specific package
./scripts/test.sh pkg sequencer

# Contract tests
./scripts/test.sh contracts

Verbose Mode

./scripts/test.sh unit true      # Verbose unit tests
./scripts/test.sh all v          # Verbose all tests

Output

  • Exit 0: All tests passed
  • Exit 1: One or more test suites failed

Coverage Report

When running coverage type:

  • Generates coverage/coverage.out
  • Generates coverage/coverage.html (open in browser)
  • Prints coverage percentage

Target: >70% coverage


Audit Scripts

audit.sh - Codebase Audit

Location: scripts/audit.sh

Purpose: Comprehensive code quality, security, and compliance audit.

Usage

./scripts/audit.sh

What It Checks

1. SPEC.md Compliance

  • Hardcoded function selectors
  • HTTP RPC usage in sequencer
  • Blocking operations in hot paths
  • Manual ABI files

2. Go Code Quality

  • go vet issues
  • TODO/FIXME comments
  • panic() in production code

3. Security

  • Hardcoded private keys
  • SQL injection risks
  • Command injection
  • Unsafe pointer usage

4. Concurrency Safety

  • Race condition risks
  • Mutex usage
  • Channel coverage

5. Error Handling

  • Ignored errors
  • Error wrapping patterns

6. Documentation

  • Exported function comments
  • Documentation coverage %

7. Test Coverage

  • Test file ratio

8. Dependencies

  • Outdated packages

9. Contract Bindings

  • Binding files present
  • Bindings used in code

10. Build Verification

  • Code compiles

11. File Organization

  • Large files (>1MB)
  • Deep nesting

12. Git Status

  • Uncommitted changes

Output

  • Exit 0: No issues found
  • Exit 1: Issues found (review required)

Issues are categorized by severity:

  • [CRITICAL] - Must fix immediately
  • [HIGH] - Fix before commit
  • [MEDIUM] - Fix soon
  • [LOW] - Consider fixing
  • [INFO] - Informational only

check-docs.sh - Documentation Coverage

Location: scripts/check-docs.sh

Purpose: Ensure all code is properly documented.

Usage

./scripts/check-docs.sh

What It Checks

1. Package Documentation

  • doc.go files in all packages

2. Exported Functions

  • Comment on line before func [A-Z]

3. Exported Types

  • Comment on line before type [A-Z]

4. README Files

  • README.md in critical directories

5. Project Documentation

  • SPEC.md
  • CLAUDE.md
  • README.md
  • docs/DEVELOPMENT_SETUP.md

6. Inline Comments

  • Comment density ratio

7. API Documentation

  • API.md for HTTP endpoints

8. Examples

  • Example code files

Output

  • Coverage percentage
  • List of undocumented items
  • Exit 0: >80% coverage
  • Exit 1: <80% coverage

check-compliance.sh - SPEC.md Compliance

Location: scripts/check-compliance.sh

Purpose: Verify code adheres to all SPEC.md requirements.

Usage

./scripts/check-compliance.sh

What It Validates

MUST DO Requirements

  1. Use Arbitrum sequencer feed
  2. Channel-based communication
  3. Official contract ABIs
  4. Generated bindings with abigen
  5. Validate all parsed data
  6. Thread-safe structures
  7. Comprehensive metrics
  8. Container-based development

MUST NOT DO Requirements

  1. HTTP RPC in sequencer
  2. Manual ABI JSON files
  3. Hardcoded function selectors
  4. Zero addresses/amounts propagation
  5. Blocking operations in hot paths
  6. Unprotected shared state
  7. Silent error failures

Architecture Requirements 🏗️

  • Channel-based concurrency
  • Sequencer isolation
  • Pool cache with RWMutex

Foundry Integration 🔨

  • foundry.toml present
  • Correct Solidity version

Development Scripts 🛠️

  • All required scripts present and executable

Output

  • Exit 0: Fully compliant or minor issues (<5)
  • Exit 1: Significant issues (>5)

Violations are categorized:

  • [CRITICAL] - Core requirement violated
  • [HIGH] - Important requirement violated
  • [MEDIUM] - Recommended practice violated
  • [LOW] - Minor issue

Contract Scripts

generate-bindings.sh - Legacy Binding Generator

Location: scripts/generate-bindings.sh

Status: Legacy script (use dev.sh bindings instead)

Purpose: Generate Go bindings from manually created ABI files.

Usage

./scripts/generate-bindings.sh

generate-bindings-in-container.sh - Container Binding Generator

Location: scripts/generate-bindings-in-container.sh

Purpose: Generate Go bindings from Foundry artifacts inside Go container.

Called by: ./scripts/dev.sh bindings

Usage

# From host (preferred)
./scripts/dev.sh bindings

# Manually inside Go container
./scripts/generate-bindings-in-container.sh

extract-official-abis.sh - ABI Extraction

Location: scripts/extract-official-abis.sh

Purpose: Extract ABIs directly from official contracts using forge inspect.

Usage

./scripts/extract-official-abis.sh

Extracts:

  • bindings/uniswap_v2/IUniswapV2Pair_abi.json
  • bindings/uniswap_v3/ISwapRouter_abi.json

generate-bindings-from-official-abis.sh - Official Binding Generator

Location: scripts/generate-bindings-from-official-abis.sh

Purpose: Generate bindings from extracted official ABIs.

Usage

# Extract ABIs first
./scripts/extract-official-abis.sh

# Generate bindings
./scripts/generate-bindings-from-official-abis.sh

Utility Scripts

dev-up.sh - Start Dev Environment

Location: scripts/dev-up.sh

Purpose: Start development containers (legacy - use dev.sh up instead).

Usage

./scripts/dev-up.sh

Starts:

  • mev-go-dev
  • mev-python-dev
  • mev-foundry

dev-down.sh - Stop Dev Environment

Location: scripts/dev-down.sh

Purpose: Stop development containers (legacy - use dev.sh down instead).

Usage

./scripts/dev-down.sh

Quick Reference

Daily Development

# Start working
./scripts/dev.sh up

# Build and test
./scripts/dev.sh build
./scripts/dev.sh test unit

# Before commit
./scripts/dev.sh test all
./scripts/dev.sh check-compliance

Before Push

# Full validation
./scripts/dev.sh test all
./scripts/dev.sh test race
./scripts/dev.sh audit
./scripts/dev.sh check-compliance
./scripts/dev.sh check-docs

Contract Development

# Build contracts
./scripts/dev.sh forge-build

# Generate bindings
./scripts/dev.sh bindings

# Test contracts
./scripts/dev.sh forge-test

Troubleshooting

# View container logs
./scripts/dev.sh logs go-dev

# Restart containers
./scripts/dev.sh restart

# Clean and rebuild
./scripts/dev.sh reset
./scripts/dev.sh up
./scripts/dev.sh build

Script Dependencies

dev.sh
├── test.sh                               # Testing suite
├── audit.sh                              # Code audit
├── check-docs.sh                         # Doc coverage
├── check-compliance.sh                   # SPEC compliance
└── generate-bindings-in-container.sh     # Binding generation

test.sh
└── (runs in mev-go-dev container)

audit.sh
└── (runs in mev-go-dev container for go vet)

generate-bindings-in-container.sh
└── (runs in mev-go-dev container)
    └── extract-official-abis.sh (optional)

Environment Variables

Scripts respect these environment variables:

# Override container names
GO_CONTAINER=mev-go-dev
FOUNDRY_CONTAINER=mev-foundry
PYTHON_CONTAINER=mev-python-dev

# Test verbosity
TEST_VERBOSE=true

# Coverage threshold
COVERAGE_THRESHOLD=70

Exit Codes

All scripts use consistent exit codes:

  • 0: Success (all checks passed)
  • 1: Failure (tests failed, violations found, etc.)
  • 127: Command not found
  • 255: Container error

Logging

Scripts use consistent logging format:

  • - Info message (blue)
  • - Success (green)
  • - Warning (yellow)
  • - Error (red)

See Also