diff --git a/AKS_DEPLOYMENT.md b/AKS_DEPLOYMENT.md new file mode 100644 index 00000000..8d111f96 --- /dev/null +++ b/AKS_DEPLOYMENT.md @@ -0,0 +1,105 @@ +# Production AKS Deployment Guide + +> **⚠️ Notice:** +> The following production deployment instructions have **not been fully tested**. +> Please proceed with caution and verify each step in your environment. +> If you encounter issues, consult the script comments and configuration files for troubleshooting. + +Full production deployment of the **Buttercup CRS** on Azure Kubernetes Service with proper networking, monitoring, and scaling for the DARPA AIxCC competition. + +## Quick Setup (Recommended) + +Use our automated setup script: + +```bash +make setup-azure +``` + +This script will check prerequisites, help create service principals, configure the environment, and validate your setup. + +## Manual Setup + +If you prefer to set up manually, follow these steps: + +### Prerequisites + +- Azure CLI installed and configured +- Terraform installed +- Active Azure subscription +- Access to competition Tailscale tailnet + +### Azure Setup + +1. **Login to Azure:** + +```bash +az login --tenant +``` + +2. **Create Service Principal:** + +```bash +# Get your subscription ID +az account show --query "{SubscriptionID:id}" --output table + +# Create service principal (replace with your subscription ID) +az ad sp create-for-rbac --name "ButtercupCRS" --role Contributor --scopes /subscriptions/ +``` + +### Production Configuration + +1. **Configure environment file:** + +```bash +cp deployment/env.template deployment/env +``` + +2. **Update `deployment/env` for production:** + +Look at the comments in the `deployment/env.template` for how to set variables. +In particular, set `TF_VAR_*` variables, and `TAILSCALE_*` if used. + +## Deploy to AKS + +**Deploy the cluster and services:** + +```bash +make deploy-azure +``` + +**Alternative manual command:** + +```bash +cd deployment && make up +``` + +## Scaling and Management + +- **Scale nodes:** Update `TF_VAR_usr_node_count` in your env file and run `make up` +- **View logs:** `kubectl logs -n crs ` +- **Monitor resources:** `kubectl top pods -A` + +## Additional Resources + +For detailed deployment instructions and advanced configuration options, see the [deployment README](deployment/README.md). + +## Troubleshooting + +### Azure Authentication Issues + +```bash +az login --tenant +az account set --subscription +``` + +### Cluster Management + +```bash +# Get cluster credentials +az aks get-credentials --name --resource-group + +# View cluster info +az aks show --name --resource-group +``` + +For more troubleshooting information, see the main [Quick Reference Guide](QUICK_REFERENCE.md). \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..4bea92ba --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,165 @@ +# Contributing to Buttercup CRS + +Thank you for your interest in contributing to the Buttercup Cyber Reasoning System! This guide will help you get started with development and understand our workflows. + +## Development Setup + +Before contributing, set up your local development environment: + +```bash +# Clone with submodules +git clone --recurse-submodules https://github.com/trailofbits/buttercup.git + +# Quick setup (recommended) +make setup-local + +# Start development environment +make deploy-local +``` + +## Development Workflow + +### Using Makefile Shortcuts + +The **Buttercup CRS** project includes a Makefile with convenient shortcuts for common tasks: + +```bash +# View all available commands +make help + +# Setup +make setup-local # Automated local development setup +make setup-azure # Automated production AKS setup +make validate # Validate current setup and configuration + +# Deployment +make deploy # Deploy to current environment (local or azure) +make deploy-local # Deploy to local Minikube environment +make deploy-azure # Deploy to production AKS environment + +# Status +make status # Check the status of the deployment + +# Testing +make send-libpng-task # Run libpng test task + +# Development +make lint # Lint all Python code +make lint-component COMPONENT=orchestrator # Lint specific component + +# Cleanup +make undeploy # Remove deployment and clean up resources +make clean-local # Delete Minikube cluster and remove local config +``` + +### Code Quality Standards + +All Python components use consistent formatting and linting standards: + +- **Formatter:** `ruff format` +- **Linter:** `ruff check` +- **Type Checker:** `mypy` (for common, patcher, and program-model components) + +### Running Tests + +```bash +# Lint all Python code +make lint + +# Lint specific component +make lint-component COMPONENT=orchestrator + +# Run test task +make send-libpng-task +``` + + +### Development Tools + + +#### Kubernetes Development + +```bash +# Port forward for local access +kubectl port-forward -n crs service/buttercup-competition-api 31323:1323 + +# View logs +kubectl logs -n crs -l app=scheduler --tail=-1 --prefix + +# Debug pods +kubectl exec -it -n crs -- /bin/bash +``` + +## Component Architecture + +The system consists of several key components: + +- **Common** (`/common/`): Shared utilities, protobuf definitions, Redis queue management, telemetry +- **Orchestrator** (`/orchestrator/`): Central coordination, task server, scheduler, competition API client +- **Fuzzer** (`/fuzzer/`): Automated vulnerability discovery (build-bot, fuzzer-bot, coverage-bot, tracer-bot) +- **Program Model** (`/program-model/`): Semantic code analysis using CodeQuery and Tree-sitter +- **Patcher** (`/patcher/`): LLM-powered automated patch generation +- **Seed Generation** (`/seed-gen/`): Intelligent input generation + +## Contribution Guidelines + +### Code Style + +- Follow existing code patterns and conventions in each component +- Use structured logging via the common logging module +- Implement proper error handling with circuit breakers for external service calls +- Use Pydantic models for data validation +- Write comprehensive tests for new functionality + +### Testing + +Each component should include: + +- Unit tests using pytest +- Mock external dependencies (Redis, LLM APIs, file system) +- Integration tests using Docker containers +- Test data stored in `/tests/data/` + +### Security Considerations + +- All untrusted code execution must happen in isolated Docker containers +- Never expose or log secrets and keys +- Never commit secrets or keys to the repository + +### Submitting Changes + +1. **Create a feature branch** from the main branch +2. **Make your changes** following the code style guidelines +3. **Test your changes** using the appropriate test commands +4. **Lint your code** using `make lint` or component-specific linting +5. **Create a pull request** with a clear description of your changes + + +### Python Package Management + +Each component uses `uv` for dependency management: + +```bash +# Install dependencies +cd && uv sync + +# Install with dev dependencies +cd && uv sync --all-extras + +# Add new dependency +cd && uv add + +# Update dependencies +cd && uv lock --upgrade +``` + +## Getting Help + +- **Validate your setup:** `make validate` - Check if your environment is ready +- Check the [Quick Reference Guide](QUICK_REFERENCE.md) for common commands and troubleshooting +- Check the [deployment README](deployment/README.md) for detailed deployment information +- Check logs: `kubectl logs -n crs ` + +## Questions? + +If you have questions about contributing, please feel free to open an issue or reach out to the development team. \ No newline at end of file diff --git a/MANUAL_SETUP.md b/MANUAL_SETUP.md new file mode 100644 index 00000000..d6d2b1f4 --- /dev/null +++ b/MANUAL_SETUP.md @@ -0,0 +1,118 @@ +# Manual Setup Guide + +This guide provides detailed manual setup instructions for the Buttercup CRS system. If you prefer automated setup, use `make setup-local` instead. + +## Prerequisites + +Before starting manual setup, ensure you have the following dependencies installed: + +### System Packages + +```bash +# Ubuntu/Debian +sudo apt-get update +sudo apt-get install -y make curl git + +# RHEL/CentOS/Fedora +sudo yum install -y make curl git +# or +sudo dnf install -y make curl git + +# MacOS +brew install make curl git +``` + +### Docker + +```bash +curl -fsSL https://get.docker.com | sh +sudo usermod -aG docker $USER +# Log out and back in for group changes to take effect +``` + +### kubectl + +```bash +curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" +sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl +``` + +### Helm + +```bash +curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash +``` + +### Minikube + +```bash +curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 +sudo install minikube-linux-amd64 /usr/local/bin/minikube +``` + +### Git LFS (for some tests) + +```bash +sudo apt-get install git-lfs +git lfs install +``` + +## Manual Configuration + +1. **Create configuration file:** + +```bash +cp deployment/env.template deployment/env +``` + +2. **Configure the environment file** (`deployment/env`): + +Look at the comments in the `deployment/env.template` for how to set variables. + +## Start Services Manually + +```bash +# Start services manually +cd deployment && make up + +# Port forward manually +kubectl port-forward -n crs service/buttercup-ui 31323:1323 + +# Test manually +./orchestrator/scripts/task_crs.sh +``` + +## Verification + +After setup, verify your installation by running: + +```bash +make status +``` + +## Troubleshooting + +### Common Manual Setup Issues + +1. **Docker permission issues:** + +```bash +sudo usermod -aG docker $USER +# Log out and back in +``` + +2. **Minikube won't start:** + +```bash +minikube delete +minikube start --driver=docker +``` + +3. **Helm chart issues:** + +```bash +helm repo update +helm dependency update deployment/k8s/ +``` + +For additional troubleshooting, see the [Quick Reference Guide](QUICK_REFERENCE.md). \ No newline at end of file diff --git a/README.md b/README.md index 0314a42c..f549e305 100644 --- a/README.md +++ b/README.md @@ -1,365 +1,140 @@ # Buttercup Cyber Reasoning System (CRS) -**Buttercup** is a Cyber Reasoning System (CRS) developed by **Trail of Bits** for the **DARPA AIxCC (AI Cyber Challenge) competition**. It's a comprehensive automated vulnerability detection and patching system designed to compete in AI-driven cybersecurity challenges. +**Buttercup** is a Cyber Reasoning System (CRS) developed by **Trail of Bits** for the **DARPA AIxCC (AI Cyber Challenge)**. Buttercup finds and patches software vulnerabilities in open-source code repositories like [example-libpng](https://github.com/tob-challenges/example-libpng). It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. **Buttercup** system consists of several components: -## Quick Start - -Clone the repo with `--recurse-submodules` as some dependencies are submodules. - -Choose your deployment method: - -- **[Local Development](#local-development)** - Quick setup for development and testing -- **[Production AKS Deployment](#production-aks-deployment)** - Full production deployment on Azure Kubernetes Service - -Note: Buttercup uses hosted LLMs, which cost money. Limit your per-deployment spend with the built-in LLM budget. +- **Orchestrator**: Coordinates the overall task process and manages the workflow +- **Seed Generator**: Creates inputs for vulnerability discovery +- **Fuzzer**: Discovers vulnerabilities through intelligent fuzzing techniques +- **Program Model**: Analyzes code structure and semantics for better understanding +- **Patcher**: Generates and applies security patches to fix vulnerabilities -## Local Development +## System Requirements -The fastest way to get started with the **Buttercup CRS** system for development and testing. +### Minimum Requirements -### Quick Setup (Recommended) +- **CPU:** 8 cores +- **Memory:** 64 GB RAM +- **Storage:** 100 GB available disk space +- **Network:** Stable internet connection for downloading dependencies -Use our automated setup script: +**Note:** Buttercup uses third-party AI providers (LLMs from companies like OpenAI, Anthropic and Google), which cost money. Please ensure that you manage per-deployment costs by using the built-in LLM budget setting. -```bash -make setup-local -``` +**Note:** Buttercup works best with access to models from OpenAI **and** Anthropic, but can be run with at least one API key from one third-party provider (support for Gemini coming soon). -This script will install all dependencies, configure the environment, and guide you through the setup process. +### Supported Systems +- **Linux x86_64** (fully supported) +- **ARM64** (partial support for upstream Google OSS-Fuzz projects) -### Manual Setup +### Required System Packages -If you prefer to set up manually, follow these steps: +Before setup, ensure you have these packages installed: ```bash -# Docker -curl -fsSL https://get.docker.com | sh -sudo usermod -aG docker $USER -# Log out and back in for group changes to take effect - -# kubectl -curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" -sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl - -# Helm -curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash +# Ubuntu/Debian +sudo apt-get update +sudo apt-get install -y make curl git -# Minikube -curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 -sudo install minikube-linux-amd64 /usr/local/bin/minikube +# RHEL/CentOS/Fedora +sudo yum install -y make curl git +# or +sudo dnf install -y make curl git -# Git LFS (for some tests) -sudo apt-get install git-lfs -git lfs install +# MacOS +brew install make curl git ``` -#### Manual Configuration +### Supported Targets -1. **Create configuration file:** +Buttercup works with: -```bash -cp deployment/env.template deployment/env -``` - -2. **Configure the environment file** (`deployment/env`): - -Look at the comments in the `deployment/env.template` for how to set variables. - -### Start Local Development Environment - -1. **Start the services:** - -```bash -make deploy-local -``` - -2. **Verify deployment:** +- **C source code repositories** that are OSS-Fuzz compatible +- **Java source code repositories** that are OSS-Fuzz compatible +- Projects that build successfully and have existing fuzzing harnesses -```bash -make status -``` - -When the deployment is succesful, you should see something like - -```shell -$ make status -----------PODS------------ -NAME READY STATUS RESTARTS AGE -buttercup-build-bot-845f5b96d9-7t8bz 1/1 Running 0 5m58s -buttercup-build-bot-845f5b96d9-bfsq9 1/1 Running 0 5m58s -buttercup-build-bot-845f5b96d9-npns4 1/1 Running 0 5m58s -buttercup-build-bot-845f5b96d9-sv5fr 1/1 Running 0 5m58s -buttercup-coverage-bot-6749f57b9d-4gzfd 1/1 Running 0 5m58s -buttercup-dind-452s6 1/1 Running 0 5m58s -buttercup-fuzzer-bot-74bc9b849d-2zkt6 1/1 Running 0 5m58s -buttercup-image-preloader-97nfb 0/1 Completed 0 5m58s -buttercup-litellm-5f87df944-2mq7z 1/1 Running 0 5m58s -buttercup-litellm-migrations-ljjcl 0/1 Completed 0 5m58s -buttercup-merger-bot-fz87v 1/1 Running 0 5m58s -buttercup-patcher-7597c965b8-6968s 1/1 Running 0 5m58s -buttercup-postgresql-0 1/1 Running 0 5m58s -buttercup-pov-reproducer-5f948bd7cc-45rgp 1/1 Running 0 5m58s -buttercup-program-model-67446b5cfc-24zfh 1/1 Running 0 5m58s -buttercup-redis-master-0 1/1 Running 0 5m58s -buttercup-registry-cache-5787f86896-czt9b 1/1 Running 0 5m58s -buttercup-scheduler-7c49bf75c5-swqkb 1/1 Running 0 5m58s -buttercup-scratch-cleaner-hdt6z 1/1 Running 0 5m58s -buttercup-seed-gen-6fdb9c94c9-4xmrp 1/1 Running 0 5m57s -buttercup-task-downloader-54cd9fb577-g4lbg 1/1 Running 0 5m58s -buttercup-task-server-7d8cd7cf49-zkt69 1/1 Running 0 5m58s -buttercup-tracer-bot-5b9fb6c8b5-zcmxd 1/1 Running 0 5m58s -buttercup-ui-5dcf7dfb8-njglh 1/1 Running 0 5m58s -----------SERVICES-------- -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -buttercup-litellm ClusterIP 10.96.88.226 4000/TCP 5m58s -buttercup-postgresql ClusterIP 10.111.161.207 5432/TCP 5m58s -buttercup-postgresql-hl ClusterIP None 5432/TCP 5m58s -buttercup-redis-headless ClusterIP None 6379/TCP 5m58s -buttercup-redis-master ClusterIP 10.108.61.77 6379/TCP 5m58s -buttercup-registry-cache ClusterIP 10.103.80.241 443/TCP 5m58s -buttercup-task-server ClusterIP 10.104.206.197 8000/TCP 5m58s -buttercup-ui ClusterIP 10.106.49.166 1323/TCP 5m58s -All CRS pods up and running. -``` +## Quick Start -3. **Submit the libpng project to the CRS (for 30mins):** +1. Clone the repository with submodules: ```bash -make send-libpng-task +git clone --recurse-submodules https://github.com/trailofbits/buttercup.git +cd buttercup ``` -**Alternative manual commands:** +2. Run automated setup (Recommended) ```bash -# Start services manually -cd deployment && make up - -# Port forward manually -kubectl port-forward -n crs service/buttercup-ui 31323:1323 - -# Test manually -./orchestrator/scripts/task_integration_test.sh +make setup-local ``` -## Production AKS Deployment - -> **⚠️ Notice:** -> The following production deployment instructions have **not been fully tested**. -> Please proceed with caution and verify each step in your environment. -> If you encounter issues, consult the script comments and configuration files for troubleshooting. - -Full production deployment of the **Buttercup CRS** on Azure Kubernetes Service with proper networking, monitoring, and scaling for the DARPA AIxCC competition. +This script will install all dependencies, configure the environment, and guide you through the setup process. -### Quick Setup (Recommended) +**Note:** If you prefer manual setup, see the [Manual Setup Guide](MANUAL_SETUP.md). -Use our automated setup script: +3. Start Buttercup locally ```bash -make setup-azure +make deploy-local ``` -This script will check prerequisites, help create service principals, configure the environment, and validate your setup. - -#### Manual Setup - -If you prefer to set up manually, follow these steps: - -##### Prerequisites - -- Azure CLI installed and configured -- Terraform installed -- Active Azure subscription -- Access to competition Tailscale tailnet - -##### Azure Setup - -1. **Login to Azure:** +4. Verify local deployment: ```bash -az login --tenant +make status ``` -2. **Create Service Principal:** +When a deployment is successful, you should see all pods in "Running" or "Completed" status. -```bash -# Get your subscription ID -az account show --query "{SubscriptionID:id}" --output table -# Create service principal (replace with your subscription ID) -az ad sp create-for-rbac --name "ButtercupCRS" --role Contributor --scopes /subscriptions/ -``` +5. Send Buttercup a simple task -##### Production Configuration +**Note:** When tasked, Buttercup will start consuming third-party AI resources. -1. **Configure environment file:** +This command will make Buttercup pull down an example repo [example-libpng](https://github.com/tob-challenges/example-libpng) with a known vulnerability. Buttercup will start fuzzing it to find and patch vulnerabilities. ```bash -cp deployment/env.template deployment/env +make send-libpng-task ``` -2. **Update `deployment/env` for production:** +6. Access Buttercup's web-based GUI -Look at the comments in the `deployment/env.template` for how to set variables. -In particular, set `TF_VAR_*` variables, and `TAILSCALE_*` if used. - -### Deploy to AKS - -**Deploy the cluster and services:** +Run: ```bash -make deploy-azure +make web-ui ``` -**Alternative manual command:** +Then navigate to `http://localhost:31323` in your web browser. -```bash -cd deployment && make up -``` +In the GUI you can monitor active tasks and see when Buttercup finds bugs and generates patches for them. -### Scaling and Management -- **Scale nodes:** Update `TF_VAR_usr_node_count` in your env file and run `make up` -- **View logs:** `kubectl logs -n crs ` -- **Monitor resources:** `kubectl top pods -A` +7. Stop Buttercup -## Run Challenges - -```bash -kubectl port-forward -n crs service/buttercup-ui 31323:1323 & - -./orchestrator/scripts/challenge.py -``` - -## Cleanup +**Note:** This is an important step to ensure Buttercup shuts down and stops consuming third-party AI resources. ```bash make undeploy ``` -**Alternative manual command:** - -```bash -cd deployment && make down -``` - -## Development Workflow - -### Using Makefile Shortcuts - -The **Buttercup CRS** project includes a Makefile with convenient shortcuts for common tasks: - -```bash -# View all available commands -make help - -# Setup -make setup-local # Automated local development setup -make setup-azure # Automated production AKS setup -make validate # Validate current setup and configuration - -# Deployment -make deploy # Deploy to current environment (local or azure) -make deploy-local # Deploy to local Minikube environment -make deploy-azure # Deploy to production AKS environment - -# Status -make status # Check the status of the deployment - -# Testing -make send-integration-task # Run integration test task -make send-libpng-task # Run libpng test task - -# Development -make lint # Lint all Python code -make lint-component COMPONENT=orchestrator # Lint specific component - -# Cleanup -make undeploy # Remove deployment and clean up resources -make clean-local # Delete Minikube cluster and remove local config -``` - -### Running Tests - -```bash -# Lint all Python code -make lint - -# Lint specific component -make lint-component COMPONENT=orchestrator -``` - -**Alternative manual commands:** - -```bash -# Lint Python code -make lint - -# Run specific component tests -make lint-component COMPONENT=orchestrator - -# Test manually -./orchestrator/scripts/task_upstream_libpng.sh -./orchestrator/scripts/challenge.py -``` - - -### Kubernetes Development - -```bash -# Port forward for local access -kubectl port-forward -n crs service/buttercup-competition-api 31323:1323 - -# View logs -kubectl logs -n crs -l app=scheduler --tail=-1 --prefix - -# Debug pods -kubectl exec -it -n crs -- /bin/bash -``` - -## Troubleshooting - -### Common Issues +## Accessing Logs -1. **Minikube won't start:** +To view system logs and telemetry you can use Langfuse or SigNoz if you configured them during setup. Otherwise you can access logs via `kubectl` commands: ```bash -minikube delete -``` +# View all pods +kubectl get pods -n crs -2. **Docker permission issues:** +# View specific pod logs +kubectl logs -n crs -```bash -sudo usermod -aG docker $USER -# Log out and back in +# Follow logs in real-time +kubectl logs -n crs -f ``` -3. **Helm chart issues:** +## Additional Resources -```bash -helm repo update -helm dependency update deployment/k8s/ -``` - -4. **Azure authentication:** - -```bash -az login --tenant -az account set --subscription -``` - -### Getting Help - -- **Validate your setup:** `make validate` - Check if your environment is ready -- Check the [Quick Reference Guide](QUICK_REFERENCE.md) for common commands and troubleshooting -- Check the [deployment README](deployment/README.md) for detailed deployment information -- Check logs: `kubectl logs -n crs ` - -## Architecture - -The **Buttercup CRS** system consists of several components designed to work together for automated vulnerability detection and patching: - -- **Orchestrator**: Coordinates the overall repair process and manages the workflow -- **Fuzzer**: Discovers vulnerabilities through intelligent fuzzing techniques -- **Patcher**: Generates and applies security patches to fix vulnerabilities -- **Program Model**: Analyzes code structure and semantics for better understanding -- **Seed Generator**: Creates targeted test cases for vulnerability discovery +- [Quick Reference Guide](QUICK_REFERENCE.md) - Common commands and troubleshooting +- [Manual Setup Guide](MANUAL_SETUP.md) - Detailed manual installation steps +- [AKS Deployment Guide](AKS_DEPLOYMENT.md) - Production deployment on Azure +- [Contributing Guidelines](CONTRIBUTING.md) - Development workflow and standards +- [Deployment Documentation](deployment/README.md) - Advanced deployment configuration \ No newline at end of file