From f1f2656937f51e28da6aad96492abacab6a33a0a Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Wed, 30 Jul 2025 14:50:24 +0000 Subject: [PATCH 01/19] Simplify README and restructure documentation - Move AKS deployment docs to separate AKS_DEPLOYMENT.md file - Move manual setup instructions to MANUAL_SETUP.md file - Create CONTRIBUTING.md with development workflow and contribution guidelines - Restructure README to focus on individual users with: - Minimum system requirements (8 CPUs, 64GB RAM, 100GB storage) - Supported systems/architectures (Linux x86_64, ARM64 in dev) - Supported targets (C/Java OSS-Fuzz compatible repositories) - Instructions for creating and running challenges - GUI component usage instructions - Required system packages info - Remove alternative manual commands (now automated in Makefile) - Add links to all new documentation files Fixes #174 Co-authored-by: Michael D Brown --- AKS_DEPLOYMENT.md | 105 +++++++++++++ CONTRIBUTING.md | 205 ++++++++++++++++++++++++++ MANUAL_SETUP.md | 112 ++++++++++++++ README.md | 366 ++++++++++++---------------------------------- 4 files changed, 519 insertions(+), 269 deletions(-) create mode 100644 AKS_DEPLOYMENT.md create mode 100644 CONTRIBUTING.md create mode 100644 MANUAL_SETUP.md diff --git a/AKS_DEPLOYMENT.md b/AKS_DEPLOYMENT.md new file mode 100644 index 00000000..8d111f96 --- /dev/null +++ b/AKS_DEPLOYMENT.md @@ -0,0 +1,105 @@ +# Production AKS Deployment Guide + +> **⚠️ Notice:** +> The following production deployment instructions have **not been fully tested**. +> Please proceed with caution and verify each step in your environment. +> If you encounter issues, consult the script comments and configuration files for troubleshooting. + +Full production deployment of the **Buttercup CRS** on Azure Kubernetes Service with proper networking, monitoring, and scaling for the DARPA AIxCC competition. + +## Quick Setup (Recommended) + +Use our automated setup script: + +```bash +make setup-azure +``` + +This script will check prerequisites, help create service principals, configure the environment, and validate your setup. + +## Manual Setup + +If you prefer to set up manually, follow these steps: + +### Prerequisites + +- Azure CLI installed and configured +- Terraform installed +- Active Azure subscription +- Access to competition Tailscale tailnet + +### Azure Setup + +1. **Login to Azure:** + +```bash +az login --tenant +``` + +2. **Create Service Principal:** + +```bash +# Get your subscription ID +az account show --query "{SubscriptionID:id}" --output table + +# Create service principal (replace with your subscription ID) +az ad sp create-for-rbac --name "ButtercupCRS" --role Contributor --scopes /subscriptions/ +``` + +### Production Configuration + +1. **Configure environment file:** + +```bash +cp deployment/env.template deployment/env +``` + +2. **Update `deployment/env` for production:** + +Look at the comments in the `deployment/env.template` for how to set variables. +In particular, set `TF_VAR_*` variables, and `TAILSCALE_*` if used. + +## Deploy to AKS + +**Deploy the cluster and services:** + +```bash +make deploy-azure +``` + +**Alternative manual command:** + +```bash +cd deployment && make up +``` + +## Scaling and Management + +- **Scale nodes:** Update `TF_VAR_usr_node_count` in your env file and run `make up` +- **View logs:** `kubectl logs -n crs ` +- **Monitor resources:** `kubectl top pods -A` + +## Additional Resources + +For detailed deployment instructions and advanced configuration options, see the [deployment README](deployment/README.md). + +## Troubleshooting + +### Azure Authentication Issues + +```bash +az login --tenant +az account set --subscription +``` + +### Cluster Management + +```bash +# Get cluster credentials +az aks get-credentials --name --resource-group + +# View cluster info +az aks show --name --resource-group +``` + +For more troubleshooting information, see the main [Quick Reference Guide](QUICK_REFERENCE.md). \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..a7f7a10d --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,205 @@ +# Contributing to Buttercup CRS + +Thank you for your interest in contributing to the Buttercup Cyber Reasoning System! This guide will help you get started with development and understand our workflows. + +## Development Setup + +Before contributing, set up your local development environment: + +```bash +# Clone with submodules +git clone --recurse-submodules + +# Quick setup (recommended) +make setup-local + +# Start development environment +make deploy-local +``` + +## Development Workflow + +### Using Makefile Shortcuts + +The **Buttercup CRS** project includes a Makefile with convenient shortcuts for common tasks: + +```bash +# View all available commands +make help + +# Setup +make setup-local # Automated local development setup +make setup-azure # Automated production AKS setup +make validate # Validate current setup and configuration + +# Deployment +make deploy # Deploy to current environment (local or azure) +make deploy-local # Deploy to local Minikube environment +make deploy-azure # Deploy to production AKS environment + +# Status +make status # Check the status of the deployment + +# Testing +make send-integration-task # Run integration test task + +# Development +make lint # Lint all Python code +make lint-component COMPONENT=orchestrator # Lint specific component + +# Cleanup +make undeploy # Remove deployment and clean up resources +make clean-local # Delete Minikube cluster and remove local config +``` + +### Code Quality Standards + +All Python components use consistent formatting and linting standards: + +- **Formatter:** `ruff format` +- **Linter:** `ruff check` +- **Type Checker:** `mypy` (for common, patcher, and program-model components) +- **Line Length:** 120 characters + +### Running Tests + +```bash +# Lint all Python code +make lint + +# Lint specific component +make lint-component COMPONENT=orchestrator + +# Run test task +make send-integration-task +``` + +**Alternative manual commands:** + +```bash +# Lint Python code +just lint-python-all + +# Run specific component tests +just lint-python orchestrator + +# Test manually +./orchestrator/scripts/task_upstream_libpng.sh +./orchestrator/scripts/challenge.sh +``` + +### Development Tools + +#### Docker Development + +```bash +# Build and run with Docker Compose (only for local development and quick testing) +docker-compose up -d +``` + +#### Kubernetes Development + +```bash +# Port forward for local access +kubectl port-forward -n crs service/buttercup-competition-api 31323:1323 + +# View logs +kubectl logs -n crs -l app=scheduler --tail=-1 --prefix + +# Debug pods +kubectl exec -it -n crs -- /bin/bash +``` + +## Component Architecture + +The system consists of several key components: + +- **Common** (`/common/`): Shared utilities, protobuf definitions, Redis queue management, telemetry +- **Orchestrator** (`/orchestrator/`): Central coordination, task server, scheduler, competition API client +- **Fuzzer** (`/fuzzer/`): Automated vulnerability discovery (build-bot, fuzzer-bot, coverage-bot, tracer-bot) +- **Program Model** (`/program-model/`): Semantic code analysis using CodeQuery and Tree-sitter +- **Patcher** (`/patcher/`): LLM-powered automated patch generation +- **Seed Generation** (`/seed-gen/`): Intelligent test case generation + +## Contribution Guidelines + +### Code Style + +- Follow existing code patterns and conventions in each component +- Use structured logging via the common logging module +- Implement proper error handling with circuit breakers for external service calls +- Use Pydantic models for data validation +- Write comprehensive tests for new functionality + +### Testing + +Each component should include: + +- Unit tests using pytest +- Mock external dependencies (Redis, LLM APIs, file system) +- Integration tests using Docker containers +- Test data stored in `/tests/data/` + +### Security Considerations + +- All untrusted code execution must happen in isolated Docker containers +- Never expose or log secrets and keys +- Never commit secrets or keys to the repository +- Use DinD (Docker-in-Docker) for additional isolation when needed + +### Submitting Changes + +1. **Create a feature branch** from the main branch +2. **Make your changes** following the code style guidelines +3. **Test your changes** using the appropriate test commands +4. **Lint your code** using `make lint` or component-specific linting +5. **Create a pull request** with a clear description of your changes + +### Running Component Tests + +For individual component testing: + +```bash +# Test specific components +cd && uv run pytest + +# Test with coverage +cd && uv run pytest --cov + +# Common component test commands: +cd common && uv run pytest +cd orchestrator && uv run pytest +cd fuzzer && uv run pytest +cd patcher && uv run pytest +cd program-model && uv run pytest +cd seed-gen && uv run pytest +``` + +### Python Package Management + +Each component uses `uv` for dependency management: + +```bash +# Install dependencies +cd && uv sync + +# Install with dev dependencies +cd && uv sync --all-extras + +# Add new dependency +cd && uv add + +# Update dependencies +cd && uv lock --upgrade +``` + +## Getting Help + +- **Validate your setup:** `make validate` - Check if your environment is ready +- Check the [Quick Reference Guide](QUICK_REFERENCE.md) for common commands and troubleshooting +- Check the [deployment README](deployment/README.md) for detailed deployment information +- Check logs: `kubectl logs -n crs ` + +## Questions? + +If you have questions about contributing, please feel free to open an issue or reach out to the development team. \ No newline at end of file diff --git a/MANUAL_SETUP.md b/MANUAL_SETUP.md new file mode 100644 index 00000000..7a93f39f --- /dev/null +++ b/MANUAL_SETUP.md @@ -0,0 +1,112 @@ +# Manual Setup Guide + +This guide provides detailed manual setup instructions for the Buttercup CRS system. If you prefer automated setup, use `make setup-local` instead. + +## Prerequisites + +Before starting manual setup, ensure you have the following system packages installed: + +- `make` - Build automation tool +- `curl` - Command-line tool for transferring data +- `git` - Version control system +- Docker (for containerization) +- kubectl (for Kubernetes management) +- Helm (for Kubernetes package management) +- Minikube (for local Kubernetes cluster) + +## Manual Installation Steps + +### Docker + +```bash +curl -fsSL https://get.docker.com | sh +sudo usermod -aG docker $USER +# Log out and back in for group changes to take effect +``` + +### kubectl + +```bash +curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" +sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl +``` + +### Helm + +```bash +curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash +``` + +### Minikube + +```bash +curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 +sudo install minikube-linux-amd64 /usr/local/bin/minikube +``` + +### Git LFS (for some tests) + +```bash +sudo apt-get install git-lfs +git lfs install +``` + +## Manual Configuration + +1. **Create configuration file:** + +```bash +cp deployment/env.template deployment/env +``` + +2. **Configure the environment file** (`deployment/env`): + +Look at the comments in the `deployment/env.template` for how to set variables. + +## Start Services Manually + +```bash +# Start services manually +cd deployment && make up + +# Port forward manually +kubectl port-forward -n crs service/buttercup-ui 31323:1323 + +# Test manually +./orchestrator/scripts/task_integration_test.sh +``` + +## Verification + +After setup, verify your installation by running: + +```bash +make status +``` + +## Troubleshooting + +### Common Manual Setup Issues + +1. **Docker permission issues:** + +```bash +sudo usermod -aG docker $USER +# Log out and back in +``` + +2. **Minikube won't start:** + +```bash +minikube delete +minikube start --driver=docker +``` + +3. **Helm chart issues:** + +```bash +helm repo update +helm dependency update deployment/k8s/ +``` + +For additional troubleshooting, see the [Quick Reference Guide](QUICK_REFERENCE.md). \ No newline at end of file diff --git a/README.md b/README.md index 4850d55a..c4ba2283 100644 --- a/README.md +++ b/README.md @@ -2,70 +2,70 @@ **Buttercup** is a Cyber Reasoning System (CRS) developed by **Trail of Bits** for the **DARPA AIxCC (AI Cyber Challenge) competition**. It's a comprehensive automated vulnerability detection and patching system designed to compete in AI-driven cybersecurity challenges. -## Quick Start - -Clone the repo with `--recurse-submodules` as some dependencies are submodules. +## System Requirements -Choose your deployment method: +### Minimum Requirements -- **[Local Development](#local-development)** - Quick setup for development and testing -- **[Production AKS Deployment](#production-aks-deployment)** - Full production deployment on Azure Kubernetes Service +- **CPU:** 8 cores +- **Memory:** 64 GB RAM +- **Storage:** 100 GB available disk space +- **Network:** Stable internet connection for downloading dependencies -## Local Development +### Supported Systems -The fastest way to get started with the **Buttercup CRS** system for development and testing. +- **Linux x86_64** (fully supported) +- **ARM64** (in development) -### Quick Setup (Recommended) +### Required System Packages -Use our automated setup script: +Before setup, ensure you have these packages installed: ```bash -make setup-local -``` - -This script will install all dependencies, configure the environment, and guide you through the setup process. +# Ubuntu/Debian +sudo apt-get update +sudo apt-get install -y make curl git -### Manual Setup +# RHEL/CentOS/Fedora +sudo yum install -y make curl git +# or +sudo dnf install -y make curl git +``` -If you prefer to set up manually, follow these steps: +## Supported Targets -```bash -# Docker -curl -fsSL https://get.docker.com | sh -sudo usermod -aG docker $USER -# Log out and back in for group changes to take effect +Buttercup CRS works with: -# kubectl -curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" -sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl +- **C source code repositories** that are OSS-Fuzz compatible +- **Java source code repositories** that are OSS-Fuzz compatible +- Projects that build successfully and have existing fuzzing harnesses +- Open source projects with standard build systems (Make, CMake, Maven, Gradle) -# Helm -curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash +## Quick Start -# Minikube -curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 -sudo install minikube-linux-amd64 /usr/local/bin/minikube +Clone the repository with submodules: -# Git LFS (for some tests) -sudo apt-get install git-lfs -git lfs install +```bash +git clone --recurse-submodules +cd buttercup ``` -#### Manual Configuration +Choose your setup method: -1. **Create configuration file:** +### Automated Setup (Recommended) ```bash -cp deployment/env.template deployment/env +make setup-local ``` -2. **Configure the environment file** (`deployment/env`): +This script will install all dependencies, configure the environment, and guide you through the setup process. + +### Manual Setup -Look at the comments in the `deployment/env.template` for how to set variables. +If you prefer manual setup, see the [Manual Setup Guide](MANUAL_SETUP.md). -### Start Local Development Environment +## Starting the System -1. **Start the services:** +1. **Deploy locally:** ```bash make deploy-local @@ -77,288 +77,108 @@ make deploy-local make status ``` -When the deployment is succesful, you should see something like - -```shell -$ make status -----------PODS------------ -NAME READY STATUS RESTARTS AGE -buttercup-build-bot-845f5b96d9-7t8bz 1/1 Running 0 5m58s -buttercup-build-bot-845f5b96d9-bfsq9 1/1 Running 0 5m58s -buttercup-build-bot-845f5b96d9-npns4 1/1 Running 0 5m58s -buttercup-build-bot-845f5b96d9-sv5fr 1/1 Running 0 5m58s -buttercup-coverage-bot-6749f57b9d-4gzfd 1/1 Running 0 5m58s -buttercup-dind-452s6 1/1 Running 0 5m58s -buttercup-fuzzer-bot-74bc9b849d-2zkt6 1/1 Running 0 5m58s -buttercup-image-preloader-97nfb 0/1 Completed 0 5m58s -buttercup-litellm-5f87df944-2mq7z 1/1 Running 0 5m58s -buttercup-litellm-migrations-ljjcl 0/1 Completed 0 5m58s -buttercup-merger-bot-fz87v 1/1 Running 0 5m58s -buttercup-patcher-7597c965b8-6968s 1/1 Running 0 5m58s -buttercup-postgresql-0 1/1 Running 0 5m58s -buttercup-pov-reproducer-5f948bd7cc-45rgp 1/1 Running 0 5m58s -buttercup-program-model-67446b5cfc-24zfh 1/1 Running 0 5m58s -buttercup-redis-master-0 1/1 Running 0 5m58s -buttercup-registry-cache-5787f86896-czt9b 1/1 Running 0 5m58s -buttercup-scheduler-7c49bf75c5-swqkb 1/1 Running 0 5m58s -buttercup-scratch-cleaner-hdt6z 1/1 Running 0 5m58s -buttercup-seed-gen-6fdb9c94c9-4xmrp 1/1 Running 0 5m57s -buttercup-task-downloader-54cd9fb577-g4lbg 1/1 Running 0 5m58s -buttercup-task-server-7d8cd7cf49-zkt69 1/1 Running 0 5m58s -buttercup-tracer-bot-5b9fb6c8b5-zcmxd 1/1 Running 0 5m58s -buttercup-ui-5dcf7dfb8-njglh 1/1 Running 0 5m58s -----------SERVICES-------- -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -buttercup-litellm ClusterIP 10.96.88.226 4000/TCP 5m58s -buttercup-postgresql ClusterIP 10.111.161.207 5432/TCP 5m58s -buttercup-postgresql-hl ClusterIP None 5432/TCP 5m58s -buttercup-redis-headless ClusterIP None 6379/TCP 5m58s -buttercup-redis-master ClusterIP 10.108.61.77 6379/TCP 5m58s -buttercup-registry-cache ClusterIP 10.103.80.241 443/TCP 5m58s -buttercup-task-server ClusterIP 10.104.206.197 8000/TCP 5m58s -buttercup-ui ClusterIP 10.106.49.166 1323/TCP 5m58s -All CRS pods up and running. -``` +When deployment is successful, you should see all pods in "Running" status. -3. **Submit the integration-test challenge to the CRS (for 30mins):** +3. **Test the system:** ```bash make send-integration-task ``` -**Alternative manual commands:** - -```bash -# Start services manually -cd deployment && make up - -# Port forward manually -kubectl port-forward -n crs service/buttercup-ui 31323:1323 - -# Test manually -./orchestrator/scripts/task_integration_test.sh -``` - -## Production AKS Deployment - -> **⚠️ Notice:** -> The following production deployment instructions have **not been fully tested**. -> Please proceed with caution and verify each step in your environment. -> If you encounter issues, consult the script comments and configuration files for troubleshooting. +## Using the GUI -Full production deployment of the **Buttercup CRS** on Azure Kubernetes Service with proper networking, monitoring, and scaling for the DARPA AIxCC competition. +The Buttercup CRS includes a web-based user interface for monitoring and managing the system. -### Quick Setup (Recommended) +### Accessing the GUI -Use our automated setup script: +1. **Start port forwarding:** ```bash -make setup-azure -``` - -This script will check prerequisites, help create service principals, configure the environment, and validate your setup. - -#### Manual Setup - -If you prefer to set up manually, follow these steps: - -##### Prerequisites - -- Azure CLI installed and configured -- Terraform installed -- Active Azure subscription -- Access to competition Tailscale tailnet - -##### Azure Setup - -1. **Login to Azure:** - -```bash -az login --tenant -``` - -2. **Create Service Principal:** - -```bash -# Get your subscription ID -az account show --query "{SubscriptionID:id}" --output table - -# Create service principal (replace with your subscription ID) -az ad sp create-for-rbac --name "ButtercupCRS" --role Contributor --scopes /subscriptions/ +kubectl port-forward -n crs service/buttercup-ui 31323:1323 & ``` -##### Production Configuration +2. **Open in browser:** -1. **Configure environment file:** +Navigate to `http://localhost:31323` in your web browser. -```bash -cp deployment/env.template deployment/env -``` +### GUI Features -2. **Update `deployment/env` for production:** +- **System Status:** View the status of all system components +- **Task Management:** Monitor active vulnerability discovery and patching tasks +- **Results Dashboard:** View discovered vulnerabilities and generated patches +- **Log Viewer:** Access system logs and debugging information -Look at the comments in the `deployment/env.template` for how to set variables. -In particular, set `TF_VAR_*` variables, and `TAILSCALE_*` if used. +## Creating and Running Challenges -### Deploy to AKS +### Running Challenges -**Deploy the cluster and services:** - -```bash -make deploy-azure -``` - -**Alternative manual command:** - -```bash -cd deployment && make up -``` - -### Scaling and Management - -- **Scale nodes:** Update `TF_VAR_usr_node_count` in your env file and run `make up` -- **View logs:** `kubectl logs -n crs ` -- **Monitor resources:** `kubectl top pods -A` - -## Run Challenges +To run challenges against the CRS: ```bash +# Start the UI port forwarding (if not already running) kubectl port-forward -n crs service/buttercup-ui 31323:1323 & +# Run the challenge script ./orchestrator/scripts/challenge.sh ``` -## Cleanup - -```bash -make undeploy -``` - -**Alternative manual command:** - -```bash -cd deployment && make down -``` - -## Development Workflow - -### Using Makefile Shortcuts - -The **Buttercup CRS** project includes a Makefile with convenient shortcuts for common tasks: - -```bash -# View all available commands -make help - -# Setup -make setup-local # Automated local development setup -make setup-azure # Automated production AKS setup -make validate # Validate current setup and configuration - -# Deployment -make deploy # Deploy to current environment (local or azure) -make deploy-local # Deploy to local Minikube environment -make deploy-azure # Deploy to production AKS environment - -# Status -make status # Check the status of the deployment - -# Testing -make send-integration-task # Run integration test task - -# Development -make lint # Lint all Python code -make lint-component COMPONENT=orchestrator # Lint specific component - -# Cleanup -make undeploy # Remove deployment and clean up resources -make clean-local # Delete Minikube cluster and remove local config -``` - -### Running Tests - -```bash -# Lint all Python code -make lint - -# Lint specific component -make lint-component COMPONENT=orchestrator +### Creating Custom Challenges -# Run test task -make send-integration-task -``` +1. **Prepare your target repository:** + - Ensure it's OSS-Fuzz compatible + - Has working build system + - Includes fuzzing harnesses -**Alternative manual commands:** +2. **Create task configuration:** + - Place repository in the appropriate format + - Configure build and fuzzing parameters -```bash -# Lint Python code -just lint-python-all +3. **Submit to CRS:** + - Use the task submission scripts in `orchestrator/scripts/` + - Monitor progress through the GUI -# Run specific component tests -just lint-python orchestrator +For detailed challenge creation instructions, see the [orchestrator documentation](orchestrator/README.md). -# Test manually -./orchestrator/scripts/task_upstream_libpng.sh -./orchestrator/scripts/challenge.sh -``` +## Common Operations -### Docker Development +### Viewing System Status ```bash -# Build and run with Docker Compose (only for local development and quick testing) -docker-compose up -d +make status ``` -### Kubernetes Development +### Accessing Logs ```bash -# Port forward for local access -kubectl port-forward -n crs service/buttercup-competition-api 31323:1323 +# View all pods +kubectl get pods -n crs -# View logs -kubectl logs -n crs -l app=scheduler --tail=-1 --prefix +# View specific pod logs +kubectl logs -n crs -# Debug pods -kubectl exec -it -n crs -- /bin/bash +# Follow logs in real-time +kubectl logs -n crs -f ``` -## Troubleshooting - -### Common Issues - -1. **Minikube won't start:** +### Stopping the System ```bash -minikube delete +make undeploy ``` -2. **Docker permission issues:** +## Advanced Usage -```bash -sudo usermod -aG docker $USER -# Log out and back in -``` +### Production Deployment -3. **Helm chart issues:** +For production deployment on Azure Kubernetes Service, see the [AKS Deployment Guide](AKS_DEPLOYMENT.md). -```bash -helm repo update -helm dependency update deployment/k8s/ -``` +### Development -4. **Azure authentication:** +For development workflows and contributing guidelines, see [CONTRIBUTING.md](CONTRIBUTING.md). -```bash -az login --tenant -az account set --subscription -``` - -### Getting Help +### Troubleshooting -- **Validate your setup:** `make validate` - Check if your environment is ready -- Check the [Quick Reference Guide](QUICK_REFERENCE.md) for common commands and troubleshooting -- Check the [deployment README](deployment/README.md) for detailed deployment information -- Check logs: `kubectl logs -n crs ` +For troubleshooting help and common commands, see the [Quick Reference Guide](QUICK_REFERENCE.md). ## Architecture @@ -369,3 +189,11 @@ The **Buttercup CRS** system consists of several components designed to work tog - **Patcher**: Generates and applies security patches to fix vulnerabilities - **Program Model**: Analyzes code structure and semantics for better understanding - **Seed Generator**: Creates targeted test cases for vulnerability discovery + +## Additional Resources + +- [Quick Reference Guide](QUICK_REFERENCE.md) - Common commands and troubleshooting +- [Manual Setup Guide](MANUAL_SETUP.md) - Detailed manual installation steps +- [AKS Deployment Guide](AKS_DEPLOYMENT.md) - Production deployment on Azure +- [Contributing Guidelines](CONTRIBUTING.md) - Development workflow and standards +- [Deployment Documentation](deployment/README.md) - Advanced deployment configuration \ No newline at end of file From ad9143f1ae7ed2c86c6ef1bd274135d6095610c1 Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Fri, 1 Aug 2025 10:06:31 +0000 Subject: [PATCH 02/19] Address review feedback: update system requirements, fix ARM support description, and clean up documentation - Reduce RAM requirement from 64GB to 16GB (with 10GB basic option) - Reduce storage requirement from 100GB to 50GB - Clarify ARM64 support is only for upstream Google OSS-Fuzz projects - Remove incorrect build system requirement statement - Update command examples to use make send-libpng-task - Remove unimplemented GUI features (Results Dashboard, Log Viewer) - Simplify challenge creation section to focus on pre-defined challenges - Clean up CONTRIBUTING.md by removing alternative manual commands - Add SigNoz reference for log monitoring - Add note about docker-compose directory requirement Co-authored-by: Riccardo Schirone --- CONTRIBUTING.md | 44 ++++---------------------------------------- README.md | 32 +++++++++++--------------------- 2 files changed, 15 insertions(+), 61 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a7f7a10d..dccb0f5c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -71,36 +71,17 @@ make lint make lint-component COMPONENT=orchestrator # Run test task -make send-integration-task +make send-libpng-task ``` -**Alternative manual commands:** - -```bash -# Lint Python code -just lint-python-all - -# Run specific component tests -just lint-python orchestrator - -# Test manually -./orchestrator/scripts/task_upstream_libpng.sh -./orchestrator/scripts/challenge.sh -``` ### Development Tools -#### Docker Development - -```bash -# Build and run with Docker Compose (only for local development and quick testing) -docker-compose up -d -``` #### Kubernetes Development ```bash -# Port forward for local access +# Port forward for local access kubectl port-forward -n crs service/buttercup-competition-api 31323:1323 # View logs @@ -110,6 +91,8 @@ kubectl logs -n crs -l app=scheduler --tail=-1 --prefix kubectl exec -it -n crs -- /bin/bash ``` +Note: For Docker Compose development, you should first `cd deployment/docker-compose` before running docker-compose commands. + ## Component Architecture The system consists of several key components: @@ -155,25 +138,6 @@ Each component should include: 4. **Lint your code** using `make lint` or component-specific linting 5. **Create a pull request** with a clear description of your changes -### Running Component Tests - -For individual component testing: - -```bash -# Test specific components -cd && uv run pytest - -# Test with coverage -cd && uv run pytest --cov - -# Common component test commands: -cd common && uv run pytest -cd orchestrator && uv run pytest -cd fuzzer && uv run pytest -cd patcher && uv run pytest -cd program-model && uv run pytest -cd seed-gen && uv run pytest -``` ### Python Package Management diff --git a/README.md b/README.md index c4ba2283..20c7e7c0 100644 --- a/README.md +++ b/README.md @@ -7,14 +7,14 @@ ### Minimum Requirements - **CPU:** 8 cores -- **Memory:** 64 GB RAM -- **Storage:** 100 GB available disk space +- **Memory:** 16 GB RAM (10 GB for basic system) +- **Storage:** 50 GB available disk space - **Network:** Stable internet connection for downloading dependencies ### Supported Systems - **Linux x86_64** (fully supported) -- **ARM64** (in development) +- **ARM64** (only for upstream Google OSS-Fuzz projects, not for AIxCC challenges) ### Required System Packages @@ -38,7 +38,6 @@ Buttercup CRS works with: - **C source code repositories** that are OSS-Fuzz compatible - **Java source code repositories** that are OSS-Fuzz compatible - Projects that build successfully and have existing fuzzing harnesses -- Open source projects with standard build systems (Make, CMake, Maven, Gradle) ## Quick Start @@ -82,7 +81,7 @@ When deployment is successful, you should see all pods in "Running" status. 3. **Test the system:** ```bash -make send-integration-task +make send-libpng-task ``` ## Using the GUI @@ -105,8 +104,6 @@ Navigate to `http://localhost:31323` in your web browser. - **System Status:** View the status of all system components - **Task Management:** Monitor active vulnerability discovery and patching tasks -- **Results Dashboard:** View discovered vulnerabilities and generated patches -- **Log Viewer:** Access system logs and debugging information ## Creating and Running Challenges @@ -122,22 +119,13 @@ kubectl port-forward -n crs service/buttercup-ui 31323:1323 & ./orchestrator/scripts/challenge.sh ``` -### Creating Custom Challenges +### Pre-defined Challenges -1. **Prepare your target repository:** - - Ensure it's OSS-Fuzz compatible - - Has working build system - - Includes fuzzing harnesses +Use the available pre-defined challenges: -2. **Create task configuration:** - - Place repository in the appropriate format - - Configure build and fuzzing parameters - -3. **Submit to CRS:** - - Use the task submission scripts in `orchestrator/scripts/` - - Monitor progress through the GUI - -For detailed challenge creation instructions, see the [orchestrator documentation](orchestrator/README.md). +```bash +make send-libpng-task +``` ## Common Operations @@ -149,6 +137,8 @@ make status ### Accessing Logs +For system logs and monitoring, use SigNoz if configured: + ```bash # View all pods kubectl get pods -n crs From a0c229fafd14b682aec4495b55a508fe4dbb87a6 Mon Sep 17 00:00:00 2001 From: Riccardo Schirone <562321+ret2libc@users.noreply.github.com> Date: Fri, 1 Aug 2025 12:20:23 +0200 Subject: [PATCH 03/19] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 20c7e7c0..6aa84361 100644 --- a/README.md +++ b/README.md @@ -137,7 +137,7 @@ make status ### Accessing Logs -For system logs and monitoring, use SigNoz if configured: +For system logs and monitoring, use SigNoz if configured, otherwise you can use kubectl: ```bash # View all pods From f16ec812a8b18ed088459cdb2da3f80f94f9d76f Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Mon, 4 Aug 2025 15:37:25 +0000 Subject: [PATCH 04/19] Fix documentation issues after merge conflict resolution - Fix test command references from send-integration-task to send-libpng-task - Remove alternative manual commands sections as requested in review - Clean up merge conflict artifacts - Update challenge description to be more accurate Co-authored-by: Michael D Brown --- CONTRIBUTING.md | 2 +- README.md | 43 +++---------------------------------------- 2 files changed, 4 insertions(+), 41 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index dccb0f5c..e0fc28cb 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -41,7 +41,7 @@ make deploy-azure # Deploy to production AKS environment make status # Check the status of the deployment # Testing -make send-integration-task # Run integration test task +make send-libpng-task # Run libpng test task # Development make lint # Lint all Python code diff --git a/README.md b/README.md index 527bac0b..37966ff1 100644 --- a/README.md +++ b/README.md @@ -122,25 +122,12 @@ buttercup-ui ClusterIP 10.106.49.166 1323/TCP All CRS pods up and running. ``` -3. **Submit the libpng project to the CRS (for 30mins):** +3. **Send a simple challenge to the system:** ```bash make send-libpng-task ``` -**Alternative manual commands:** - -```bash -# Start services manually -cd deployment && make up - -# Port forward manually -kubectl port-forward -n crs service/buttercup-ui 31323:1323 - -# Test manually -./orchestrator/scripts/task_integration_test.sh -``` - ## Production AKS Deployment > **⚠️ Notice:** @@ -229,7 +216,6 @@ make deploy-azure # Deploy to production AKS environment make status # Check the status of the deployment # Testing -make send-integration-task # Run integration test task make send-libpng-task # Run libpng test task # Development @@ -241,32 +227,9 @@ make undeploy # Remove deployment and clean up resources make clean-local # Delete Minikube cluster and remove local config ``` -### Running Tests - -```bash -# Lint all Python code -make lint - -# Lint specific component -make lint-component COMPONENT=orchestrator -``` - -**Alternative manual commands:** - -```bash -# Lint Python code -make lint - -# Run specific component tests -make lint-component COMPONENT=orchestrator - -# Test manually -./orchestrator/scripts/task_upstream_libpng.sh -./orchestrator/scripts/challenge.py -``` - +### Accessing Logs -### Kubernetes Development +For system logs and monitoring, use SigNoz if configured, otherwise you can use kubectl: ```bash # View all pods From 0c112694568b3d65d239ff8176a1e4e789d4b134 Mon Sep 17 00:00:00 2001 From: "Michael D. Brown" Date: Mon, 4 Aug 2025 20:00:45 +0000 Subject: [PATCH 05/19] WIP manual edits --- README.md | 76 ++++++++++++++----------------------------------------- 1 file changed, 19 insertions(+), 57 deletions(-) diff --git a/README.md b/README.md index 37966ff1..c731c1e3 100644 --- a/README.md +++ b/README.md @@ -2,22 +2,24 @@ **Buttercup** is a Cyber Reasoning System (CRS) developed by **Trail of Bits** for the **DARPA AIxCC (AI Cyber Challenge) competition**. It's a comprehensive automated vulnerability detection and patching system designed to compete in AI-driven cybersecurity challenges. +Buttercup finds and patches software vulnerabilities in open-source code repositories like [example-libpng](https://github.com/tob-challenges/example-libpng). It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. + ## System Requirements ### Minimum Requirements - **CPU:** 8 cores -- **Memory:** 16 GB RAM (10 GB for basic system) +- **Memory:** 16 GB RAM - **Storage:** 50 GB available disk space - **Network:** Stable internet connection for downloading dependencies -Note: Buttercup uses hosted LLMs, which cost money. Limit your per-deployment spend with the built-in LLM budget. +**Note:** Buttercup uses third-party AI providers (LLMs from companies like OpenAI, Anthropic and Google), which cost money. Please ensure that you manage per-deployment costs by using the built-in LLM budget setting. ## Local Development ### Supported Systems - **Linux x86_64** (fully supported) -- **ARM64** (only for upstream Google OSS-Fuzz projects, not for AIxCC challenges) +- **ARM64** (partial support for upstream Google OSS-Fuzz projects) ### Required System Packages @@ -32,6 +34,9 @@ sudo apt-get install -y make curl git sudo yum install -y make curl git # or sudo dnf install -y make curl git + +# MacOS +brew install make curl git ``` ## Supported Targets @@ -44,16 +49,14 @@ Buttercup CRS works with: ## Quick Start -Clone the repository with submodules: +1. Clone the repository with submodules: ```bash git clone --recurse-submodules cd buttercup ``` -Choose your setup method: - -### Automated Setup (Recommended) +2. Run automated setup (Recommended) ```bash make setup-local @@ -61,73 +64,32 @@ make setup-local This script will install all dependencies, configure the environment, and guide you through the setup process. -### Manual Setup - -If you prefer manual setup, see the [Manual Setup Guide](MANUAL_SETUP.md). - -## Starting the System +**Note:** If you prefer manual setup, see the [Manual Setup Guide](MANUAL_SETUP.md). -1. **Deploy locally:** +3. Start Buttercup locally ```bash make deploy-local ``` -2. **Verify deployment:** +4. Verify local deployment: ```bash make status ``` -When deployment is successful, you should see all pods in "Running" status: - -```shell -$ make status -----------PODS------------ -NAME READY STATUS RESTARTS AGE -buttercup-build-bot-845f5b96d9-7t8bz 1/1 Running 0 5m58s -buttercup-build-bot-845f5b96d9-bfsq9 1/1 Running 0 5m58s -buttercup-build-bot-845f5b96d9-npns4 1/1 Running 0 5m58s -buttercup-build-bot-845f5b96d9-sv5fr 1/1 Running 0 5m58s -buttercup-coverage-bot-6749f57b9d-4gzfd 1/1 Running 0 5m58s -buttercup-dind-452s6 1/1 Running 0 5m58s -buttercup-fuzzer-bot-74bc9b849d-2zkt6 1/1 Running 0 5m58s -buttercup-image-preloader-97nfb 0/1 Completed 0 5m58s -buttercup-litellm-5f87df944-2mq7z 1/1 Running 0 5m58s -buttercup-litellm-migrations-ljjcl 0/1 Completed 0 5m58s -buttercup-merger-bot-fz87v 1/1 Running 0 5m58s -buttercup-patcher-7597c965b8-6968s 1/1 Running 0 5m58s -buttercup-postgresql-0 1/1 Running 0 5m58s -buttercup-pov-reproducer-5f948bd7cc-45rgp 1/1 Running 0 5m58s -buttercup-program-model-67446b5cfc-24zfh 1/1 Running 0 5m58s -buttercup-redis-master-0 1/1 Running 0 5m58s -buttercup-registry-cache-5787f86896-czt9b 1/1 Running 0 5m58s -buttercup-scheduler-7c49bf75c5-swqkb 1/1 Running 0 5m58s -buttercup-scratch-cleaner-hdt6z 1/1 Running 0 5m58s -buttercup-seed-gen-6fdb9c94c9-4xmrp 1/1 Running 0 5m57s -buttercup-task-downloader-54cd9fb577-g4lbg 1/1 Running 0 5m58s -buttercup-task-server-7d8cd7cf49-zkt69 1/1 Running 0 5m58s -buttercup-tracer-bot-5b9fb6c8b5-zcmxd 1/1 Running 0 5m58s -buttercup-ui-5dcf7dfb8-njglh 1/1 Running 0 5m58s -----------SERVICES-------- -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -buttercup-litellm ClusterIP 10.96.88.226 4000/TCP 5m58s -buttercup-postgresql ClusterIP 10.111.161.207 5432/TCP 5m58s -buttercup-postgresql-hl ClusterIP None 5432/TCP 5m58s -buttercup-redis-headless ClusterIP None 6379/TCP 5m58s -buttercup-redis-master ClusterIP 10.108.61.77 6379/TCP 5m58s -buttercup-registry-cache ClusterIP 10.103.80.241 443/TCP 5m58s -buttercup-task-server ClusterIP 10.104.206.197 8000/TCP 5m58s -buttercup-ui ClusterIP 10.106.49.166 1323/TCP 5m58s -All CRS pods up and running. -``` +When a deployment is successful, you should see all pods in "Running" or "Completed" status. + -3. **Send a simple challenge to the system:** +5. Send Buttercup a simple task: ```bash make send-libpng-task ``` +6. Access Buttercup's web-based GUI + + ## Production AKS Deployment > **⚠️ Notice:** From 17da843f804b6e54cbfd407079fd646e7d22d737 Mon Sep 17 00:00:00 2001 From: "Michael D. Brown" Date: Mon, 4 Aug 2025 20:17:27 +0000 Subject: [PATCH 06/19] WIP manual edits --- README.md | 37 ++++++------------------------------- 1 file changed, 6 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index c731c1e3..fb09e154 100644 --- a/README.md +++ b/README.md @@ -88,21 +88,16 @@ make send-libpng-task ``` 6. Access Buttercup's web-based GUI +```bash +make web-ui +``` +7. Stop Buttercup -## Production AKS Deployment - -> **⚠️ Notice:** -> The following production deployment instructions have **not been fully tested**. -> Please proceed with caution and verify each step in your environment. -> If you encounter issues, consult the script comments and configuration files for troubleshooting. - -Full production deployment of the **Buttercup CRS** on Azure Kubernetes Service with proper networking, monitoring, and scaling for the DARPA AIxCC competition. - -3. **Test the system:** +**Note:** This is an important step to ensure Buttercup shuts down and stops consuming third-party AI resources. ```bash -make send-libpng-task +make undeploy ``` ## Using the GUI @@ -204,26 +199,6 @@ kubectl logs -n crs kubectl logs -n crs -f ``` -### Stopping the System - -```bash -make undeploy -``` - -## Advanced Usage - -### Production Deployment - -For production deployment on Azure Kubernetes Service, see the [AKS Deployment Guide](AKS_DEPLOYMENT.md). - -### Development - -For development workflows and contributing guidelines, see [CONTRIBUTING.md](CONTRIBUTING.md). - -### Troubleshooting - -For troubleshooting help and common commands, see the [Quick Reference Guide](QUICK_REFERENCE.md). - ## Architecture The **Buttercup CRS** system consists of several components designed to work together for automated vulnerability detection and patching: From 995a8ab48531e9ddbc9ffe305af4c2f19795eedf Mon Sep 17 00:00:00 2001 From: "Michael D. Brown" Date: Mon, 4 Aug 2025 20:35:55 +0000 Subject: [PATCH 07/19] WIP manual edits --- CONTRIBUTING.md | 3 --- README.md | 35 +++++++++++++---------------------- 2 files changed, 13 insertions(+), 25 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e0fc28cb..0cd74953 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -91,8 +91,6 @@ kubectl logs -n crs -l app=scheduler --tail=-1 --prefix kubectl exec -it -n crs -- /bin/bash ``` -Note: For Docker Compose development, you should first `cd deployment/docker-compose` before running docker-compose commands. - ## Component Architecture The system consists of several key components: @@ -128,7 +126,6 @@ Each component should include: - All untrusted code execution must happen in isolated Docker containers - Never expose or log secrets and keys - Never commit secrets or keys to the repository -- Use DinD (Docker-in-Docker) for additional isolation when needed ### Submitting Changes diff --git a/README.md b/README.md index fb09e154..45a1f0db 100644 --- a/README.md +++ b/README.md @@ -81,46 +81,37 @@ make status When a deployment is successful, you should see all pods in "Running" or "Completed" status. -5. Send Buttercup a simple task: +5. Send Buttercup a simple task + +**Note:** When tasked, Buttercup will start consuming third-party AI resources. + +This command will make Buttercup pull down an example repo [example-libpng](https://github.com/tob-challenges/example-libpng) with a known vulnerability. Buttercup will start fuzzing it to find and patch vulnerabilities. ```bash make send-libpng-task ``` 6. Access Buttercup's web-based GUI -```bash -make web-ui -``` -7. Stop Buttercup - -**Note:** This is an important step to ensure Buttercup shuts down and stops consuming third-party AI resources. +Run: ```bash -make undeploy +make web-ui ``` -## Using the GUI +Then navigate to `http://localhost:31323` in your web browser. -The Buttercup CRS includes a web-based user interface for monitoring and managing the system. +In the GUI you can monitor active tasks and see when Buttercup finds bugs and generates patches for them. -### Accessing the GUI -1. **Start port forwarding:** +7. Stop Buttercup + +**Note:** This is an important step to ensure Buttercup shuts down and stops consuming third-party AI resources. ```bash -kubectl port-forward -n crs service/buttercup-ui 31323:1323 & +make undeploy ``` -2. **Open in browser:** - -Navigate to `http://localhost:31323` in your web browser. - -### GUI Features - -- **System Status:** View the status of all system components -- **Task Management:** Monitor active vulnerability discovery and patching tasks - ## Creating and Running Challenges ### Running Challenges From 35c8be89a099e5e588d4ea3199e503351ebafd20 Mon Sep 17 00:00:00 2001 From: "Michael D. Brown" Date: Mon, 4 Aug 2025 20:39:34 +0000 Subject: [PATCH 08/19] WIP manual edits --- README.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 45a1f0db..f1160faa 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,6 @@ # Buttercup Cyber Reasoning System (CRS) -**Buttercup** is a Cyber Reasoning System (CRS) developed by **Trail of Bits** for the **DARPA AIxCC (AI Cyber Challenge) competition**. It's a comprehensive automated vulnerability detection and patching system designed to compete in AI-driven cybersecurity challenges. - -Buttercup finds and patches software vulnerabilities in open-source code repositories like [example-libpng](https://github.com/tob-challenges/example-libpng). It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. +**Buttercup** is a Cyber Reasoning System (CRS) developed by **Trail of Bits** for the **DARPA AIxCC (AI Cyber Challenge)**. Buttercup finds and patches software vulnerabilities in open-source code repositories like [example-libpng](https://github.com/tob-challenges/example-libpng). It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. ## System Requirements @@ -15,8 +13,6 @@ Buttercup finds and patches software vulnerabilities in open-source code reposit **Note:** Buttercup uses third-party AI providers (LLMs from companies like OpenAI, Anthropic and Google), which cost money. Please ensure that you manage per-deployment costs by using the built-in LLM budget setting. -## Local Development - ### Supported Systems - **Linux x86_64** (fully supported) - **ARM64** (partial support for upstream Google OSS-Fuzz projects) @@ -39,7 +35,7 @@ sudo dnf install -y make curl git brew install make curl git ``` -## Supported Targets +### Supported Targets Buttercup CRS works with: From a6ac13387169d42a1fa0f38811e43b47a7f3ffc3 Mon Sep 17 00:00:00 2001 From: "Michael D. Brown" Date: Mon, 4 Aug 2025 20:47:48 +0000 Subject: [PATCH 09/19] WIP manual edits --- README.md | 87 +++++++------------------------------------------------ 1 file changed, 10 insertions(+), 77 deletions(-) diff --git a/README.md b/README.md index f1160faa..d1604d07 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,12 @@ # Buttercup Cyber Reasoning System (CRS) -**Buttercup** is a Cyber Reasoning System (CRS) developed by **Trail of Bits** for the **DARPA AIxCC (AI Cyber Challenge)**. Buttercup finds and patches software vulnerabilities in open-source code repositories like [example-libpng](https://github.com/tob-challenges/example-libpng). It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. +**Buttercup** is a Cyber Reasoning System (CRS) developed by **Trail of Bits** for the **DARPA AIxCC (AI Cyber Challenge)**. Buttercup finds and patches software vulnerabilities in open-source code repositories like [example-libpng](https://github.com/tob-challenges/example-libpng). It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. **Buttercup** system consists of several components: + +- **Orchestrator**: Coordinates the overall task process and manages the workflow +- **Seed Generator**: Creates targeted test cases for vulnerability discovery +- **Fuzzer**: Discovers vulnerabilities through intelligent fuzzing techniques +- **Program Model**: Analyzes code structure and semantics for better understanding +- **Patcher**: Generates and applies security patches to fix vulnerabilities ## System Requirements @@ -37,7 +43,7 @@ brew install make curl git ### Supported Targets -Buttercup CRS works with: +Buttercup works with: - **C source code repositories** that are OSS-Fuzz compatible - **Java source code repositories** that are OSS-Fuzz compatible @@ -108,72 +114,9 @@ In the GUI you can monitor active tasks and see when Buttercup finds bugs and ge make undeploy ``` -## Creating and Running Challenges - -### Running Challenges +## Accessing Logs -To run challenges against the CRS: - -```bash -# Start the UI port forwarding (if not already running) -kubectl port-forward -n crs service/buttercup-ui 31323:1323 & - - -# Run the challenge script -./orchestrator/scripts/challenge.py -``` - -### Pre-defined Challenges - -Use the available pre-defined challenges: - -```bash -make send-libpng-task -``` - -## Common Operations - -### Viewing System Status - -```bash -make status -``` - -### Accessing Logs - -For system logs and monitoring, use SigNoz if configured, otherwise you can use kubectl: -======= -# View all available commands -make help - -# Setup -make setup-local # Automated local development setup -make setup-azure # Automated production AKS setup -make validate # Validate current setup and configuration - -# Deployment -make deploy # Deploy to current environment (local or azure) -make deploy-local # Deploy to local Minikube environment -make deploy-azure # Deploy to production AKS environment - -# Status -make status # Check the status of the deployment - -# Testing -make send-libpng-task # Run libpng test task - -# Development -make lint # Lint all Python code -make lint-component COMPONENT=orchestrator # Lint specific component - -# Cleanup -make undeploy # Remove deployment and clean up resources -make clean-local # Delete Minikube cluster and remove local config -``` - -### Accessing Logs - -For system logs and monitoring, use SigNoz if configured, otherwise you can use kubectl: +To view system logs and telemtry you can use Langfuse or SigNoz if you configured them during setup. Otherwise you can access logs via `kubectl` commands: ```bash # View all pods @@ -186,16 +129,6 @@ kubectl logs -n crs kubectl logs -n crs -f ``` -## Architecture - -The **Buttercup CRS** system consists of several components designed to work together for automated vulnerability detection and patching: - -- **Orchestrator**: Coordinates the overall repair process and manages the workflow -- **Fuzzer**: Discovers vulnerabilities through intelligent fuzzing techniques -- **Patcher**: Generates and applies security patches to fix vulnerabilities -- **Program Model**: Analyzes code structure and semantics for better understanding -- **Seed Generator**: Creates targeted test cases for vulnerability discovery - ## Additional Resources - [Quick Reference Guide](QUICK_REFERENCE.md) - Common commands and troubleshooting From 3d334b3cdf51b0e91a788026d92a5470c732fdec Mon Sep 17 00:00:00 2001 From: "Michael D. Brown" Date: Mon, 4 Aug 2025 20:57:31 +0000 Subject: [PATCH 10/19] WIP manual edits --- MANUAL_SETUP.md | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/MANUAL_SETUP.md b/MANUAL_SETUP.md index 7a93f39f..d6d2b1f4 100644 --- a/MANUAL_SETUP.md +++ b/MANUAL_SETUP.md @@ -4,17 +4,23 @@ This guide provides detailed manual setup instructions for the Buttercup CRS sys ## Prerequisites -Before starting manual setup, ensure you have the following system packages installed: +Before starting manual setup, ensure you have the following dependencies installed: -- `make` - Build automation tool -- `curl` - Command-line tool for transferring data -- `git` - Version control system -- Docker (for containerization) -- kubectl (for Kubernetes management) -- Helm (for Kubernetes package management) -- Minikube (for local Kubernetes cluster) +### System Packages -## Manual Installation Steps +```bash +# Ubuntu/Debian +sudo apt-get update +sudo apt-get install -y make curl git + +# RHEL/CentOS/Fedora +sudo yum install -y make curl git +# or +sudo dnf install -y make curl git + +# MacOS +brew install make curl git +``` ### Docker @@ -73,7 +79,7 @@ cd deployment && make up kubectl port-forward -n crs service/buttercup-ui 31323:1323 # Test manually -./orchestrator/scripts/task_integration_test.sh +./orchestrator/scripts/task_crs.sh ``` ## Verification From 3207d0d82923141c329a6242936d244137e3a6dc Mon Sep 17 00:00:00 2001 From: Michael D Brown Date: Mon, 4 Aug 2025 17:18:24 -0400 Subject: [PATCH 11/19] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d1604d07..3f944b75 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ - **CPU:** 8 cores - **Memory:** 16 GB RAM -- **Storage:** 50 GB available disk space +- **Storage:** 100 GB available disk space - **Network:** Stable internet connection for downloading dependencies **Note:** Buttercup uses third-party AI providers (LLMs from companies like OpenAI, Anthropic and Google), which cost money. Please ensure that you manage per-deployment costs by using the built-in LLM budget setting. From 728fda598c5a1041f20a4cfa1e0e85aeb28f2eb5 Mon Sep 17 00:00:00 2001 From: Michael D Brown Date: Mon, 4 Aug 2025 17:19:13 -0400 Subject: [PATCH 12/19] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3f944b75..0870d843 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ ### Minimum Requirements - **CPU:** 8 cores -- **Memory:** 16 GB RAM +- **Memory:** 64 GB RAM - **Storage:** 100 GB available disk space - **Network:** Stable internet connection for downloading dependencies From e7cc0a7b94d41c551eb32a3e53e557819ce36765 Mon Sep 17 00:00:00 2001 From: Michael D Brown Date: Mon, 4 Aug 2025 17:23:48 -0400 Subject: [PATCH 13/19] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 0870d843..e70c9f79 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,8 @@ **Note:** Buttercup uses third-party AI providers (LLMs from companies like OpenAI, Anthropic and Google), which cost money. Please ensure that you manage per-deployment costs by using the built-in LLM budget setting. +**Note:** Buttercup works best with access to models from OpenAI **and** Anthropic, but can be run with at least one API key from one third-party provider (support for Gemini coming soon). + ### Supported Systems - **Linux x86_64** (fully supported) - **ARM64** (partial support for upstream Google OSS-Fuzz projects) From 20bc67fbfcce564942bb2f2f42228716ec80022b Mon Sep 17 00:00:00 2001 From: Michael D Brown Date: Mon, 4 Aug 2025 17:27:21 -0400 Subject: [PATCH 14/19] Update README.md Co-authored-by: Ronald Eytchison <58823072+reytchison@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e70c9f79..d30bf1d6 100644 --- a/README.md +++ b/README.md @@ -56,7 +56,7 @@ Buttercup works with: 1. Clone the repository with submodules: ```bash -git clone --recurse-submodules +git clone --recurse-submodules https://github.com/trailofbits/buttercup.git cd buttercup ``` From fd0abe597e1461321a16826bf49e4bfb2dee8f25 Mon Sep 17 00:00:00 2001 From: Michael D Brown Date: Mon, 4 Aug 2025 18:57:28 -0400 Subject: [PATCH 15/19] Update README.md Co-authored-by: Ronald Eytchison <58823072+reytchison@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d30bf1d6..91f3bb52 100644 --- a/README.md +++ b/README.md @@ -118,7 +118,7 @@ make undeploy ## Accessing Logs -To view system logs and telemtry you can use Langfuse or SigNoz if you configured them during setup. Otherwise you can access logs via `kubectl` commands: +To view system logs and telemetry you can use Langfuse or SigNoz if you configured them during setup. Otherwise you can access logs via `kubectl` commands: ```bash # View all pods From 7db481be194299b2234ea0b2f66e1c2b7e834078 Mon Sep 17 00:00:00 2001 From: Michael D Brown Date: Mon, 4 Aug 2025 18:58:17 -0400 Subject: [PATCH 16/19] Update CONTRIBUTING.md --- CONTRIBUTING.md | 1 - 1 file changed, 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0cd74953..bee8eea8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -59,7 +59,6 @@ All Python components use consistent formatting and linting standards: - **Formatter:** `ruff format` - **Linter:** `ruff check` - **Type Checker:** `mypy` (for common, patcher, and program-model components) -- **Line Length:** 120 characters ### Running Tests From 3ee9891a27c0906983e4339a0740c68ce0615751 Mon Sep 17 00:00:00 2001 From: Michael D Brown Date: Mon, 4 Aug 2025 18:58:29 -0400 Subject: [PATCH 17/19] Update CONTRIBUTING.md Co-authored-by: Ronald Eytchison <58823072+reytchison@users.noreply.github.com> --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bee8eea8..bb519168 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -8,7 +8,7 @@ Before contributing, set up your local development environment: ```bash # Clone with submodules -git clone --recurse-submodules +git clone --recurse-submodules https://github.com/trailofbits/buttercup.git # Quick setup (recommended) make setup-local From 75a42229b1ee30f59188222dc05e4480c0c275d7 Mon Sep 17 00:00:00 2001 From: Michael D Brown Date: Mon, 4 Aug 2025 18:59:45 -0400 Subject: [PATCH 18/19] Update README.md Co-authored-by: Ronald Eytchison <58823072+reytchison@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 91f3bb52..f549e305 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ **Buttercup** is a Cyber Reasoning System (CRS) developed by **Trail of Bits** for the **DARPA AIxCC (AI Cyber Challenge)**. Buttercup finds and patches software vulnerabilities in open-source code repositories like [example-libpng](https://github.com/tob-challenges/example-libpng). It starts by running an AI/ML-assisted fuzzing campaign (built on oss-fuzz) for the program. When vulnerabilities are found, Buttercup analyzes them and uses a multi-agent AI-driven patcher to repair the vulnerability. **Buttercup** system consists of several components: - **Orchestrator**: Coordinates the overall task process and manages the workflow -- **Seed Generator**: Creates targeted test cases for vulnerability discovery +- **Seed Generator**: Creates inputs for vulnerability discovery - **Fuzzer**: Discovers vulnerabilities through intelligent fuzzing techniques - **Program Model**: Analyzes code structure and semantics for better understanding - **Patcher**: Generates and applies security patches to fix vulnerabilities From f2f10637e67f2283be3bb14bb0dd86928c30978f Mon Sep 17 00:00:00 2001 From: Michael D Brown Date: Mon, 4 Aug 2025 18:59:55 -0400 Subject: [PATCH 19/19] Update CONTRIBUTING.md Co-authored-by: Ronald Eytchison <58823072+reytchison@users.noreply.github.com> --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bb519168..4bea92ba 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -99,7 +99,7 @@ The system consists of several key components: - **Fuzzer** (`/fuzzer/`): Automated vulnerability discovery (build-bot, fuzzer-bot, coverage-bot, tracer-bot) - **Program Model** (`/program-model/`): Semantic code analysis using CodeQuery and Tree-sitter - **Patcher** (`/patcher/`): LLM-powered automated patch generation -- **Seed Generation** (`/seed-gen/`): Intelligent test case generation +- **Seed Generation** (`/seed-gen/`): Intelligent input generation ## Contribution Guidelines