Mitigating the AI Safety Impact of Multi-Agent Scaffolds via Self-Improvement
Interested in collaborating on AI safety research? We're actively looking for collaborators to extend AgentBreeder and explore new frontiers in multi-agent safety. Whether you're working on:
- 🔬 Safety evaluation benchmarks - Help us develop better safety metrics
- 🏗️ Multi-agent architectures - Explore novel scaffold designs
- 🎯 Red/blue team methodologies - Advance adversarial testing approaches
- 📊 Evaluation frameworks - Improve our assessment capabilities
Reach out to us: jrosser.co.uk
AgentBreeder was accepted as a spotlight paper at NeurIPS 2025 and previouly an Oral at the ICLR 2025 SSI-FM Workshop!
- 🎯 79.4% average uplift in safety benchmark performance while maintaining capability
- 🔬 First framework for multi-objective evolutionary search over multi-agent scaffolds
- ⚖️ Balanced optimization of safety and capability through Pareto optimization
- 🔴 Red-team insights revealing how capable scaffolds can become highly vulnerable
- 📊 Comprehensive evaluation across DROP, MMLU, GPQA, and SaladData benchmarks
Scaffolding Large Language Models (LLMs) into multi-agent systems often improves performance on complex tasks, but the safety impact of such scaffolds has not been thoroughly explored. We introduce AgentBreeder, a framework for multi-objective self-improving evolutionary search over scaffolds, specifically targeting scaffolds' safety impact on large language models in multi-agent systems.
We evaluate discovered scaffolds on widely recognized reasoning, mathematics, and safety benchmarks. In 'blue' mode, we see a 79.4% average uplift in safety benchmark performance while maintaining or improving capability scores. In 'red' mode, we find adversarially weak scaffolds emerging concurrently with capability optimization. Our work demonstrates the risks of multi-agent scaffolding and provides a framework for mitigating them.
AgentBreeder operates in three distinct modes, each serving different research and deployment needs:
- Objective: Maximize both safety and capability
- Use case: Develop robust, safe multi-agent systems
- Result: 79.4% average safety improvement while maintaining capability
- Objective: Maximize capability while minimizing safety
- Use case: Red-team testing and vulnerability discovery
- Result: Reveals how scaffolding can inadvertently expose safety weaknesses
- Objective: Maximize capability only
- Use case: Baseline comparison and pure performance optimization
- Result: Competitive performance with existing approaches