Dynamic Ensemble-Based Architectures Controlled by LLMs

This is a replication package for the paper Dynamic Ensemble-Based Architectures Controlled by LLMs submitted to ECSA 2025.

Contents

• experiments -- experiments (Jupyter notebooks) ran to produce results in the paper
• results -- the raw results (logs) of the experiments
• DSL -- the domain-specific language for the solution architecture (note that here, we use a YAML syntax instead of the syntax described in paper to simplify parsing), and the Jinja templates for prompt generation
• common implementation
  • base classes -- base classes for components, simulation, adaptation (ensemble assignment strategy)
  • adaptations -- the implementation of calls to the LLM (via different APIs) for "online prompting", and the prompt generation and response parsing (jinja.py uses the Jinja templates for generating prompts)
  • main.py -- the main entry point for running the simulation
• use cases -- implementation of the simulation, hand-coded adaptation strategies, configuration files
  • Smart Farm
  • Dragon Hunt
• configuration files -- for selecting a particular LLM and other parameters that are common to all use cases (the configuration files specific for each use case are in the use case directories)
• generated_adaptations -- the adaptation strategies generated by LLMs in the "offline prompting" approach, accompanied by Jupyter notebooks and necessary configurations for running them
  • results -- the raw results (logs) of the offline-prompting experiments

Installation

1. (Optional): create virtual environment: python3 -m venv .venv and activate it: source .venv/bin/activate
2. install requirements: pip install -r requirements.txt

To use OpenAI (paid API):

• create API key: https://platform.openai.com/api-keys
• rename .env.example to .env and save the API key there

To use Google AI (free or paid API):

• create API key: https://aistudio.google.com/app/u/1/apikey
• rename .env.example to .env and save the API key there

To use Anthropic (paid API):

• create API key: https://console.anthropic.com
• rename .env.example to .env and save the API key there

Run

The simulation is run via the main.py file. It is necessary to specify the configuration with command line arguments (yaml files). See experiments for examples of configurations.

To list all main.py file options, use python main.py --help.