ALAS: First Automated, Dynamic, General-Purpose Multi-Agent Workflow/Pipeline/Framework for Planning and Optimization

A General-Purpose Operating System for Dynamic Planning, Multi-Agent Communication, Multi-Thread Job Execution and Goal optimization. **
--- ALAS Authors

⬇️ Github 📃 Paper 🌐 Project Page

This repository ALAS provides:

1) Dynamic Planning: Supports both static and dynamic (disruption-prone) tasks.
2) Multi-Agent Communication: Robust inter-agent dependency management and coordination.
3) Multi-Thread Job Execution: Modular, concurrent, and resilient execution with rollback and adaptation.
4) Self-Validation: Ensures plan and schedule is valid by structural, constraint, and compensation soundness at every step.
5) Goal Optimization: Ensures plan and schedule get optimized with user's prompt.
6) Global Replanning: Handles stochastic, random or preset disruption.

ALAS is as far as we know, the first comprehensive middleware for agent application layers and multi-agent databases, supporting real-world use cases in planning, scheduling, optimization, orchestration, and more.

Key Functions of `ALAS`

Three-Layer Architecture:
1. Specification Construction: Parses and builds workflow/task graphs from high-level specifications.
2. Inter-Agent Coordination: Manages agent instantiation, dependency resolution, and communication.
3. Execution, Adaptation, and Validation: Executes agents, handles disruptions with dynamic adaptation (local compensation and global replanning), supports rollback, and performs self-validation.

Function	Description	Function Name	Input
Workflow Construction	Build workflow/task graph from specification.	`WorkflowSpecification`	Task specification (dict/JSON)
Agent Coordination	Set up agents, dependencies, and communication.	`InterAgentCoordinator`	Workflow nodes and edges
Execution Manager	Execute agents, support rollback, validate, adapt.	`ExecutionManager`	Agent list, adaptation manager
Dynamic Adaptation	Handle disruptions, compensation, and replanning.	`DynamicAdaptationManager`	Workflow, coordinator, executor
Self-Validation	Validate structure, constraints, and compensation.	`self_validate()` (in ExecutionManager)	Execution context
Context Management	Query/restore agent execution context.	`select_context`, `restore_context`	Agent name

🔹 Key Features of ALAS vs. Previous Saga

Feature	Saga	ALAS
Architecture	Transactional, rollback	Three-layer: specification, coordination, execution
Dynamic Adaptation	Rollback only	Local compensation + global replanning
Validation	Manual/context-based	Automated self-validation at every step
Optimization	-	General-purpose optimization task
Disruption Handling	Rollback	Compensation, replanning, and rollback
Use Case	Transactional flows	General-purpose, static/dynamic, multi-agent planning
Task/Thread	Single	Single, but extendable to multiple

Results

Benchmark Comparison: Mean Gap to Upper Bound

Validity (~100%)

Optimality: 1) Demirkol-DMU Dataset (19.09%), 2) Real-world TA Dataset (0.86%), 3) Adams, Balas & Zawack Job Shop (ABZ) (0%), 4) Swv Job Shop Benchmark Set (SWZ) (0%), and 5) Yamada and Nakano Benchmark Set (YN) (0%).

MAPLE System Algorithm Workflow

📂 Project Structure

src/workflow_integrations/ — optional Argo / ASL / BPMN exporters (Python verification remains default)
applications/export_workflow_integrations.py — CLI to generate those artifacts
examples/workflow_integrations/ — sample canonical workflow JSON
src/workflow_integrations/ — optional Argo / ASL / BPMN exporters (Python verification remains default)
applications/export_workflow_integrations.py — CLI to generate those artifacts
examples/workflow_integrations/ — sample canonical workflow JSON

🔄 Multi-Agent Framework Comparison

Traditional Multi-Agent Frameworks vs. ALAS

This project compares four established multi-agent frameworks with our novel ALAS (Automated, Dynamic, General-Purpose Multi-Agent Workflow) system:

1. AutoGen Framework

Architecture: Conversational multi-agent system with group chat
Coordination: Manual agent registration and message passing
Validation: External validation required
Optimization: Not built-in, requires custom implementation
Disruption Handling: Manual rollback and recovery

2. CrewAI Framework

Architecture: Task-based multi-agent system with role assignment
Coordination: Hierarchical task delegation
Validation: External validation required
Optimization: Limited optimization capabilities
Disruption Handling: Basic error handling

3. LangGraph Framework

Architecture: Graph-based state machine for agent workflows
Coordination: State-based agent transitions
Validation: External validation required
Optimization: Not built-in
Disruption Handling: State rollback capabilities

4. OpenAI Swarm Framework

Architecture: Swarm-based multi-agent coordination
Coordination: Decentralized agent communication
Validation: External validation required
Optimization: Limited optimization
Disruption Handling: Basic error recovery

5. ALAS (Our Framework)

Architecture: Three-layer automated workflow system
Coordination: Automated dependency resolution and communication
Validation: Built-in self-validation at every step
Optimization: Integrated optimization with multiple strategies
Disruption Handling: Local compensation + global replanning + rollback

Framework Comparison Table

Feature	AutoGen	CrewAI	LangGraph	OpenAI Swarm	ALAS
Architecture	Conversational	Task-based	Graph-based	Swarm-based	Three-layer automated
Coordination	Manual	Hierarchical	State-based	Decentralized	Automated dependency resolution
Validation	External	External	External	External	Built-in self-validation
Optimization	Custom	Limited	None	Limited	Integrated optimization
Disruption Handling	Manual	Basic	State rollback	Basic	Local + Global + Rollback
Scalability	Medium	Medium	High	High	Highly scalable
Ease of Use	Medium	High	Medium	Medium	High (automated)
JSSP Performance	Variable	Variable	Variable	Variable	Consistently optimal

Workflow Verification (optional)

Default (always on): Verification and orchestration are Python-based—no external workflow engine is required.

Layer	Implementation
Schedule / constraint checks	`src/utils/validation_tools.py`, `ValidationTools` in MAPLE runners
Batch validation	`validate_all_initial_schedules_comprehensive.py`
In-loop self-check	`ExecutionManager.self_validate()` in `src/multi_agent/MAPLE.py`
Agent prompts	Plain-text task specifications (not BPMN)

Optional (off by default): Export the same canonical workflow graph to other formats for Kubernetes execution, cloud state machines, or documentation diagrams. These do not replace Python verification; they are adapters you can enable when deploying outside a single Python process.

Integration	Format	Purpose	Enable
Python spec (default export)	`*_python_verification.json`	Documents which Python verifier applies to each step	Default; use `--no-python-spec` to skip
Argo Workflows	`*_argo_workflow.yaml`	Run steps as a DAG on Kubernetes	`ALAS_ENABLE_ARGO=1` or `--enable-argo`
Amazon States Language (ASL)	`*_asl.json`	JSON state machine (e.g. Step Functions)	`ALAS_ENABLE_ASL=1` or `--enable-asl`
BPMN 2.0	`*_bpmn20.xml`	Documentation-quality process diagrams	`ALAS_ENABLE_BPMN=1` or `--enable-bpmn`

# Export all optional artifacts for the built-in MAPLE JSSP full workflow
PYTHONPATH=src python3 applications/export_workflow_integrations.py \
  --enable-argo --enable-asl --enable-bpmn \
  --out-dir artifacts/workflow_integrations

# Or from a JSON task_spec / canonical workflow
PYTHONPATH=src python3 applications/export_workflow_integrations.py \
  --workflow examples/workflow_integrations/maple_jssp_full.json \
  --enable-bpmn

# Kubernetes: submit generated manifest (cluster must have Argo Workflows installed)
# kubectl create -f artifacts/workflow_integrations/maple-jssp-full_argo_workflow.yaml

Module layout: src/workflow_integrations/ (canonical.py, argo_export.py, asl_export.py, bpmn_export.py). Step containers/Lambdas can call applications/run_workflow_step.py as a stub entrypoint and wire it to MAPLE agents in your image.

Preset workflow JSON: examples/workflow_integrations/maple_jssp_full.json.

ALAS Advantages

Automated Workflow Construction: No manual agent setup required
Self-Validation: Built-in constraint checking and validation
Integrated Optimization: Multiple optimization strategies built-in
Robust Disruption Handling: Three-tier approach (local compensation, global replanning, rollback)
Multi-LLM Support: Seamless integration with multiple LLM providers
Real-time Adaptation: Dynamic workflow modification based on disruptions

🔧 ALAS Workflow Documentation

ALAS (Automated, Dynamic, General-Purpose Multi-Agent Workflow) Architecture

ALAS implements a three-layer architecture for dynamic multi-agent planning and optimization:

1. Specification Construction Layer

Purpose: Parses and builds workflow/task graphs from high-level specifications
Components:
- WorkflowSpecification: Converts task specifications into executable workflow graphs
- Task parsing and dependency resolution
- Graph construction with nodes (agents) and edges (dependencies)

2. Inter-Agent Coordination Layer

Purpose: Manages agent instantiation, dependency resolution, and communication
Components:
- InterAgentCoordinator: Sets up agents, dependencies, and communication protocols
- Agent lifecycle management
- Dependency resolution and scheduling
- Inter-agent communication protocols

3. Execution, Adaptation, and Validation Layer

Purpose: Executes agents, handles disruptions, and performs self-validation
Components:
- ExecutionManager: Executes agents with rollback support and validation
- DynamicAdaptationManager: Handles disruptions with local compensation and global replanning
- self_validate(): Automated validation at every step
- Context management and restoration

ALAS Workflow Process

graph TD
    A[Task Specification] --> B[Workflow Construction]
    B --> C[Agent Coordination]
    C --> D[Execution Manager]
    D --> E{Validation}
    E -->|Valid| F[Continue Execution]
    E -->|Invalid| G[Local Compensation]
    G --> H{Compensation Success?}
    H -->|Yes| F
    H -->|No| I[Global Replanning]
    I --> J[Rollback & Restart]
    J --> C
    F --> K{All Agents Complete?}
    K -->|No| D
    K -->|Yes| L[Success]
    
    M[Disruption] --> N[Adaptation Manager]
    N --> G

MAPLE-Integrated JSSP Optimization Workflow

MAPLE Workflow Diagram

MAPLE-INTEGRATED JSSP OPTIMIZATION WORKFLOW
============================================

AGENTS USED:
-----------
1. MAPLEJSSPQueryAgent (1 instance) - CALLS LLM
2. MAPLESupervisorAgent (1 instance) - CALLS LLM

WORKFLOW:
---------

┌─────────────────────────────────────────────────────────────────┐
│                    MAPLE-INTEGRATED WORKFLOW                    │
└─────────────────────────────────────────────────────────────────┘

For each dataset (rcmax_20_15_5, TA01, abz07, swv01, yn01):
│
├─ 1. MAPLEJSSPQueryAgent.generate_schedule()
│   ├─ CALLS LLM: Generates valid schedule
│   ├─ Fallback: Static schedule generation if LLM unavailable
│   └─ Returns: initial schedule
│
├─ 2. ValidationTools.validate_schedule()
│   ├─ Validates schedule against dataset constraints
│   ├─ Checks job precedence, machine capacity, completeness
│   ├─ If invalid: Go back to step 1 (max 3 attempts)
│   └─ If valid: Continue to step 3
│
└─ For each optimization method (6 methods):
    │
<<<<<<< HEAD
    ├─ 3. OptimizationTools.run_optimization_schedule() (optional)
=======
<<<<<<< HEAD
    ├─ 3. OptimizationTools.run_optimization_schedule() (optional)
=======
    ├─ 3. OptimizationTools.run_optimization_schedule() （optional)
>>>>>>> 867d34a51df476c442275c5963030d18087d88d0
>>>>>>> a8de420a5961303908f5c2b53c2c4a620764d537
    │   ├─ Creates optimizer (SA, GA, TS, VNS, MA, LRCP)
    │   ├─ Runs optimization (max 5 iterations)
    │   ├─ Validates result against upper bounds
    │   └─ Returns: optimized schedule, makespan, validation results
    │
    └─ 4. FileStorageTools.save_schedule()
        ├─ Saves optimized schedule to JSON file
        ├─ Includes dataset, method, makespan, timestamp
        └─ Returns filepath for reference

TOTAL AGENTS: 3
- 1 MAPLEJSSPQueryAgent (schedule generation) - CALLS LLM
- 1 ValidationTools (validation) - NO LLM
- 1 FileStorageAgent (file storage) - NO LLM

LLM USAGE:
- MAPLEJSSPQueryAgent: 1-3 LLM calls per dataset (5-15 total)
- ValidationTools: 0 LLM calls (static validation)
- FileStorageAgent: 0 LLM calls (file operations)
- TOTAL LLM CALLS: 5-15 (depends on validation success)

TOTAL METHODS TESTED: 6
- Simulated Annealing (optional)
- Genetic Algorithm   (optional)
- Tabu Search (optional)
- Variable Neighborhood Search (optional)
- Memetic Algorithm (optional)
- LRCP

TOTAL DATASETS: 5
- rcmax_20_15_5 (20x15)
- TA01 (15x15)
- abz07 (20x15)
- swv01 (20x10)
- yn01 (20x20)

TOTAL TESTS: 30 (5 datasets × 6 methods)

Key ALAS Features

Dynamic Adaptation

Local Compensation: Fixes issues at the agent level without affecting the entire workflow
Global Replanning: Reconstructs the entire workflow when local compensation fails
Rollback Support: Automatically reverts to previous valid states

Self-Validation

Structural Validation: Ensures precedence constraints are maintained
Resource Validation: Verifies machine capacity and resource availability
Temporal Validation: Checks time windows and scheduling constraints
Compensation Soundness: Validates that compensation actions don't introduce new violations

Multi-Agent Coordination

Dependency Management: Handles complex inter-agent dependencies
Communication Protocols: Ensures reliable information exchange between agents
Conflict Resolution: Manages resource conflicts and scheduling conflicts
Consistency Maintenance: Ensures all agents maintain consistent state

ALAS vs. Traditional Saga Pattern

Feature	Saga Pattern	ALAS
Architecture	Transactional, rollback only	Three-layer: specification, coordination, execution
Adaptation	Rollback only	Local compensation + global replanning
Validation	Manual/context-based	Automated self-validation at every step
Optimization	Not supported	General-purpose optimization tasks
Disruption Handling	Rollback	Compensation, replanning, and rollback
Use Cases	Transactional flows	General-purpose, static/dynamic, multi-agent planning
Scalability	Limited	Highly scalable with multi-thread support

ALAS Implementation for JSSP

The ALAS framework is specifically implemented for Job Shop Scheduling Problems (JSSP) with the following components:

JSSP-Specific Agents

Job Agents: Individual agents responsible for scheduling specific jobs
Supervisor Agent: Coordinates and aggregates schedules from all job agents
Validation Agent: Validates schedules for constraint violations and optimality
Optimization Agent: Performs makespan optimization and resource utilization

JSSP Workflow Steps

Initial Schedule Generation: Each job agent generates an initial schedule
Schedule Aggregation: Supervisor agent combines all job schedules
Validation: Validation agent checks for constraint violations
Optimization: Optimization agent improves makespan and resource utilization
Repair Iterations: If violations exist, repair agents fix them iteratively
Final Validation: Ensures the final schedule is valid and optimal

ALAS JSSP Execution Flow

# Initialize ALAS workflow
maple = MAPLE(task_specification)

# Run with full ALAS capabilities
maple.run(
    with_rollback=True,      # Enable rollback on failures
    validate=True,          # Enable self-validation
    optimize=True,          # Enable optimization
    repair_iterations=5     # Maximum repair iterations
)

ALAS Configuration Options

Workflow Types

Full ALAS: Complete workflow with all features (validation, repair, optimization)
No Repair: Disables repair iterations for faster execution
No Validation: Skips validation steps (faster but less reliable)
No Optimization: Disables optimization (faster but suboptimal results)

LLM Provider Support

OpenAI: GPT-4o, GPT-4, GPT-3.5-turbo
Anthropic: Claude-4, Claude-3.5-Sonnet
Google: Gemini-2.5, Gemini-1.5-Pro
DeepSeek: DeepSeek-V3, DeepSeek-Coder

Dataset Support

DMU (Demirkol): 16 datasets with varying complexity
TA (Real-world): 71 datasets from real manufacturing
ABZ (Adams-Balas-Zawack): 10 classic job shop problems
SWV (Swv): 20 benchmark datasets
YN (Yamada-Nakano): 4 additional benchmark datasets

📊 Experimental Results Summary

Performance Metrics Overview

Our comprehensive evaluation compares ALAS with traditional multi-agent frameworks and single-agent LLMs across 5 dataset categories (DMU, TA, ABZ, SWV, YN) using 6 key performance metrics:

1. Success Rate Report ✅

Metric: Percentage of valid schedules generated
Script: generate_success_rate_report.py
Key Findings:
- ALAS workflows achieve 100% success rate for full optimization
- Traditional MAS frameworks: 0-100% (highly variable)
- Single-agent models: 0-100% (inconsistent)
- ALAS consistently outperforms all baselines

2. Error Rate Report ✅

Metric: Error steps / total steps in schedule (lower is better)
Script: generate_error_rate_report.py
Key Findings:
- ALAS full workflow: 0% error rate across all datasets
- Traditional MAS: 0-100% error rate (highly variable)
- Single-agent: 0-100% error rate (inconsistent)
- ALAS repair iterations effectively eliminate all errors

3. Optimal Rate Report ✅

Metric: (makespan / upper_bound) × 100% (100% is optimal)
Script: generate_optimal_rate_report.py
Key Findings:
- ALAS achieves 100% optimal rate for ABZ, SWV, YN datasets
- Real-world TA datasets: 0.86% gap from optimal
- DMU datasets: 19.09% gap (challenging instances)
- Significantly outperforms all baseline methods

4. Execution Time Report ✅

Metric: Wall time for complete workflow execution
Script: generate_execution_time_report.py
Key Findings:
- ALAS execution time: 2.5-3.0 seconds (including LLM generation)
- Traditional MAS: 10-100+ seconds (highly variable)
- Single-agent: 5-50 seconds (model dependent)
- ALAS provides optimal balance of speed and quality

5. Token Usage Report ✅

Metric: Token consumption and cost analysis
Script: generate_token_number_report.py
Key Findings:
- ALAS token usage: Optimized for efficiency
- Traditional MAS: High token consumption (multiple agent interactions)
- Single-agent: Variable token usage (model dependent)
- ALAS achieves better results with lower token costs

6. Makespan Report ✅

Metric: Final schedule makespan values
Script: generate_makespan_report.py
Key Findings:
- ALAS consistently achieves lowest makespan values
- Traditional MAS: Highly variable makespan results
- Single-agent: Inconsistent makespan performance
- ALAS optimization strategies significantly improve schedule quality

Dataset Category Performance

Dataset Category	ALAS Success Rate	ALAS Optimal Rate	Best Traditional Method	ALAS Advantage
DMU (Demirkol)	100%	80.91%	Variable (0-100%)	Consistent high performance
TA (Real-world)	100%	99.14%	Variable (0-100%)	Near-optimal on real data
ABZ (Adams-Balas-Zawack)	100%	100%	Variable (0-100%)	Perfect optimization
SWV (Swv)	100%	100%	Variable (0-100%)	Perfect optimization
YN (Yamada-Nakano)	100%	100%	Variable (0-100%)	Perfect optimization

ALAS Performance Summary Table

Metric	ALAS Performance	Traditional Methods	ALAS Advantage
Success Rate	100% across all dataset categories	0-100% (highly variable)	Consistent reliability
Optimal Rate	100% for ABZ/SWV/YN 99.14% for TA 80.91% for DMU	0-100% (inconsistent)	Near-perfect optimization
Execution Time	2.5-3.0 seconds	10-100+ seconds	10-40x faster
Error Rate	0% (full workflow)	0-100% (variable)	Perfect error elimination
Token Usage	Optimized efficiency	High consumption (MAS) Variable (single-agent)	Better results, lower cost

Key Performance Insights

Consistency: ALAS provides consistent, reliable performance across all dataset categories
Optimality: Achieves optimal or near-optimal solutions for most problem instances
Efficiency: Fast execution with low token usage and computational overhead
Robustness: Handles disruptions and errors through automated repair mechanisms
Scalability: Performance remains consistent across different problem sizes

Repair Iteration Analysis

Iteration 1-2: Most error reduction occurs
Iteration 3-5: Fine-tuning and optimization
Error Rate Reduction: 100% → 0% across all datasets
Convergence: ALAS consistently converges to valid, optimal solutions

Additional Experimental Results and Analysis

Table 1: Success Rates (%) across Benchmarks

Method	DMU	TA	ABZ	SWV	YN	Overall
Multi-Agent Systems (GPT-4o)
AutoGen	0.0	0.0	0.0	0.0	0.0	0.0
CrewAI	25.0	57.1	33.3	13.3	75.0	31.1
LangGraph	6.2	28.6	66.7	0.0	0.0	11.1
OpenAI Swarm	43.8	28.6	0.0	33.3	25.0	33.3
Multi-Agent Systems (Claude-4)
AutoGen	0.0	0.0	0.0	0.0	0.0	0.0
CrewAI	43.8	71.4	33.3	13.3	50.0	37.8
LangGraph	6.2	28.6	33.3	0.0	0.0	8.9
OpenAI Swarm	18.8	14.3	33.3	20.0	50.0	22.2
Single-Agent Models
GPT-4o	68.8	85.7	66.7	53.3	100.0	68.9
Claude-Sonnet-4	0.0	28.6	0.0	0.0	0.0	4.4
Gemini-2.5	6.2	0.0	33.3	0.0	25.0	6.7
DeepSeek-V3	6.2	14.3	100.0	6.7	0.0	13.3
ALAS (Ours, Best Variant per Backbone)
ALAS(GPT-4o)	68.8	71.4*	66.7	53.3	100.0	66.7
ALAS(Claude-4)	93.8†	28.6*	66.7	6.7*	50.0*	48.9*
ALAS(DeepSeek-V3)	6.2*	0.0*	100.0†	6.7*	0.0*	11.1*
ALAS(Gemini-2.5)	6.2*	0.0*	33.3*	0.0*	25.0*	6.7*
ALAS (Ours, Best Variant per Dataset)
ALAS(aggregated)	93.8†	71.4*	100.0†	53.3	100.0	83.7†

ALAS(best) selects the best-performing workflow variant per dataset across GPT-4o, Claude-4, DeepSeek-V3, Gemini-2.5.
† = significantly better than baseline (p<0.05), * = significantly better than baseline (p<0.01)

Table 2: Optimal Rates (%) across Benchmarks

Method	DMU	TA	ABZ	SWV	YN	Overall
Multi-Agent Systems (GPT-4o Backbone)
AutoGen	1.4	10.2	1.5	6.0	2.9	4.4
CrewAI	71.8	42.3	88.9	63.7	43.0	63.1
LangGraph	94.3	60.4	42.1	87.8	58.9	80.2
OpenAI Swarm	60.5	73.7	68.5	66.0	51.4	64.1
Multi-Agent Systems (Claude-4 Backbone)
AutoGen	69.8	95.9	100.0	100.0	95.0	92.1
CrewAI	72.7	53.5	99.6	94.2	70.2	78.5
LangGraph	48.3	87.9	57.6	86.3	68.6	69.6
OpenAI Swarm	80.6	87.5	68.5	72.6	80.5	78.2
ALAS Variants (Full Workflows)
ALAS (GPT-4o)	100.0*	78.5*	100.0*	100.0*	100.0*	96.7*
ALAS (Claude-4)	54.9	78.5†	84.5	100.0*	73.3	77.2†
ALAS (Gemini-2.5)	97.4†	100.0*	100.0*	96.8*	100.0†	98.0*
ALAS (DeepSeek-V3)	100.0*	93.6*	100.0*	100.0*	100.0*	98.7*
ALAS (Ours, Best Variant per Dataset)
ALAS (Best)	100.0*	100.0*	100.0*	100.0*	100.0*	100.0*

† = significantly better than baseline (p<0.05), * = significantly better than baseline (p<0.01)

Table 3: Execution Time (s) across Benchmarks

Framework / Model	DMU	TA	ABZ	SWV	YN	Overall
Multi-Agent Systems (GPT-4o Backbone)
AutoGen	33.4±12.8	29.6±7.5	24.7±10.3	33.0±12.1	23.4±5.6	31.20
CrewAI	45.6±11.5	35.6±4.6	43.5±19.6	38.7±9.4	46.4±15.7	41.67
LangGraph	210.5±114.0	183.4±179.9	157.8±107.4	145.6±108.8	201.2±128.4	180.32
OpenAI Swarm	29.1±13.6	24.5±3.6	26.9±12.2	32.3±12.1	24.0±7.7	28.86
MAS (Average)	79.7	68.3	63.2	62.4	73.8	70.51
Multi-Agent Systems (Claude-4 Backbone)
AutoGen	225.1±90.6	218.8±74.0	262.5±77.5	201.1±73.6	184.9±56.7	215.04
CrewAI	168.3±54.3	134.6±71.5	208.0±131.3	147.1±68.1	189.4±79.0	160.50
LangGraph	193.6±33.7	194.2±65.6	208.7±27.4	150.1±52.9	141.9±94.8	175.58
OpenAI Swarm	30.3±19.4	76.2±91.4	43.0±6.1	42.5±13.6	50.1±33.1	44.10
MAS (Average)	154.3	155.9	180.6	135.2	141.6	148.81
ALAS (Variants)
ALAS (GPT-4o)	57.6±77.1	31.5±8.0	152.5±184.4	92.7±100.8	35.5±16.7	69.59
ALAS (Claude-4)	83.9±13.4	73.2±19.4	81.9±7.7	85.9±19.2	83.9±9.5	82.78
ALAS (Gemini-2.5)	39.6±9.1	33.9±13.5	34.1±11.2	36.6±8.2	37.4±8.0	37.17
ALAS (DeepSeek-V3)	61.7±95.6	70.2±76.5	38.4±11.5	72.0±102.1	102.4±166.0	68.52
ALAS (Average)	60.7	52.2	76.7	71.8	64.8	64.52

Table 4: Token Usage across Benchmarks

Framework / Model	DMU	TA	ABZ	SWV	YN	Overall
Multi-Agent Systems (GPT-4o Backbone)
AutoGen	49,850	39,159	26,091	36,483	37,864	41,082
CrewAI	302	283	261	401	622	358
LangGraph	12,996	8,731	4,566	12,279	13,216	11,551
OpenAI Swarm	2,038	2,335	2,176	3,036	2,671	2,482
Multi-Agent Systems (Claude-4 Backbone)
AutoGen	89,690	80,242	94,033	64,920	56,079	77,266
CrewAI	715	882	622	661	609	708
LangGraph	7,734	7,133	6,134	7,414	7,152	7,375
OpenAI Swarm	1,608	3,432	2,565	2,408	2,237	2,278
MAS (Average)	21,054	18,384	17,306	16,190	14,847	17,577
ALAS Variants (Full Workflows)
ALAS (GPT-4o)	8,498	6,774	6,004	5,832	5,634	6,920
ALAS (Claude-4)	12,208	10,033	8,926	8,872	9,980	10,341
ALAS (Gemini-2.5)	11,719	9,927	7,991	8,524	9,657	9,943
ALAS (DeepSeek-V3)	7,762	6,543	4,305	5,184	6,227	6,346
ALAS (Average)	10,047	8,319	6,806	7,103	7,875	8,393

Table 5: Token Cost Summary

Source	Total Tokens	Total Cost	Avg Cost/Instance
Multi-Agent Systems
MAS-GPT4o	2,496,295	$74.89	$0.4160
MAS-Claude4	3,943,206	$118.30	$0.6572
MAS (Average)	3,219,751	$96.60	$0.5366
ALAS Variants
ALAS-GPT4o	1,038,000	$31.14	$0.1730
ALAS-Claude4	1,551,150	$46.53	$0.2590
ALAS-DeepSeek-V3	951,900	$28.55	$0.1590
ALAS-Gemini-2.5	1,491,450	$44.74	$0.2490
ALAS (Average)	1,258,625	$37.74	$0.2100

Table 6: Ablation Study - Optimal Rates (%) of ALAS Workflow Variants

Workflow Variant	DMU	TA	ABZ	SWV	YN	Overall
ALAS (GPT-4o Backbone)
No Repair	32.4	23.3	76.2	60.8	55.0	45.4
No Validation	25.2	12.9	30.9	35.4	6.0	25.4
Full Workflow	100.0*	87.8*	100.0*	100.0*	100.0*	98.1*
ALAS (Claude-4 Backbone)
No Repair	59.2	36.6	99.0	63.0	61.0	63.8
No Validation	53.8	30.2	77.5	69.7	48.1	55.9
Full Workflow	61.9	88.2†	99.2	94.0	84.1	85.5†
ALAS (DeepSeek-V3 Backbone)
No Repair	86.5†	86.7†	31.2	94.4†	93.2*	86.1*
No Validation	67.3	78.5	10.3	90.9	87.1†	74.9
Full Workflow	100.0*	93.6*	100.0*	100.0*	100.0*	99.0*
ALAS (Gemini-2.5 Backbone)
No Repair	83.6†	100.0*	98.5	95.5*	75.3	90.6*
No Validation	83.9†	100.0*	63.0	96.9†	75.3	83.8†
Full Workflow	97.8*	100.0*	100.0*	96.8*	100.0*	98.2*

† = significantly better than baseline (p<0.05), * = significantly better than baseline (p<0.01)

Table 7: Summary Comparison of Multi-Agent Systems (MAS) vs ALAS

Metric	MAS (Average)	ALAS (Average)	Improvement
Token Usage	17,577	8,393	-52.3%
Token Cost	$0.5366	$0.2100	-60.9%
Execution Time (s)	117.6	64.5	1.82× Faster

🚀 How To Run

1️⃣ Quick Start

Clone the repository:

git clone https://github.com/yourusername/M-APPLE-OS.git
cd M-APPLE-OS

Set up environment:

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Configure API keys: Create a .env file in the root directory:

OPENAI_API_KEY=your_openai_key_here
ANTHROPIC_API_KEY=your_anthropic_key_here
GOOGLE_API_KEY=your_google_key_here
DEEPSEEK_API_KEY=your_deepseek_key_here

2️⃣ Run ALAS Workflows

Option A: Run Complete ALAS Evaluation

# Run ALAS with GPT-4o
python3 applications/run_all_maples_optimization_comparison_ablation.py

# Run ALAS with Claude-4
python3 applications/run_all_maples_optimization_comparison_ablation_claude-4.py

# Run ALAS with DeepSeek-V3
python3 applications/run_all_maples_optimization_comparison_ablation_deepseek-v3.py

# Run ALAS with Gemini-2.5
python3 applications/run_all_maples_optimization_comparison_ablation_gemini-2.5.py

Option B: Run Framework Comparisons

# Compare Multi-Agent Systems
python3 applications/run_jssp_framework_comparison.py

# Compare Single-Agent Models
python3 applications/run_jssp_framework_comparison_singleagent.py

3️⃣ Generate Reports

After running the experiments, generate comprehensive reports:

# Success rate analysis
python3 generate_success_rate_report.py

# Error rate analysis
python3 generate_error_rate_report.py

# Optimal rate analysis
python3 generate_optimal_rate_report.py

# Execution time analysis
python3 generate_execution_time_report.py

# Token usage analysis
python3 generate_token_number_report.py

# Makespan analysis
python3 generate_makespan_report.py

# Repair iteration plots
python3 generate_repair_iteration_plots.py

# Scalability analysis
python3 generate_scalability_plot_report.py

4️⃣ ALAS Configuration Options

Workflow Types

Full ALAS: Complete workflow with all features

maple.run(with_rollback=True, validate=True, optimize=True, repair_iterations=5)

No Repair: Disable repair iterations

maple.run(with_rollback=True, validate=True, optimize=True, repair_iterations=0)

No Validation: Skip validation steps

maple.run(with_rollback=True, validate=False, optimize=True)

No Optimization: Disable optimization

maple.run(with_rollback=True, validate=True, optimize=False)

LLM Provider Selection

# Use specific LLM provider
maple = MAPLE(task_spec, model_type="gpt-4o")      # OpenAI
maple = MAPLE(task_spec, model_type="claude-4")    # Anthropic
maple = MAPLE(task_spec, model_type="gemini-2.5") # Google
maple = MAPLE(task_spec, model_type="deepseek-v3") # DeepSeek

5️⃣ Dataset Support

The system supports 5 dataset categories:

DMU (Demirkol): 16 datasets with varying complexity
TA (Real-world): 71 datasets from real manufacturing
ABZ (Adams-Balas-Zawack): 10 classic job shop problems
SWV (Swv): 20 benchmark datasets
YN (Yamada-Nakano): 4 additional benchmark datasets

6️⃣ Expected Results

After running the experiments, you should see:

ALAS Success Rate: 100% across all datasets
ALAS Optimal Rate: 100% for ABZ/SWV/YN, 99.14% for TA, 80.91% for DMU
ALAS Execution Time: 2.5-3.0 seconds
ALAS Error Rate: 0% (full workflow)
ALAS Token Usage: Optimized efficiency

7️⃣ Troubleshooting

API Key Issues: Ensure all API keys are correctly set in .env file
Memory Issues: Use smaller batch sizes for large datasets
Timeout Issues: Increase timeout values in configuration
Dependency Issues: Ensure all packages are installed with pip install -r requirements.txt

Contributing

Fork the repository
Create a feature branch
Commit your changes
Push to the branch
Create a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

Based on the ALAS (Multi-Agent Planning and Learning Environment) framework
Inspired by various JSSP solving approaches and multi-agent systems

✅ Final Thoughts

If everything succeeds, all agents complete. ✅
If any agent fails, local compensation or global replanning is attempted; if not possible, all completed agents roll back automatically, or by inputting a specific node. ✅
Ensures multi-agent consistency in real-world applications (e.g., operating system, stock trading, planning, scheduling, transaction, or payments). ✅

📜 Citation

If you find this repository helpful, please cite the following paper:

ALAS: A Dynamic Multi-LLM Agent Framework for Disruption-Aware Planning and Optimization
Anonymous Author(s)

Name		Name	Last commit message	Last commit date
Latest commit History 53 Commits
.idea		.idea
__pycache__		__pycache__
agent_frameworks		agent_frameworks
agent_frameworks_jssp		agent_frameworks_jssp
agent_frameworks_p10		agent_frameworks_p10
agent_frameworks_p4		agent_frameworks_p4
applications		applications
artifacts/workflow_integrations/demo		artifacts/workflow_integrations/demo
examples/workflow_integrations		examples/workflow_integrations
figures		figures
img		img
logs		logs
outdated_files		outdated_files
output		output
plots		plots
results copy		results copy
results		results
results0		results0
results_baselines_abzswvyn		results_baselines_abzswvyn
results_baselines_dmu		results_baselines_dmu
results_baselines_optimized		results_baselines_optimized
results_baselines_ta		results_baselines_ta
results_maple(claude-4)		results_maple(claude-4)
results_maple(gpt-4o)		results_maple(gpt-4o)
results_mas(claude-4)		results_mas(claude-4)
results_mas(claude-4)_converted		results_mas(claude-4)_converted
results_mas(gpt-4o)		results_mas(gpt-4o)
results_mas(gpt-4o)_converted		results_mas(gpt-4o)_converted
results_optimized(claude-4)		results_optimized(claude-4)
results_optimized(deepseek-v3)		results_optimized(deepseek-v3)
results_optimized(gemini-2.5)		results_optimized(gemini-2.5)
results_optimized(gpt-4o)		results_optimized(gpt-4o)
results_optimized(sagaLLM)		results_optimized(sagaLLM)
results_single(gpt-4o)		results_single(gpt-4o)
src		src
.DS_Store		.DS_Store
.gitattributes		.gitattributes
CORRECTED_RESULTS_SUMMARY.md		CORRECTED_RESULTS_SUMMARY.md
README.md		README.md
assignmentA.tex		assignmentA.tex
benchmark_utils.py		benchmark_utils.py
claude4_validation_results.json		claude4_validation_results.json
comprehensive_initial_schedule_validation.json		comprehensive_initial_schedule_validation.json
comprehensive_initial_schedule_validation_with_alas.json		comprehensive_initial_schedule_validation_with_alas.json
comprehensive_single_agent_results.tex		comprehensive_single_agent_results.tex
convert_autogen_json.py		convert_autogen_json.py
corrected_results_summary.tex		corrected_results_summary.tex
dataflow.tex		dataflow.tex
evaluate_p4_p10_benchmarks.py		evaluate_p4_p10_benchmarks.py
extract_final_report_with_fallback.py		extract_final_report_with_fallback.py
extract_makespan_data.py		extract_makespan_data.py
extract_maple_iterations.py		extract_maple_iterations.py
extract_maple_iterations_simple.py		extract_maple_iterations_simple.py
extract_results_summary.py		extract_results_summary.py
generate_error_rate_report.py		generate_error_rate_report.py
generate_execution_time_report.py		generate_execution_time_report.py
generate_gantt_rcmax_50_20_9.py		generate_gantt_rcmax_50_20_9.py
generate_makespan_report.py		generate_makespan_report.py
generate_optimal_rate_detailed_report.py		generate_optimal_rate_detailed_report.py
generate_optimal_rate_report.py		generate_optimal_rate_report.py
generate_repair_iteration_plots.py		generate_repair_iteration_plots.py
generate_results_tex.py		generate_results_tex.py
generate_scalability_plot_report.py		generate_scalability_plot_report.py
generate_scalability_summary_report.py		generate_scalability_summary_report.py
generate_success_rate_report.py		generate_success_rate_report.py
generate_token_number_report.py		generate_token_number_report.py
maple_iterations_analysis.csv		maple_iterations_analysis.csv
maple_workflow_diagram.txt		maple_workflow_diagram.txt
optimization_comparison.log		optimization_comparison.log
optimization_iteration_data.json		optimization_iteration_data.json
optimization_iteration_summary.csv		optimization_iteration_summary.csv
p4_p10_evaluation.json		p4_p10_evaluation.json
prompt_setup.tex		prompt_setup.tex
requirements.txt		requirements.txt
results.tex		results.tex
single_agent_summary_table.tex		single_agent_summary_table.tex
success_rate_report_by_category.tex		success_rate_report_by_category.tex
validate_all_initial_schedules.py		validate_all_initial_schedules.py
validate_all_initial_schedules_comprehensive.py		validate_all_initial_schedules_comprehensive.py
validate_claude4_schedules.py		validate_claude4_schedules.py

Folders and files

Latest commit

History

Repository files navigation

ALAS: First Automated, Dynamic, General-Purpose Multi-Agent Workflow/Pipeline/Framework for Planning and Optimization

Key Functions of ALAS

🔹 Key Features of ALAS vs. Previous Saga

Results

Benchmark Comparison: Mean Gap to Upper Bound

Validity (~100%)

Optimality: 1) Demirkol-DMU Dataset (19.09%), 2) Real-world TA Dataset (0.86%), 3) Adams, Balas & Zawack Job Shop (ABZ) (0%), 4) Swv Job Shop Benchmark Set (SWZ) (0%), and 5) Yamada and Nakano Benchmark Set (YN) (0%).

MAPLE System Algorithm Workflow

📂 Project Structure

🔄 Multi-Agent Framework Comparison

Traditional Multi-Agent Frameworks vs. ALAS

1. AutoGen Framework

2. CrewAI Framework

3. LangGraph Framework

4. OpenAI Swarm Framework

5. ALAS (Our Framework)

Framework Comparison Table

Workflow Verification (optional)

ALAS Advantages

🔧 ALAS Workflow Documentation

ALAS (Automated, Dynamic, General-Purpose Multi-Agent Workflow) Architecture

1. Specification Construction Layer

2. Inter-Agent Coordination Layer

3. Execution, Adaptation, and Validation Layer

ALAS Workflow Process

MAPLE-Integrated JSSP Optimization Workflow

MAPLE Workflow Diagram

Key ALAS Features

Dynamic Adaptation

Self-Validation

Multi-Agent Coordination

ALAS vs. Traditional Saga Pattern

ALAS Implementation for JSSP

JSSP-Specific Agents

JSSP Workflow Steps

ALAS JSSP Execution Flow

ALAS Configuration Options

Workflow Types

LLM Provider Support

Dataset Support

📊 Experimental Results Summary

Performance Metrics Overview

1. Success Rate Report ✅

2. Error Rate Report ✅

3. Optimal Rate Report ✅

4. Execution Time Report ✅

5. Token Usage Report ✅

6. Makespan Report ✅

Dataset Category Performance

ALAS Performance Summary Table

Key Performance Insights

Repair Iteration Analysis

Additional Experimental Results and Analysis

Table 1: Success Rates (%) across Benchmarks

Table 2: Optimal Rates (%) across Benchmarks

Table 3: Execution Time (s) across Benchmarks

Table 4: Token Usage across Benchmarks

Table 5: Token Cost Summary

Table 6: Ablation Study - Optimal Rates (%) of ALAS Workflow Variants

Table 7: Summary Comparison of Multi-Agent Systems (MAS) vs ALAS

🚀 How To Run

1️⃣ Quick Start

2️⃣ Run ALAS Workflows

Option A: Run Complete ALAS Evaluation

Option B: Run Framework Comparisons

3️⃣ Generate Reports

4️⃣ ALAS Configuration Options

Workflow Types

LLM Provider Selection

5️⃣ Dataset Support

6️⃣ Expected Results

7️⃣ Troubleshooting

Contributing

License

Acknowledgments

✅ Final Thoughts

Key Functions of `ALAS`

Packages