diff --git a/.github/workflows/README.md b/.github/workflows/README.md new file mode 100644 index 000000000..a2adcb1cc --- /dev/null +++ b/.github/workflows/README.md @@ -0,0 +1,117 @@ +# GitHub Actions Workflows + +This directory contains all GitHub Actions workflows for the Aria repository. Workflows are organized by purpose and execution pattern. + +## Workflow Organization + +### 🔄 Continuous Integration (CI) +- **`ci-pipeline.yml`** - Main CI pipeline + - Runs on: Push/PR to main/dev, daily schedule + - Purpose: Code validation, tests, daily training, model deployment + - Duration: ~15-30 minutes + +### ✅ Testing Workflows +- **`aria-tests.yml`** - Comprehensive Aria testing + - Runs on: Changes to `aria_web/` or Aria test files + - Purpose: Multi-version (3.10-3.12), multi-browser E2E tests + - Duration: ~20-30 minutes + - Note: Path-filtered, more thorough + +- **`e2e-tests.yml`** - Quick regression testing + - Runs on: Any push/PR to main + - Purpose: Fast Aria regression tests + - Duration: ~10-15 minutes + - Note: No path filtering, catches broader regressions + +### 🔬 Validation Workflows +- **`auto-validation.yml`** - Orchestrator validation + - Runs on: Changes to orchestrator configs/scripts, daily schedule + - Purpose: Validates `autotrain.yaml` and `quantum_autorun.yaml` + - Duration: ~5-10 minutes + +### ☁️ Cloud Workflows +- **`azureml-train.yml`** - Azure ML training + - Runs on: Manual trigger only + - Purpose: Submit LoRA fine-tuning jobs to Azure ML + - Requires: Azure credentials, ML workspace + +- **`quantum-orchestration.yml`** - Azure Quantum + - Runs on: Push to main, manual trigger + - Purpose: Execute quantum workflows on Azure Quantum + - Requires: Azure credentials, Quantum workspace + +## Workflow Patterns + +### Automatic Triggers +```yaml +on: + push: + branches: [main] # Runs on commits to main + pull_request: + branches: [main] # Runs on PRs targeting main + schedule: + - cron: '0 2 * * *' # Runs daily at 2 AM UTC +``` + +### Manual Triggers +```yaml +on: + workflow_dispatch: # Allows manual execution via GitHub UI + inputs: + parameter: ... # Custom parameters +``` + +### Path Filtering +```yaml +on: + push: + paths: + - 'aria_web/**' # Only runs if these paths change + - 'tests/**' +``` + +## Best Practices + +### When to Use Each Workflow +- **Local Development**: Run tests locally first with `pytest` +- **PR Review**: `e2e-tests.yml` provides quick validation +- **Aria Changes**: `aria-tests.yml` runs comprehensive tests automatically +- **Daily CI**: `ci-pipeline.yml` catches integration issues overnight +- **Training**: Use `azureml-train.yml` for GPU-accelerated cloud training +- **Quantum**: Use `quantum-orchestration.yml` for quantum computing tasks + +### Workflow Naming Convention +- Use descriptive names that indicate purpose +- Suffix with type: `-tests.yml`, `-validation.yml`, `-train.yml` +- Group related workflows with consistent prefixes + +### Adding New Workflows +1. Choose appropriate trigger pattern (push/PR/schedule/manual) +2. Add header comment block explaining purpose +3. Use path filtering when possible to reduce unnecessary runs +4. Set appropriate timeout limits +5. Upload artifacts for debugging +6. Update this README with workflow description + +## Troubleshooting + +### Workflow Not Running +- Check branch filters match your target branch +- Verify path filters include your changed files +- Ensure required secrets are configured + +### Workflow Failing +- Check job logs in GitHub Actions tab +- Download artifacts for detailed results +- Run equivalent commands locally for debugging + +### Reducing CI Time +- Use path filtering to avoid unnecessary runs +- Cache dependencies (pip, npm) +- Run expensive jobs only on schedule or manual trigger +- Parallelize independent jobs with `needs:` dependencies + +## Resources +- [GitHub Actions Documentation](https://docs.github.com/en/actions) +- [Workflow Syntax](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions) +- [Repository Root README](../../README.md) diff --git a/.github/workflows/aria-tests.yml b/.github/workflows/aria-tests.yml index dacb22819..82c04a89d 100644 --- a/.github/workflows/aria-tests.yml +++ b/.github/workflows/aria-tests.yml @@ -1,3 +1,17 @@ +# ============================================================================= +# Aria Comprehensive Tests - Full Aria character system testing +# ============================================================================= +# Purpose: Comprehensive testing of Aria web interface with multiple browsers +# Triggers: Changes to aria_web or test files (path-filtered) +# Jobs: +# - unit-integration-tests: Tests across Python 3.10, 3.11, 3.12 +# - playwright-e2e: Playwright browser automation tests +# - pyppeteer-e2e: Pyppeteer-based E2E tests +# - containerized-chrome-e2e: Selenium containerized tests +# - test-summary: Aggregates results from all test jobs +# Note: More comprehensive but slower than e2e-tests.yml +# ============================================================================= + name: Aria E2E Tests on: diff --git a/.github/workflows/auto-validation.yml b/.github/workflows/auto-validation.yml index b39ed05fb..7032ddbbb 100644 --- a/.github/workflows/auto-validation.yml +++ b/.github/workflows/auto-validation.yml @@ -1,3 +1,12 @@ +# ============================================================================= +# Auto Validation - Orchestrator configuration validation +# ============================================================================= +# Purpose: Validates autotrain.yaml and quantum_autorun.yaml configurations +# Triggers: Changes to YAML configs, training scripts, or daily schedule +# Jobs: +# - dry-run: Validates orchestrator configs without execution +# ============================================================================= + name: Auto Validation on: diff --git a/.github/workflows/azureml-train.yml b/.github/workflows/azureml-train.yml index 62e5ceb0e..f8da4d602 100644 --- a/.github/workflows/azureml-train.yml +++ b/.github/workflows/azureml-train.yml @@ -1,3 +1,13 @@ +# ============================================================================= +# AzureML LoRA Training - Cloud-based model fine-tuning +# ============================================================================= +# Purpose: Submits LoRA fine-tuning jobs to Azure ML +# Triggers: Manual workflow_dispatch only +# Jobs: +# - submit-job: Submits Azure ML training job with specified parameters +# Note: Requires Azure credentials and ML workspace configuration +# ============================================================================= + name: AzureML LoRA Train on: diff --git a/.github/workflows/ci-pipeline.yml b/.github/workflows/ci-pipeline.yml index c4b8fda49..e78526674 100644 --- a/.github/workflows/ci-pipeline.yml +++ b/.github/workflows/ci-pipeline.yml @@ -1,3 +1,14 @@ +# ============================================================================= +# QAI CI Pipeline - Main continuous integration workflow +# ============================================================================= +# Purpose: Validates code quality, runs tests, and performs daily training +# Triggers: Push/PR to main/dev branches, daily scheduled runs +# Jobs: +# - validate: Code validation, unit & integration tests +# - train: Daily scheduled training workflow (requires validate pass) +# - deploy: Model deployment after successful training +# ============================================================================= + name: QAI CI Pipeline on: diff --git a/.github/workflows/e2e-tests.yml b/.github/workflows/e2e-tests.yml index 9cd0f4692..49b16c3cc 100644 --- a/.github/workflows/e2e-tests.yml +++ b/.github/workflows/e2e-tests.yml @@ -1,3 +1,15 @@ +# ============================================================================= +# E2E + Integration Tests - Quick Aria regression testing +# ============================================================================= +# Purpose: Fast regression tests for Aria character system +# Triggers: Any push/PR to main (no path filtering) +# Jobs: +# - integration: Quick unit & integration tests +# - e2e_playwright: Playwright E2E tests +# - containerized_chrome: Pyppeteer containerized tests +# Note: Faster subset of aria-tests.yml, runs on all changes +# ============================================================================= + name: E2E + Integration Tests on: diff --git a/.github/workflows/quantum-orchestration.yml b/.github/workflows/quantum-orchestration.yml index 8ff82f68e..1290e0917 100644 --- a/.github/workflows/quantum-orchestration.yml +++ b/.github/workflows/quantum-orchestration.yml @@ -1,3 +1,13 @@ +# ============================================================================= +# Quantum Automation - Azure Quantum workflow orchestration +# ============================================================================= +# Purpose: Executes quantum computing workflows on Azure Quantum +# Triggers: Push to main or manual workflow_dispatch +# Jobs: +# - run-quantum: Runs quantum orchestration scripts +# Note: Requires Azure credentials for quantum workspace access +# ============================================================================= + name: Quantum Automation on: