Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 60 additions & 22 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,93 @@
# Greengage CI Workflow

This repository contains the main CI pipeline for the Greengage project, orchestrating the build, test, and upload stages for containerized environments. The pipeline is designed to be flexible, with parameterized inputs for version and target operating systems, allowing it to adapt to different branches and configurations.
This directory contains the CI pipelines for the Greengage project,
orchestrating the build, test, and upload stages for containerized
environments. The pipeline is designed to be flexible, with parameterized
inputs for version and target operating systems, allowing it to adapt to
different branches and configurations.

## ⚠️ Important Notice

Whenever the list of **NAMES of required jobs** in the workflow (including any
**reusable workflows**) is **added, removed, or renamed**, you must contact a
repository administrator to update the **Branch Protection Rules** accordingly.
Without this, new, deleted, or renamed jobs will not be recognized as required
when checking Pull Requests.

## Overview

The `Greengage CI` workflow triggers on:

- **Push events** to `main` branch (after merged PR) or versioned release tags (`6.*`).
- **Push events** to `main` branch (after merged PR) or versioned release tags
(`6.*`).
- **Pull requests** to any branch.

It executes the following jobs in a matrix strategy for multiple target operating systems:
It executes the following jobs in a matrix strategy for multiple target
operating systems:

- **Build**: Constructs and pushes Docker images to the GitHub Container Registry (GHCR) with development commit SHA tag and branchname tag. Runs for pull requests and all push events (main and tags).
- **Build**: Constructs and pushes Docker images to the GitHub Container
Registry (GHCR) with development commit SHA tag and branchname tag. Runs for
pull requests and all push events (main and tags).
- **Tests**: Runs multiple test suites only for pull requests, including:
- Behave tests
- Regression tests
- Orca tests
- Resource group tests
- **Upload**: Retags and pushes final Docker images to GHCR and optionally DockerHub. Runs for push to `main` (retags to `latest`) and tags (uses tag like `6.28.2`) after build.
- **Upload**: Retags and pushes final Docker images to GHCR and optionally
DockerHub. Runs for push to `main` (retags to `latest`) and tags (uses tag
like `6.28.2`) after build.

## Configuration

The workflow is parameterized to support flexibility:

- **Version**: Specifies the Greengage version (e.g., `6`), configurable per branch.
- **Target OS**: Supports multiple operating systems, defined in the matrix strategy.
- **Version**: Specifies the Greengage version (e.g., `6`), configurable per
branch.
- **Target OS**: Supports multiple operating systems, defined in the matrix
strategy.

All jobs use reusable workflows stored in the `greengagedb/greengage-ci` repository, accessible publicly for detailed inspection.
All jobs use reusable workflows stored in the `greengagedb/greengage-ci`
repository, accessible publicly for detailed inspection.

## Usage

To use this pipeline:

1. Ensure the repository has a valid `GITHUB_TOKEN` with `packages: write` permissions for GHCR access.
2. Optionally configure `DOCKERHUB_TOKEN` and `DOCKERHUB_USERNAME` for DockerHub uploads.
3. Configure the version and target OS parameters in the branch-specific workflow configuration.
1. Ensure the repository has a valid `GITHUB_TOKEN` with `packages: write`
permissions for GHCR access.
2. Optionally configure `DOCKERHUB_TOKEN` and `DOCKERHUB_USERNAME` for
DockerHub uploads.
3. Configure the version and target OS parameters in the branch-specific
workflow configuration.
4. Create a pull request or push a tag (`6.*`) to trigger the pipeline.

## Additional Documentation

Detailed README files for each process are available in the `README` directory of the `greengagedb/greengage-ci` repository. For example:

- Build process: [README/REUSABLE-BUILD.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-BUILD.md)
- Behave tests: [README/REUSABLE-TESTS-BEHAVE.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-TESTS-BEHAVE.md)
- Regression tests: [README/REUSABLE-TESTS-REGRESSION.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-TESTS-REGRESSION.md)
- Orca tests: [README/REUSABLE-TESTS-ORCA.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-TESTS-ORCA.md)
- Resource group tests: [README/REUSABLE-TESTS-RESGROUP.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-TESTS-RESGROUP.md)
- Upload process: [README/REUSABLE-UPLOAD.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-UPLOAD.md)
Detailed README files for each process are available in the `README` directory
of the `greengagedb/greengage-ci` repository. For example:

- Build process:
[README/REUSABLE-BUILD.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-BUILD.md)
- Behave tests:
[README/REUSABLE-TESTS-BEHAVE.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-TESTS-BEHAVE.md)
- Regression tests:
[README/REUSABLE-TESTS-REGRESSION.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-TESTS-REGRESSION.md)
- Orca tests:
[README/REUSABLE-TESTS-ORCA.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-TESTS-ORCA.md)
- Resource group tests:
[README/REUSABLE-TESTS-RESGROUP.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-TESTS-RESGROUP.md)
- Upload process:
[README/REUSABLE-UPLOAD.md](https://github.com/greengagedb/greengage-ci/blob/main/README/REUSABLE-UPLOAD.md)

## Notes

- The pipeline uses a `fail-fast: true` strategy to stop on any matrix job failure, ensuring quick feedback.
- The full process, including build, tests, and upload, runs only before pull request approval. For push events (main or tags), a build occurs to ensure correct commit references and product version, using the closest tag to HEAD, followed by upload. If DockerHub credentials (`DOCKERHUB_TOKEN`, `DOCKERHUB_USERNAME`) are missing or invalid, DockerHub upload is skipped, but other processes (GHCR upload, etc.) are unaffected.
- For specific details on each stage, refer to the respective reusable workflow files and their READMEs in the `greengagedb/greengage-ci` repository.
- The pipeline uses a `fail-fast: true` strategy to stop on any matrix job
failure, ensuring quick feedback.
- The full process, including build, tests, and upload, runs only before pull
request approval. For push events (main or tags), a build occurs to ensure
correct commit references and product version, using the closest tag to HEAD,
followed by upload. If DockerHub credentials (`DOCKERHUB_TOKEN`,
`DOCKERHUB_USERNAME`) are missing or invalid, DockerHub upload is skipped,
but other processes (GHCR upload, etc.) are unaffected.
- For specific details on each stage, refer to the respective reusable workflow
files and their READMEs in the `greengagedb/greengage-ci` repository.
5 changes: 5 additions & 0 deletions .github/workflows/greengage-abi-tests.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
name: Greengage ABI Tests

# Concurrency control to cancel previous runs on new push to same PR/branch
concurrency:
group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.event.pull_request.number || github.ref }}
cancel-in-progress: true # Automatically cancel any previous runs for the same group

on:
workflow_dispatch:
pull_request:
Expand Down
20 changes: 11 additions & 9 deletions .github/workflows/greengage-ci.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# Main CI pipeline orchestrating build, test, and upload stages
name: Greengage CI

env:
version: 6

on:
push:
branches: ['main'] # Trigger on push to main (after merged PR)
tags: ['6.*'] # Trigger on tags for versioned releases
pull_request:
branches: ['*'] # Trigger on pull requests for all branches

# Concurrency control to cancel previous runs on new push to same PR/branch
concurrency:
group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.event.pull_request.number || github.ref }}
cancel-in-progress: true # Automatically cancel any previous runs for the same group

jobs:
build:
strategy:
Expand All @@ -21,7 +23,7 @@ jobs:
contents: read # Explicit for default behavior
packages: write # Required for GHCR access
actions: write # Required for artifact upload
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-build.yml@main
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-build.yml@v1
with:
version: 6
target_os: ${{ matrix.target_os }}
Expand All @@ -39,7 +41,7 @@ jobs:
contents: read # Explicit for default behavior
packages: read # Explicit for GHCR access clarity
actions: write # Required for artifact upload
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-tests-behave.yml@main
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-tests-behave.yml@v1
with:
version: 6
target_os: ${{ matrix.target_os }}
Expand All @@ -57,7 +59,7 @@ jobs:
contents: read # Explicit for default behavior
packages: read # Explicit for GHCR access clarity
actions: write # Required for artifact upload
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-tests-regression.yml@main
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-tests-regression.yml@v1
with:
version: 6
target_os: ${{ matrix.target_os }}
Expand All @@ -75,7 +77,7 @@ jobs:
contents: read # Explicit for default behavior
packages: read # Explicit for GHCR access clarity
actions: write # Required for artifact upload
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-tests-orca.yml@main
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-tests-orca.yml@v1
with:
version: 6
target_os: ${{ matrix.target_os }}
Expand All @@ -93,7 +95,7 @@ jobs:
contents: read # Explicit for default behavior
packages: read # Explicit for GHCR access clarity
actions: write # Required for artifact upload
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-tests-resgroup.yml@main
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-tests-resgroup.yml@v1
with:
version: 6
target_os: ${{ matrix.target_os }}
Expand All @@ -111,7 +113,7 @@ jobs:
contents: read # Explicit for default behavior
packages: write # Required for GHCR access
actions: write # Required for artifact upload
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-upload.yml@main
uses: greengagedb/greengage-ci/.github/workflows/greengage-reusable-upload.yml@v1
with:
version: 6
target_os: ${{ matrix.target_os }}
Expand Down
2 changes: 1 addition & 1 deletion ci/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ RUN yum makecache && \
# repository can no longer update itself.
curl https://bootstrap.pypa.io/pip/2.7/get-pip.py | python && \
pip install psi pytest==3.10.1 && \
pip install behave==1.2.6 allure-behave==2.4.0 && \
pip install allure-behave==2.4.0 && \
yum clean all

# setup ssh configuration
Expand Down
2 changes: 1 addition & 1 deletion ci/Dockerfile.ubuntu
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ RUN set -eux; \
apt install -y krb5-kdc krb5-admin-server fakeroot sudo python-pip \
openjdk-11-jdk protobuf-compiler; \
# Install allure-behave for behave tests
pip2 install behave==1.2.6 allure-behave==2.4.0; \
pip2 install allure-behave==2.4.0; \
pip2 cache purge; \
# ADCC extension dependencies
$adb_python3_bin -m pip install protobuf==3.20.0; \
Expand Down
Loading
Loading