From 405b11549b15c3afa962aeea41413e81567384c0 Mon Sep 17 00:00:00 2001 From: mbloechli Date: Fri, 3 Jul 2026 15:39:33 +0200 Subject: [PATCH 01/14] docs: add CLAUDE.md Repo guidance for Claude Code covering architecture, versioning conventions, and known GitHub Actions quirks. --- CLAUDE.md | 122 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..7f3b9b6 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,122 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this repo is + +`ci-workflows` is the **single source of truth for CI** across every Duatic ROS 2 repository +(`duatic_helpers`, `duatic_dynaarm`, `duatic_duarover`, etc.). It replaces what used to be a +copy-pasted, drift-prone `reusable_ici.yml` / `build-.yml` / `pre-commit.yml` set of files +duplicated in every repo's `.github/workflows/`. Consumer repos call one workflow from here instead +of authoring their own CI logic. + +This repo is **public** (org policy blocks creating org repos as public directly, but the visibility +was changed after creation via the API). Public is deliberate, not incidental: a private version of +this repo had unreliable cross-repo reusable-workflow resolution — GitHub's access-control indexing +for a *newly created* private repo was flaky even with **Settings → Actions → General → Access** set +to "organization" (calls intermittently failed with "workflow was not found" for several minutes +after each change). Making the repo public sidesteps that indexing path entirely and resolved +instantly. There are no secrets in this repo, so public is safe. If this repo is ever made private +again, re-enable that Access setting and expect to re-verify resolution carefully. + +## Design principle: push complexity here, not into consumer repos + +The whole point of this repo is that a consumer's `.github/workflows/ci.yml` should be almost +nothing — triggers plus one job call. Concretely, a consumer repo should **not** need to know or +configure: +- the distro list or the `ROS_REPO` channel per distro (`rolling` → `testing`, everything else → + `main` — this mapping lives in `ci_orchestrator.yml`'s `prepare` job and nowhere else) +- which distro gates which (jazzy always builds first; the rest only run if it succeeds) +- whether it has an `UPSTREAM_WORKSPACE` (`reusable_ici.yml` auto-detects a `repos.list` file) +- extra `apt` install steps (auto-detected from an optional `.github/ci/apt_dependencies` file in the + consumer repo, one package per line, `#` comments allowed) +- extra `pip` deps that rosdep can't resolve (auto-detected from an optional `requirements.txt` file + at the consumer repo root, installed with full deps via `AFTER_INSTALL_TARGET_DEPENDENCIES`) +- concurrency/cancellation behavior (declared once, centrally, in `ci_orchestrator.yml`) +- what to do about draft PRs (see below) + +When you're asked to add a new capability, default to adding it **here** with a sane built-in +default, not as a new input every consumer has to set. The bar for adding a new `workflow_call` +input is: "does this genuinely vary per repo in a way that can't be inferred?" (e.g. `runner` for the +handful of repos needing `self-hosted`, or `CI_PAT` for repos with private upstream deps) — not "some +repo might someday want to tweak this." + +## Architecture: three workflow files, one nesting level + +``` +.github/workflows/ + ci_orchestrator.yml # consumer-facing entry point — the ONLY workflow product repos call + reusable_ici.yml # leaf: builds ONE distro/channel combination (industrial_ci) + pre-commit.yml # leaf: runs pre-commit across all files +``` + +`ci_orchestrator.yml` calls the two leaves via **local** refs (`uses: ./.github/workflows/reusable_ici.yml`), +which resolve at whatever ref the consumer pinned (e.g. `@v1`) — so a consumer pinned to `v1` always +gets a self-consistent set of all three files, never a mix of versions. `secrets: inherit` chains +consumer → orchestrator → leaf. + +`ci_orchestrator.yml` jobs, in order: +1. **`draft_guard`** — runs only if the triggering event is a draft PR; immediately fails with a + `::error::` message telling the author to mark it ready for review. This exists because the full + build matrix is expensive, but a draft PR still needs a visibly failing required check so nobody + mistakes "no CI ran" for "CI passed." +2. **`prepare`** — skipped on drafts. A single bash step computes two JSON matrices (`primary`, + `secondary`) from the `ros_distro` input. `ros_distro: all` (the default) puts `jazzy` alone in + `primary` and `kilted`/`lyrical`/`rolling` in `secondary`; a specific distro name puts only that + distro in `primary` and empties `secondary`. This is also where the `ROS_REPO` channel is decided + per distro. +3. **`primary`** — matrix over `prepare`'s `primary` output, calls `reusable_ici.yml`. +4. **`secondary`** — `needs: [prepare, primary]`, skipped via `if:` when its matrix is `'[]'` (empty + matrices are avoided deliberately — GitHub Actions handles a job with zero matrix entries poorly + in this context, so the job is skipped outright instead of given an empty `include:`). This is the + gate: it only starts once `primary` (jazzy) has succeeded. +5. **`pre-commit`** — skipped on drafts and on a focused single-distro dispatch (`ros_distro != all`), + since a targeted manual rebuild of one distro usually isn't about linting. + +`reusable_ici.yml` is close to the upstream `ros-industrial/industrial_ci` reusable-workflow +template (see the `original author` comment) — resist the urge to restructure it away from that +shape, since it's the pattern most contributors will already recognize from other ROS orgs. + +## Versioning + +Semver tags. `v1` is a **lightweight tag pointing directly at a commit** — not at another tag, and +not annotated. This matters: an earlier `v1` was created as a lightweight tag pointing at the +annotated `v1.0.0` tag object (a tag-of-a-tag chain via `git tag v1 v1.0.0`), and GitHub's reusable +workflow resolver could not reliably dereference that chain (`workflow was not found` errors that +had nothing to do with permissions). Always recreate `v1` as `git tag -f v1 ` — never +`git tag -f v1 `. + +Release flow for a change: +```bash +git tag vX.Y.Z +git tag -f v1 # move the moving major tag (only if this is a v1-compatible change) +git push origin vX.Y.Z +git push -f origin v1 +``` +Breaking changes to `workflow_call` inputs (removing/renaming an input, changing default behavior in +a way existing consumers rely on) bump the major and get their own `v2` line; don't silently change +`v1`'s behavior underneath existing callers. + +## Testing changes + +There's no CI on this repo itself (it only ships workflow definitions) — validate by: +1. `python3 -c "import yaml; yaml.safe_load(open('path/to/file.yml'))"` for a quick syntax check. +2. Actually dispatching a consumer workflow against the new commit/tag and watching it with + `gh run watch --repo Duatic/ --exit-status`. `duatic_helpers` is the pilot + repo (`.github/workflows/ci.yml`, PR-based branch `ci/reusable-workflows` was used for the initial + validation) — cheapest place to test since it has no private deps and a small distro set. +3. When testing something version-sensitive, point the consumer's `uses:` at the exact commit SHA + first, confirm, then move `v1`/cut a tag — don't debug against a moving tag. + +## Known GitHub quirks worth remembering + +- `schedule`-triggered runs check out the default branch automatically; don't add a + `ref_for_scheduled_build`-style input to work around this (an earlier version of `reusable_ici.yml` + had one — it was removed as dead complexity). +- The `github` context inside a called reusable workflow reflects the **caller** (repository, + workflow name, ref, and event payload including `github.event.pull_request.draft`), which is what + makes the central `concurrency:` group and the `draft_guard` job possible without any consumer-side + code. +- Accessing a missing nested key on the `github.event` object (e.g. `github.event.pull_request.draft` + when the trigger wasn't a `pull_request`) evaluates to `null`/`false` in an `if:` expression rather + than erroring — this is relied on throughout `ci_orchestrator.yml`'s `if:` conditions. From 7e6aef9480e5b47c53eb1b752b6f57b29d25ca69 Mon Sep 17 00:00:00 2001 From: mbloechli Date: Fri, 3 Jul 2026 15:39:43 +0200 Subject: [PATCH 02/14] feat(ci): auto-detect and install root requirements.txt via pip Some rosdep keys aren't resolvable (e.g. python3-open3d-pip). Mirror the existing apt_dependencies auto-detection: an optional requirements.txt at the consumer repo root is pip installed (full dependency tree, --break-system-packages) via AFTER_INSTALL_TARGET_DEPENDENCIES, after rosdep target deps are in place. No-op when the file is absent, so existing @v1 consumers are unaffected. --- .github/workflows/reusable_ici.yml | 8 +++++++- README.md | 3 +++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/.github/workflows/reusable_ici.yml b/.github/workflows/reusable_ici.yml index 30b5833..cc55214 100644 --- a/.github/workflows/reusable_ici.yml +++ b/.github/workflows/reusable_ici.yml @@ -46,7 +46,7 @@ jobs: steps: - name: Checkout uses: actions/checkout@v7 - - name: Detect upstream workspace and extra apt dependencies + - name: Detect upstream workspace, extra apt dependencies, and pip requirements id: detect run: | if [ -f repos.list ]; then @@ -60,6 +60,11 @@ jobs: else echo "before_install=" >> "$GITHUB_OUTPUT" fi + if [ -f requirements.txt ]; then + echo "pip_install=python3 -m pip install --break-system-packages -r requirements.txt" >> "$GITHUB_OUTPUT" + else + echo "pip_install=" >> "$GITHUB_OUTPUT" + fi - name: Cache ccache uses: actions/cache@v6 with: @@ -74,4 +79,5 @@ jobs: ROS_DISTRO: ${{ inputs.ros_distro }} ROS_REPO: ${{ inputs.ros_repo }} BEFORE_INSTALL_UPSTREAM_DEPENDENCIES: ${{ steps.detect.outputs.before_install }} + AFTER_INSTALL_TARGET_DEPENDENCIES: ${{ steps.detect.outputs.pip_install }} BEFORE_INIT: ${{ secrets.CI_PAT != '' && format('apt-get update -qq && apt-get install -y -qq git && git config --global url."https://{0}@github.com/".insteadOf "https://github.com/"', secrets.CI_PAT) || '' }} diff --git a/README.md b/README.md index af82a44..746b88e 100644 --- a/README.md +++ b/README.md @@ -41,6 +41,9 @@ What's **not** an input, because it's derived or auto-detected rather than repo- - **Extra `apt` dependencies** (e.g. `libboost-regex-dev`) — drop a `.github/ci/apt_dependencies` file in the consumer repo (one package per line, `#` comments allowed); picked up automatically. Most repos don't need this file at all. +- **Extra `pip` dependencies not resolvable via rosdep** (e.g. `open3d`) — drop a `requirements.txt` + file at the consumer repo root; `pip install`ed (full dependency tree, `--break-system-packages`) + after target dependencies are installed, before build/test. Most repos don't need this file at all. - **Concurrency** — every orchestrator run is keyed on `--` with `cancel-in-progress: true`, so pushing a new commit to a branch/PR cancels the superseded run. No consumer configuration needed. From f84c491278fc48e962405b682c800edbed5d26a0 Mon Sep 17 00:00:00 2001 From: mbloechli Date: Fri, 3 Jul 2026 17:25:54 +0200 Subject: [PATCH 03/14] debug(ci): add temporary host resource diagnostics around industrial_ci duatic_skills' jazzy build hangs ~29min inside industrial_ci's colcon_setup apt-get step on the self-hosted runner, both before and after the requirements.txt change, so it isn't caused by this PR. Add before/after df/free/docker snapshots plus a 15s-interval background poller (uploaded as an artifact via if: always(), since that survives job cancellation) to see whether disk/memory is exhausted during the hang. Remove once root-caused. --- .github/workflows/reusable_ici.yml | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/.github/workflows/reusable_ici.yml b/.github/workflows/reusable_ici.yml index cc55214..82d7f63 100644 --- a/.github/workflows/reusable_ici.yml +++ b/.github/workflows/reusable_ici.yml @@ -73,6 +73,20 @@ jobs: restore-keys: | ccache-${{ inputs.ros_distro }}-${{ steps.detect.outputs.upstream_workspace }}-${{ inputs.ros_repo }}-${{ github.job }}-${{ github.sha }} ccache-${{ inputs.ros_distro }}-${{ steps.detect.outputs.upstream_workspace }}-${{ inputs.ros_repo }}-${{ github.job }} + # TEMPORARY: diagnosing a self-hosted-runner hang inside industrial_ci's colcon_setup + # apt-get step (duatic_skills#13). Remove once root-caused. + - name: DEBUG - host resources before industrial_ci + run: | + echo "== df -h =="; df -h + echo "== free -h =="; free -h + echo "== docker system df =="; docker system df -v || true + - name: DEBUG - start background resource monitor + run: | + ( while true; do + { date -u +%FT%TZ; df -h /; free -h; echo '---'; } >> /tmp/resource-monitor.log + sleep 15 + done ) & + echo $! > /tmp/monitor.pid - uses: 'ros-industrial/industrial_ci@master' env: UPSTREAM_WORKSPACE: ${{ steps.detect.outputs.upstream_workspace }} @@ -81,3 +95,19 @@ jobs: BEFORE_INSTALL_UPSTREAM_DEPENDENCIES: ${{ steps.detect.outputs.before_install }} AFTER_INSTALL_TARGET_DEPENDENCIES: ${{ steps.detect.outputs.pip_install }} BEFORE_INIT: ${{ secrets.CI_PAT != '' && format('apt-get update -qq && apt-get install -y -qq git && git config --global url."https://{0}@github.com/".insteadOf "https://github.com/"', secrets.CI_PAT) || '' }} + - name: DEBUG - stop background resource monitor + if: always() + run: kill "$(cat /tmp/monitor.pid)" 2>/dev/null || true + - name: DEBUG - host resources after industrial_ci + if: always() + run: | + echo "== df -h =="; df -h + echo "== free -h =="; free -h + echo "== docker system df =="; docker system df -v || true + - name: DEBUG - upload resource monitor log + if: always() + uses: actions/upload-artifact@v4 + with: + name: resource-monitor-${{ inputs.ros_distro }}-${{ inputs.ros_repo }} + path: /tmp/resource-monitor.log + if-no-files-found: ignore From afb60f9202e1e44420274821854677ad4172c1f3 Mon Sep 17 00:00:00 2001 From: mbloechli Date: Fri, 3 Jul 2026 18:00:03 +0200 Subject: [PATCH 04/14] debug(ci): snapshot process tree inside the industrial_ci container Host resources (disk/memory) stayed completely flat for the full ~29min of the duatic_skills hang, which means colcon_setup's apt-get is genuinely blocked, not slow. builder_setup in industrial_ci does nothing but that single apt-get call, and debconf is already set to noninteractive during init, so the likely culprit is a maintainer script blocking on something like systemctl/dbus during dpkg trigger processing (common when a container has no init system running). Poll `docker exec ps auxf` alongside the existing resource monitor to see which process is actually stuck next run. --- .github/workflows/reusable_ici.yml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/.github/workflows/reusable_ici.yml b/.github/workflows/reusable_ici.yml index 82d7f63..31a0954 100644 --- a/.github/workflows/reusable_ici.yml +++ b/.github/workflows/reusable_ici.yml @@ -84,6 +84,9 @@ jobs: run: | ( while true; do { date -u +%FT%TZ; df -h /; free -h; echo '---'; } >> /tmp/resource-monitor.log + for cid in $(docker ps -q); do + { echo "== docker exec $cid ps auxf =="; docker exec "$cid" ps auxf 2>&1; echo '---'; } >> /tmp/container-procs.log + done sleep 15 done ) & echo $! > /tmp/monitor.pid @@ -109,5 +112,7 @@ jobs: uses: actions/upload-artifact@v4 with: name: resource-monitor-${{ inputs.ros_distro }}-${{ inputs.ros_repo }} - path: /tmp/resource-monitor.log + path: | + /tmp/resource-monitor.log + /tmp/container-procs.log if-no-files-found: ignore From 70e63a6067d64eed3d72ef1d37f4186e632c9f22 Mon Sep 17 00:00:00 2001 From: mbloechli Date: Sat, 4 Jul 2026 18:10:55 +0200 Subject: [PATCH 05/14] debug(ci): check whether CI_PAT reaches the job and applied vcs import still hangs identically on the same private repos' git ls-remote after CI_PAT was added to duatic_skills, which could mean either the secret isn't reaching this job (misconfigured scope, wrong repo/org level) or it reaches but doesn't have access to those repos (fails auth, falls back to the same interactive-prompt hang). Add a boolean-only echo of secrets.CI_PAT presence, plus a count of git global insteadOf rules configured inside the container (from BEFORE_INIT), to tell the two cases apart without printing the token. --- .github/workflows/reusable_ici.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/reusable_ici.yml b/.github/workflows/reusable_ici.yml index 31a0954..828302e 100644 --- a/.github/workflows/reusable_ici.yml +++ b/.github/workflows/reusable_ici.yml @@ -80,12 +80,16 @@ jobs: echo "== df -h =="; df -h echo "== free -h =="; free -h echo "== docker system df =="; docker system df -v || true + - name: DEBUG - is CI_PAT reaching this job? + run: | + echo "CI_PAT set: ${{ secrets.CI_PAT != '' }}" - name: DEBUG - start background resource monitor run: | ( while true; do { date -u +%FT%TZ; df -h /; free -h; echo '---'; } >> /tmp/resource-monitor.log for cid in $(docker ps -q); do { echo "== docker exec $cid ps auxf =="; docker exec "$cid" ps auxf 2>&1; echo '---'; } >> /tmp/container-procs.log + { echo -n "insteadOf rules configured: "; docker exec "$cid" git config --global --get-regexp 'url\..*\.insteadof' 2>/dev/null | wc -l; } >> /tmp/container-procs.log done sleep 15 done ) & From 4ffec9822f60f84a26db18f113b34ba288ae1b35 Mon Sep 17 00:00:00 2001 From: mbloechli Date: Sat, 4 Jul 2026 18:44:42 +0200 Subject: [PATCH 06/14] debug(ci): remove temporary host/container diagnostics Root-caused: duatic_skills' hang was two separate, repo-side issues, not a defect in this shared workflow -- a missing/misnamed CI_PAT secret (git blocked on an unanswerable credential prompt) and a stale repos.list pointing at two repos that had since been merged into duatic_dynaarm. Neither needed a change here, so drop the scaffolding. --- .github/workflows/reusable_ici.yml | 39 ------------------------------ 1 file changed, 39 deletions(-) diff --git a/.github/workflows/reusable_ici.yml b/.github/workflows/reusable_ici.yml index 828302e..cc55214 100644 --- a/.github/workflows/reusable_ici.yml +++ b/.github/workflows/reusable_ici.yml @@ -73,27 +73,6 @@ jobs: restore-keys: | ccache-${{ inputs.ros_distro }}-${{ steps.detect.outputs.upstream_workspace }}-${{ inputs.ros_repo }}-${{ github.job }}-${{ github.sha }} ccache-${{ inputs.ros_distro }}-${{ steps.detect.outputs.upstream_workspace }}-${{ inputs.ros_repo }}-${{ github.job }} - # TEMPORARY: diagnosing a self-hosted-runner hang inside industrial_ci's colcon_setup - # apt-get step (duatic_skills#13). Remove once root-caused. - - name: DEBUG - host resources before industrial_ci - run: | - echo "== df -h =="; df -h - echo "== free -h =="; free -h - echo "== docker system df =="; docker system df -v || true - - name: DEBUG - is CI_PAT reaching this job? - run: | - echo "CI_PAT set: ${{ secrets.CI_PAT != '' }}" - - name: DEBUG - start background resource monitor - run: | - ( while true; do - { date -u +%FT%TZ; df -h /; free -h; echo '---'; } >> /tmp/resource-monitor.log - for cid in $(docker ps -q); do - { echo "== docker exec $cid ps auxf =="; docker exec "$cid" ps auxf 2>&1; echo '---'; } >> /tmp/container-procs.log - { echo -n "insteadOf rules configured: "; docker exec "$cid" git config --global --get-regexp 'url\..*\.insteadof' 2>/dev/null | wc -l; } >> /tmp/container-procs.log - done - sleep 15 - done ) & - echo $! > /tmp/monitor.pid - uses: 'ros-industrial/industrial_ci@master' env: UPSTREAM_WORKSPACE: ${{ steps.detect.outputs.upstream_workspace }} @@ -102,21 +81,3 @@ jobs: BEFORE_INSTALL_UPSTREAM_DEPENDENCIES: ${{ steps.detect.outputs.before_install }} AFTER_INSTALL_TARGET_DEPENDENCIES: ${{ steps.detect.outputs.pip_install }} BEFORE_INIT: ${{ secrets.CI_PAT != '' && format('apt-get update -qq && apt-get install -y -qq git && git config --global url."https://{0}@github.com/".insteadOf "https://github.com/"', secrets.CI_PAT) || '' }} - - name: DEBUG - stop background resource monitor - if: always() - run: kill "$(cat /tmp/monitor.pid)" 2>/dev/null || true - - name: DEBUG - host resources after industrial_ci - if: always() - run: | - echo "== df -h =="; df -h - echo "== free -h =="; free -h - echo "== docker system df =="; docker system df -v || true - - name: DEBUG - upload resource monitor log - if: always() - uses: actions/upload-artifact@v4 - with: - name: resource-monitor-${{ inputs.ros_distro }}-${{ inputs.ros_repo }} - path: | - /tmp/resource-monitor.log - /tmp/container-procs.log - if-no-files-found: ignore From e09205b7633c75cbb85a96be526221e3bc93c929 Mon Sep 17 00:00:00 2001 From: mbloechli Date: Sat, 4 Jul 2026 19:08:17 +0200 Subject: [PATCH 07/14] fix(ci): use --ignore-installed for the requirements.txt pip install open3d's dependency tree wants a newer typing_extensions than the one apt/debian installed on the base image. pip can't uninstall a debian-managed package (no RECORD file), so it fails with "Cannot uninstall typing_extensions ..., RECORD file not found". --ignore-installed makes pip install its own copy alongside instead of trying to uninstall the OS-managed one first -- the standard workaround for --break-system-packages installs colliding with apt. --- .github/workflows/reusable_ici.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/reusable_ici.yml b/.github/workflows/reusable_ici.yml index cc55214..1d0812d 100644 --- a/.github/workflows/reusable_ici.yml +++ b/.github/workflows/reusable_ici.yml @@ -61,7 +61,7 @@ jobs: echo "before_install=" >> "$GITHUB_OUTPUT" fi if [ -f requirements.txt ]; then - echo "pip_install=python3 -m pip install --break-system-packages -r requirements.txt" >> "$GITHUB_OUTPUT" + echo "pip_install=python3 -m pip install --break-system-packages --ignore-installed -r requirements.txt" >> "$GITHUB_OUTPUT" else echo "pip_install=" >> "$GITHUB_OUTPUT" fi From 464ffed348075877cd3e361256c6c331f082135c Mon Sep 17 00:00:00 2001 From: mbloechli Date: Sat, 4 Jul 2026 21:08:45 +0200 Subject: [PATCH 08/14] fix(ci): always run pre-commit regardless of ros_distro selection pre-commit was skipped whenever a single distro was chosen (manual dispatch or targeted rebuild), so linting only ran on the default "all" path. Lint checks aren't tied to which distro is being built, so run them unconditionally (still skipped on draft PRs). --- .github/workflows/ci_orchestrator.yml | 2 +- README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci_orchestrator.yml b/.github/workflows/ci_orchestrator.yml index 23ac541..38da458 100644 --- a/.github/workflows/ci_orchestrator.yml +++ b/.github/workflows/ci_orchestrator.yml @@ -99,5 +99,5 @@ jobs: pre-commit: name: pre-commit - if: ${{ (inputs.ros_distro == '' || inputs.ros_distro == 'all') && !(github.event_name == 'pull_request' && github.event.pull_request.draft) }} + if: ${{ !(github.event_name == 'pull_request' && github.event.pull_request.draft) }} uses: ./.github/workflows/pre-commit.yml diff --git a/README.md b/README.md index 746b88e..f1039d1 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ workflows (`reusable_ici.yml`, `pre-commit.yml`) that most repos never reference | Input | Required | Default | Description | |---|---|---|---| -| `ros_distro` | no | `all` | `all` runs the full gated matrix (jazzy, then kilted/lyrical/rolling once jazzy succeeds); a single distro name (e.g. `kilted`) builds only that one and skips `pre-commit`. | +| `ros_distro` | no | `all` | `all` runs the full gated matrix (jazzy, then kilted/lyrical/rolling once jazzy succeeds); a single distro name (e.g. `kilted`) builds only that one. `pre-commit` always runs regardless of this input. | | `runner` | no | `ubuntu-latest` | Runner label(s) for the build jobs, e.g. `self-hosted`. | | Secret | Required | Description | From 492b8fe06789d019d5d4cca0bd703a4b74a8405a Mon Sep 17 00:00:00 2001 From: mbloechli Date: Sat, 4 Jul 2026 21:14:13 +0200 Subject: [PATCH 09/14] docs: add LICENSE and CONTRIBUTING Match duatic_dynaarm's public-facing repo conventions: BSD-3-Clause LICENSE and a CONTRIBUTING.md covering contribution/licensing terms. The tooling section is adapted for this repo specifically, since it has no pre-commit config of its own -- points at the syntax-check and consumer-dispatch validation steps instead. --- CONTRIBUTING.md | 18 ++++++++++++++++++ LICENSE | 11 +++++++++++ 2 files changed, 29 insertions(+) create mode 100644 CONTRIBUTING.md create mode 100644 LICENSE diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..1e0d2b3 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,18 @@ +# Contributing + +We encourage community contributions to this repository. Feel free to open an issue or provide a pull request. + +# Licensing + +Any contribution to this repository will be under the BSD License. + +## Testing changes + +This repo has no CI of its own — it only ships workflow definitions. Before opening a pull request: + +1. Run a quick syntax check on any changed workflow file: + `python3 -c "import yaml; yaml.safe_load(open('path/to/file.yml'))"`. +2. Actually dispatch a consumer workflow against your branch/commit and watch it run, e.g. + `gh run watch --repo Duatic/ --exit-status`. +3. When testing something version-sensitive, point the consumer's `uses:` at the exact commit SHA + first, confirm it works, then move a tag — don't debug against a moving tag. diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..552656a --- /dev/null +++ b/LICENSE @@ -0,0 +1,11 @@ +Copyright 2026 Duatic AG + +Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. From 7a53777e3865b4a5abaf71cf34fcba5287aad471 Mon Sep 17 00:00:00 2001 From: mbloechli Date: Sat, 4 Jul 2026 21:17:00 +0200 Subject: [PATCH 10/14] ci: run pre-commit against this repo's own files Adds a .pre-commit-config.yaml scoped to what this repo actually contains (workflow YAML, docs) -- standard hygiene hooks, actionlint for GitHub Actions-specific linting, and codespell. A new self-check.yml workflow dogfoods the repo's own pre-commit.yml leaf workflow on every PR/push to main, since previously this repo had no CI of its own. --- .github/workflows/self-check.yml | 12 ++++++++++ .pre-commit-config.yaml | 41 ++++++++++++++++++++++++++++++++ CONTRIBUTING.md | 27 ++++++++++++++++----- 3 files changed, 74 insertions(+), 6 deletions(-) create mode 100644 .github/workflows/self-check.yml create mode 100644 .pre-commit-config.yaml diff --git a/.github/workflows/self-check.yml b/.github/workflows/self-check.yml new file mode 100644 index 0000000..51e5d3a --- /dev/null +++ b/.github/workflows/self-check.yml @@ -0,0 +1,12 @@ +name: Self-check +# Runs pre-commit against this repo's own files (workflow YAML, docs), dogfooding the +# pre-commit.yml leaf workflow that every consumer repo gets via ci_orchestrator.yml. + +on: + pull_request: + push: + branches: [main] + +jobs: + pre-commit: + uses: ./.github/workflows/pre-commit.yml diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 0000000..e27b4af --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,41 @@ +# To use: +# +# pre-commit run -a +# +# Or: +# +# pre-commit install # (runs every time you commit in git) +# +# To update this file: +# +# pre-commit autoupdate +# +# See https://github.com/pre-commit/pre-commit + +repos: + # Standard hooks + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v5.0.0 + hooks: + - id: check-added-large-files + - id: check-case-conflict + - id: check-merge-conflict + - id: check-symlinks + - id: check-yaml + - id: end-of-file-fixer + - id: mixed-line-ending + - id: trailing-whitespace + - id: check-byte-order-marker # Forbid UTF-8 byte-order markers + + # GitHub Actions workflow linting + - repo: https://github.com/rhysd/actionlint + rev: v1.7.7 + hooks: + - id: actionlint + + # Spellcheck in comments and docs + - repo: https://github.com/codespell-project/codespell + rev: v2.3.0 + hooks: + - id: codespell + args: ["--write-changes"] diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1e0d2b3..f1221c9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,13 +6,28 @@ We encourage community contributions to this repository. Feel free to open an is Any contribution to this repository will be under the BSD License. -## Testing changes +## Tooling -This repo has no CI of its own — it only ships workflow definitions. Before opening a pull request: +Please make sure for any pull request that it passes the `pre-commit` checks (see Tooling section). -1. Run a quick syntax check on any changed workflow file: - `python3 -c "import yaml; yaml.safe_load(open('path/to/file.yml'))"`. -2. Actually dispatch a consumer workflow against your branch/commit and watch it run, e.g. +### pre-commit + +The [pre-commit](https://pre-commit.com/) tool is used for running certain checks and formatters every time before a commit is done. +Please use it on any pull requests for this repository. It also runs automatically in CI via `.github/workflows/self-check.yml`. + +__Installation:__ + +`apt install pipx && pipx install pre-commit`. Open a new terminal afterwards. + +__Usage:__ +In the repository folder run: `pre-commit run --all`. This can be automated with `pre-commit install`, so all checks are run every time a commit is done. + +## Testing workflow changes + +Beyond `pre-commit` (which lints the YAML itself via `actionlint`), workflow *behavior* changes still +need to be validated against a real consumer, since this repo has no build/test of its own: + +1. Actually dispatch a consumer workflow against your branch/commit and watch it run, e.g. `gh run watch --repo Duatic/ --exit-status`. -3. When testing something version-sensitive, point the consumer's `uses:` at the exact commit SHA +2. When testing something version-sensitive, point the consumer's `uses:` at the exact commit SHA first, confirm it works, then move a tag — don't debug against a moving tag. From c45e7a0df81e1437baddd5970d8ca15d31a14ce2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Marc=20Bl=C3=B6chlinger?= <91188043+mbloechli@users.noreply.github.com> Date: Sat, 4 Jul 2026 21:19:19 +0200 Subject: [PATCH 11/14] Refactor README for improved clarity Updated README.md for clarity and consistency. --- README.md | 52 ++++++++++++++-------------------------------------- 1 file changed, 14 insertions(+), 38 deletions(-) diff --git a/README.md b/README.md index f1039d1..fc198ca 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,6 @@ # ci-workflows -Central, versioned GitHub Actions reusable workflows for the [Duatic](https://github.com/Duatic) -organization's ROS 2 repositories. Consolidates what used to be copy-pasted, drift-prone -`reusable_ici.yml` / `build-.yml` / `pre-commit.yml` files across every product and -library repo into one entry point. +Central, versioned GitHub Actions reusable workflows. ## Versioning @@ -18,15 +15,15 @@ for rollback/bisection. Breaking input changes bump the major (`@v2`). ## What consumers call -Repos only ever call **`ci_orchestrator.yml`** — it owns the distro matrix, the `ROS_REPO` channel +Repos only ever call **`ci_orchestrator.yml`** - it owns the distro matrix, the `ROS_REPO` channel per distro, gating, concurrency, and the draft-PR policy, and internally drives the two leaf workflows (`reusable_ici.yml`, `pre-commit.yml`) that most repos never reference directly. -### `ci_orchestrator.yml` — consumer-facing entry point +### `ci_orchestrator.yml` - consumer-facing entry point | Input | Required | Default | Description | |---|---|---|---| -| `ros_distro` | no | `all` | `all` runs the full gated matrix (jazzy, then kilted/lyrical/rolling once jazzy succeeds); a single distro name (e.g. `kilted`) builds only that one. `pre-commit` always runs regardless of this input. | +| `ros_distro` | no | `all` | `all` runs the full gated matrix (jazzy, then kilted/lyrical/rolling once jazzy succeeds); a single distro name (e.g. `kilted`) builds only that one. | | `runner` | no | `ubuntu-latest` | Runner label(s) for the build jobs, e.g. `self-hosted`. | | Secret | Required | Description | @@ -34,28 +31,14 @@ workflows (`reusable_ici.yml`, `pre-commit.yml`) that most repos never reference | `CI_PAT` | no | PAT used to clone private upstream dependencies from a `repos.list` file at the repo root. Pass via `secrets: inherit`; omit entirely for public-only repos. | What's **not** an input, because it's derived or auto-detected rather than repo-specific config: -- **`ROS_REPO` channel** — `rolling` always builds against `testing`, every other distro against - `main`. Not overridable per repo. -- **`upstream_workspace`** — auto-detected: `reusable_ici.yml` uses `repos.list` if it exists at the - repo root, otherwise nothing. -- **Extra `apt` dependencies** (e.g. `libboost-regex-dev`) — drop a `.github/ci/apt_dependencies` - file in the consumer repo (one package per line, `#` comments allowed); picked up automatically. - Most repos don't need this file at all. -- **Extra `pip` dependencies not resolvable via rosdep** (e.g. `open3d`) — drop a `requirements.txt` - file at the consumer repo root; `pip install`ed (full dependency tree, `--break-system-packages`) - after target dependencies are installed, before build/test. Most repos don't need this file at all. -- **Concurrency** — every orchestrator run is keyed on `--` - with `cancel-in-progress: true`, so pushing a new commit to a branch/PR cancels the superseded run. - No consumer configuration needed. -- **Draft PRs** — the full matrix is too expensive to run on drafts. On a draft PR the orchestrator - runs a single job that fails immediately with a message to mark the PR ready for review, instead of - building anything. Marking the PR "Ready for review" (which the consumer's `pull_request` trigger - must include as a `ready_for_review` type) runs the real matrix. +- **`upstream_workspace`** - auto-detected: `reusable_ici.yml` uses `repos.list` if it exists at the repo root, otherwise nothing. +- **Extra `apt` dependencies** (e.g. `libboost-regex-dev`) - drop a `.github/ci/apt_dependencies` file in the consumer repo (one package per line, `#` comments allowed); picked up automatically. Most repos don't need this file at all. +- **Extra `pip` dependencies not resolvable via rosdep** (e.g. `open3d`) - drop a `requirements.txt` file at the consumer repo root; `pip install`ed (full dependency tree, `--break-system-packages`) after target dependencies are installed, before build/test. Most repos don't need this file at all. +- **Draft PRs** - the full matrix is too expensive to run on drafts. On a draft PR the orchestrator runs a single job that fails immediately with a message to mark the PR ready for review, instead of building anything. Marking the PR "Ready for review" runs the real matrix. ## Example: library repo, no private deps -This is the *entire* CI file a typical repo needs. The only things meant to be edited per repo are -the `cron`/`timezone` and, at rollout, whether it has a `CI_PAT` secret to inherit. +This is the *entire* CI file a typical repo needs. The only things meant to be edited per repo are the `cron`/`timezone` and, whether it has a secret to inherit. ```yaml name: CI @@ -85,8 +68,7 @@ jobs: ## Example: product repo with a private-dependency PAT, self-hosted runner -Same file, plus `runner: self-hosted`. The `repos.list` file at the repo root and the `CI_PAT` -secret (repo or org level) are all that's needed for private upstream deps — no extra workflow input. +Same file, plus `runner: self-hosted`. The `repos.list` file at the repo root and the secret (repo or org level) are all that's needed for private upstream deps - no extra workflow input. ```yaml jobs: @@ -95,20 +77,14 @@ jobs: with: ros_distro: ${{ inputs.ros_distro }} runner: self-hosted - secrets: inherit # forwards CI_PAT + secrets: inherit ``` ## Leaf workflows (internal, not called directly by product repos) -- **`reusable_ici.yml`** — the upstream `ros-industrial` industrial_ci template, builds one +- **`reusable_ici.yml`** - the upstream `ros-industrial` industrial_ci template, builds one distro/channel combination. Auto-detects `repos.list` and `.github/ci/apt_dependencies`. -- **`pre-commit.yml`** — runs `pre-commit` across all files. +- **`pre-commit.yml`** - runs `pre-commit` across all files. ## Requirements on consumer repos - -- This repo is public, so no per-repo "Access" toggle is needed to resolve - `uses: Duatic/ci-workflows/...`. (A private version of this repo was tried first; cross-repo - reusable-workflow resolution was unreliable for a newly created private repo even with the - organization-access setting enabled, so it was made public instead — it holds no secrets.) -- Repos using a private-dependency PAT must have a `CI_PAT` secret available (repo or org level) - and pass `secrets: inherit`. +- Repos using a private-dependency PAT must have a secret available (repo or org level) and pass `secrets: inherit`. From bd6002f7e087d70de49cf63b0ee54a3f446399bb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Marc=20Bl=C3=B6chlinger?= <91188043+mbloechli@users.noreply.github.com> Date: Sat, 4 Jul 2026 21:21:31 +0200 Subject: [PATCH 12/14] Fix formatting in CONTRIBUTING.md --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f1221c9..74841b8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -30,4 +30,4 @@ need to be validated against a real consumer, since this repo has no build/test 1. Actually dispatch a consumer workflow against your branch/commit and watch it run, e.g. `gh run watch --repo Duatic/ --exit-status`. 2. When testing something version-sensitive, point the consumer's `uses:` at the exact commit SHA - first, confirm it works, then move a tag — don't debug against a moving tag. + first, confirm it works, then move a tag - don't debug against a moving tag. From defebbf1ed9e16e74593a79eed13dc11358f89ed Mon Sep 17 00:00:00 2001 From: mbloechli Date: Sat, 4 Jul 2026 21:19:50 +0200 Subject: [PATCH 13/14] refactor(ci): auto-detect apt deps from root Aptfile, not .github/ci/ Nested .github/ci/apt_dependencies didn't match the requirements.txt convention (repo root). Move it to an Aptfile at the consumer repo root, same format (one package per line, # comments allowed), so both optional dependency files live in the same obvious place. No existing consumer repo has the old file, so this is a clean rename. --- .github/workflows/reusable_ici.yml | 4 ++-- CLAUDE.md | 4 ++-- README.md | 5 ++--- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/.github/workflows/reusable_ici.yml b/.github/workflows/reusable_ici.yml index 1d0812d..46fddc9 100644 --- a/.github/workflows/reusable_ici.yml +++ b/.github/workflows/reusable_ici.yml @@ -54,8 +54,8 @@ jobs: else echo "upstream_workspace=" >> "$GITHUB_OUTPUT" fi - if [ -f .github/ci/apt_dependencies ]; then - packages=$(grep -v '^\s*#' .github/ci/apt_dependencies | tr '\n' ' ') + if [ -f Aptfile ]; then + packages=$(grep -v '^\s*#' Aptfile | tr '\n' ' ') echo "before_install=sudo apt-get install -y -qq ${packages}" >> "$GITHUB_OUTPUT" else echo "before_install=" >> "$GITHUB_OUTPUT" diff --git a/CLAUDE.md b/CLAUDE.md index 7f3b9b6..f3d5656 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,8 +28,8 @@ configure: `main` — this mapping lives in `ci_orchestrator.yml`'s `prepare` job and nowhere else) - which distro gates which (jazzy always builds first; the rest only run if it succeeds) - whether it has an `UPSTREAM_WORKSPACE` (`reusable_ici.yml` auto-detects a `repos.list` file) -- extra `apt` install steps (auto-detected from an optional `.github/ci/apt_dependencies` file in the - consumer repo, one package per line, `#` comments allowed) +- extra `apt` install steps (auto-detected from an optional `Aptfile` at the consumer repo root, one + package per line, `#` comments allowed) - extra `pip` deps that rosdep can't resolve (auto-detected from an optional `requirements.txt` file at the consumer repo root, installed with full deps via `AFTER_INSTALL_TARGET_DEPENDENCIES`) - concurrency/cancellation behavior (declared once, centrally, in `ci_orchestrator.yml`) diff --git a/README.md b/README.md index fc198ca..88e0e48 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ workflows (`reusable_ici.yml`, `pre-commit.yml`) that most repos never reference What's **not** an input, because it's derived or auto-detected rather than repo-specific config: - **`upstream_workspace`** - auto-detected: `reusable_ici.yml` uses `repos.list` if it exists at the repo root, otherwise nothing. -- **Extra `apt` dependencies** (e.g. `libboost-regex-dev`) - drop a `.github/ci/apt_dependencies` file in the consumer repo (one package per line, `#` comments allowed); picked up automatically. Most repos don't need this file at all. +- **Extra `apt` dependencies** (e.g. `libboost-regex-dev`) - drop an `Aptfile` at the consumer repo root (one package per line, `#` comments allowed); picked up automatically. Most repos don't need this file at all. - **Extra `pip` dependencies not resolvable via rosdep** (e.g. `open3d`) - drop a `requirements.txt` file at the consumer repo root; `pip install`ed (full dependency tree, `--break-system-packages`) after target dependencies are installed, before build/test. Most repos don't need this file at all. - **Draft PRs** - the full matrix is too expensive to run on drafts. On a draft PR the orchestrator runs a single job that fails immediately with a message to mark the PR ready for review, instead of building anything. Marking the PR "Ready for review" runs the real matrix. @@ -82,8 +82,7 @@ jobs: ## Leaf workflows (internal, not called directly by product repos) -- **`reusable_ici.yml`** - the upstream `ros-industrial` industrial_ci template, builds one - distro/channel combination. Auto-detects `repos.list` and `.github/ci/apt_dependencies`. +- **`reusable_ici.yml`** - the upstream `ros-industrial` industrial_ci template, builds one distro/channel combination. Auto-detects `repos.list`, `Aptfile`, and `requirements.txt`. - **`pre-commit.yml`** - runs `pre-commit` across all files. ## Requirements on consumer repos From 856e2a1e79ac2bb444408b27d79a2f74bb2ac8bb Mon Sep 17 00:00:00 2001 From: mbloechli Date: Sat, 4 Jul 2026 21:35:37 +0200 Subject: [PATCH 14/14] chore: remove CLAUDE.md --- CLAUDE.md | 122 ------------------------------------------------------ 1 file changed, 122 deletions(-) delete mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index f3d5656..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,122 +0,0 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## What this repo is - -`ci-workflows` is the **single source of truth for CI** across every Duatic ROS 2 repository -(`duatic_helpers`, `duatic_dynaarm`, `duatic_duarover`, etc.). It replaces what used to be a -copy-pasted, drift-prone `reusable_ici.yml` / `build-.yml` / `pre-commit.yml` set of files -duplicated in every repo's `.github/workflows/`. Consumer repos call one workflow from here instead -of authoring their own CI logic. - -This repo is **public** (org policy blocks creating org repos as public directly, but the visibility -was changed after creation via the API). Public is deliberate, not incidental: a private version of -this repo had unreliable cross-repo reusable-workflow resolution — GitHub's access-control indexing -for a *newly created* private repo was flaky even with **Settings → Actions → General → Access** set -to "organization" (calls intermittently failed with "workflow was not found" for several minutes -after each change). Making the repo public sidesteps that indexing path entirely and resolved -instantly. There are no secrets in this repo, so public is safe. If this repo is ever made private -again, re-enable that Access setting and expect to re-verify resolution carefully. - -## Design principle: push complexity here, not into consumer repos - -The whole point of this repo is that a consumer's `.github/workflows/ci.yml` should be almost -nothing — triggers plus one job call. Concretely, a consumer repo should **not** need to know or -configure: -- the distro list or the `ROS_REPO` channel per distro (`rolling` → `testing`, everything else → - `main` — this mapping lives in `ci_orchestrator.yml`'s `prepare` job and nowhere else) -- which distro gates which (jazzy always builds first; the rest only run if it succeeds) -- whether it has an `UPSTREAM_WORKSPACE` (`reusable_ici.yml` auto-detects a `repos.list` file) -- extra `apt` install steps (auto-detected from an optional `Aptfile` at the consumer repo root, one - package per line, `#` comments allowed) -- extra `pip` deps that rosdep can't resolve (auto-detected from an optional `requirements.txt` file - at the consumer repo root, installed with full deps via `AFTER_INSTALL_TARGET_DEPENDENCIES`) -- concurrency/cancellation behavior (declared once, centrally, in `ci_orchestrator.yml`) -- what to do about draft PRs (see below) - -When you're asked to add a new capability, default to adding it **here** with a sane built-in -default, not as a new input every consumer has to set. The bar for adding a new `workflow_call` -input is: "does this genuinely vary per repo in a way that can't be inferred?" (e.g. `runner` for the -handful of repos needing `self-hosted`, or `CI_PAT` for repos with private upstream deps) — not "some -repo might someday want to tweak this." - -## Architecture: three workflow files, one nesting level - -``` -.github/workflows/ - ci_orchestrator.yml # consumer-facing entry point — the ONLY workflow product repos call - reusable_ici.yml # leaf: builds ONE distro/channel combination (industrial_ci) - pre-commit.yml # leaf: runs pre-commit across all files -``` - -`ci_orchestrator.yml` calls the two leaves via **local** refs (`uses: ./.github/workflows/reusable_ici.yml`), -which resolve at whatever ref the consumer pinned (e.g. `@v1`) — so a consumer pinned to `v1` always -gets a self-consistent set of all three files, never a mix of versions. `secrets: inherit` chains -consumer → orchestrator → leaf. - -`ci_orchestrator.yml` jobs, in order: -1. **`draft_guard`** — runs only if the triggering event is a draft PR; immediately fails with a - `::error::` message telling the author to mark it ready for review. This exists because the full - build matrix is expensive, but a draft PR still needs a visibly failing required check so nobody - mistakes "no CI ran" for "CI passed." -2. **`prepare`** — skipped on drafts. A single bash step computes two JSON matrices (`primary`, - `secondary`) from the `ros_distro` input. `ros_distro: all` (the default) puts `jazzy` alone in - `primary` and `kilted`/`lyrical`/`rolling` in `secondary`; a specific distro name puts only that - distro in `primary` and empties `secondary`. This is also where the `ROS_REPO` channel is decided - per distro. -3. **`primary`** — matrix over `prepare`'s `primary` output, calls `reusable_ici.yml`. -4. **`secondary`** — `needs: [prepare, primary]`, skipped via `if:` when its matrix is `'[]'` (empty - matrices are avoided deliberately — GitHub Actions handles a job with zero matrix entries poorly - in this context, so the job is skipped outright instead of given an empty `include:`). This is the - gate: it only starts once `primary` (jazzy) has succeeded. -5. **`pre-commit`** — skipped on drafts and on a focused single-distro dispatch (`ros_distro != all`), - since a targeted manual rebuild of one distro usually isn't about linting. - -`reusable_ici.yml` is close to the upstream `ros-industrial/industrial_ci` reusable-workflow -template (see the `original author` comment) — resist the urge to restructure it away from that -shape, since it's the pattern most contributors will already recognize from other ROS orgs. - -## Versioning - -Semver tags. `v1` is a **lightweight tag pointing directly at a commit** — not at another tag, and -not annotated. This matters: an earlier `v1` was created as a lightweight tag pointing at the -annotated `v1.0.0` tag object (a tag-of-a-tag chain via `git tag v1 v1.0.0`), and GitHub's reusable -workflow resolver could not reliably dereference that chain (`workflow was not found` errors that -had nothing to do with permissions). Always recreate `v1` as `git tag -f v1 ` — never -`git tag -f v1 `. - -Release flow for a change: -```bash -git tag vX.Y.Z -git tag -f v1 # move the moving major tag (only if this is a v1-compatible change) -git push origin vX.Y.Z -git push -f origin v1 -``` -Breaking changes to `workflow_call` inputs (removing/renaming an input, changing default behavior in -a way existing consumers rely on) bump the major and get their own `v2` line; don't silently change -`v1`'s behavior underneath existing callers. - -## Testing changes - -There's no CI on this repo itself (it only ships workflow definitions) — validate by: -1. `python3 -c "import yaml; yaml.safe_load(open('path/to/file.yml'))"` for a quick syntax check. -2. Actually dispatching a consumer workflow against the new commit/tag and watching it with - `gh run watch --repo Duatic/ --exit-status`. `duatic_helpers` is the pilot - repo (`.github/workflows/ci.yml`, PR-based branch `ci/reusable-workflows` was used for the initial - validation) — cheapest place to test since it has no private deps and a small distro set. -3. When testing something version-sensitive, point the consumer's `uses:` at the exact commit SHA - first, confirm, then move `v1`/cut a tag — don't debug against a moving tag. - -## Known GitHub quirks worth remembering - -- `schedule`-triggered runs check out the default branch automatically; don't add a - `ref_for_scheduled_build`-style input to work around this (an earlier version of `reusable_ici.yml` - had one — it was removed as dead complexity). -- The `github` context inside a called reusable workflow reflects the **caller** (repository, - workflow name, ref, and event payload including `github.event.pull_request.draft`), which is what - makes the central `concurrency:` group and the `draft_guard` job possible without any consumer-side - code. -- Accessing a missing nested key on the `github.event` object (e.g. `github.event.pull_request.draft` - when the trigger wasn't a `pull_request`) evaluates to `null`/`false` in an `if:` expression rather - than erroring — this is relied on throughout `ci_orchestrator.yml`'s `if:` conditions.