Skip to content

Scheduled drift detection: weekly full-suite re-run against fresh resolves #107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AlexanderFengler merged 2 commits into from
Aug 9, 2026
Merged

Scheduled drift detection: weekly full-suite re-run against fresh resolves #107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
045d6cb
ci: scheduled drift detection workflow (drift.yml)
AlexanderFengler Aug 6, 2026
bbc02ff
drift.yml: don't treat a cancelled run as green; unpin uv in test_not…
AlexanderFengler Aug 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 98 additions & 0 deletions .github/workflows/drift.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
name: Drift

# Scheduled drift detection: re-run the full test workflow (unit matrix +
# notebook execution) against a fresh dependency resolve of an unchanged
# main. No lockfile is committed AND this repo has no push-to-main CI at
# all, so between PRs nothing else proves main still works.
# Failures file ONE deduped `drift`-labeled issue; green runs close it.
on:
schedule:
- cron: "20 5 * * 1" # weekly
workflow_dispatch:
inputs:
force_fail:
description: "Exercise the drift-issue path without a real failure"
type: boolean
default: false

permissions:
contents: read

jobs:
tests:
# Only `report` needs issue-write; the token that runs freshly-resolved
# third-party test code should not carry it.
permissions:
contents: read
uses: ./.github/workflows/run_tests.yml
Comment on lines +22 to +27

@coderabbitai coderabbitai Bot Aug 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== files =="
git ls-files .github/workflows || true

echo "== drift workflow =="
sed -n '1,220p' .github/workflows/drift.yml 2>/dev/null || true

echo "== tests workflow outline =="
if [ -f .github/workflows/run_tests.yml ]; then
  wc -l .github/workflows/run_tests.yml
  ast-grep outline .github/workflows/run_tests.yml --view expanded || true
  sed -n '1,260p' .github/workflows/run_tests.yml
fi

echo "== references to issue/drift/main/reusable workflow =="
rg -n "issue|drift|main|workflow_dispatch|uses: \./\.github/workflows/run_tests|needs:|result|close|open|gh" .github/workflows/. 2>/dev/null || true

Repository: lnccbrown/LANfactory

Length of output: 8701

🌐 Web query:

GitHub reusable workflows github.ref default branch input ref workflow_dispatch workflow_call

💡 Result:

In GitHub Actions, the github.ref context and the calling mechanism for reusable workflows have specific behaviors you should be aware of regarding branches, workflow dispatch, and workflow calls. ### Using References in Reusable Workflows You cannot use expressions or context variables (like ${{ github.ref }} or ${{ github.sha }}) inside the uses keyword when calling a reusable workflow [1][2][3]. GitHub Actions requires a static reference—a branch name, tag, or commit SHA—to parse the workflow file at execution time [2][3]. * Same Repository: If your reusable workflow is in the same repository, you should omit the @{ref} suffix entirely (e.g., uses: ./.github/workflows/my-workflow.yml) [1][2][3]. When you omit this, GitHub automatically executes the workflow version from the same commit as the caller workflow [1][2]. * External Repositories: If calling a workflow from another repository, you must hardcode the version (e.g., uses: octo-org/repo/.github/workflows/my-workflow.yml@v1) [1][3]. ### The github Context in Reusable Workflows When a reusable workflow is triggered, the github context (including github.ref) is associated with the caller workflow, not the called reusable workflow [4][5]. This means the reusable workflow does not inherently "know" which specific branch or tag was used to call it if it was referenced by a tag or branch name [5]. If your reusable workflow needs to act on a specific branch or tag (e.g., for actions/checkout), you must explicitly pass that reference as an input [5][3]: 1. Define an input in the reusable workflow's on.workflow_call.inputs section [6][4]. 2. Pass the value from the caller using ${{ github.ref }} or ${{ github.ref_name }} inside the jobs.<job_id>.with block [4][5]. ### Workflow Dispatch vs. Workflow Call * Workflow Dispatch: The workflow_dispatch event allows you to trigger a workflow manually [6][7]. It is distinct from workflow_call (reusable workflows) and only receives events when the workflow file exists on the default branch [6]. * Inputs: While both workflow_dispatch and workflow_call support inputs, they serve different purposes [6][4]. workflow_call allows one workflow to use another as a component, whereas workflow_dispatch is typically used for on-demand execution [6][7]. Remember that the inputs context inside a reusable workflow is populated only by the values defined in its own on.workflow_call configuration [4][8]. Attempting to access inputs in the caller workflow's job-level if conditions will fail, as those inputs exist only within the scope of the called workflow [8].

Citations:

🌐 Web query:

site:docs.github.com GitHub Actions reusable workflows workflow_dispatch branch workflow_call ref input github.ref default branch

💡 Result:

GitHub Actions utilizes specific mechanisms for triggering, referencing, and passing data to workflows, with distinct behaviors for reusable and manually dispatched workflows. 1. Reusable Workflows (workflow_call) Reusable workflows are defined using the workflow_call trigger [1][2]. They can be called from another workflow (the caller) by referencing their file path and a specific version (ref) [1][3]. - Referencing: You can reference a reusable workflow using the syntax {owner}/{repo}/.github/workflows/{filename}@{ref} [1][4]. The {ref} can be a branch name, a release tag, or a commit SHA [4][5]. - Context and Inputs: When a reusable workflow is triggered, the github context is associated with the caller workflow, not the called workflow [5][6]. Inputs are defined in the called workflow under on.workflow_call.inputs [2] and passed from the caller workflow using the with keyword in a job [1][2]. - Repository Context: Actions within a reusable workflow run as if they were part of the caller workflow; for example, actions/checkout will check out the caller repository's code, not the reusable workflow's repository [3]. 2. Manually Triggered Workflows (workflow_dispatch) The workflow_dispatch trigger allows workflows to be run manually [7]. - Default Branch Requirement: Traditionally, for a workflow to be triggered via workflow_dispatch, the workflow file must exist on the repository's default branch [8][7]. - Branch Execution: While the file must reside on the default branch to be visible and available for manual triggering, you can execute the workflow on a different branch by specifying the ref (e.g., using the --ref flag in the GitHub CLI: gh workflow run WORKFLOW --ref BRANCH) [7]. - Inputs: Inputs for workflow_dispatch are defined in the workflow file and accessed within the workflow via the inputs context or github.event.inputs [8][2][9]. 3. Key Considerations for github.ref - github.ref: This context variable represents the fully-formed ref (e.g., refs/heads/<branch_name> or refs/tags/<tag_name>) of the branch or tag that triggered the workflow [10]. In the context of reusable workflows, because the github context is inherited from the caller [5][6], github.ref will point to the branch or tag of the caller repository that initiated the run, not the branch of the repository hosting the reusable workflow [10].

Citations:

Restrict drift reporting to the default branch.

workflow_dispatch can run this call from a non-default branch when a ref is supplied. The tests job then runs that branch, but report: always() uses needs.tests.result to create or close the shared drift issue, so a passing branch can close an open issue and a failing branch can create a false drift issue. Drive both jobs from the default branch before using the result for issue updates.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/drift.yml around lines 23 - 24, Update the drift
workflow’s tests and reporting jobs so they only execute when the workflow ref
is the repository’s default branch, including manually dispatched runs with a
supplied ref. Apply the same default-branch condition to the job that uses
needs.tests.result for shared drift issue updates, while preserving the existing
test and reporting behavior on the default branch.


Comment on lines +22 to +28
report:
needs: [tests]
if: always()
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
# One issue reconciler at a time: the read at `gh issue list` and the write
# below are not atomic, so an overlapping dispatch could file a duplicate.
concurrency:
group: drift-issue-${{ github.repository }}
cancel-in-progress: false
steps:
- name: Create, update, or close the deduped drift issue
Comment thread
coderabbitai[bot] marked this conversation as resolved.
env:
GH_TOKEN: ${{ github.token }}
RESULT_TESTS: ${{ needs.tests.result }}
FORCE_FAIL: ${{ inputs.force_fail }}
run: |
set -euo pipefail
KEY="drift-key: LANfactory-scheduled"
TITLE="drift: LANfactory scheduled checks failing"
RUN_URL="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"

FAILED_JOBS=""
if [ "$RESULT_TESTS" = "failure" ]; then
FAILED_JOBS=" tests"
fi
if [ "$FORCE_FAIL" = "true" ]; then
FAILED_JOBS="$FAILED_JOBS (force_fail rehearsal)"
fi

EXISTING=$(gh issue list --repo "$GITHUB_REPOSITORY" --label drift \
--state open --search "in:body \"$KEY\"" \
--json number --jq '.[0].number // empty')

# Only a genuine success closes the issue. `report` runs under
# `if: always()`, which includes cancellation, and a cancelled or
# never-started test job is no evidence of health — treating it as
# green would silently retire a real drift issue.
if [ -z "$FAILED_JOBS" ]; then
if [ "$RESULT_TESTS" = "success" ]; then
if [ -n "$EXISTING" ]; then
gh issue close "$EXISTING" --repo "$GITHUB_REPOSITORY" \
--comment "Scheduled drift checks green again: $RUN_URL"
fi
else
echo "::notice::Inconclusive run (tests: $RESULT_TESTS) —" \
"drift issue state left unchanged."
fi
exit 0
Comment thread
coderabbitai[bot] marked this conversation as resolved.
fi

if [ -n "$EXISTING" ]; then
gh issue comment "$EXISTING" --repo "$GITHUB_REPOSITORY" \
--body "Still failing (${FAILED_JOBS# }): $RUN_URL"
else
gh label create drift --repo "$GITHUB_REPOSITORY" \
--description "scheduled drift detection failure" \
--force 2>/dev/null || true
BODY=$(printf '%s\n' \
"<!-- $KEY -->" \
"Scheduled drift run failed in:${FAILED_JOBS}" \
"Run: $RUN_URL" \
"" \
"This re-runs the existing test workflow against a fresh dependency resolve of an unchanged main — a failure usually means an upstream or toolchain release, not a code change." \
"Triage via the spine's audit-drift skill.")
gh issue create --repo "$GITHUB_REPOSITORY" --label drift \
--title "$TITLE" --body "$BODY"
fi
32 changes: 16 additions & 16 deletions .github/workflows/run_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ name: Run tests

on:
pull_request:
workflow_call: # Called by drift.yml (scheduled drift detection)

jobs:
run_tests:
Expand All @@ -27,22 +28,19 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
version: "0.6.5"
# enable-cache: true
# cache-dependency-glob: "pyproject.toml pdm.lock"

- name: Clear all caches
run: |
rm -rf ~/.cache/pip
rm -rf ~/.cache/uv
rm -rf ~/.cache/conda
rm -rf ~/.cache/npm

version: "latest"
enable-cache: true
cache-dependency-glob: "pyproject.toml"
Comment on lines 28 to +33

# (a cache-nuking block used to sit here: rm -rf of every cache plus
# `uv sync --reinstall` with PIP_NO_CACHE_DIR — every PR run cold-
# downloaded the multi-GB torch/bayesflow/keras/sbi stack, and uv was
# pinned to the ancient 0.6.5. It was added during an undiagnosed hour
# of trial and error in June 2025; the SIGILL it was flailing at came
# from ssm-simulators' `-march=native` wheels and is pinned out by
# `ssm-simulators>=0.13.1`. Don't reinstate it on a hunch.)
- name: Install package
run: uv sync --all-groups --reinstall
env:
# UV_CACHE_DIR: ""
PIP_NO_CACHE_DIR: "1"
run: uv sync --all-groups

- name: Run pytest
run: uv run pytest
Expand Down Expand Up @@ -75,7 +73,9 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
version: "0.6.5"
version: "latest"
enable-cache: true
cache-dependency-glob: "pyproject.toml"

- name: Install package (with notebook + backend deps)
run: uv sync --all-groups
Expand Down
Loading