Sync shared package via repository_dispatch and stop Dependabot duplicates#774
Sync shared package via repository_dispatch and stop Dependabot duplicates#774edvilme wants to merge 14 commits into
Conversation
…icates Adds .github/workflows/shared-package-release.yml to handle repository_dispatch (shared-package-release) events from vscode-common-python-lsp: it bumps the npm dep, recompiles the pip pin via uv, pushes a branch, and opens a tracking issue with a manual-PR link (org settings block auto-created PRs). Removes @vscode/common-python-lsp (npm) and vscode-common-python-lsp (pip) from Dependabot via the ignore lists so Dependabot no longer opens duplicate update PRs for the shared package. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the inline Python rewrite of requirements.in with a sed one-liner, keeping uv pip compile --generate-hashes --upgrade to regenerate the hash-locked requirements.txt (matching the command documented in requirements.in). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🔒 Automated review in progress — @rchiodo is auto-reviewing this PR via the Review Center. Other automated reviewers defer to this claim to avoid duplicate, conflicting feedback. The claim is refreshed while the review is active; if it is not refreshed within the ownership timeout it is treated as vacant and another reviewer may take over. This is an automated marker. |
|
Solid approach overall. Main themes to address: make the workflow idempotent/re-runnable (branch push + issue creation), validate the incoming payload, and align the Python version with the documented lockfile-compile process. |
|
🔒 Automated review in progress — @rchiodo is auto-reviewing this PR via the Review Center. Other automated reviewers defer to this claim to avoid duplicate, conflicting feedback. The claim is refreshed while the review is active; if it is not refreshed within the ownership timeout it is treated as vacant and another reviewer may take over. This is an automated marker. |
- Pin uv pip compile to the Python version documented in requirements.in (--python-version) so markers like exceptiongroup are not dropped - Scope the recompile with --upgrade-package so unrelated deps don't move - Assert the pip pin actually changed after sed; fail fast otherwise - Validate release_tag is present and version-shaped before any work - Add concurrency group, --force-with-lease, tracking-issue dedup, and tolerate Issues being disabled (surface compare URL via job summary) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thanks @rchiodo — addressed the feedback in the latest commit:
|
|
Most of the first-pass concerns are already handled in this revision (concurrency group, payload validation, |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
# Conflicts: # package-lock.json
|
GitHub cannot anchor PR review comments to unchanged lines in the diff. Falling back to a general PR comment for requirements.txt:L188. 📍 requirements.txt:185 [verified] |
|
Main themes: (1) the submodule-sync workflow's |
|
🔒 Automated review in progress — @rchiodo is auto-reviewing this PR. |
Consolidate the sync steps into a single script, add a retry wrapper for network operations, fail when the released tag is missing instead of silently falling back to main, and surface failures as workflow annotations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a Development section explaining that the shared vscode-common-python-lsp library is a git submodule and how to initialize it (clone with --recurse-submodules, or git submodule update --init --recursive for an existing clone) before running npm install. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| echo 'exists=false' >>"${GITHUB_OUTPUT}" | ||
| fi | ||
|
|
||
| - name: Create the release branch from the latest main |
There was a problem hiding this comment.
The Python-compat guard validates the wrong "minimum": EXT_MIN is parsed from runtime.txt (the bundling Python), not the extension's declared minimum supported interpreter in src/common/constants.ts/noxfile.py. If runtime.txt is 3.13 but the extension still supports 3.9, the guard computes 3.13 >= PKG_FLOOR and passes — yet a user on 3.9 hits an ImportError from the bundled lib, the exact failure this guard claims to prevent. The error message names three files but only inspects one. Compare PKG_FLOOR against the declared minimum, not runtime.txt.
| echo 'exists=false' >>"${GITHUB_OUTPUT}" | ||
| fi | ||
|
|
||
| - name: Create the release branch from the latest main |
There was a problem hiding this comment.
Two structural issues in the compat guard: (1) PKG_FLOOR is parsed from ${SUBMODULE_PATH}/python/pyproject.toml's requires-python, so any upstream repackaging (moved file, setup.cfg, or a dropped requires-python) yields an empty value and hard-fails every future sync until this workflow is edited. (2) It runs before the no-op short-circuit (git diff --quiet), so a re-dispatch that genuinely has nothing to do can still fail loudly. Consider treating an unparseable floor as warn-and-skip, and reordering the no-op check ahead of the drift guard.
| return 1 | ||
| fi | ||
| echo "Attempt ${attempt} failed: $* — retrying..." >&2 | ||
| attempt=$((attempt + 1)) |
There was a problem hiding this comment.
Narrow notification gap: if git push succeeds but the runner dies before the tracking-issue/summary step, a re-dispatch hits this guard-and-skip and exit 0s before ever emitting a tracking issue or compare URL — the branch is left orphaned with no maintainer signal. On the skip path, consider checking for an existing tracking issue for ${BRANCH} and re-emitting the compare URL to $GITHUB_STEP_SUMMARY before exiting. Low priority; the guard-and-skip itself is correct.
| break | ||
| fi | ||
| done | ||
| if [ -z "${TARGET}" ]; then |
There was a problem hiding this comment.
npm install --package-lock-only --ignore-scripts is not wrapped in the retry helper used for fetch/push, so a flaky registry fails the job late. Because the root manifest carries ^/range deps, install (not ci) can also resolve newer transitives and commit lockfile churn beyond the gitlink bump. Consider wrapping it in retry and/or asserting the lockfile delta is limited to the linked package. Low priority.
|
Overall the submodule-sync approach looks solid. A few robustness notes on the sync workflow's Python-floor guard and dependency-constraint coupling are worth a look, but nothing blocking. |
Add a guard that fails the sync if the extension's pinned Node (.nvmrc) is older than the shared package's pinned Node (submodule .nvmrc), since the extension builds the shared package's TypeScript during install. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🔒 Automated review in progress — @rchiodo is auto-reviewing this PR. |
| working-directory: ${{ env.SUBMODULE_PATH }} | ||
| run: | | ||
| set -euo pipefail | ||
| retry git fetch --tags --force origin |
There was a problem hiding this comment.
This Node guard validates against the wrong "minimum." It parses PKG_NODE from the submodule's .nvmrc (an exact dev pin) and requires EXT_NODE >= PKG_NODE, but the submodule's actual buildability floor is engines.node (>=18.0.0). If the submodule bumps its .nvmrc above the extension's Node version while still declaring engines.node >=18, the extension can build it fine yet this step hard-fails the whole sync. The strict full-X.Y.Z regex also hard-fails every future sync if the submodule .nvmrc is absent or partial (22, lts/*). Prefer comparing against engines.node (or drop the guard and let npm ci/postinstall fail loudly), and treat an unparseable floor as warn-and-skip.
| @@ -0,0 +1,221 @@ | |||
| name: Shared Package Submodule Sync | |||
There was a problem hiding this comment.
The PR description is stale and now misleading: it narrates a shared-package-release.yml doing uv pip compile/sed/requirements.in rewriting, but the shipped file is shared-package-submodule-sync.yml doing git-submodule vendoring. Only the Dependabot section is accurate. Please rewrite the description to match the submodule-sync mechanism before merge.
| TARGET="${CANDIDATE}" | ||
| break | ||
| fi | ||
| done |
There was a problem hiding this comment.
Two structural points on the compat guards. (1) They reach across the repo boundary into the submodule's internal layout (${SUBMODULE_PATH}/python/pyproject.toml, ${SUBMODULE_PATH}/.nvmrc), so any upstream repackaging (moved file, setup.cfg, dropped requires-python) hard-fails every future sync until this workflow is edited. (2) Both guards run before the step-5 git diff --quiet no-op check, so a genuinely no-op re-dispatch does the extra work and can fail loudly with nothing to sync. Consider reordering the no-op detection ahead of the drift guards, and treating an unparseable floor as warn-and-skip.
| node-version-file: .nvmrc | ||
|
|
||
| # Expose a `retry` helper to every later bash step via BASH_ENV so the | ||
| # network operations below (fetch/push) stay readable one-liners. |
There was a problem hiding this comment.
The branch-exists guard treats any git ls-remote failure as "branch absent" — a transient network error falls into the else → exists=false → the workflow proceeds to create and push. Unlike the later network ops, this ls-remote is not retried. Consider wrapping it in retry, or distinguishing exit code 2 (no ref) from a genuine failure, so a network blip can't bypass the guard.
|
Overall this is sound. Two themes worth addressing before merge: (1) the Node/Python compatibility guards in the sync workflow validate against the submodule's exact dev pins ( |
Address reviewer feedback on the sync workflow: - Retry the branch-existence ls-remote and only treat a definitive 'no such ref' (exit 2) as absent, so a transient network error can no longer bypass the guard and clobber an existing branch. - Run the no-op detection before the compatibility guards so a re-dispatch with nothing to sync exits cleanly instead of failing loudly. - Compare the Python floor against the extension's declared minimum in src/common/constants.ts (not the bundling interpreter in runtime.txt), and the Node floor against the shared package's engines.node build floor (not the exact .nvmrc dev pin). - Treat an unparseable package floor as a warning instead of hard-failing every future sync. - Wrap 'npm install --package-lock-only' in the retry helper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🔒 Automated review in progress — @rchiodo is auto-reviewing this PR. |
| set +e | ||
| git ls-remote --exit-code --heads origin "refs/heads/${BRANCH}" >/dev/null 2>&1 | ||
| rc=$? | ||
| set -e |
There was a problem hiding this comment.
📍 .github/workflows/shared-package-submodule-sync.yml:88
Notification gap on the guard-skip path: if git push succeeds but the runner dies before the "Open or update the tracking issue" step, a re-dispatch hits the branch-exists guard (exists=true) and skips all gated steps — so the pushed branch is left with no tracking issue or compare URL. The guard conflates "maintainer is editing" with "prior run crashed mid-flight." On the exists=true path, consider re-emitting the compare URL to $GITHUB_STEP_SUMMARY (and/or checking for an existing tracking issue) before finishing. Low priority; the guard-and-skip itself is correct.
|
Overall this is sound. One real structural gap worth tracking (submodule Python runtime deps are satisfied only by hand-pins in requirements.in with no drift enforcement) plus a stale PR description. Nothing blocking. |
Point external/vscode-common-python-lsp at the v0.8.1 release and refresh package-lock.json to keep the branch npm ci-mergeable. Python (>=3.10) and Node (engines.node >=18.0.0) compatibility floors are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🔒 Automated review in progress — @rchiodo is auto-reviewing this PR. |
|
|
||
| # The extension must not claim support for an older Python than the shared | ||
| # package requires, or the bundled library would fail to import at runtime. | ||
| # Compare the package floor against the extension's *declared* minimum in |
There was a problem hiding this comment.
📍 .github/workflows/shared-package-submodule-sync.yml:145
The Node guard parses PKG_NODE as the first integer sequence in engines.node, which has two latent edge cases: (1) sort -V orders a bare-major .nvmrc (e.g. 18) before a full 18.0.0 floor, spuriously hard-failing a compatible sync; and (2) a compound range like >=20 || ^18.18.0 or >=18 <=20 is mis-classified (first-integer-wins, no upper-bound check). Neither trips today (.nvmrc is full 24.17.0, engines.node is open-ended >=18.0.0). Normalize both operands to a common component count (or use semver) if you want to harden the floor comparison.
|
Overall this looks solid and is safe to merge. A few optional hardening notes are left inline (Python dep drift, workflow version-parsing edge cases, and a crash-window notification gap); none are blocking. |
Summary
Two related changes so the shared package (
@vscode/common-python-lspnpm /vscode-common-python-lsppip) is updated via arepository_dispatchevent fromvscode-common-python-lspinstead of by Dependabot.1. New
.github/workflows/shared-package-release.ymlHandles
repository_dispatch(shared-package-release): branches offmain, bumps the npm dependency, rewrites the pip pin inrequirements.inand recompilesrequirements.txtwithuv, then pushes the branch and opens a tracking issue with a manual compare/PR link (org settings prevent the workflow from opening PRs automatically).2.
.github/dependabot.ymlMoves the shared package from the group
exclude-patterns(which still produced a standalone Dependabot PR) into the npm and pipignorelists, so Dependabot no longer opens duplicate PRs for it.