From 98f729d70d3170afec7c0e950ca512e18de072f8 Mon Sep 17 00:00:00 2001 From: Elijah Zupancic Date: Wed, 3 Jun 2026 15:40:00 -0700 Subject: [PATCH 1/3] feat(ci): publish bca CLI to PyPI as pip wheel Add a standalone python-cli-wheels.yml workflow plus a -b bin pyproject.toml so `pip install big-code-analysis-cli` drops the compiled `bca` binary onto PATH (no Rust toolchain), the way `pip install ruff` installs `ruff`. The dist name (big-code-analysis-cli) differs from the installed command (bca) and from the importable library bindings (big-code-analysis); README, book, and CHANGELOG state the split explicitly. Mirrors python-wheels.yml: numeric-tag trigger, opt-in PR label, aggregate gate, PyPI Trusted Publishing in a distinct `pypi-cli` environment. Differs in shipping a py3-none- bin wheel (no abi3 / no per-Python matrix) across Linux manylinux_2_28 (x86_64/aarch64), macOS (x86_64/arm64), and Windows (x86_64). The CLI crate already pins all-languages (#252), so the bin build carries every grammar with no maturin feature wiring. Per-binary THIRD-PARTY-LICENSES-bca.md (cargo-about) + LICENSE ride in .dist-info/licenses/ via [project].license-files; man pages are bundled via [tool.maturin] include (globbed so clean builds and sdist still work). Staged artefacts are gitignored. RELEASING.md documents the one-time PyPI Trusted Publisher and pypi-cli environment setup. Fixes #408 --- .github/workflows/python-cli-wheels.yml | 424 ++++++++++++++++++ .gitignore | 10 + CHANGELOG.md | 14 + RELEASING.md | 92 +++- big-code-analysis-book/src/commands/README.md | 33 ++ big-code-analysis-cli/README.md | 31 +- big-code-analysis-cli/pyproject.toml | 60 +++ 7 files changed, 662 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/python-cli-wheels.yml create mode 100644 big-code-analysis-cli/pyproject.toml diff --git a/.github/workflows/python-cli-wheels.yml b/.github/workflows/python-cli-wheels.yml new file mode 100644 index 00000000..51400986 --- /dev/null +++ b/.github/workflows/python-cli-wheels.yml @@ -0,0 +1,424 @@ +name: Python CLI wheels + +# Build, smoke-test, and publish pip-installable wheels for the `bca` +# command-line tool (distribution name `big-code-analysis-cli`, console +# command `bca`). See #408. +# +# This is the binary-distribution sibling of `python-wheels.yml`, which +# ships the PyO3 *library* bindings (`big-code-analysis`). The two are +# independent PyPI projects that happen to share maturin as a build +# backend: +# +# * `python-wheels.yml` → `import big_code_analysis` (extension wheel, +# abi3, one wheel per arch covering every CPython 3.x). +# * this workflow → `bca` on PATH (a `-b bin` wheel: the compiled +# binary is packaged as a console script). +# +# Design notes specific to the bin wheel: +# +# * **Not an extension module — abi3 is irrelevant.** A bin wheel is +# tagged `py3-none-`: one wheel per (OS, arch), but a single +# wheel covers every CPython 3.x (and PyPy) on that platform. There is +# no per-Python-version matrix and no libpython link. +# +# * **Bindings mode is fixed in `big-code-analysis-cli/pyproject.toml`** +# (`[tool.maturin] bindings = "bin"`). The crate is already a single +# `[[bin]] name = "bca"` with no pyo3/cdylib target, which is exactly +# the shape maturin auto-detects for bin bindings; the explicit key +# guards against a future PyO3 addition silently flipping the mode. +# +# * **Full grammar set is inherited from the crate.** The CLI crate +# pins `big-code-analysis` with `default-features = false, features = +# ["all-languages"]` (see #252), so a plain `-b bin` build already +# compiles every grammar in — no `[tool.maturin] features` wiring is +# needed here, and a default-features build cannot silently drop a +# grammar. +# +# * **manylinux_2_28** floor, matching `python-wheels.yml`: the MSRV +# (1.94) toolchain lives in the 2_28 container and RHEL/CentOS Stream +# 8 (glibc 2.28) is the oldest realistic target. +# +# * **Compliance artefacts ride in the wheel.** The per-binary +# `THIRD-PARTY-LICENSES-bca.md` (rendered by cargo-about, the same +# tool `release.yml` uses for the deb/rpm/archive TPLs) plus the +# workspace `LICENSE` are staged into the crate directory and picked +# up by `[project].license-files`, landing in the wheel's standard +# `.dist-info/licenses/`. The `bca` man pages are bundled for +# reference via `[tool.maturin] include`. maturin additionally emits a +# CycloneDX SBOM into `.dist-info/sboms/` automatically. +# +# * **PyPI Trusted Publishing** (OIDC), no long-lived token — identical +# posture to `python-wheels.yml`. The deployment environment is +# `pypi-cli`, intentionally distinct from the library's `pypi` +# environment so the two projects' Trusted-Publisher OIDC claims +# (matched on repo + workflow filename + environment) do not overlap. + +on: + push: + # Numeric-prefix glob (not bare `v*`) so a word-prefixed debugging tag + # cannot reach the publish job — mirrors python-wheels.yml. + tags: ['v[0-9]*'] + pull_request: + paths: + - 'big-code-analysis-cli/**' + - '.github/workflows/python-cli-wheels.yml' + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: ${{ github.ref != 'refs/heads/main' && !startsWith(github.ref, 'refs/tags/') }} + +# Every `uses:` is pinned to a commit SHA (trailing comment = the tag it +# points at); Dependabot bumps both in lockstep. SHAs are kept in sync +# with python-wheels.yml / release.yml. +env: + CARGO_TERM_COLOR: always + CARGO_INCREMENTAL: 0 + CARGO_NET_RETRY: 10 + RUSTUP_MAX_RETRIES: 10 + PYTHON_VERSION_HOST: "3.12" + # cargo-about renders the per-binary TPL; pinned to the version + # release.yml uses so the deb/rpm/archive/wheel TPLs are byte-identical + # for a given tag. + CARGO_ABOUT_VERSION: "0.8.4" + +jobs: + # Build the wheel matrix. PR-time builds are opt-in via the + # `python-cli-wheels` label to keep the cost off Rust-only PRs that + # merely brush a path-filter neighbour. Tag pushes and + # workflow_dispatch always run. + build: + name: build (${{ matrix.target }}) + if: >- + github.event_name != 'pull_request' + || contains(github.event.pull_request.labels.*.name, 'python-cli-wheels') + strategy: + fail-fast: false + matrix: + include: + - target: x86_64-unknown-linux-gnu + runs-on: ubuntu-latest + manylinux: "2_28" + wheel_tag: manylinux_2_28_x86_64 + - target: aarch64-unknown-linux-gnu + runs-on: ubuntu-24.04-arm + manylinux: "2_28" + wheel_tag: manylinux_2_28_aarch64 + - target: x86_64-apple-darwin + runs-on: macos-latest + manylinux: "auto" + wheel_tag: macosx + - target: aarch64-apple-darwin + runs-on: macos-latest + manylinux: "auto" + wheel_tag: macosx + - target: x86_64-pc-windows-msvc + runs-on: windows-latest + manylinux: "auto" + wheel_tag: win_amd64 + runs-on: ${{ matrix.runs-on }} + timeout-minutes: 45 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + fetch-depth: 1 + submodules: false + + - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 + with: + python-version: ${{ env.PYTHON_VERSION_HOST }} + + # Host toolchain — needed for the cargo-about staging step below. + # The Linux legs build the binary inside maturin-action's + # manylinux_2_28 container (its own toolchain), but cargo-about runs + # on the host; the `targets:` add lets cargo-about's + # `--filter-platform` resolve a cross target (e.g. x86_64 on the + # arm64 macOS runner) without a full target install. + - uses: dtolnay/rust-toolchain@3c5f7ea28cd621ae0bf5283f0e981fb97b8a7af9 # stable tip + with: + toolchain: stable + targets: ${{ matrix.target }} + + - name: Install cargo-about + uses: taiki-e/install-action@50b4a718b59c718df4ef27a3b445f86cd57b9f00 # v2.80.0 + with: + tool: cargo-about@${{ env.CARGO_ABOUT_VERSION }} + + # Stage the compliance + doc artefacts into the CLI crate directory + # so maturin picks them up (TPL + LICENSE via + # `[project].license-files`; man pages via `[tool.maturin] include`). + # These are .gitignore'd; the canonical sources live at the repo + # root and `man/`. cargo-about is target-filtered so the TPL reflects + # the dependency closure actually shipped in this wheel. + - name: Stage license + man-page artefacts + shell: bash + env: + TARGET: ${{ matrix.target }} + run: | + set -euo pipefail + cargo about generate --locked \ + --config about.toml \ + --target "$TARGET" \ + --manifest-path big-code-analysis-cli/Cargo.toml \ + about.hbs \ + > big-code-analysis-cli/THIRD-PARTY-LICENSES-bca.md + test -s big-code-analysis-cli/THIRD-PARTY-LICENSES-bca.md + cp LICENSE big-code-analysis-cli/LICENSE + # Bundle only the `bca` pages — `bca-web.1` belongs to the web + # server binary, which this wheel does not ship. + mkdir -p big-code-analysis-cli/man + for f in man/bca*.1; do + [ "$f" = "man/bca-web.1" ] && continue + cp "$f" big-code-analysis-cli/man/ + done + test -f big-code-analysis-cli/man/bca.1 + test ! -f big-code-analysis-cli/man/bca-web.1 + + # maturin-action builds the bin wheel. On Linux it pulls the + # manylinux_2_28 container matching ${{ matrix.target }}; on + # macOS/Windows the `manylinux: auto` value is a no-op and the build + # runs natively. `--strip` drops debug symbols from the binary; + # `--locked` honours the workspace Cargo.lock byte-for-byte so the + # wheel's transitive-dep versions are reproducible from the tag. + # Bindings mode is set in pyproject (`bindings = "bin"`). + - name: Build bin wheel (${{ matrix.target }}) + uses: PyO3/maturin-action@e83996d129638aa358a18fbd1dfb82f0b0fb5d3b # v1.51.0 + with: + working-directory: big-code-analysis-cli + target: ${{ matrix.target }} + manylinux: ${{ matrix.manylinux }} + args: --release --strip --locked --out dist + + # Catch a tag / interpreter / arch regression here instead of at + # PyPI upload time. A bin wheel must be `py3-none-`: the + # `py3-none` segment proves it was not built as a per-version + # extension wheel (the most plausible regression if the pyproject + # bindings key were lost), and `${{ matrix.wheel_tag }}` proves the + # platform tag matches this leg. + - name: Verify wheel is py3-none / ${{ matrix.wheel_tag }} + shell: bash + working-directory: big-code-analysis-cli + env: + WHEEL_TAG: ${{ matrix.wheel_tag }} + run: | + set -euo pipefail + ls -la dist/ + shopt -s nullglob + all=(dist/*.whl) + if [[ ${#all[@]} -ne 1 ]]; then + echo "::error::expected exactly one wheel in dist/, found ${#all[@]}" + exit 1 + fi + name=$(basename "${all[0]}") + case "$name" in + *-py3-none-*"${WHEEL_TAG}"*.whl) : ;; + *) + echo "::error::wheel '$name' is not py3-none-*${WHEEL_TAG}*" + exit 1 + ;; + esac + + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: cli-wheels-${{ matrix.target }} + path: big-code-analysis-cli/dist/*.whl + if-no-files-found: error + + # Source distribution. PyPI fallback for niche architectures and a + # reproducibility anchor. A `pip install` from this sdist rebuilds the + # binary, so it needs a Rust toolchain on the consumer side — expected + # for a bin crate; the prebuilt wheels above cover the common platforms. + sdist: + name: sdist + if: >- + github.event_name != 'pull_request' + || contains(github.event.pull_request.labels.*.name, 'python-cli-wheels') + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + fetch-depth: 1 + submodules: false + + - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 + with: + python-version: ${{ env.PYTHON_VERSION_HOST }} + + # No artefact staging: the TPL/LICENSE/man pages are `format = + # "wheel"` / generated, so they are intentionally absent from the + # sdist (a source snapshot). The workspace Cargo.lock travels in the + # tarball, so a downstream rebuild resolves identical dep versions. + - name: Build sdist + uses: PyO3/maturin-action@e83996d129638aa358a18fbd1dfb82f0b0fb5d3b # v1.51.0 + with: + working-directory: big-code-analysis-cli + command: sdist + args: --out dist + + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: cli-sdist + path: big-code-analysis-cli/dist/*.tar.gz + if-no-files-found: error + + # Install each wheel into a clean venv on a runner of the matching + # platform and exercise the binary end-to-end. The acceptance criteria + # (#408) require `bca --version` and a parse that proves the + # `all-languages` grammar set shipped. The x86_64-apple-darwin wheel is + # cross-built on the arm64 macOS runner and cannot be executed here + # (GitHub's macOS pool is arm64); its structural integrity is covered by + # the build job's verify step, mirroring release.yml's handling of the + # non-executable aarch64-windows lane. + smoke-test: + name: smoke-test (${{ matrix.target }}) + needs: build + strategy: + fail-fast: false + matrix: + include: + - target: x86_64-unknown-linux-gnu + runs-on: ubuntu-latest + - target: aarch64-unknown-linux-gnu + runs-on: ubuntu-24.04-arm + - target: aarch64-apple-darwin + runs-on: macos-latest + - target: x86_64-pc-windows-msvc + runs-on: windows-latest + runs-on: ${{ matrix.runs-on }} + timeout-minutes: 15 + steps: + - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 + with: + python-version: ${{ env.PYTHON_VERSION_HOST }} + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: cli-wheels-${{ matrix.target }} + path: dist + + # Resolve the expected version from the workspace Cargo.toml so the + # assertion is not hard-coded — the wheel version is dynamic + # (maturin reads `version.workspace = true`), and this check proves + # the published binary reports the same lockstep version. + - name: Install wheel + assert bca on PATH + shell: bash + env: + PYTHONUNBUFFERED: "1" + run: | + set -euo pipefail + ls -la dist/ + python -m pip install --upgrade pip + # --no-index makes an accidental PyPI fallback (e.g. a wheel + # whose platform tag does not match this runner) fail loudly + # rather than silently installing a stale published version. + python -m pip install --no-index --find-links=dist big-code-analysis-cli + # The console script lands in the venv/Scripts dir on PATH. + bca --version + bca list-metrics names | head -n1 + # Parse two unrelated languages to prove the all-languages + # grammar set is compiled in, not just the host language. + printf 'def add(a, b):\n if a > b:\n return a\n return b\n' > smoke.py + printf 'fn main() { if true { println!("x"); } }\n' > smoke.rs + py_cc=$(bca --paths smoke.py metrics -O json | python -c "import sys,json; print(json.load(sys.stdin)['metrics']['cyclomatic']['sum'])") + rs_cc=$(bca --paths smoke.rs metrics -O json | python -c "import sys,json; print(json.load(sys.stdin)['metrics']['cyclomatic']['sum'])") + test "$py_cc" = "3.0" || { echo "::error::python cyclomatic.sum=$py_cc, expected 3.0"; exit 1; } + test "$rs_cc" = "3.0" || { echo "::error::rust cyclomatic.sum=$rs_cc, expected 3.0"; exit 1; } + echo "smoke OK" + + # Aggregate gate so branch protection can require a single check name. + # Every dependency is label-gated at PR time, so on an unlabelled PR all + # `needs` resolve to `skipped`; the predicate fails on `skipped` too + # (not just failure/cancelled) so an unlabelled run cannot green-tick + # the required check without having built anything — same wrinkle as + # python-wheels.yml's `wheels` gate. + cli-wheels: + name: cli-wheels + if: always() + needs: + - build + - sdist + - smoke-test + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - name: Fail if any dependency did not succeed + if: >- + contains(needs.*.result, 'failure') + || contains(needs.*.result, 'cancelled') + || contains(needs.*.result, 'skipped') + run: exit 1 + + # PyPI publish — only on a numeric `v*` tag push. Prerelease tags + # (`-rc`/`-beta`/`-alpha`) are skipped to stay aligned with + # release.yml's crates.io policy and python-wheels.yml. The Trusted + # Publisher for `big-code-analysis-cli` must be registered on PyPI + # (repo + this workflow filename + the `pypi-cli` environment) before + # the first tagged release. + publish: + name: publish to PyPI + needs: [build, sdist, smoke-test] + if: >- + github.event_name == 'push' + && startsWith(github.ref, 'refs/tags/v') + && !contains(github.ref, '-rc') + && !contains(github.ref, '-beta') + && !contains(github.ref, '-alpha') + runs-on: ubuntu-latest + environment: + name: pypi-cli + url: https://pypi.org/project/big-code-analysis-cli/ + permissions: + # A job-level `permissions:` block replaces the workflow default + # rather than augmenting it; restate `contents: read` for any future + # step that needs repo access. + contents: read + # Exchanged for a one-off PyPI upload credential — no long-lived + # token. `attestations: write` powers the PEP 740 Sigstore + # attestations the publish action generates by default. + id-token: write + attestations: write + steps: + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + pattern: cli-wheels-* + path: dist + merge-multiple: true + + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: cli-sdist + path: dist + + - name: Inventory artefacts + run: | + set -euo pipefail + ls -la dist/ + # Refuse to publish a half-matrix release: assert every + # platform wheel plus the sdist arrived. A download-artifact + # silently dropping one leg would otherwise leave PyPI users on + # that platform with no wheel. + shopt -s nullglob + require() { + local label=$1; shift + local matches=("$@") + if [[ ${#matches[@]} -eq 0 ]]; then + echo "::error::Missing expected artefact: ${label}" + exit 1 + fi + } + require "manylinux_2_28 x86_64 wheel" dist/*-py3-none-manylinux_2_28_x86_64.whl + require "manylinux_2_28 aarch64 wheel" dist/*-py3-none-manylinux_2_28_aarch64.whl + require "macOS x86_64 wheel" dist/*-py3-none-macosx_*_x86_64.whl + require "macOS arm64 wheel" dist/*-py3-none-macosx_*_arm64.whl + require "Windows x86_64 wheel" dist/*-py3-none-win_amd64.whl + require "sdist tarball" dist/*.tar.gz + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0 + with: + packages-dir: dist + attestations: true diff --git a/.gitignore b/.gitignore index fbe4ac2a..d4ad9649 100644 --- a/.gitignore +++ b/.gitignore @@ -21,3 +21,13 @@ big-code-analysis-py/python/big_code_analysis/_native*.so # tracking policy and the regenerate workflow (`make py-relock`). *.pyc abuild-keys/ + +# CLI pip-wheel build artefacts. `python-cli-wheels.yml` stages the +# generated TPL, a copy of the workspace LICENSE, and the `man/` pages +# into the CLI crate directory and builds wheels under `dist/` there; +# none of those staged copies are source and must not be checked in +# (the canonical originals live at the repo root / `man/`). +big-code-analysis-cli/dist/ +big-code-analysis-cli/LICENSE +big-code-analysis-cli/THIRD-PARTY-LICENSES-bca.md +big-code-analysis-cli/man/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ec40756..fad75f5c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,20 @@ for historical reference. ### Added +- The `bca` CLI is now pip-installable. `pip install big-code-analysis-cli` + drops the compiled `bca` binary onto your `PATH` (no Rust toolchain + required), the way `pip install ruff` installs the `ruff` command. The + PyPI **distribution name is `big-code-analysis-cli`** while the installed + **command stays `bca`** — distinct from the importable library bindings + published as `big-code-analysis`. A new + [`python-cli-wheels.yml`](.github/workflows/python-cli-wheels.yml) + workflow builds `-b bin` wheels for Linux (`manylinux_2_28` `x86_64` / + `aarch64`), macOS (`x86_64` / `arm64`), and Windows (`x86_64`), + smoke-tests each, and publishes to PyPI via Trusted Publishing in + lockstep with the workspace version. Each wheel carries the full + `all-languages` grammar set and bundles the per-binary + `THIRD-PARTY-LICENSES-bca.md` + `LICENSE` (in `.dist-info/licenses/`) + and the `bca` man pages. (#408) - `bca report markdown|html` now honors in-source suppression markers (`bca: suppress`, `bca: suppress-file`, `#lizard forgives`) **by default**, omitting a function from a metric's hotspot table when that diff --git a/RELEASING.md b/RELEASING.md index 95d70160..c407c497 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -639,6 +639,15 @@ turns into a foot-gun on the *next* release. - [ ] **`python-wheels` PR label.** Create the label (see the Python wheels section) so contributors can opt PRs into the wheel matrix. +- [ ] **PyPI Trusted Publisher and `pypi-cli` GH environment (CLI + wheel).** Claim `big-code-analysis-cli` on PyPI via the + pending-publisher flow, registering a TP for workflow + `python-cli-wheels.yml` + environment `pypi-cli`, and create the + `pypi-cli` GitHub environment. See [CLI wheels + (PyPI)](#cli-wheels-pypi). +- [ ] **`python-cli-wheels` PR label.** Create the label (see the CLI + wheels section) so contributors can opt PRs into the CLI wheel + matrix. - [ ] **Shared Homebrew tap reachable.** Confirm `dekobon/homebrew-tap` exists and the configured PAT can push to it. The release workflow appends `Formula/big-code-analysis.rb` @@ -799,7 +808,88 @@ into a production-shaped flow. The wheel pipeline ships Linux only (x86_64 + aarch64). macOS and Windows wheels are tracked separately under [#103](https://github.com/dekobon/big-code-analysis/issues/103)'s -"Out of scope" section. +"Out of scope" section. (This Linux-only scope is for the *library* +bindings wheel above; the **CLI** `bca` wheel — see below — does ship +macOS and Windows.) + +## CLI wheels (PyPI) + +The `bca` command-line tool ships as its own pip-installable +distribution via `.github/workflows/python-cli-wheels.yml`, separate +from both `release.yml` (crates.io / native packages) and +`python-wheels.yml` (the library bindings). All three trigger on the +same `v[0-9]*` tag push and run in parallel; a failure in one does not +block the others. + +This is a `maturin -b bin` wheel: the compiled `bca` binary is packaged +as a console script, so `pip install big-code-analysis-cli` drops `bca` +onto the user's `PATH`. Key differences from the library wheel: + +- **No abi3 / no per-Python matrix.** A bin wheel is tagged + `py3-none-`; one wheel per (OS, arch) covers every CPython + 3.x and PyPy. The matrix is per-platform, not per-Python-version. +- **Distribution name `big-code-analysis-cli`, command `bca`.** The + installed command intentionally differs from the dist name (the `bca` + name on PyPI is taken; `big-code-analysis` is the library bindings). +- **Full grammar set is inherited from the crate** (`all-languages`, via + #252) — no `[tool.maturin] features` wiring. +- **Wider platform matrix:** Linux `manylinux_2_28` (`x86_64` / + `aarch64`), macOS (`x86_64` / `arm64`), Windows (`x86_64`). +- **Compliance artefacts ride in the wheel:** the per-binary + `THIRD-PARTY-LICENSES-bca.md` (cargo-about) and `LICENSE` land in + `.dist-info/licenses/`; the `bca` man pages are bundled; maturin emits + a CycloneDX SBOM into `.dist-info/sboms/`. + +### One-time PyPI setup + +Mirror the library-wheel setup above, with CLI-specific values: + +1. **Claim the project name** at + `https://pypi.org/project/big-code-analysis-cli/` (the + pending-publisher flow reserves the name in the same step as the TP + registration). The name was confirmed available when #408 was filed. + +2. **Register a Trusted Publisher** at + `https://pypi.org/manage/account/publishing/` with: + + - PyPI Project Name: `big-code-analysis-cli`. + - Owner: `dekobon`. + - Repository name: `big-code-analysis`. + - Workflow filename: `python-cli-wheels.yml` (basename only). + - Environment name: `pypi-cli`. + + `pypi-cli` is intentionally distinct from the library's `pypi` + environment and the crates.io `release` environment so each + registry/project's OIDC `environment` claim is unambiguous. + +3. **Create the `pypi-cli` GitHub Environment** (Settings → + Environments → New environment → `pypi-cli`) before the first `v*` + tag if you want a manual approval gate on the first publish — GitHub + auto-creates a referenced-but-undefined environment with no + protection rules otherwise. + +4. **Create the `python-cli-wheels` PR label** so contributors can opt a + PR into the wheel matrix (Rust-only PRs that merely brush a + path-filter neighbour skip it): + + ```bash + gh label create python-cli-wheels \ + --color 1d76db \ + --description "PR opts in to the bca CLI wheel CI matrix" + ``` + + Tag pushes and `workflow_dispatch` runs ignore the label. + +5. **First tagged release validates the path**, exactly as for the + library wheel — Trusted Publishing cannot be rehearsed via + `workflow_dispatch`. + +### Version coupling + +`big-code-analysis-cli` inherits its version from +`[workspace.package] version` (`version.workspace = true`), and its +`pyproject.toml` reads the same value at build time (`dynamic = +["version"]`). No separate version field to maintain. ## Rotating the minisign key diff --git a/big-code-analysis-book/src/commands/README.md b/big-code-analysis-book/src/commands/README.md index 6e3c4513..a3b060b5 100644 --- a/big-code-analysis-book/src/commands/README.md +++ b/big-code-analysis-book/src/commands/README.md @@ -5,6 +5,39 @@ information from source code. Each command **may** include parameters specific to the task it performs. Below, we describe the core types of commands available in **bca**. +## Installation + +The `bca` command-line tool is available as a pip-installable wheel. +The **distribution name is `big-code-analysis-cli`** and the installed +**command is `bca`** — the two differ deliberately (the `bca` name on +PyPI belongs to an unrelated project, and `big-code-analysis` is this +project's importable *library* bindings): + +```bash +pip install big-code-analysis-cli # installs the `bca` command on PATH +bca --version +``` + +This drops the compiled `bca` binary onto your `PATH` the way +`pip install ruff` gives you the `ruff` command — no Rust toolchain +required. The wheel carries the full `all-languages` grammar set, so +every [supported language](../languages.md) works out of the box. A +single `py3-none-` wheel covers every CPython 3.x (and PyPy) +on that platform; prebuilt wheels ship for Linux (`manylinux_2_28` +`x86_64` / `aarch64`), macOS (`x86_64` / `arm64`), and Windows +(`x86_64`). On any other platform `pip` falls back to a source build, +which needs a Rust toolchain. + +This is the binary CLI, distinct from the importable +[Python bindings](../python/installation.md) +(`pip install big-code-analysis`). Other install paths — Homebrew, +`.deb` / `.rpm` / `.apk` packages, prebuilt release archives, or +`cargo install big-code-analysis-cli` — are described in the +repository README. + +The wheel build and publish matrix is defined in +[`.github/workflows/python-cli-wheels.yml`](https://github.com/dekobon/big-code-analysis/blob/main/.github/workflows/python-cli-wheels.yml). + ## Metrics Metrics provide quantitative measures about source code, which can help in: diff --git a/big-code-analysis-cli/README.md b/big-code-analysis-cli/README.md index 744d2a57..d49c33ed 100644 --- a/big-code-analysis-cli/README.md +++ b/big-code-analysis-cli/README.md @@ -9,9 +9,38 @@ aggregated reports, AST dumps, node lookups, and more. ## Installation +### From PyPI (pip) + +The fastest path on Linux, macOS, and Windows — no Rust toolchain +required: + +```sh +pip install big-code-analysis-cli # installs the `bca` command on PATH +bca --version +``` + +Note the deliberate split between the **distribution name** and the +**command name**: you `pip install big-code-analysis-cli`, but the +installed executable is `bca`. The `bca` name on PyPI belongs to an +unrelated project, and `big-code-analysis` is this project's importable +*library* bindings (`pip install big-code-analysis` — a different +deliverable). The wheel ships the full `all-languages` grammar set; a +single `py3-none-` wheel covers every CPython 3.x (and PyPy) +on that platform. Prebuilt wheels are published for Linux +(`manylinux_2_28` `x86_64` / `aarch64`), macOS (`x86_64` / `arm64`), and +Windows (`x86_64`); other platforms fall back to a source build. + +### From crates.io (cargo) + +```sh +cargo install big-code-analysis-cli +``` + +### From source + ```sh cd big-code-analysis-cli/ -cargo build +cargo build --release ``` ## Usage diff --git a/big-code-analysis-cli/pyproject.toml b/big-code-analysis-cli/pyproject.toml new file mode 100644 index 00000000..e0c2c5bd --- /dev/null +++ b/big-code-analysis-cli/pyproject.toml @@ -0,0 +1,60 @@ +[build-system] +requires = ["maturin>=1.7,<2.0"] +build-backend = "maturin" + +[project] +name = "big-code-analysis-cli" +description = "The bca command-line tool: compute and export code metrics for many languages" +readme = "README.md" +# A `-b bin` wheel carries a compiled binary launched as a console +# script, not a CPython extension module — there is no Python ABI to +# match, so a single `py3-none-` wheel serves every CPython +# 3.x and PyPy on that platform. The floor is therefore set low and +# generously (3.8) rather than tracking the library bindings' 3.12 +# requirement; nothing here imports Python. +requires-python = ">=3.8" +license = "MPL-2.0" +license-files = ["LICENSE", "THIRD-PARTY-LICENSES-bca.md"] +authors = [ + { name = "Calixte Denizet", email = "cdenizet@mozilla.com" }, + { name = "Elijah Zupancic", email = "elijah@zupancic.name" }, +] +keywords = ["metrics", "tree-sitter", "static-analysis", "cli"] +classifiers = [ + "Development Status :: 5 - Production/Stable", + "Environment :: Console", + "Intended Audience :: Developers", + "Programming Language :: Rust", + "Topic :: Software Development :: Libraries", + "Topic :: Software Development :: Quality Assurance", +] +# Version is read from the Rust workspace's Cargo.toml at build time by +# maturin (the package inherits `version.workspace = true`), keeping the +# wheel version in lockstep with the crate / library / web versions. +dynamic = ["version"] + +[project.urls] +Repository = "https://github.com/dekobon/big-code-analysis" +Documentation = "https://dekobon.github.io/big-code-analysis/" + +[tool.maturin] +# `bin` bindings package the compiled `bca` binary into the wheel as a +# console script so `pip install big-code-analysis-cli` drops `bca` onto +# the user's PATH. maturin auto-detects this mode for a single-binary +# crate with no pyo3 / cdylib target (which this crate is), but it is +# spelled out so the contract is explicit and a future PyO3 addition to +# the crate cannot silently flip the build into extension-module mode. +bindings = "bin" +# The `bca` man pages (workspace `man/`, generated by `cargo xtask`) are +# copied into this crate directory at wheel-build time by +# `.github/workflows/python-cli-wheels.yml` and bundled for reference. +# They are listed as a glob — not literal paths — so a clean local +# `maturin build` / `maturin sdist`, where the staged copies are absent, +# silently omits them instead of failing. `format = "wheel"` keeps them +# out of the sdist (a source snapshot; a downstream rebuild regenerates +# them from the clap definitions via `cargo xtask`). The compliance +# artefacts (LICENSE + per-binary THIRD-PARTY-LICENSES-bca.md) are NOT +# bundled here — they ride in `[project].license-files`, which places +# them in the wheel's standard `.dist-info/licenses/` directory and +# records `License-File:` metadata. +include = [{ path = "man/*.1", format = "wheel" }] From 059178509b32f1780010407f9396ced0ed5e1e8a Mon Sep 17 00:00:00 2001 From: Elijah Zupancic Date: Wed, 3 Jun 2026 17:12:57 -0700 Subject: [PATCH 2/3] docs(release): wire PyPI wheel publishes into per-release flow The release narrative described only release.yml. Note that the same v* tag also fires python-wheels.yml and python-cli-wheels.yml in parallel, that they publish in lockstep on every bump, that prerelease tags skip the PyPI publish (matching crates.io), and add a post-release step to verify both wheels landed. Fixes #408 --- RELEASING.md | 57 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 54 insertions(+), 3 deletions(-) diff --git a/RELEASING.md b/RELEASING.md index c407c497..459ee754 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -86,6 +86,23 @@ If any stage fails, nothing downstream runs. `publish` and repo; they run in parallel so a crates.io failure does not block the GitHub Release's `verify` step (and vice versa). +The **same `v*` tag push also triggers two independent PyPI wheel +workflows** that are *not* part of `release.yml` and run fully in +parallel with it (a failure in one does not block the others): + +- **`python-wheels.yml`** publishes the importable library bindings + (`big-code-analysis`) — an abi3 extension wheel. See + [Python wheels (PyPI)](#python-wheels-pypi). +- **`python-cli-wheels.yml`** publishes the `bca` command-line tool + (`big-code-analysis-cli`) — a `-b bin` wheel that drops `bca` onto + `PATH`. See [CLI wheels (PyPI)](#cli-wheels-pypi). + +Both read the workspace version (`dynamic = ["version"]`), so they +publish in lockstep with the crates above on every bump — no separate +version step. Their one-time Trusted-Publisher setup is in the +[Post-public-release checklist](#post-public-release-checklist); after +that they fire automatically on each tag. + ## Defer-and-gate state for public publication The repository is staging for a future public release. Until the @@ -550,15 +567,25 @@ git tag -a v1.2.0 -m "v1.2.0" git push origin v1.2.0 ``` -That's it — the push of the tag triggers `release.yml`. Watch it in -the Actions tab: +That's it — the push of the tag triggers `release.yml` **and** the two +PyPI wheel workflows (`python-wheels.yml` for the library bindings, +`python-cli-wheels.yml` for the `bca` CLI), all in parallel. Watch all +three in the Actions tab: ```bash gh run watch -# or +# or, per workflow: gh run list --workflow=Release +gh run list --workflow="Python wheels" +gh run list --workflow="Python CLI wheels" ``` +The wheel workflows publish to PyPI automatically once their one-time +Trusted Publishers are registered (see the +[Post-public-release checklist](#post-public-release-checklist)); no +per-release action beyond the tag is needed. Confirm both wheels landed +in [Post-release verification](#post-release-verification). + ## Cutting a pre-release Pre-release tags match `vX.Y.Z-` where `` is @@ -579,6 +606,12 @@ Signed artefacts, SBOMs, and SLSA provenance still publish normally, so a pre-release is a full test of everything except the external pushes. +The two PyPI wheel workflows follow the same policy: a `-rc` / `-beta` +/ `-alpha` tag still builds and smoke-tests every wheel but **skips the +PyPI publish step**, so a pre-release never lands a wheel on PyPI. The +crates.io and PyPI postures stay aligned — one tag cannot publish a +prerelease to one registry while skipping the other. + ## Post-release verification The pipeline's own `verify` job downloads the musl tarball from the @@ -612,6 +645,24 @@ once the corresponding gating variable is on): - Scoop bucket: new commit on `dekobon/scoop-bucket` touching `bucket/big-code-analysis.json`. +Confirm both PyPI wheels published at the new version (these ship on +every tag once their Trusted Publishers are registered). Either check +the project pages — and + — or verify the CLI +end-to-end from a clean environment: + +```bash +VERSION=0.1.0 +python -m venv /tmp/bca-rel && . /tmp/bca-rel/bin/activate +# Library bindings (importable module): +pip install "big-code-analysis==${VERSION}" +python -c "import big_code_analysis as bca; print(bca.__version__)" +# CLI tool (drops `bca` on PATH): +pip install "big-code-analysis-cli==${VERSION}" +bca --version # must print the tagged version +deactivate +``` + ## Post-public-release checklist The first time the repository goes public and a stable release is From a84904b1e45f82a1af1916bf8f148691ff92f7c9 Mon Sep 17 00:00:00 2001 From: Elijah Zupancic Date: Wed, 3 Jun 2026 17:30:59 -0700 Subject: [PATCH 3/3] fix(ci): address review findings on CLI wheel workflow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - smoke-test: replace `bca list-metrics names | head -n1` with a redirect to /dev/null — the pipe SIGPIPEs the producer under pipefail (the anti-pattern release.yml documents). - smoke-test: implement the version-parity assertion the comment promised — on a tag build, assert `bca --version` contains the tag version; skip on PR/dispatch (no tag, no checkout). - build/verify: make the macOS per-leg wheel check arch-specific (`macosx_*_x86_64` / `macosx_*_arm64`) so a cross-build emitting the host arch fails at the leg, not only at the publish inventory. - build: pin the host toolchain (RUST_TOOLCHAIN 1.94.0, matching release.yml) so macOS/Windows wheel binaries are reproducible instead of tracking a floating stable. - publish: widen the prerelease guard to `!contains(github.ref, '-')` to match release.yml's `*-*` rule exactly, so a single tag cannot land a prerelease on PyPI while skipping crates.io; RELEASING.md updated to describe the actual guards. - fix stale "venv" wording in smoke-test comments (no venv is created). Fixes #408 --- .github/workflows/python-cli-wheels.yml | 80 +++++++++++++++++-------- RELEASING.md | 14 +++-- 2 files changed, 65 insertions(+), 29 deletions(-) diff --git a/.github/workflows/python-cli-wheels.yml b/.github/workflows/python-cli-wheels.yml index 51400986..361a90cb 100644 --- a/.github/workflows/python-cli-wheels.yml +++ b/.github/workflows/python-cli-wheels.yml @@ -80,6 +80,12 @@ env: CARGO_NET_RETRY: 10 RUSTUP_MAX_RETRIES: 10 PYTHON_VERSION_HOST: "3.12" + # Pinned toolchain so the macOS / Windows wheel binaries (built on the + # host, outside maturin's manylinux container) are reproducible from a + # tag rather than tracking a floating `stable`. Matches release.yml's + # RUST_TOOLCHAIN; the Linux legs build inside the 2_28 container with + # its own toolchain, unaffected by this value. + RUST_TOOLCHAIN: "1.94.0" # cargo-about renders the per-binary TPL; pinned to the version # release.yml uses so the deb/rpm/archive/wheel TPLs are byte-identical # for a given tag. @@ -110,11 +116,14 @@ jobs: - target: x86_64-apple-darwin runs-on: macos-latest manylinux: "auto" - wheel_tag: macosx + # Glob (the macOS minor version varies) so the verify step + # below distinguishes x86_64 from arm64 — a bare `macosx` + # would accept a wrong-arch wheel from a cross-build slip. + wheel_tag: 'macosx_*_x86_64' - target: aarch64-apple-darwin runs-on: macos-latest manylinux: "auto" - wheel_tag: macosx + wheel_tag: 'macosx_*_arm64' - target: x86_64-pc-windows-msvc runs-on: windows-latest manylinux: "auto" @@ -139,7 +148,7 @@ jobs: # arm64 macOS runner) without a full target install. - uses: dtolnay/rust-toolchain@3c5f7ea28cd621ae0bf5283f0e981fb97b8a7af9 # stable tip with: - toolchain: stable + toolchain: ${{ env.RUST_TOOLCHAIN }} targets: ${{ matrix.target }} - name: Install cargo-about @@ -196,8 +205,11 @@ jobs: # PyPI upload time. A bin wheel must be `py3-none-`: the # `py3-none` segment proves it was not built as a per-version # extension wheel (the most plausible regression if the pyproject - # bindings key were lost), and `${{ matrix.wheel_tag }}` proves the - # platform tag matches this leg. + # bindings key were lost), and `${{ matrix.wheel_tag }}` (which + # carries the architecture, e.g. `macosx_*_x86_64`) proves the + # platform AND arch match this leg — so a cross-build that silently + # emitted the host arch fails here rather than only at the publish + # job's inventory. - name: Verify wheel is py3-none / ${{ matrix.wheel_tag }} shell: bash working-directory: big-code-analysis-cli @@ -213,8 +225,12 @@ jobs: exit 1 fi name=$(basename "${all[0]}") + # WHEEL_TAG is matrix-controlled and may contain a `*` glob + # (the macOS minor version varies), so it is intentionally + # unquoted here to act as a case pattern, not a literal. + # shellcheck disable=SC2254 case "$name" in - *-py3-none-*"${WHEEL_TAG}"*.whl) : ;; + *-py3-none-*${WHEEL_TAG}*.whl) : ;; *) echo "::error::wheel '$name' is not py3-none-*${WHEEL_TAG}*" exit 1 @@ -265,8 +281,9 @@ jobs: path: big-code-analysis-cli/dist/*.tar.gz if-no-files-found: error - # Install each wheel into a clean venv on a runner of the matching - # platform and exercise the binary end-to-end. The acceptance criteria + # Install each wheel into a fresh runner Python environment on a runner + # of the matching platform and exercise the binary end-to-end. The + # acceptance criteria # (#408) require `bca --version` and a parse that proves the # `all-languages` grammar set shipped. The x86_64-apple-darwin wheel is # cross-built on the arm64 macOS runner and cannot be executed here @@ -300,14 +317,14 @@ jobs: name: cli-wheels-${{ matrix.target }} path: dist - # Resolve the expected version from the workspace Cargo.toml so the - # assertion is not hard-coded — the wheel version is dynamic - # (maturin reads `version.workspace = true`), and this check proves - # the published binary reports the same lockstep version. - name: Install wheel + assert bca on PATH shell: bash env: PYTHONUNBUFFERED: "1" + # On a tag push this is the release version (`vX.Y.Z`); empty + # on PR / workflow_dispatch runs, where there is no tag to + # compare against. + EXPECTED_TAG: ${{ github.ref_type == 'tag' && github.ref_name || '' }} run: | set -euo pipefail ls -la dist/ @@ -316,9 +333,23 @@ jobs: # whose platform tag does not match this runner) fail loudly # rather than silently installing a stale published version. python -m pip install --no-index --find-links=dist big-code-analysis-cli - # The console script lands in the venv/Scripts dir on PATH. - bca --version - bca list-metrics names | head -n1 + # The console script lands in the setup-python interpreter's + # scripts dir, which is on PATH. Capture the version rather than + # piping to `head` (a `| head -n1` would SIGPIPE the producer + # under pipefail — see release.yml's note). + ver_out=$(bca --version) + echo "$ver_out" + # On a tag build, prove the binary reports the lockstep release + # version (maturin reads `version.workspace = true`, so the + # wheel version must equal the tag minus its leading `v`). + if [[ -n "$EXPECTED_TAG" ]]; then + want="${EXPECTED_TAG#v}" + case "$ver_out" in + *"$want"*) : ;; + *) echo "::error::bca --version '$ver_out' does not contain tag version $want"; exit 1 ;; + esac + fi + bca list-metrics names >/dev/null # Parse two unrelated languages to prove the all-languages # grammar set is compiled in, not just the host language. printf 'def add(a, b):\n if a > b:\n return a\n return b\n' > smoke.py @@ -352,21 +383,22 @@ jobs: || contains(needs.*.result, 'skipped') run: exit 1 - # PyPI publish — only on a numeric `v*` tag push. Prerelease tags - # (`-rc`/`-beta`/`-alpha`) are skipped to stay aligned with - # release.yml's crates.io policy and python-wheels.yml. The Trusted - # Publisher for `big-code-analysis-cli` must be registered on PyPI - # (repo + this workflow filename + the `pypi-cli` environment) before - # the first tagged release. + # PyPI publish — only on a numeric `v*` tag push with NO pre-release + # suffix. `!contains(github.ref, '-')` mirrors release.yml's prerelease + # rule exactly (it classifies any tag matching `*-*` as a prerelease + # and skips the crates.io publish), so a single tag cannot land a + # prerelease on PyPI while skipping crates.io. A release tag + # (`refs/tags/v1.2.0`) has no hyphen; any suffix (`-rc1`, `-beta2`, + # `-pre1`, …) does. The Trusted Publisher for `big-code-analysis-cli` + # must be registered on PyPI (repo + this workflow filename + the + # `pypi-cli` environment) before the first tagged release. publish: name: publish to PyPI needs: [build, sdist, smoke-test] if: >- github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') - && !contains(github.ref, '-rc') - && !contains(github.ref, '-beta') - && !contains(github.ref, '-alpha') + && !contains(github.ref, '-') runs-on: ubuntu-latest environment: name: pypi-cli diff --git a/RELEASING.md b/RELEASING.md index 459ee754..665ea7f8 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -606,11 +606,15 @@ Signed artefacts, SBOMs, and SLSA provenance still publish normally, so a pre-release is a full test of everything except the external pushes. -The two PyPI wheel workflows follow the same policy: a `-rc` / `-beta` -/ `-alpha` tag still builds and smoke-tests every wheel but **skips the -PyPI publish step**, so a pre-release never lands a wheel on PyPI. The -crates.io and PyPI postures stay aligned — one tag cannot publish a -prerelease to one registry while skipping the other. +Both PyPI wheel workflows still **build and smoke-test** every wheel on +a pre-release tag but **skip the PyPI publish step**, so a pre-release +never lands a wheel on PyPI. The CLI wheel (`python-cli-wheels.yml`) +skips publish for *any* hyphenated suffix — `!contains(github.ref, +'-')`, matching `release.yml`'s `*-*` prerelease rule exactly; the +library wheel (`python-wheels.yml`) skips the recognised `-rc` / +`-beta` / `-alpha` suffixes. For the suffixes this project actually +uses (above), all three pipelines stay aligned — one tag cannot publish +a prerelease to one registry while skipping another. ## Post-release verification