ci: detect public API breakage with swift-api-digester - #1090
Merged
Conversation
Adds an `api-check` job that builds LiveKit for distribution at HEAD and at the PR base, dumps each module's API surface with `swift-api-digester -dump-sdk`, and diagnoses the delta. Anything removed or changed incompatibly fails the job and lands in the run summary. Runs for macOS and iOS, since a good chunk of the public surface is behind `#if os(iOS)`. Both builds share one package cache so the WebRTC xcframework is downloaded once. swiftlang's `soundness` workflow can't be reused: its api-breakage job is pinned to `ubuntu-latest`, and `swift package diagnose-api-breaking-changes` fails on this package even on macOS — SwiftPM gives the digester no framework search path for the xcframework dependencies, so the module won't load. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Temporary. Drops a public conformance and its `id` property so the new `api-check` job reports a real breakage on CI. Revert before merging. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Matches how cocoapods-lint and codeql are already split out: self-contained PR gate, own tool install, no artifacts shared with build-and-test. Being its own workflow buys three things it can't have inside ci.yaml: `workflow_dispatch` inputs, so any two refs can be compared on demand; a `paths` filter, so the repo's most expensive job is skipped for pull requests that can't move the public API; and no push trigger, since the check is a comparison and only a pull request carries a base. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The digester only reports what a consumer would trip over, never additions. Diffing the other way round surfaces them: whatever disappears going from HEAD back to the base is new in HEAD. A reversed report is not purely additive, though — changed types, renames and the like appear mirrored in both directions — so only the two sections that mean "present here, absent there" are read as additions, with their wording flipped back. Breaking changes still fail the job; additions are reported for review, and an additions-only run now writes a summary and passes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Temporary, alongside the Identifiable removal. Also demonstrates the known blind spot: the new case shifts every later @objc raw value and the digester reports only the addition, never the shift. Revert before merging. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hiroshihorie
approved these changes
Aug 18, 2026
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
A checkout leaves only the checked-out branch as a local ref; every other branch arrives as `origin/<name>`, and `git worktree add --detach` disables the DWIM that would otherwise resolve the bare name. So a manually dispatched run died on `fatal: invalid reference: main` unless it happened to be dispatched from main. Try the ref as given, then `origin/<ref>`. Only branches were affected — the pull_request path passes a SHA, and a checkout with fetch-depth 0 fetches `+refs/tags/*:refs/tags/*` regardless of `fetch-tags: false`, so tags resolved already. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Protocol Conformance Change is not one-directional. Besides the removals the reversed report is read for, it files diagnostics that are themselves worded as additions — `has added inherited protocol`, `has added a conformance to an existing protocol` — plus `is now/no longer an optional requirement`. All are breaking, and none were touched by the relabelling, so a mirror of a forward finding could print verbatim under "Added". Gate on the relabelling instead: a line survives only if inverting it actually changed the text. What can't be inverted is dropped, and the forward report already covers it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ShellOut joins its arguments into one shell command without quoting them, so a dispatched `base` reached git as source text. `nosuchref; touch FILE` created the file — and then resolve() accepted the ref, because the injected command exited 0 and ShellOut only sees the exit status, so the build went on into a worktree that was never created. Two changes: the workflow passes the value through env instead of interpolating it into the run script, so it is never substituted as shell text; and the ref is checked against the characters git actually permits before any shell call. That excludes nothing valid — git rejects refs containing space, `~`, `^`, `:`, `?`, `*`, `[` or `\` — and a bad ref now fails with a message naming it. Only `base` was ever user-supplied; the platform comes from the matrix literal. Dispatching needs write access, so this was never reachable from a fork. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an
api-checkworkflow that catches accidental public API changes on every PR.It builds LiveKit for distribution (
BUILD_LIBRARY_FOR_DISTRIBUTION=YES) at HEAD and at the PR base, dumps each module's API surface withswift-api-digester -dump-sdk, and diagnoses the delta both ways round. Breaking changes fail the job; additions are reported alongside them for review. Runs for macOS and iOS, since a good chunk of the public surface sits behind#if os(iOS); both builds share one package cache so the WebRTC xcframework is downloaded once. ~5 min per leg..github/api-check/api-check.swiftfollows the sameswift-sh+swiftly run +xcodeshape as.github/size-check. It's a standalone workflow rather than aci.yamljob, matching howcocoapods-lintandcodeqlare already split out — which also buysworkflow_dispatchinputs for comparing any two refs on demand, and apathsfilter so the repo's most expensive job is skipped for PRs that can't move the public API.Reporting additions
The digester only reports what a consumer would trip over. Diffing the other way round surfaces additions: whatever disappears going from HEAD back to the base is new in HEAD. A reversed report isn't purely additive — changed types, renames and the like appear mirrored — so only the two sections meaning "present here, absent there" are read as additions, with their wording flipped back.
Why not swiftlang's
soundnessIts api-breakage job is pinned to
runs-on: ubuntu-latestin a Swift container, which a caller can't override — and this package is Apple-only. Running its command directly on macOS fails too:SwiftPM gives the digester no framework search path for the xcframework dependencies, and the command has no flag to pass one. This job supplies
-Ffrom xcodebuild's products dir instead.Known blind spot
Inserting a case into the middle of an
@objc enumis reported as an addition, but the raw-value shift it causes for every later case is not — the digester doesn't see raw values.Verified on CI
Two throwaway commits deliberately broke and extended
ConnectionState's public API to see both halves reported live (run):Both have since been reverted in 7c6dccf —
Sources/matchesmain, so this PR now only adds the check.