Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 27 additions & 8 deletions .github/actions/fetch-canary/action.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,18 @@
name: fetch-canary
description: >
Cache + fetch the private canary GGUFs and export TRANSCRIBE_SMOKE_MODEL /
TRANSCRIBE_SMOKE_STREAMING_MODEL. Skips cleanly when hf-token is empty
(forks have no secret): the model tests then skip, exactly as before this
action existed. Always fetches BOTH canaries (~95 MB total) under one cache
key — a per-consumer subset would let one job save the shared key with only
its subset in it, and every other consumer would re-download forever
(exact-key hits are never re-saved). Requires a checkout (composite actions
resolve from the repo) and uv on PATH (uvx fetches via huggingface_hub).
TRANSCRIBE_SMOKE_STREAMING_MODEL / the two parakeet family canaries. Skips
cleanly when hf-token is empty (forks have no secret): the model tests then
skip, exactly as before this action existed. Always fetches ALL CI canaries
(whisper-tiny + moonshine-streaming ~95 MB, plus the two parakeet 0.6b
family-extension canaries — cache-aware + buffered — at Q4_K_M ~0.95 GB)
under one cache key — a per-consumer subset would let one job save the shared
key with only its subset in it, and every other consumer would re-download
forever (exact-key hits are never re-saved), which is also why the key carries
a version that bumps whenever the fetched set changes. Voxtral-realtime is NOT
fetched (~2.5 GB, too heavy for CI; its test stays local-only). Requires a
checkout (composite actions resolve from the repo) and uv on PATH (uvx fetches
via huggingface_hub).

inputs:
hf-token:
Expand All @@ -29,7 +34,7 @@ runs:
uses: actions/cache@v5
with:
path: canary
key: canary-models-v1
key: canary-models-v2
- name: Fetch canary models (cache miss only)
if: inputs.hf-token != ''
shell: bash
Expand All @@ -42,9 +47,23 @@ runs:
[ -f canary/moonshine-streaming-tiny-Q8_0.gguf ] || \
uvx --from huggingface_hub hf download handy-computer/moonshine-streaming-tiny-gguf \
moonshine-streaming-tiny-Q8_0.gguf --local-dir canary
# Parakeet family-extension canaries (cache-aware + buffered streaming).
# Q4_K_M keeps the CI canary set light (~0.95 GB for the pair); the
# family-ext tests are content-lenient (assert non-empty, not accuracy),
# so a lighter quant than the local Q8_0 default is sufficient.
[ -f canary/nemotron-speech-streaming-en-0.6b-Q4_K_M.gguf ] || \
uvx --from huggingface_hub hf download \
handy-computer/nemotron-speech-streaming-en-0.6b-gguf \
nemotron-speech-streaming-en-0.6b-Q4_K_M.gguf --local-dir canary
[ -f canary/parakeet-unified-en-0.6b-Q4_K_M.gguf ] || \
uvx --from huggingface_hub hf download \
handy-computer/parakeet-unified-en-0.6b-gguf \
parakeet-unified-en-0.6b-Q4_K_M.gguf --local-dir canary
# GITHUB_WORKSPACE (not bash's $PWD) so Windows exports a path
# Python can open rather than an MSYS one.
prefix='${{ inputs.model-path-prefix }}'
if [ -z "$prefix" ]; then prefix="$GITHUB_WORKSPACE"; fi
echo "TRANSCRIBE_SMOKE_MODEL=$prefix/canary/whisper-tiny-Q5_K_M.gguf" >> "$GITHUB_ENV"
echo "TRANSCRIBE_SMOKE_STREAMING_MODEL=$prefix/canary/moonshine-streaming-tiny-Q8_0.gguf" >> "$GITHUB_ENV"
echo "TRANSCRIBE_SMOKE_PARAKEET_STREAM_MODEL=$prefix/canary/nemotron-speech-streaming-en-0.6b-Q4_K_M.gguf" >> "$GITHUB_ENV"
echo "TRANSCRIBE_SMOKE_PARAKEET_BUFFERED_MODEL=$prefix/canary/parakeet-unified-en-0.6b-Q4_K_M.gguf" >> "$GITHUB_ENV"
171 changes: 153 additions & 18 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,11 @@ name: publish
#
# RELEASE (push tag v0.0.X): same full build, then with the `pypi`
# environment (add a required-reviewer rule once the repo is public):
# dist-* → PyPI, cu12 wheels → GitHub release assets for the tag, and
# the wheel-index workflow is dispatched so /whl/cu12 picks them up.
# cu12 → PyPI additionally when the repo variable CU12_ON_PYPI is
# "true" (set it once the PyPI file-size request is granted).
# dist-* → PyPI, cu12 wheels → draft GitHub release assets for the tag,
# Rust crates → crates.io, Swift → the draft release, then the draft is
# published and wheel-index.yml is dispatched so /whl/cu12 picks it up.
# cu12 → PyPI additionally when the repo variable CU12_ON_PYPI is "true"
# (set it once the PyPI file-size request is granted).

on:
push:
Expand All @@ -43,10 +44,28 @@ jobs:
# arches under MSVC, ~3 h on 16vcpu), and the TestPyPI rehearsal neither
# publishes nor smokes its output (cu12 wheels exceed TestPyPI's file
# cap). Validate it on demand with its own workflow_dispatch.
if: startsWith(github.ref, 'refs/tags/')
if: startsWith(github.ref, 'refs/tags/v')
uses: ./.github/workflows/cuda-windows.yml
secrets: inherit

create-release:
# Tags only. Owns creation of the GitHub Release object. Keep it draft
# until the mandatory publishers have succeeded and assets are verified.
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Create the draft release for the tag
env:
GH_TOKEN: ${{ github.token }}
run: |
set -e
tag="${GITHUB_REF_NAME}"
gh release view "$tag" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1 || \
gh release create "$tag" --repo "$GITHUB_REPOSITORY" \
--title "$tag" --notes "transcribe.cpp $tag" --verify-tag --draft

publish-testpypi:
# Rehearsal target (dispatch only — real tags go to PyPI).
# Blacksmith, not the Hetzner box: gh-action-pypi-publish is a Docker
Expand Down Expand Up @@ -164,12 +183,11 @@ jobs:
# cu12's primary distribution home: wheels as GitHub release assets,
# served to pip through the PEP 503 index on Pages (wheel-index.yml).
if: startsWith(github.ref, 'refs/tags/v')
needs: [wheels, cuda-windows]
needs: [create-release, wheels, cuda-windows]
runs-on: [self-hosted, Linux, X64, hetzner]
timeout-minutes: 30
permissions:
contents: write # create the release + upload assets
actions: write # dispatch wheel-index
contents: write # upload release assets
steps:
- uses: actions/download-artifact@v8
with:
Expand All @@ -185,15 +203,12 @@ jobs:
pattern: native-*
merge-multiple: true
path: native-bundles
- name: Create the release for the tag and attach the cu12 wheels
- name: Attach the cu12 wheels
env:
GH_TOKEN: ${{ github.token }}
run: |
set -e
tag="${GITHUB_REF#refs/tags/}"
gh release view "$tag" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1 || \
gh release create "$tag" --repo "$GITHUB_REPOSITORY" \
--title "$tag" --notes "transcribe.cpp $tag" --verify-tag
gh release upload "$tag" cu12/*.whl --repo "$GITHUB_REPOSITORY" --clobber
- name: Attach the native bundles (versioned names)
env:
Expand All @@ -210,12 +225,6 @@ jobs:
done
ls -la upload/
gh release upload "$tag" upload/*.tar.gz --repo "$GITHUB_REPOSITORY" --clobber
- name: Refresh the PEP 503 index (requires Pages enabled on the repo)
env:
GH_TOKEN: ${{ github.token }}
run: |
gh workflow run wheel-index.yml --repo "$GITHUB_REPOSITORY" || \
echo "::warning::wheel-index dispatch failed — is the workflow on the default branch and Pages enabled?"

# ---- Rust crates (crates.io) -------------------------------------------------
# The Rust release path mirrors the Python one: a dispatch REHEARSAL that
Expand Down Expand Up @@ -272,3 +281,129 @@ jobs:
run: cargo publish -p transcribe-cpp-sys
- name: Publish transcribe-cpp (the safe wrapper; resolves the just-published sys)
run: cargo publish -p transcribe-cpp

# ---------------------------------------------------------------------------
# Swift binding (TranscribeCpp) — xcframework release.
#
# The Swift package is consumed as a prebuilt static `.xcframework`
# binaryTarget (notes/swift-bindings-plan.md; requirements §5). Releasing it
# means: build the four Apple slices, zip + checksum the xcframework, attach
# the zip as a release asset, then point the mirror repo's Package.swift
# `binaryTarget(url:checksum:)` at it. "Releases are cut from CI, never a
# laptop." macOS runner: the xcframework needs Xcode (libtool/xcodebuild).
# ---------------------------------------------------------------------------
swift-rehearsal:
# The shipped-artifact gate (§4): build the real xcframework and run the
# suite against it (transcribes the canary through the published shape).
if: github.event_name == 'workflow_dispatch'
runs-on: macos-15
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v8.2.0
- run: brew install ninja
- name: Build the full xcframework (macOS + iOS device + iOS simulator)
run: scripts/ci/build_xcframework.sh
- name: Package + checksum (proves the release artifact + licenses)
run: scripts/ci/package_xcframework.sh
- uses: ./.github/actions/fetch-canary
with:
hf-token: ${{ secrets.HF_TOKEN }}
- name: swift test against the built xcframework
working-directory: bindings/swift
run: swift test

swift-release:
# Tags only. Builds the artifact, attaches the zip to the tag's release, and
# emits the checksum. The mirror-repo Package.swift bump is the final step
# (CJ-gated — needs the dedicated SwiftPM repo + a deploy key; see the plan).
if: startsWith(github.ref, 'refs/tags/v')
needs: [create-release]
runs-on: macos-15
permissions:
contents: write # attach the release asset
steps:
- uses: actions/checkout@v4
- run: brew install ninja
- name: Build the full xcframework
run: scripts/ci/build_xcframework.sh
- name: Package + checksum
id: pkg
run: |
scripts/ci/package_xcframework.sh | tee pkg.txt
echo "checksum=$(awk '/checksum:/ {print $2}' pkg.txt)" >> "$GITHUB_OUTPUT"
- name: Attach the xcframework zip to the release
env:
GH_TOKEN: ${{ github.token }}
run: |
set -e
tag="${GITHUB_REF_NAME}"
gh release upload "$tag" \
bindings/swift/build-apple/TranscribeCpp.xcframework.zip \
--repo "$GITHUB_REPOSITORY" --clobber
- name: Checksum for the mirror repo's Package.swift
run: |
echo "binaryTarget(url: .../TranscribeCpp.xcframework.zip,"
echo " checksum: \"${{ steps.pkg.outputs.checksum }}\")"
# TODO(CJ): push the thin Swift sources + the url/checksum-bearing
# Package.swift to the dedicated mirror repo (transcribe-cpp-swift) and tag
# it, so `swift package add` resolves the release. Needs the mirror repo +
# a deploy key secret. Until then, the asset + checksum above are produced
# but not wired into a resolvable SwiftPM tag.

finalize-release:
# Tags only. Publish the draft GitHub Release only after its hosted assets
# and the core PyPI publish have completed. crates.io has its own approval
# gate and does not host artifacts on this release.
if: startsWith(github.ref, 'refs/tags/v')
needs: [publish-pypi, release-assets, swift-release]
runs-on: ubuntu-latest
permissions:
contents: write # publish the draft release
actions: write # dispatch wheel-index
steps:
- name: Verify assets and publish the release
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
tag="${GITHUB_REF_NAME}"
ver="${tag#v}"
assets_file="$(mktemp)"

gh release view "$tag" --repo "$GITHUB_REPOSITORY" \
--json assets --jq '.assets[].name' | sort > "$assets_file"

echo "Release assets:"
sed 's/^/ /' "$assets_file"

require_asset() {
name="$1"
if ! grep -Fxq "$name" "$assets_file"; then
echo "::error::missing release asset: $name"
exit 1
fi
}

require_asset "TranscribeCpp.xcframework.zip"
require_asset "transcribe-native-${ver}-linux-x86_64-cpu-vulkan.tar.gz"
require_asset "transcribe-native-${ver}-linux-aarch64-cpu-vulkan.tar.gz"
require_asset "transcribe-native-${ver}-macos-arm64-metal.tar.gz"
require_asset "transcribe-native-${ver}-macos-x86_64-cpu.tar.gz"
require_asset "transcribe-native-${ver}-windows-x86_64-cpu-vulkan.tar.gz"

cu12_count="$(grep -Ec '^transcribe_cpp_native_cu12-.*\.whl$' "$assets_file" || true)"
if [ "$cu12_count" -lt 2 ]; then
echo "::error::expected at least two cu12 provider wheels, found $cu12_count"
exit 1
fi

gh release edit "$tag" --repo "$GITHUB_REPOSITORY" --draft=false

- name: Refresh the PEP 503 index (requires Pages enabled on the repo)
env:
GH_TOKEN: ${{ github.token }}
run: |
gh workflow run wheel-index.yml --repo "$GITHUB_REPOSITORY" || \
echo "::warning::wheel-index dispatch failed — is the workflow on the default branch and Pages enabled?"
118 changes: 118 additions & 0 deletions .github/workflows/swift-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
name: swift-ci

# Every-PR gates for the Swift binding (TranscribeCpp). Thin per-binding
# workflow on the shared rails: the binding-agnostic C contracts are certified
# in native-ci.yml; this file adds only what the Swift layer introduces.
#
# - swift-gates: the public-ABI drift gate — the pinned hash in
# ABIHash.swift compared against include/transcribe.abihash
# (scripts/ci/swift_abihash_check.py). No native build; fast,
# runs everywhere. (Swift has no generated FFI layer: the
# Clang importer reads the headers directly, so the gate is a
# pinned constant, not a regen check. Version-sync is the git
# tag + the load-time gate, not check_version_sync.py.)
# - swift-macos: build the macOS slice of the xcframework
# (scripts/ci/build_xcframework.sh), then `swift test` the
# no-model tier against the real binaryTarget — the Swift
# analog of Rust's no_model.rs (version/ABI/device discovery).
# - swift-ios: cross-compile the iOS device + simulator slices
# (build-verify only — no iOS runner executes models here).
#
# Two test tiers (requirements §4): the no-model tests always run; the
# model-gated tier (real transcription/streaming/cancel/family ext) un-skips
# only when the canary GGUFs are fetched (fetch-canary + HF_TOKEN). The model
# tier lands with M4 — this workflow ships the no-model tier first.
#
# Path filters follow native-ci.yml's shape: the binding's own tree plus the
# native paths it compiles from (binding behavior depends on the C side).

on:
push:
branches: [main]
paths: &paths
- "bindings/swift/**"
- "src/**"
- "include/**"
- "ggml/**"
- "cmake/**"
- "CMakeLists.txt"
- "CMakePresets.json"
- "scripts/ci/build_xcframework.sh"
- "scripts/ci/swift_abihash_check.py"
- ".github/workflows/swift-ci.yml"
pull_request:
paths: *paths
workflow_dispatch:

concurrency:
group: swift-ci-${{ github.ref }}
cancel-in-progress: true

jobs:
swift-gates:
runs-on: blacksmith-2vcpu-ubuntu-2404
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v8.2.0
- name: Public-ABI drift gate (pinned hash vs include/transcribe.abihash)
run: uv run --no-project scripts/ci/swift_abihash_check.py

swift-macos:
runs-on: [self-hosted, macOS, ARM64]
env:
# The CMake builds inside build_xcframework.sh honor these launcher env
# vars (CMake initializes CMAKE_<LANG>_COMPILER_LAUNCHER from them).
CMAKE_C_COMPILER_LAUNCHER: ccache
CMAKE_CXX_COMPILER_LAUNCHER: ccache
# Present on this repo's runs, empty on forks — fetch-canary skips cleanly
# when empty and the model tier of `swift test` then XCTSkips.
HF_TOKEN: ${{ secrets.HF_TOKEN }}
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v8.2.0 # fetch-canary fetches via uvx
- name: Install build deps
run: brew install ninja ccache
- uses: actions/cache@v4
with:
path: ~/Library/Caches/ccache
key: ccache-swift-macos-${{ github.sha }}
restore-keys: ccache-swift-macos-
- name: Build the macOS xcframework slice
run: TRANSCRIBE_XCFRAMEWORK_SLICES="macos" scripts/ci/build_xcframework.sh
# Fetch the canary GGUFs (whisper-tiny + moonshine-streaming-tiny) and
# export TRANSCRIBE_SMOKE_MODEL / _STREAMING_MODEL. Audio falls back to the
# in-repo samples/jfk.wav. Skips cleanly without HF_TOKEN (the model tier
# then XCTSkips — the two-tier scheme, requirements §4).
- uses: ./.github/actions/fetch-canary
with:
hf-token: ${{ secrets.HF_TOKEN }}
- name: swift test (no-model + model tiers)
working-directory: bindings/swift
run: swift test
# The 5 canonical examples (§6) run on every leg under the same skip
# rules as the model tier: each transcribes with the canary or exits 0
# with a skip note (models/ is gitignored, so forks skip cleanly).
- name: Run the canonical examples
working-directory: bindings/swift
run: |
for example in transcribe-file streaming batch backend-select error-handling; do
echo "== $example =="
swift run "$example"
done

swift-ios:
runs-on: [self-hosted, macOS, ARM64]
env:
CMAKE_C_COMPILER_LAUNCHER: ccache
CMAKE_CXX_COMPILER_LAUNCHER: ccache
steps:
- uses: actions/checkout@v4
- name: Install build deps
run: brew install ninja ccache
- uses: actions/cache@v4
with:
path: ~/Library/Caches/ccache
key: ccache-swift-ios-${{ github.sha }}
restore-keys: ccache-swift-ios-
- name: Cross-compile the iOS device + simulator slices (build-verify)
run: TRANSCRIBE_XCFRAMEWORK_SLICES="ios-device ios-sim" scripts/ci/build_xcframework.sh
Loading
Loading