Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ Runnable samples that complement [docs/api-guide](../docs/api-guide/).

| Directory | Purpose |
|-----------|---------|
| [IrisDemo](IrisDemo/) | Full SwiftUI macOS app: bundled Python sklearn service, Charts, JSON boundary |
| [Studio](Studio/) | **Flagship app.** Native SwiftUI macOS app on the canonical V2 `pool.<module>` surface — races four scikit-learn models in parallel across Python workers, with live charts, worker activity, and an embedded "behind the curtain" console (+ `studio-cli`). Ships as a notarized DMG with a self-contained Python ([SHIPPING.md](Studio/SHIPPING.md)). |
| [IrisDemo](IrisDemo/) | Minimal SwiftUI macOS app: bundled Python sklearn service via the raw runtime + JSON boundary (the simplest "hello world"; predates the `pool.<module>` surface) |
| [CoreRuntimeSmoke](CoreRuntimeSmoke/) | Minimal CLI: `Python.run` + `sys` / `json` |
| [ProcessPoolSmoke](ProcessPoolSmoke/) | Minimal CLI: `withProcessPool`, `invokeResult`, `math.sqrt` |
| [BridgingRing](BridgingRing/) | Showcase: callbacks + **`WorkerCallbackContext` reentrant** Python↔Swift + **`evalEvents`** generator/progress streams over the pool IPC |
Expand Down
104 changes: 104 additions & 0 deletions Examples/Studio/Package.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
// swift-tools-version: 6.0
import PackageDescription
import Foundation

// SwiftPython Studio — a flagship demo app built on the commercial
// SwiftPythonRuntime xcframework. It showcases the canonical V2 `pool.<module>`
// surface (vendored as `SwiftPythonKit`) by racing several scikit-learn models
// across Python worker processes, live, from Swift.

private func parseSemVer(_ raw: String) -> Version? {
let core = raw.split(separator: "-", maxSplits: 1).first.map(String.init) ?? raw
let parts = core.split(separator: ".", omittingEmptySubsequences: false)
guard parts.count == 3, let a = Int(parts[0]), let b = Int(parts[1]), let c = Int(parts[2]) else { return nil }
return Version(a, b, c)
}

private let swiftPythonPackageName = "swiftpython-commercial"

private struct SwiftPythonDependencyConfig {
let dependency: Package.Dependency
let packageName: String
}

private func packageNameFromDependencyURL(_ rawURL: String) -> String {
var trimmed = rawURL.trimmingCharacters(in: .whitespacesAndNewlines)
while trimmed.hasSuffix("/") { trimmed.removeLast() }
let base = (trimmed as NSString).lastPathComponent
return base.hasSuffix(".git") ? String(base.dropLast(4)) : base
}

private func swiftPythonConfig() -> SwiftPythonDependencyConfig {
let env = ProcessInfo.processInfo.environment
if let url = env["SWIFTPYTHON_COMMERCIAL_PACKAGE_URL"]?.trimmingCharacters(in: .whitespacesAndNewlines),
!url.isEmpty,
let versionRaw = env["SWIFTPYTHON_COMMERCIAL_PACKAGE_VERSION"]?.trimmingCharacters(in: .whitespacesAndNewlines),
!versionRaw.isEmpty {
guard let version = parseSemVer(versionRaw) else {
fatalError("SWIFTPYTHON_COMMERCIAL_PACKAGE_VERSION must be MAJOR.MINOR.PATCH")
}
return SwiftPythonDependencyConfig(
dependency: .package(url: url, exact: version),
packageName: packageNameFromDependencyURL(url)
)
}
return SwiftPythonDependencyConfig(
dependency: .package(name: swiftPythonPackageName, path: "../.."),
packageName: swiftPythonPackageName
)
}

private func pythonLibraryDirectory() -> String {
let env = ProcessInfo.processInfo.environment
if let explicit = env["SWIFTPYTHON_PYTHON_LIB_DIR"]?.trimmingCharacters(in: .whitespacesAndNewlines), !explicit.isEmpty {
return explicit
}
if let home = (env["PYTHON_HOME"] ?? env["PYTHONHOME"])?.trimmingCharacters(in: .whitespacesAndNewlines), !home.isEmpty {
let frameworkLib = "\(home)/Frameworks/Python.framework/Versions/3.13/lib"
return FileManager.default.fileExists(atPath: frameworkLib) ? frameworkLib : "\(home)/lib"
}
let candidates = [
"/opt/homebrew/opt/python@3.13/Frameworks/Python.framework/Versions/3.13/lib",
"/usr/local/opt/python@3.13/Frameworks/Python.framework/Versions/3.13/lib",
"/opt/homebrew/opt/python@3.13/lib",
"/usr/local/opt/python@3.13/lib",
]
return candidates.first { FileManager.default.fileExists(atPath: $0) } ?? candidates[0]
}

private let pythonLinkerSettings: [LinkerSetting] = [
.unsafeFlags(["-L\(pythonLibraryDirectory())", "-lpython3.13"])
]

private let config = swiftPythonConfig()

let package = Package(
name: "Studio",
platforms: [.macOS(.v15)],
dependencies: [config.dependency],
targets: [
// Vendored canonical V2 pool bindings (generated by `swift-python-gen --surface pool`).
.target(
name: "SwiftPythonKit",
dependencies: [.product(name: "SwiftPythonRuntime", package: config.packageName)]
),
.executableTarget(
name: "Studio",
dependencies: [
"SwiftPythonKit",
.product(name: "SwiftPythonRuntime", package: config.packageName),
],
path: "Sources/Studio",
linkerSettings: pythonLinkerSettings
),
.executableTarget(
name: "studio-cli",
dependencies: [
"SwiftPythonKit",
.product(name: "SwiftPythonRuntime", package: config.packageName),
],
path: "Sources/studio-cli",
linkerSettings: pythonLinkerSettings
),
]
)
54 changes: 54 additions & 0 deletions Examples/Studio/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# SwiftPython Studio

The flagship demo for the SwiftPython runtime: **Swift driving scikit-learn across
real Python worker processes, live, in a native macOS app.**

Studio races four classifiers (Random Forest, Gradient Boosting, Logistic
Regression, K-Neighbors) in parallel — each pinned to its own Python worker —
through the canonical V2 `pool.<module>` surface, and shows the accuracy bars,
confusion matrix, worker activity, and the raw pool calls streaming in an embedded
"behind the curtain" console.

```swift
let model = try await pool.sklearn.ensemble.RandomForestClassifier(n_estimators: 300)
let accuracy = try await model.fit(X: xTrain, y: yTrain).score(X: xTest, y: yTest)
```

## What it shows off

- **`pool.<module>` canonical surface** — `pool.sklearn.ensemble.…`, owned remote
handles, typed results, method chaining (`fit(…).score(…)`), all `.handle`-free.
- **True multi-process parallelism** — wall time ≈ the slowest model, not the sum.
- **The commercial xcframework** — built exactly the way a customer consumes it.

## Run it (dev)

Requires Homebrew `python@3.13` with `numpy pandas scikit-learn scipy`.

```bash
cd Examples/Studio

# GUI
./scripts/build_app.sh --open

# Terminal demo (same engine, designed output)
SWIFTPYTHON_WORKER_PATH="$(cd ../.. && pwd)/SwiftPythonWorker" \
swift run studio-cli
```

## Layout

| Path | Role |
|------|------|
| `Sources/Studio/` | SwiftUI app (bake-off screen + embedded console) |
| `Sources/studio-cli/` | Standalone terminal bake-off |
| `Sources/SwiftPythonKit/` | Vendored V2 pool bindings (generated by `swift-python-gen --surface pool`) |
| `scripts/build_app.sh` | Assemble `.app` (`--bundle-python`, `--sign`, `--open`) |
| `scripts/bundle_python.sh` | Bundle a self-contained Python + scientific stack |
| `SHIPPING.md` | Notarized DMG runbook (download-and-run for anyone) |

## Distribution

Studio is meant to ship as a notarized, stapled DMG with a **self-contained Python**
(no system install needed) — see [`SHIPPING.md`](SHIPPING.md). The `IrisDemo`
sample remains the minimal "hello world"; Studio is the flagship on the V2 surface.
Binary file added Examples/Studio/Resources/AppIcon-1024.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Examples/Studio/Resources/AppIcon-source.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Examples/Studio/Resources/AppIcon.icns
Binary file not shown.
134 changes: 134 additions & 0 deletions Examples/Studio/SHIPPING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Shipping SwiftPython Studio

How to turn `SwiftPython Studio.app` into a notarized, stapled DMG that anyone can
download and run with **no Python installed**. This is a standard macOS Developer ID
notarization flow with the extra steps a bundled-Python app needs.

## What ships

- A native macOS app built on the **commercial `SwiftPythonRuntime.xcframework`**
(v0.5.14) — the exact artifact a customer consumes.
- The bundled **`SwiftPythonWorker`** sidecar (auto-discovered by `PythonProcessPool`).
- A **bundled, relocatable Python 3.13 + numpy/pandas/scikit-learn/scipy** so the
app is self-contained (the IRIS sample, by contrast, requires Homebrew Python).
- The canonical V2 `pool.<module>` bindings, vendored as `SwiftPythonKit`.
- The `studio-cli` terminal demo (in `Contents/MacOS/`, also droppable in the DMG).

## Prerequisites (release machine)

- A `Developer ID Application` identity in the login Keychain
(`security find-identity -v -p codesigning`).
- A notary keychain profile created once with:
```bash
xcrun notarytool store-credentials "<YourNotaryProfile>" \
--apple-id "<APPLE_ID>" --team-id "<TEAM_ID>" --password "<APP_SPECIFIC_PASSWORD>"
```
The commands below use `"$NOTARY_PROFILE"`; export it to your profile name.
- Homebrew `python@3.13` (source for the relocatable bundle) with the four wheels.

## Build → sign → notarize → DMG

```bash
cd Examples/Studio
export NOTARY_PROFILE="<YourNotaryProfile>"

# 1. Assemble, bundle Python, and sign (Developer ID, hardened runtime, inside-out).
STUDIO_APP_SHORT_VERSION=0.1.0 STUDIO_APP_BUILD_VERSION=1 \
./scripts/build_app.sh --bundle-python --sign

# 2. Notarize the app, then staple.
APP="build/SwiftPython Studio.app"
ditto -c -k --keepParent "$APP" build/Studio.zip
xcrun notarytool submit build/Studio.zip --keychain-profile "$NOTARY_PROFILE" --wait
xcrun stapler staple "$APP"
spctl -a -vv --type execute "$APP" # → "Notarized Developer ID"

# 3. Build a DMG (with an Applications drop-target), sign + notarize + staple it.
rm -rf build/dmg && mkdir build/dmg
cp -R "$APP" build/dmg/ && ln -s /Applications build/dmg/Applications
hdiutil create -volname "SwiftPython Studio" -srcfolder build/dmg -ov -format UDZO build/SwiftPythonStudio.dmg
codesign --force --sign "Developer ID Application" build/SwiftPythonStudio.dmg
xcrun notarytool submit build/SwiftPythonStudio.dmg --keychain-profile "$NOTARY_PROFILE" --wait
xcrun stapler staple build/SwiftPythonStudio.dmg
```

## Signing order (load-bearing)

`build_app.sh --sign` signs **inside-out**, which is mandatory or notarization
returns `Invalid`:

1. Every nested Mach-O (wheel `.so`/`.dylib`, framework dylibs, the Python binary)
under `Contents/Frameworks` + `Contents/Resources/Python`
(Developer ID + `--options runtime` + `--timestamp`).
2. Non-Mach-O Python payloads with executable bits are stripped (`chmod -x`) or a
quarantined launch shows **"App is damaged"** even when `spctl` passes.
3. The `Python.framework` bundle (re-sealed).
4. The `SwiftPythonWorker` sidecar + `studio-cli`.
5. The outer `.app` last (no `--deep`), with `scripts/studio.entitlements`
(`disable-library-validation`, `allow-dyld-environment-variables`).

## Wiring the worker to the bundled interpreter

The shipped `SwiftPythonWorker` must use the **bundled** Python, not a system one.
Two pieces, both implemented:

- **Relink** the three binaries (`Studio`, `studio-cli`, `SwiftPythonWorker`) from
the absolute Homebrew Python load path to `@rpath/Python.framework/...` with an
added rpath of `@executable_path/../Frameworks` (done in `build_app.sh --bundle-python`).
Re-sign after this (`install_name_tool` invalidates the signature).
- **Env**: `StudioRuntime.configureBundledPythonIfPresent()` sets, relative to the
running executable, `PYTHONHOME = …/Contents/Frameworks/Python.framework/Versions/3.13`
and `PYTHONPATH = …/Contents/Resources/Python/site-packages` before the pool spawns
the worker (which inherits them). `studio.entitlements` enables
`allow-dyld-environment-variables` so this is honored under the hardened runtime.

Verify on a Mac with **no Homebrew / no Python** (or simulate locally with
`env -i HOME="$HOME" "$APP/Contents/MacOS/studio-cli"`):

```bash
xcrun stapler validate "build/SwiftPython Studio.app"
# also launch from Finder (not Terminal) — the GUI launchd env has no Homebrew PATH,
# so this proves the bundled interpreter + env wiring are what make it work for strangers.
```

## Hosting

- Upload `SwiftPythonStudio.dmg` to a **public GitHub release** (a dedicated
releases repo, or this repo's Releases).
- Link to it from your download/marketing page.
- Sparkle auto-update is **deferred** for v1 (static DMG). To add it later, wire a
standard Sparkle appcast under a `studio/` feed path; nothing in the build here
changes.

## Gotchas hit while producing the first notarized DMG (do not repeat)

1. **Relinked binaries must be re-signed.** `install_name_tool` (used to point the
3 binaries at `@rpath/Python.framework`) invalidates the signature; on Apple
Silicon the binary then fails to launch with `Bad executable`. The `--sign`
step runs *after* relink for this reason.
2. **Do NOT sign the app's main executable separately.** Signing
`Contents/MacOS/Studio` directly *and* sealing the `.app` yields "The signature
of the binary is invalid" at notarization. Sign only the `.app` (it signs the
main executable + applies entitlements). Helper executables (`SwiftPythonWorker`,
`studio-cli`) DO need their own `codesign`.
3. **Sign the framework as a bundle**, not its `Versions/3.13` directory:
`codesign … Contents/Frameworks/Python.framework`.
4. **Strip symlinks that escape the framework.** Homebrew's framework links
`lib/python3.13/site-packages` up into the Cellar; Gatekeeper rejects it with
"invalid destination for symbolic link in bundle." `bundle_python.sh` removes
escaping/absolute symlinks (we use `Contents/Resources/Python/site-packages`
via `PYTHONPATH` anyway).

## Status — DONE (first release built)

The full path is implemented, run, and verified on the build machine:

- `numpy 2.5 / pandas 3.0 / scikit-learn 1.9 / scipy 1.18` bundled (lean tier).
- The bake-off runs in a **fully sanitized environment** (`env -i`, no Homebrew,
no inherited `PYTHONHOME`) — proof it works on a Python-less Mac.
- App: notarized (Accepted) + stapled + `spctl … → "Notarized Developer ID"`.
- **`build/SwiftPythonStudio.dmg`** (~182 MB): notarized + stapled + Gatekeeper
accepted (`source=Notarized Developer ID`).

Remaining before public download: host the DMG on a public GitHub release + link it
from the download page, and one real download-and-run smoke on a *different* Mac.
Loading