Skip to content

Add mediated automation surface#79

Open
jmcte wants to merge 5 commits into
mainfrom
codex/issue-50-automation-surface
Open

Add mediated automation surface#79
jmcte wants to merge 5 commits into
mainfrom
codex/issue-50-automation-surface

Conversation

@jmcte
Copy link
Copy Markdown
Contributor

@jmcte jmcte commented May 24, 2026

Summary

  • Add a shared BrokerAutomation request/response path for mediated login and fill automation.
  • Expose macOS Shortcuts actions for APW login/fill through App Intents.
  • Run Shortcuts broker requests through an async detached broker path instead of synchronously on MainActor.
  • Publish an AppleScript dictionary in the built app bundle and wire its commands to NSScriptCommand implementations that call BrokerAutomation.
  • Run bundled serve launches under NSApplication so Cocoa scripting can dispatch the SDEF command classes while the broker socket server runs on a background thread.
  • Keep AppleScript command requests off the main thread while pumping the AppKit run loop, preserving credential UI dispatch for non-demo broker requests.
  • Keep AuthenticationServices callable from synchronous automation contexts by pumping the main run loop when the broker is invoked on the main thread.
  • Add a fast native automation contract test so CI guards Shortcuts/AppIntents wiring, AppleScript command wiring, dictionary packaging, and broker automation test coverage.

Closes #50.

Verification

  • bash scripts/test-native-automation-config.sh
  • bash scripts/ci/run-fast-checks.sh
  • swift test --package-path native-app --filter BrokerCoreTests/testAutomation (outside sandbox; SwiftPM/Xcode need user cache access)
  • swift test --package-path native-app (outside sandbox; 18 tests passed)
  • ./scripts/build-native-app.sh (outside sandbox; Swift emits the existing AuthenticationServices exhaustive-switch warning)
  • plutil -p native-app/dist/APW.app/Contents/Info.plist confirmed NSAppleScriptEnabled=true and OSAScriptingDefinition=APW.sdef
  • test -f native-app/dist/APW.app/Contents/Resources/APW.sdef
  • xmllint --noout native-app/Resources/APW.sdef
  • sdp -fh -o /private/tmp/apw-sdef.h native-app/Resources/APW.sdef
  • nm -m native-app/dist/APW.app/Contents/MacOS/APW | rg 'APWRequest(Login|Fill)Command|BrokerApplicationDelegate'
  • git diff --check

Notes

  • I verified the compiled App Intents and packaged AppleScript dictionary locally. I did not manually open the Shortcuts app UI in this run.

@jmcte jmcte requested a review from pheidon as a code owner May 24, 2026 00:04
@athena-omt athena-omt added area:ai AI/model/agent behavior surface. lane:daedalus Daedalus implementation/forge lane. review:athena Athena review governance requested. risk:medium Medium-risk change; normal care required. state:waiting-checks Waiting for CI/check status to settle. status:needs-review PR is ready for Athena review. labels May 24, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 527d2e34a4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +23 to +25
@MainActor
func perform() async throws -> some IntentResult & ReturnsValue<String> {
let data = try BrokerAutomation.performResponseData(
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid running broker request synchronously on MainActor

Calling BrokerAutomation.performResponseData from an @MainActor perform() can deadlock the non-demo credential flow: AppleAuthenticationServicesBroker.login enqueues work onto DispatchQueue.main and then blocks on a semaphore, so when the caller is already on the main actor/thread the enqueued block cannot run and the request times out. This makes Shortcuts login/fill fail in normal (non-APW_DEMO) use.

Useful? React with 👍 / 👎.

Comment thread native-app/Resources/APW.sdef Outdated
Comment on lines +5 to +6
<command name="request login" code="APWalgon" description="Request a login credential through the APW broker. The user still approves the credential picker before credential material is returned.">
<direct-parameter type="text" description="The HTTPS URL to request."/>
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Wire AppleScript commands to an implementation

This commit publishes custom AppleScript verbs in the sdef, but there is no corresponding command handler path in the APW app target to execute broker automation for these events (no NSScriptCommand subclass/selector wiring, and main.swift only launches runBrokerAppMain()). As shipped, request login/request fill Apple events have terminology but no executable bridge to BrokerAutomation, so the new AppleScript surface is non-functional.

Useful? React with 👍 / 👎.

@pheidon pheidon added state:needs-repair PR needs repair before review can proceed. codex:P1 Codex Connector P1 finding. priority:P1 Codex Connector P1; blocks execution until Athena and Ares validate. risk:high High-risk change; validation required. and removed state:waiting-checks Waiting for CI/check status to settle. labels May 24, 2026
@athena-omt athena-omt added the state:waiting-checks Waiting for CI/check status to settle. label May 24, 2026
@jmcte jmcte removed state:waiting-checks Waiting for CI/check status to settle. state:needs-repair PR needs repair before review can proceed. labels May 24, 2026
pheidon
pheidon previously approved these changes May 24, 2026
Copy link
Copy Markdown
Contributor

@pheidon pheidon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved from Pheidon sweep. Mediated automation surface is covered by native automation config tests and green PR Fast CI, Rust, lint, secrets, and Swift checks.

@hephaestus-omt hephaestus-omt force-pushed the codex/issue-50-automation-surface branch from 2e99fe6 to 4289405 Compare May 24, 2026 16:20
@athena-omt athena-omt added the state:waiting-checks Waiting for CI/check status to settle. label May 24, 2026
@hephaestus-omt hephaestus-omt force-pushed the codex/issue-50-automation-surface branch from 4289405 to 65feadf Compare May 24, 2026 18:29
@hephaestus-omt hephaestus-omt enabled auto-merge (squash) May 24, 2026 18:30
@hephaestus-omt hephaestus-omt force-pushed the codex/issue-50-automation-surface branch from 65feadf to 27992b9 Compare May 24, 2026 18:31
Copy link
Copy Markdown
Contributor

@pheidon pheidon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second-pass approval after Hephaestus repair: current head has green required checks, the prior connector comments are stale against older commits, and the branch is no longer blocked by failing CI. Approved for merge readiness pending GitHub merge-state refresh.

jmcte added 5 commits May 24, 2026 20:19
Start the broker from an AppKit-backed bundled launch so AppleScript command classes can be dispatched from the app's scripting definition.

Keep AppleScript broker requests off the main thread while pumping the run loop, preserving credential UI dispatch for non-demo requests.
Document that the AppleScript direct parameters are routed to the concrete NSScriptCommand bridge classes added for broker automation.
@hephaestus-omt hephaestus-omt force-pushed the codex/issue-50-automation-surface branch from 27992b9 to 2e6d6dc Compare May 24, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ai AI/model/agent behavior surface. codex:P1 Codex Connector P1 finding. lane:daedalus Daedalus implementation/forge lane. priority:P1 Codex Connector P1; blocks execution until Athena and Ares validate. review:athena Athena review governance requested. risk:high High-risk change; validation required. risk:medium Medium-risk change; normal care required. state:waiting-checks Waiting for CI/check status to settle. status:needs-review PR is ready for Athena review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Roadmap: AppleScript / Shortcuts automation surface

3 participants