fix(tests): align marketplace e2e guard's credential filter with runtime#344
Open
johnhain-msft wants to merge 3 commits into
Open
fix(tests): align marketplace e2e guard's credential filter with runtime#344johnhain-msft wants to merge 3 commits into
johnhain-msft wants to merge 3 commits into
Conversation
The Playwright Electron test guard `canAccessRepo` previously iterated every
`keytar.findCredentials('copilot-cli')` entry — including EMU/enterprise-shape
accounts (`https://github.com/enterprises/microsoft:<login>`) that Chamber's
runtime filters out. As a result, the guard could return true on the strength
of an EMU token that the in-app marketplace fetcher would never use, opening
the test gate so specs ran but then failed in the runtime with `Unable to
access marketplace <nwo>`.
Reuse `listStoredGitHubCredentials` from `@chamber/services` to filter the
guard's credential candidates to the same Chamber-format
`https://github.com:<login>` set the runtime considers. The guard and the
runtime now consult the same filtered credential set by construction.
- New `tests/e2e/electron/chamberRepoAccess.ts` exports
`canAccessRepoWithChamberCredentials(nwo, credentialStore, options?)`. Pure,
unit-testable, no Electron dependency.
- `tests/e2e/electron/electronApp.ts:canAccessRepo` is now a thin
keytar+helper delegation. The lazy `import('keytar')` is preserved
(CHANGELOG ianphil#250 — Windows EPERM on `keytar.node`). The helper is imported
statically because Playwright's TS loader handles static `.ts` imports but
routes dynamic `import('./helper')` through Node's native loader, which
rejects the helper's top-level ESM `import` statements. Static import is
safe only as long as `@chamber/services` has no `keytar` transitive
imports (verified at commit time); the JSDoc on `canAccessRepo` documents
this invariant.
- Inline `canFetchRepo` is removed (no other callers).
- New `tests/regression/electronAppChamberRepoAccess.test.ts` covers
anon-public, chamber-credential-succeeds, only-EMU-can-reach-repo (bug
repro), no-credential-can-reach-repo, and credential-store-errors (parity
with runtime's `safeCredentials` swallow).
Verified by re-running the four affected Electron Playwright specs
(`genesis-landing-add-marketplace`, `genesis-marketplace-aggregation`,
`settings-marketplace-management`, `a365-release-tools-smoke`): all skip
cleanly with the documented skip reasons. Before this fix, the first three
would have run and then failed in runtime.
No production credential-store change.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/e2e/electron/electronApp.ts'scanAccessRepoaccepts stored Copilot credentials that the Chamber runtime filters out. The result: marketplace e2e specs can be allowed to run when the runtime cannot actually reach the marketplace repo, causing the specs to fail with a runtime "Unable to access marketplace …" error instead of skipping cleanly.This PR aligns the test guard's credential set with the runtime's by reusing
listStoredGitHubCredentialsfrom@chamber/services. Test-side change only — no production code, no version bump, no CHANGELOG entry.Closes #343
Root cause
canAccessRepoiterated every entry returned bykeytar.findCredentials('copilot-cli'). The Chamber runtime reads from the same keytar service but filters vialistStoredGitHubCredentials(packages/services/src/auth/AuthService.ts:36), which keeps onlyhttps://github.com:<login>accounts.GitHubRegistryClientandGitHubReleaseAssetClientnever see EMU/enterprise-shape accounts (https://github.com/enterprises/microsoft:<login>).So when a developer's keychain holds an EMU credential that can reach the private marketplace repo but no chamber-format credential that can, the guard probes the EMU token, returns
true, and lets the spec run. The runtime then fails with:Approach — same keytar service, same filter
Test-side only. The guard now uses the exact same filter as the runtime.
Files changed
tests/e2e/electron/chamberRepoAccess.ts(new)Exports
canAccessRepoWithChamberCredentials(nwo, credentialStore, options?). Top-level importslistStoredGitHubCredentialsandCredentialStorefrom@chamber/services. Anonfetchfirst, then iterates the filtered set. A credential-store rejection is swallowed (returnsfalse) to mirrorGitHubRegistryClient.safeCredentials()atpackages/services/src/genesis/GitHubRegistryClient.ts:143.tests/e2e/electron/electronApp.tscanAccessRepobecomes a thin delegation to the helper. Two invariants preserved:import('keytar')is kept. Per CHANGELOG Desktop smoke fails on Windows: keytar lock + PR #244 sidebar selector regressions + ProfileMarkdownEditor viewport #250, eagerly loadingkeytar.nodein the Playwright runner process pins the addon and breakselectron-forge's rebuild on Windows (EPERM on unlink).await import('./helper')). Playwright's TypeScript loader handles static.tsimports of sibling files in the test graph, but routesawait import('./helper')through Node's native loader, which then rejects the helper's top-level ESMimportstatements withCannot use import statement outside a module. The static-import invariant is only safe while@chamber/serviceshas zero transitivekeytarimports — that invariant is verified and called out in the JSDoc oncanAccessRepoas a tripwire for future regressions.Inline
canFetchRepowas removed (no other callers).The single helper covers both runtime paths (
GitHubRegistryClient.withCredentialStoreandGitHubReleaseAssetClient.withCredentialStore) because both go throughlistStoredGitHubCredentials.Tests
tests/regression/electronAppChamberRepoAccess.test.ts— 5 vitest cases, no Electron required:anon-publictruewhen anonfetch(/repos/{nwo})succeeds.chamber-credential-succeedstruewhen a singlehttps://github.com:<login>token reaches the repo.only-EMU-can-reach-repo(bug repro)falsewhen only EMU-format tokens can reach the repo.no-credential-can-reach-repofalsewhen neither anon nor any Chamber-format credential works.credential-store-errorsfalsewhenfindCredentialsrejects (parity with runtimesafeCredentials).The fake satisfies the full
CredentialStoreinterface;setPassword/deletePasswordthrow to assert they're never called by the helper.Verification
Static checks against current
masternpx tsc --noEmit— cleannpm run lint— clean (TS + ESLint +deps:check(446 modules, 0 violations) + the newlint:yamlandlint:md)npm test— 164 files, 1801 tests pass (1796 baseline + 5 new)Empirical A/B repro
On a workstation with the exact bug-repro credential state — chamber-format credential present but lacking access (HTTP 404 against
/repos/agency-microsoft/genesis-minds), EMU credential present and granting access (HTTP 200):master(pre-fix)canAccessReporeturnstruevia EMU probe → Electron launches → runtime tries to add marketplaceUnable to access marketplace agency-microsoft/genesis-minds. Chamber tried the public GitHub API and any stored Chamber GitHub credentials.canAccessReporeturnsfalsebecause EMU is filtered out and chamber-format can't reach the repoStored Chamber GitHub credentials cannot access agency-microsoft/genesis-minds.Same machine, same credential state, only the test-side guard changed. The runtime error in the pre-fix row is the literal error string from issue #343.
Notes for the reviewer
safeCredentials(). Mirroring that exact filter keeps the guard's behavior identical to the runtime's by construction./git/treesfor actual marketplace fetches; the guard uses/repos/{nwo}. Closing that gap is a separate concern and was intentionally left out.AbortControlleron the guard'sfetchcalls was considered and left out for the same reason — separate concern, not in scope for this fix.Out of scope
Anything other than aligning the two credential paths.