Skip to content

Add scenario demo verification and bug bash report - #97

Merged
DanFashauer merged 12 commits into
mainfrom
chatgpt/scenario-demo-verification-bug-bash
May 18, 2026
Merged

Add scenario demo verification and bug bash report#97
DanFashauer merged 12 commits into
mainfrom
chatgpt/scenario-demo-verification-bug-bash

Conversation

@DanFashauer

Copy link
Copy Markdown
Owner

Summary:

  • Add scenario-specific demo verification for compliant, non-compliant, unknown, and all scenarios.
  • Preserve the existing deny-path demo verifier when no scenario query is provided.
  • Harden the v1 compatibility API so runtime auth and signing fail closed when required secrets are missing.
  • Pin the demo workflow Bun version and add Node 22 .nvmrc.
  • Add a bug bash report and readiness status updates.

Validation reported by Codex:

  • npm ci
  • npm run typecheck
  • npm run lint
  • npm run build
  • npm run test:run
  • npm run test:api:server
  • npx vitest run tests/api/demo-session-start.test.ts
  • npm run demo:media with storyboard fallback
  • git diff --check

Dependabot notes:

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7dfb4bdcf4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +11 to +12
const apiKey = process.env.ADMIN_API_KEY || 'dev-admin-key-12345';
const signingSecret = process.env.DEVICE_WEBHOOK_SECRET || process.env.BACKEND_SIGNING_SECRET || 'dev-secret';

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Set server env for v1 integration defaults

When this suite is run directly via npm run test:api:server:only or against a local server that was not launched by scripts/test-api-server.ts, these fallback client credentials do not configure the Next.js server. The updated v1 route now fails closed with API_KEY_NOT_CONFIGURED / SIGNING_SECRET_NOT_CONFIGURED, so the authenticated and signed-request tests send dev-admin-key-12345/dev-secret but receive 500s instead of the expected 200s. Move these defaults into the server bootstrap for every server-dependent test entry point or require the env vars before running the assertions.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bee30e6641

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/e2e.yml
Comment on lines +32 to +36
- name: Setup Node.js
uses: actions/setup-node@v4
with:
bun-version: latest
node-version: '22'
cache: npm

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Install Bun before running Playwright

In this workflow I checked playwright.config.ts, where the Playwright web server is still configured as command: 'bun run dev'; after replacing the Bun setup with actions/setup-node, no step installs Bun before npm run test:e2e. On a clean GitHub runner this means Playwright cannot start the app server and the E2E job fails with bun not found before any tests run.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f2610cc975

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

bodyText.includes('retry') ||
bodyText.includes('replay') ||
bodyText.includes('delete') ||
dlqUrlPattern.test(url);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require DLQ content instead of the current URL

The first test already covers acceptable URLs; in this second test, adding dlqUrlPattern.test(url) makes hasDLQContent true for any response that stays on /admin/dlq, including a 404 or blank page. I checked src/app and DLQView is only rendered from the /admin tab switch at src/app/admin/page.tsx:113, so this assertion no longer proves the replay/delete controls or an auth redirect are present.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc94754d2c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +23 to +24
'not found',
'404',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Disallow 404s in the policies smoke test

When /admin/policies is missing or misrouted, the Next 404 page includes not found/404, so this test now passes instead of catching that the policies page is unavailable. I checked the app routing with rg and src/app/admin/page.tsx, where PoliciesView is only rendered inside the /admin tab switch, so accepting these signals masks the exact broken-route case this E2E should protect.

Useful? React with 👍 / 👎.

@DanFashauer
DanFashauer merged commit efb9636 into main May 18, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant