Skip to content

Implement comprehensive testing setup and CI improvements for extension features#19

Merged
navidshad merged 28 commits into
mainfrom
dev
May 4, 2026
Merged

Implement comprehensive testing setup and CI improvements for extension features#19
navidshad merged 28 commits into
mainfrom
dev

Conversation

@navidshad
Copy link
Copy Markdown
Collaborator

@navidshad navidshad commented May 4, 2026

📋 Summary

This PR introduces a full testing infrastructure including unit, Vue component, and end-to-end (E2E) tests for the extension. It refactors core scripts for better modularity and communication, adds CI enhancements like cleaner typecheck outputs, Playwright report uploads, and gating releases based on test verification. Additional improvements include documentation updates and UX enhancements in the popup translate feature.

🔗 Related Tasks

#86exfn1e3 - Implement test series for the extension, including Tier 1-3 tests and CI pipeline improvements
#86exfjner - Implement single translate on popup page with loading states and UI refresh
#86et9bk39 - Add support for learning from articles/Wikipedia with event-based messaging bridge refactor

📝 Additional Details

  • Added Vitest scaffold and unit tests for core logic
  • Created Tier 2 Vue component and store tests
  • Developed Tier 3 Playwright E2E tests covering popup translate input and Persian translate-save flow
  • Refactored ConsoleCrane into standalone content script with cross-bundle communication bridge
  • Enhanced CI to produce cleaner typecheck summaries and upload reports on success
  • Updated documentation to cover testing setup and environment routing in release pipeline
  • Introduced bootstrap loader to improve popup initial load experience

📜 Commit List

  • ddeb5e4 docs: document the testing setup in CLAUDE.md #86exfn1e3
  • e63f078 ci: clean typecheck summary output and upload playwright report on success too #86exfn1e3
  • eae7598 ci: write non-empty stub env values + fix typecheck filter for prefixed log lines #86exfn1e3
  • 120bc19 ci: restore --headless=new and add Linux sandbox flags so extensions load #86exfn1e3
  • aed4dc3 ci: drop --headless=new arg and emit playwright HTML report in CI #86exfn1e3
  • 13f31b0 ci: filter dashboard-app paths from typecheck and gate playwright report upload #86exfn1e3
  • a2d4e7d test: cover popup translate input and Persian translate-save E2E #86exfn1e3
  • 3b5471d test: round-out Tier-3 with route-params fix, visual + lifecycle E2E, typecheck gate #86exfn1e3
  • be85f93 ci: gate releases on a verify job that runs unit and e2e tests #86exfn1e3
  • ed950e4 test: add Tier-3 Playwright E2E with unpacked extension #86exfn1e3
  • 22c4a37 docs: note env-routing in release-pipeline step 3
  • 3c67b53 test: add Tier-2 Vue component and store tests #86exfn1e3
  • 1f58e69 test: add Vitest scaffold and Tier-1 unit tests #86exfn1e3
  • 24f087b feat: add prerelease support for dev branch with environment-specific changelogs and configs
  • 9c89f83 feat: add loading state and section divider to popup translate
  • a0968ec feat: refresh popup help view to cover web text and quick translate #86exfjner
  • efb435c feat: add translate input on popup home view
  • 76ad53a refactor: decouple ConsoleCrane into a standalone content script and implement a cross-bundle bridge for modal communication
  • 5122962 refactor: decouple ConsoleCrane from UI components by implementing an event-based messaging bridge #86et9bk39
  • 02c59f8 feat: add bootstrap loader to popup for improved initial load experience

navidshad and others added 28 commits May 2, 2026 14:54
…Learning-from-ArticlesWikipedia_Navid-Shad

Refactor ConsoleCrane to use event-based messaging bridge for UI decoupling #86et9bk39
…implement a cross-bundle bridge for modal communication
Adds an inline translation card at the top of the popup so users can
translate ad-hoc text without needing a webpage. The result reuses the
existing WordDetailModule + SaveWordSectionV2 to render detailed
linguistic data and the bundle-save flow, so the popup matches the UX
users already know from selecting a word inside a video subtitle.

Translation works for both logged-in and logged-out users; only Save is
gated. The popup's auth guard no longer redirects logged-out users away
from `/`, and HomeView surfaces a "Log in" footer link when signed out.

Also extracts `encodeRouteParams` / `decodeRouteParams` into a standalone
`src/console-crane/route-params.ts` module to break a circular import
that surfaced once WordDetailModule was bundled into popup.js
(WordDetailModule -> store -> router -> WordDetailModule). WordDetailModule
now accepts optional `word` / `context` props with the existing `:data`
route-param path as a fallback, so it can be mounted from the popup
without inventing a fake route. Console-crane's call paths are unchanged.

#86exfjner

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…86exfjner

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Translate button shows a spinner + "Translating…" while the fetch
  is in flight; disabled while loading. WordDetailModule emits a
  `loading: boolean` event mirroring its internal pending ref so any
  caller can reflect the state, and TranslateCard skips no-op
  resubmits (same word) that wouldn't trigger the prop watcher and
  would otherwise leave the spinner stuck.
- HomeView gets a subtle horizontal gradient divider between the
  translate card and the settings cards below so the result detail
  doesn't visually bleed into the rest of the popup.
- CLAUDE.md: document the new WordDetailModule prop-driven mounting
  mode, the cross-bundle reuse carve-out (WordDetailModule etc. are
  reusable from non-content-script bundles like the popup), the new
  route-params.ts helper file location, and the popup translate
  verification flow.
- Ignore /.claude config dir in .gitignore.

#86exfjner

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# [1.11.0-dev.1](v1.10.1...v1.11.0-dev.1) (2026-05-03)

### Features

* add bootstrap loader to popup for improved initial load experience ([02c59f8](02c59f8))
* add prerelease support for dev branch with environment-specific changelogs and configs ([24f087b](24f087b))
…-translate-on-popup-page_Navid-Shad

Enhance Popup Translation UI with Loading State, Input, and Help View Refresh #86exfjner
# [1.11.0-dev.2](v1.11.0-dev.1...v1.11.0-dev.2) (2026-05-03)

### Features

* add loading state and section divider to popup translate ([9c89f83](9c89f83)), closes [#86exfjner](https://github.com/codebridger/subturtle-extension-apps/issues/86exfjner)
* add translate input on popup home view ([efb435c](efb435c)), closes [#86exfjner](https://github.com/codebridger/subturtle-extension-apps/issues/86exfjner)
* refresh popup help view to cover web text and quick translate [#86](https://github.com/codebridger/subturtle-extension-apps/issues/86)exfjner ([a0968ec](a0968ec)), closes [#86exfjner](https://github.com/codebridger/subturtle-extension-apps/issues/86exfjner)
Bring the repo from zero automated tests to a baseline of regression
cover on the highest-signal pure-logic surfaces. All 48 tests run in
under 500ms.

- vitest.config.ts: happy-dom env; bypasses the webpack-targeted
  postcss config (which Vite's loader rejects) since unit tests don't
  import CSS.
- tests/setup.ts: minimal hand-rolled chrome.* shim, mixpanel module
  mock, console.log silencer.
- tests/route-params.test.ts: parametrized round-trip across ASCII,
  accented Latin, Persian, CJK, emoji, mixed - the regression net for
  the documented btoa/Latin1 InvalidCharacterError class.
- tests/console-crane-bridge.test.ts: payload integrity, multi-listener
  fanout, unsubscribe semantics, channel independence.
- tests/translate.service.test.ts: cache hit/miss across (text,
  context, type, language) and 24h TTL eviction via vi.useFakeTimers.
- tests/settings-host.test.ts: setNibbleDisabledForHost host
  normalization (case fold, www. strip, dedup, removal, subdomain
  isolation, sendMessage fan-out).
- tests/language-detection.test.ts: RTL detection, title lookup,
  supported-codes registry, setLanguage storage write.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Layer Vue component coverage on top of the Tier-1 unit baseline. 18 new
tests across 3 files; the full suite is now 66 tests in ~630ms.

- vitest.config.ts: register @vitejs/plugin-vue so .vue SFCs compile
  through the same Vite pipeline.
- @vue/test-utils, @pinia/testing, @vitejs/plugin-vue added as devDeps.
- tests/console-crane-store.test.ts: toggleConsoleCrane activation
  flow, history push/dedup, force-active, Unicode-safe params, goBack
  pop+route, canGoBack, isOnMainPage, resetHistory. Router mocked via
  vi.hoisted so the test doesn't drag in the real page component
  graph.
- tests/selection-popup.test.ts: regression for the "popup deselects
  page text" bug — root mousedown must call preventDefault (.prevent)
  and must not bubble to document (.stop).
- tests/nibble-surface.test.ts: regression for the "modal closes
  Nibble" bug class — emitState({isActive:true}) must hide the popup,
  emitState({isActive:false}) must show it again, and the bridge
  unsubscribe must be clean across remounts. useTextSelection is
  mocked with real Vue refs so isVisible can be driven from tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 3 of "How a release runs" now references the job's `environment:`
line and which keys it routes to per-branch, with a forward link to the
"Required GitHub Actions config" section that already documents the
prod/dev environment vs repo-level split in detail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Layer end-to-end coverage on top of the Tier-1 unit and Tier-2
component tests. 6 specs across 2 files run against a real Chromium
with `dist/` loaded as an unpacked MV3 extension; the suite is ~5s
once the browser is warm.

- playwright.config.ts: testDir tests/e2e, single worker (extensions
  don't parallelize cleanly), webServer boots the fixtures server.
  spec/test split keeps Vitest and Playwright from fighting over file
  ownership.
- tests/e2e/server.mjs: ~30-line static-file server, no extra dep.
- tests/e2e/extension-fixture.ts: chromium.launchPersistentContext
  with --load-extension=dist plus --disable-extensions-except so the
  test browser only sees this extension. Exposes serviceWorker and
  extensionId fixtures.
- tests/e2e/fixtures/{index,persian,large-font}.html: minimal HTML
  fixtures for English, RTL Persian, and large-html-font-size pages.
  Generic content-script paths (nibble + console-crane match
  <all_urls>) so we never need to touch real youtube.com /
  netflix.com.
- tests/e2e/dist-artifacts.spec.ts: fs-only checks (required entry
  files, no orphan numeric chunks, manifest declares main / nibble /
  console-crane content scripts).
- tests/e2e/nibble-flow.spec.ts: nibble + console-crane roots mount
  on a generic page, exactly one console-crane root in the DOM,
  emitOpen with Persian + emoji + CJK params throws no
  InvalidCharacterError, double-clicking a word shows the Subturtle
  icon.
- vitest.config.ts: exclude tests/e2e/** and **/*.spec.ts so the
  Vitest run doesn't try to collect the Playwright specs.
- package.json: add `test:e2e` script.
- .gitignore: ignore /playwright-report and /test-results.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…n1e3

Add a verify job that runs on every push and pull_request to main/dev,
and make the release job depend on it. Failing tests now block the
release commit, tag, and zip upload.

- New verify job: checkout + dashboard-app sibling clone + Node 22 +
  yarn install + cached Playwright Chromium + `yarn test` (Vitest) +
  stub .env.production from .env.example + `yarn build` +
  `yarn test:e2e` (Playwright).
- ~/.cache/ms-playwright cached on yarn.lock hash so cold runs pay
  the ~150MB Chromium download once and warm runs skip it.
- The verify build copies .env.example to .env.production verbatim;
  dotenv-webpack `safe: true` only requires the keys to exist, and
  empty values are fine for a dist that's only loaded into Playwright.
- The release job is unchanged in body but now has `needs: verify`
  and `if: github.event_name == 'push'` so it skips on PRs and only
  fires after verify is green.
- Top-level `on:` adds `pull_request` to main/dev so PRs get the same
  gate before merge.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… typecheck gate #86exfn1e3

Three follow-ups bundled to keep the PR focused: a small route-params
production fix that the Tier-1 tests had surfaced, two new E2E specs
against the loaded extension, and a typecheck round that fixes 8 of 9
pre-existing TS errors and wires the check into the verify gate.

Test totals after this lands: 68 unit + 9 E2E (was 66 + 6).

route-params undefined round-trip fix:
- src/console-crane/route-params.ts: encode skips when JSON.stringify
  returns undefined; decode returns undefined for empty input.
  Production path: toggleConsoleCrane(page) without explicit params no
  longer throws "Unexpected end of JSON input" on the next decode.
- tests/route-params.test.ts: restores the two undefined-round-trip
  cases removed during Tier 1.

E2E specs:
- tests/e2e/visual-scale.spec.ts: opens ConsoleCrane on a 16px-html
  fixture and a 24px-html fixture, reads computed font-size of a
  text-sm element inside the modal, asserts the rendered px is
  identical. Regression net for the postcss rem→px rewrite.
- tests/e2e/console-crane-lifecycle.spec.ts: opens the modal, writes
  nibbleDisabledDomains:['localhost'] via the extension service worker,
  asserts the modal stays visible and body scroll stays locked. Paired
  test confirms scroll is restored when the modal is later closed.
  Regression net for the "modal lifecycle is decoupled from the Nibble
  per-host gate" contract documented in CLAUDE.md.

Typecheck round (8 of 9 errors fixed; 1 third-party suppressed):
- src/vue-shim.d.ts: replace the Vue-2-style default export with the
  canonical Vue 3 DefineComponent ambient declaration. Clears the 7
  errors in src/console-crane/router.ts and src/popup/router.ts where
  imported .vue files were typed as the bare vue module namespace.
- src/background.ts: cast chrome.tabs.sendMessage to Promise<unknown>.
  @types/chrome@0.0.193 still types it as void; in MV3 the no-callback
  overload returns a Promise. Clears 1 error.
- scripts/typecheck.mjs: wrap tsc --noEmit and filter out the one
  remaining third-party error in node_modules/pilotui/src/vue.ts.
  pilotui's package.json points exports.types at raw TS source, so tsc
  follows into a file with a mismatched plugin signature against
  vue3-perfect-scrollbar — not our code to fix. Real errors in our own
  code still fail the script.
- package.json: "typecheck": "node scripts/typecheck.mjs".
- .github/workflows/release.yml: new "Type check" step in the verify
  job, runs before unit tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…xfn1e3

Two final wish-list items: a Vitest component test for the popup's
TranslateCard input, and a Playwright E2E that stubs the modular-rest
endpoint to cement the full Persian translate-and-save flow. Test
totals after this lands: 79 unit + 11 E2E (was 68 + 9).

tests/translate-card.test.ts (11 tests):
- Auto-focus on mount; disabled when input is empty; disabled on
  whitespace-only input; enabled with text; trims surrounding
  whitespace; renders WordDetailModule with the typed word on submit;
  resets when a different word is submitted; shows the spinner +
  "Translating…" label after submit; clears the spinner when the
  child emits loading=false; disables while pending; ignores
  re-submitting the same word (no double-fetch on enter mash).
- WordDetailModule is stubbed at module-resolve time so the
  modular-rest + auth chain never evaluates.

tests/e2e/translate-flow.spec.ts (2 tests):
- Intercepts POST /function/run via Playwright page.route to return
  a Persian-payload stub for translateWithContext (both simple and
  detailed types); also stubs /user/** to keep anonymous-login error
  noise out of the console capture.
- Spec 1: select 'سلام' on persian.html → click Subturtle icon →
  translated card renders the stubbed Persian content, no
  InvalidCharacterError.
- Spec 2: same flow + click Save → ConsoleCrane modal opens with
  WordDetail rendering 'سلام'. Proves Persian survives
  encodeRouteParams end-to-end through the bridge → store →
  vue-router pipeline that previously crashed under btoa.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ort upload #86exfn1e3

The verify job's typecheck step was failing because src/stores/profile.ts
imports types from the sibling dashboard-app repo. CI clones dashboard-app
without installing its dependencies, so when tsc walks the import chain
(database.type.ts → server/src/modules/phrase_bundle/db.ts) it can't
resolve mongoose / stripe / @modular-rest/server.

Locally the typecheck passes because maintainers usually have
dashboard-app's own node_modules installed.

scripts/typecheck.mjs:
- Extend the suppression list (previously just node_modules/pilotui/)
  to also drop ../dashboard-app/* and dashboard-app/* errors. Same
  rationale as the pilotui filter — upstream code we don't own. Real
  errors in our own src/* still surface.

.github/workflows/release.yml:
- Guard the "Upload Playwright report on failure" step with hashFiles
  so it skips silently when an earlier step (typecheck or unit tests)
  fails before Playwright runs. The previous run produced a "no files
  found" warning artifact that obscured the actual failure.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…exfn1e3

Two changes targeting the verify job's e2e failure:

1. tests/e2e/extension-fixture.ts: drop the explicit --headless=new
   chromium arg. Playwright 1.40+ with Chromium 121+ runs extensions
   cleanly under managed headless; the explicit flag works on macOS
   but appears to mis-toggle on Ubuntu CI runners with the bundled
   Chromium 147 that Playwright pulls. Letting Playwright pick the
   headless mode itself is the documented happy path.

2. playwright.config.ts: emit the HTML report unconditionally
   (previously only in local runs). The verify workflow uploads
   playwright-report/ on failure, but the previous CI run had nothing
   to upload because the list reporter doesn't write artifacts. With
   the html reporter always on, the next failure (if any) will have
   a downloadable report we can actually open.

Local sanity: 11/11 e2e still pass (12.5s).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…load #86exfn1e3

The previous run on Linux CI failed every browser-loading e2e test at
toBeAttached for #subturtle-nibble-root / #subturtle-console-crane-root
— the extension's content scripts never injected. Root cause:

When `headless: true` is in effect (Playwright's default) and no
`--headless=new` arg is passed, Playwright selects the lightweight
`chrome-headless-shell` binary, which does NOT support extensions.
The previous commit dropping `--headless=new` based on macOS behaviour
was wrong; on Linux that flag is what forces the *full* Chromium
binary in new-headless mode, which DOES run content scripts.

Also adding three flags that are standard CI hygiene for Chromium
under GitHub Actions runners — harmless on macOS, sometimes required
on Linux:
- --no-sandbox
- --disable-setuid-sandbox
- --disable-dev-shm-usage

Local sanity: 11/11 e2e still pass (12.4s).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ed log lines #86exfn1e3

Two fixes for the verify gate after the previous --headless=new restore.

scripts/typecheck.mjs:
- The CI run still failed typecheck because GitHub Actions prefixes
  workflow log lines with `Error: `, so the previous filter's
  `startsWith("../dashboard-app/")` check never matched. Replace it
  with a regex that captures the file path before
  `(line,col): error TS<num>:` regardless of any prefix tokens, then
  checks that path against the suppressed-fragment list. Pilotui +
  dashboard-app errors are suppressed; real errors in our code still
  surface.

.github/workflows/release.yml:
- Verify e2e tests were all timing out at toBeAttached on
  #subturtle-nibble-root / #subturtle-console-crane-root despite
  --headless=new. Tracing the failed runs showed content scripts
  start (Trusted Types polyfill logs fire from both nibble.js and
  console-crane.js) but never reach the next top-level statement,
  with zero subsequent network activity. Root cause: the CI build
  was using an empty .env.production (cp from .env.example), and
  `mixpanel.init("")` in src/plugins/mixpanel.ts throws
  synchronously during the content-script import chain — silently
  halting every Vue mount.
- Replace `cp .env.example .env.production` with a heredoc that
  writes non-empty placeholders. SUBTURTLE_API_URL points at the
  local fixtures server (http://localhost:4173) so any auth or
  translate calls hit 404 instead of escaping to the real backend.
  Other vars use harmless ci_e2e_stub_* tokens.

Local sanity: typecheck clean, 11/11 e2e still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ccess too #86exfn1e3

Two cosmetic but real follow-ups after the verify gate went green:

scripts/typecheck.mjs:
- The previous version printed the suppressed pilotui / dashboard-app
  errors verbatim to stdout on clean runs so a future maintainer
  could notice them. GitHub Actions' log parser auto-prefixes any
  line matching `<file>(line,col): error TS<num>:` with a red
  `Error:` annotation, making a passing job look broken.
- Replace the verbose dump with a one-line summary
  ("typecheck clean. (N upstream errors ... suppressed)"). On real
  failures we still print the full tsc output for context. The
  suppression list and rationale stay documented in the file header.

.github/workflows/release.yml:
- Upload the Playwright HTML report on every run (success or
  failure), gated only by hashFiles so we still skip silently when
  typecheck / unit tests fail before Playwright produces any output.
  Makes green-run debugging possible without re-running the suite.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four updates, all aligned with the doc's existing voice:

- Quick start: add yarn test / test:watch / test:e2e / typecheck to
  the script list. Flip npm → yarn to match the actual lockfile.
- New ## Testing section before ## Verification checklist. Covers:
  stack table (Vitest unit/component, Playwright e2e, tsc wrapper);
  Vitest setup (chrome shim, postcss bypass rationale); Playwright
  setup (extension fixture, fixtures server, fixture pages, no real
  YouTube/Netflix); the critical Chromium flags with a "changing
  them breaks CI silently" warning around --headless=new (forces
  full Chromium, otherwise chrome-headless-shell skips extensions);
  the CI verify gate step list with a heredoc-stub-or-mixpanel-throws
  warning around .env.production; typecheck wrapper rationale (the
  pilotui + dashboard-app suppression list); full test file map; and
  totals (79 unit + 11 e2e, ~15s warm).
- ## Verification checklist: re-framed as "most of this is automated"
  with each bullet now cross-referencing the spec file that pins it.
  Residual manual items reduced to YouTube/Netflix subtitle behaviour
  and the popup full-page lifecycle.
- ## Useful pointers: add Vitest config, tests/setup.ts, playwright
  config, extension fixture, fixtures server, typecheck wrapper, and
  the Vue 3 SFC ambient declaration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eries-for-the-extension_Navid-Shad

implement test series for the extension navid shad #86exfn1e3
@navidshad
Copy link
Copy Markdown
Collaborator Author

@navidshad navidshad changed the title Dev Implement comprehensive testing setup and CI improvements for extension features May 4, 2026
@navidshad navidshad merged commit 7847f0e into main May 4, 2026
5 checks passed
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 4, 2026

🎉 This PR is included in version 1.11.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants