test(axe): expand accessibility regression coverage - #14713
Merged
Conversation
#14677) PR #14677 vendored axe-core to drop a Skypack CDN dependency so axe checking works offline and never phones home, but nothing guarded against a future revert. Adds two complementary tests: a Playwright behavioral guard that fails if an axe-enabled page makes any request to a known package-CDN host, and a Deno static guard that scans axe-check.js and format-html-axe.ts for hard-coded CDN host references (catching a CDN fallback path the behavioral test can't exercise, since the vendored file always returns 200 in tests).
Commit 4a528a2 made the fixed HTML overlay a focusable native scroll region (overflow-y: auto + tabindex=0, no keydown handler). Coverage was static only: attribute checks plus the axe self-scan. Neither catches the overlay silently losing keyboard scroll while the attributes stay intact (overflow-y flipped away from auto, or a future keydown handler swallowing the keys). Add a Playwright test on the existing axe-overlay-scroll fixture that focuses the overlay, presses PageDown and ArrowDown, and asserts the overlay's scrollTop actually changes.
The report chrome is injected after the page scan runs, so like the HTML overlay it never audits itself. Scan .quarto-axe-report-slide (navigated present, since it is otherwise a hidden future slide) and #quarto-axe-offcanvas plus its sibling .quarto-axe-toggle, asserting zero violations. Guards against a template refactor dropping a button aria-label, alt text, or aria wiring. The dashboard offcanvas body already overflows on this fixture and axe flags scrollable-region-focusable on it (#14710) -- filtered out of this test's assertion rather than using test.fail(), so the test still catches any other regression (e.g. a dropped aria-label) while that issue is open. The reveal report slide has the same latent gap once it overflows (confirmed manually with a throwaway long fixture, not committed here since the rule doesn't fire on this short fixture) but is out of scope for a permanent guard test per #14710 pending a fix decision.
Whole-violation filtering hid a hypothetical second, unrelated scrollable-region-focusable node in the same scan scope as long as the known .offcanvas-body node remained. Filter targets instead so any other node under the rule still fails the test while #14710 is open.
Collaborator
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recent axe-core changes (#14607, #14655, #14667, #14677, #14680) reworked the
accessibility-report feature repeatedly with no dedicated regression tests
guarding several of its behaviors. A coverage audit found three gaps; this
closes them.
Vendor axe-core, drop the Skypack CDN dependency #14677 vendoring it specifically to drop the Skypack CDN.
checked for static
tabindex/roleattributes, never an actual keypress.not the RevealJS report slide or dashboard offcanvas that render the same
violation markup.
Extending the self-scan to RevealJS and the dashboard surfaced a real
pre-existing issue: both report-chrome regions are scrollable but not
keyboard-focusable once they overflow, the same class of gap already fixed
for the plain-HTML overlay. Filed as #14710, not fixed here — the dashboard
test filters that one known node out of the
scrollable-region-focusableviolation (not the whole violation), so it still catches other regressions
in that scope.
Related to #14710