-
Notifications
You must be signed in to change notification settings - Fork 45
feat(atomic-a11y): add OpenACR converter modules #7124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
7bd623e
2cf42d6
af3262b
94a34f0
74af535
efa402b
e05f330
8e7bbed
f95508f
b5dc51a
461ce15
eea174c
471838d
b97a25b
7f6c675
f6fd40c
87801d2
0b3caf0
1aedae1
f7b57ef
b4afa13
0042dd9
16d6250
4aee3cd
e0527c0
3d90b19
a6a88bb
bee703d
585cc7d
5ffceab
f25233d
7f145a8
6354e0e
dcba2f0
fcf5b83
e60d0b4
d2744d1
a1d62f2
862c439
23112a0
840e926
0ce8238
7039603
4e8111a
9626d40
61496ce
086864e
0abcf4c
6ec3281
dd2dd0c
56ae711
463405a
6f08cb4
314dcf7
b33828f
7299b3c
39a678c
594b23c
8cd45b3
e5c1445
2f17078
6c4c49f
0450337
bb8b9fd
4a87d4f
698a34c
bf73e99
ca5de58
3e48554
9a05a2c
dd726e3
bc507ee
182b421
232d7c8
dd5493c
5014c96
322e8a6
9d72599
ab20258
8547fa6
9e1eb96
804674f
8ac2a1b
6b3e2d5
d26e8a3
ed2cebe
7760e8d
bbe73d4
f058663
670ca42
a1fa5f5
97f3d67
3c996cc
1f2003e
af007e8
12887e8
b7d4772
0e8ff0b
b672834
790aed8
61404a6
c8d5964
c78af92
691a8a3
0387978
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| # @coveo/atomic-a11y | ||
|
|
||
| **Generated:** 2026-02-17 | ||
| **Commit:** 56ae711d8 | ||
| **Branch:** feat/a11y-reporter | ||
|
|
||
| ## OVERVIEW | ||
|
|
||
| Accessibility auditing and reporting package for Coveo Atomic components. Captures axe-core results from Storybook/Vitest tests, maps them to WCAG 2.2 AA criteria, and emits structured JSON reports. | ||
|
|
||
| ## STRUCTURE | ||
|
|
||
| ``` | ||
| atomic-a11y/ | ||
| ├── src/ | ||
| │ ├── data/ # WCAG criteria definitions + axe-to-WCAG mappings | ||
| │ ├── reporter/ # Vitest reporter (main business logic) → see src/reporter/AGENTS.md | ||
| │ ├── shared/ # Types, constants, guards, sorting utilities | ||
| │ ├── __tests__/ # Empty — tests removed during refactoring | ||
| │ └── index.ts # Public API surface | ||
| ├── scripts/ # WCAG criteria code generator (fetches W3C JSON) | ||
| ├── reports/ # Generated a11y report artifacts (JSON) | ||
| ├── turbo.json # Turbo task config (build outputs only) | ||
| └── package.json # Private package, ESM ("type": "module") | ||
| ``` | ||
|
|
||
| ## WHERE TO LOOK | ||
|
|
||
| | Task | Location | Notes | | ||
| |------|----------|-------| | ||
| | Understand the public API | `src/index.ts` | All exported symbols | | ||
| | Modify report structure | `src/shared/types.ts` | `A11yReport`, `A11yComponentReport`, `A11yCriterionReport`, `A11ySummary` | | ||
| | Change how axe results become WCAG criteria | `src/reporter/axe-integration.ts`, `src/data/axe-rule-mappings.ts` | Tag parsing + prebuilt map | | ||
| | Change how the final JSON is assembled | `src/reporter/report-builder.ts` | `buildA11yReport()` — transforms accumulators into `A11yReport` | | ||
| | Adjust report output paths/defaults | `src/shared/constants.ts` | `DEFAULT_A11Y_REPORT_OUTPUT_DIR`, `DEFAULT_WCAG_22_AA_CRITERIA_COUNT` (55) | | ||
| | Update WCAG criteria data | Run `pnpm generate:wcag` | Auto-generates `src/data/wcag-criteria.ts` from W3C JSON | | ||
| | Change component/category extraction | `src/reporter/storybook-extraction.ts` | Regex-based path parsing | | ||
| | Read package metadata (versions) | `src/reporter/reporter-utils.ts` | `readPackageMetadata()` reads `package.json` for axe-core/storybook versions | | ||
|
|
||
| ## CODE MAP | ||
|
|
||
| | Symbol | Type | Location | Role | | ||
| |--------|------|----------|------| | ||
| | `VitestA11yReporter` | class | `src/reporter/vitest-a11y-reporter.ts` | Core: implements Vitest `Reporter`, accumulates results, writes JSON | | ||
| | `buildA11yReport` | function | `src/reporter/report-builder.ts` | Transforms `Map<string, ComponentAccumulator>` → `A11yReport` | | ||
| | `A11yReport` | interface | `src/shared/types.ts` | Root report shape: `{report, components, criteria, summary}` | | ||
| | `A11yComponentReport` | interface | `src/shared/types.ts` | Per-component automated results | | ||
| | `A11yCriterionReport` | interface | `src/shared/types.ts` | Per-WCAG-criterion coverage/conformance | | ||
| | `A11ySummary` | interface | `src/shared/types.ts` | Rollup metrics: counts, coverage percentages | | ||
| | `ComponentAccumulator` | interface | `src/reporter/reporter-utils.ts` | Mutable per-component state with `Set<string>` for deduplication | | ||
| | `wcagCriteriaDefinitions` | const | `src/data/wcag-criteria.ts` | Auto-generated WCAG 2.2 A+AA criteria (DO NOT EDIT) | | ||
| | `getCriterionMetadata` | function | `src/data/criterion-metadata.ts` | Lookup criterion name/level/version by ID | | ||
| | `buildAxeRuleCriteriaMap` | function | `src/data/axe-rule-mappings.ts` | Builds `Map<ruleId, criteriaIds[]>` from axe-core rules | | ||
| | `isA11yReport` / `isRecord` | function | `src/shared/guards.ts` | Runtime type guards | | ||
| | `compareByNumericId` / `compareByName` | function | `src/shared/sorting.ts` | Locale-aware sorting for criteria IDs and component names | | ||
|
|
||
| ## CONVENTIONS | ||
|
|
||
| - **ESM only**: `"type": "module"` — all imports use `.js` extensions even for `.ts` source files | ||
| - **No path aliases**: Uses relative imports (no `@/` configured in tsconfig) | ||
| - **Strict TypeScript**: `strict: true`, target ES2022, `moduleResolution: NodeNext` | ||
| - **`catalog:` versions**: `typescript` and `vitest` use pnpm catalog references (workspace-level pinning) | ||
| - **Private package**: Not published (`"private": true`), consumed internally within the monorepo | ||
| - **Build = codegen + tsc**: `pnpm build` runs `generate:wcag` first, then `tsc` | ||
| - **Reporter never throws**: All Vitest hooks use `try/catch` with `this.warn()` to avoid breaking test runs | ||
|
|
||
| ## ANTI-PATTERNS (THIS PROJECT) | ||
|
|
||
| - **DO NOT** manually edit `src/data/wcag-criteria.ts` — it is auto-generated; run `pnpm generate:wcag` instead | ||
| - **DO NOT** throw from reporter lifecycle hooks — use `this.warn()` instead | ||
| - Conformance values on criteria are always `'notEvaluated'` in automated reports — manual audit sets final conformance | ||
| - Stencil components are tracked but `stencilExcluded: true` in summary — Lit is the target framework | ||
|
|
||
| ## DATA FLOW | ||
|
|
||
| ``` | ||
| Storybook Test Run | ||
| → Vitest calls onTestCaseResult() per test case | ||
| → Filter: only project.name.startsWith('storybook') | ||
| → Extract: axe results from meta.reports (populated by @storybook/addon-a11y) | ||
| → Map: axe rules → WCAG criteria via tag parsing (wcagXYZ → X.Y.Z) | ||
| → Accumulate: per-component violations/passes/incomplete/inapplicable + criteria coverage | ||
| → onTestRunEnd(): buildA11yReport() → components[] + criteria[] + summary → write JSON | ||
| → Output: a11y-report.json (+ shard variant if --shard CLI flag) | ||
| ``` | ||
|
|
||
| ## COMMANDS | ||
|
|
||
| ```bash | ||
| pnpm build # Generate WCAG data + compile TypeScript | ||
| pnpm test # Run vitest (unit + integration) — currently no test files | ||
| pnpm generate:wcag # Re-fetch WCAG criteria from W3C and regenerate src/data/wcag-criteria.ts | ||
| ``` | ||
|
|
||
| ## NOTES | ||
|
|
||
| - **No tests currently**: `src/__tests__/` is empty — test file was removed during refactoring | ||
| - **Shard support**: Only via `--shard=N/M` CLI flag (env var support was removed); writes both `a11y-report.json` and `a11y-report.shard-N.json` | ||
| - **Component detection**: Relies on `atomic-*` naming convention in module paths and story IDs; non-atomic components silently skipped | ||
| - **Category detection**: Extracted from path segments (`commerce/`, `search/`, `insight/`, `ipx/`, `common/`, `recommendations/`) or story ID prefixes | ||
| - **Framework detection**: `.new.stories.tsx` = Lit, `.stories.tsx` = Stencil | ||
| - **PR chain**: This package is being built incrementally — see `PR-DEPENDENCY-GRAPH.md` for merge order | ||
| - **Report consumers**: Downstream PRs (#7124-#7126) add OpenACR generation, CLI scripts, and wiring into `packages/atomic` | ||
| - **Report builder throws**: Unlike the reporter class, `buildA11yReport()` throws if `axe-core` or `storybook` versions are missing from package metadata |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| # @coveo/atomic-a11y | ||
|
|
||
| Accessibility auditing and reporting for Coveo Atomic components. Captures axe-core results from Storybook/Vitest tests, maps them to WCAG 2.2 AA criteria, and produces structured reports in JSON and [OpenACR](https://github.com/GSA/openacr) YAML (VPAT 2.5). | ||
|
|
||
| > This is a **private** internal package — not published to npm. | ||
|
|
||
| ## Pipeline | ||
|
|
||
| ``` | ||
| Storybook tests (axe-core) | ||
| → VitestA11yReporter → a11y-report.json (per shard) | ||
| → mergeA11yShardReports() → a11y-report.json (merged) | ||
| → transformJsonToOpenAcr() → openacr.yaml → VPAT markdown | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| ```ts | ||
| // vitest.config.ts — capture axe results during Storybook tests | ||
| import { VitestA11yReporter } from '@coveo/atomic-a11y'; | ||
|
|
||
| export default defineConfig({ | ||
| test: { | ||
| reporters: [new VitestA11yReporter({ outputDir: 'reports' })], | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| ```ts | ||
| // After test run — merge shards (if using --shard) | ||
| import { mergeA11yShardReports } from '@coveo/atomic-a11y'; | ||
|
|
||
| await mergeA11yShardReports({ inputDir: 'reports' }); | ||
| ``` | ||
|
|
||
| ```ts | ||
| // Generate OpenACR YAML for VPAT | ||
| import { transformJsonToOpenAcr } from '@coveo/atomic-a11y'; | ||
|
|
||
| await transformJsonToOpenAcr({ | ||
| inputFile: 'reports/a11y-report.json', | ||
| outputFile: 'reports/openacr.yaml', | ||
| }); | ||
| ``` | ||
|
|
||
| ## Scripts | ||
|
|
||
| ```bash | ||
| pnpm build # Generate WCAG data + compile TypeScript | ||
| pnpm test # Run unit tests | ||
| pnpm a11y:merge-shards # Merge shard reports from parallel CI runs | ||
| pnpm a11y:vpat # Generate OpenACR YAML + VPAT markdown | ||
| ``` | ||
|
|
||
| ## Manual audits | ||
|
|
||
| Automated testing covers ~30-40% of WCAG criteria. The rest requires human review. QA creates JSON baseline files that feed into the OpenACR pipeline alongside automated results. | ||
|
|
||
| **→ [Manual Audit Guide](docs/manual-audit-guide.md)** | ||
|
|
||
| ## Structure | ||
|
|
||
| ``` | ||
| src/ | ||
| ├── data/ WCAG criteria definitions (auto-generated) | ||
| ├── reporter/ Vitest reporter + shard merging | ||
| ├── openacr/ JSON → OpenACR YAML converter | ||
| ├── shared/ Types, constants, guards, sorting | ||
| ├── __tests__/ Unit tests | ||
| └── index.ts Public API | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
|
y-lakhdar marked this conversation as resolved.
|
||
| "overrides": [ | ||
| { | ||
| "criterion": "1.4.2", | ||
| "conformance": "not-applicable", | ||
| "reason": "Atomic components do not produce audio output." | ||
| } | ||
| ] | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| [] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The file as is doesn't seem super useful 😅 Should we fill it with sample data? Or just remove it? Or is it just there so that the reports folder is kept? If so we can just add a .gitkeep file instead.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. good point. this is something i forgot to fill. even tho it is explained in the readme. I will update the file and commit it. good catch |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this? We have pnpm a11y:merge-shards and pnpm a11y:vpat
If we don't then we could just remove mergeA11yShardReports and transformJsonToOpenAcr from index.ts.