feat(.ai): ai validation in CI by marissahuysentruyt · Pull Request #6136 · adobe/spectrum-web-components

marissahuysentruyt · 2026-04-03T16:17:09Z

Description

⚠️ SECOND PARALLEL MERGE ⚠️
Can be merged directly into feat-agnostic-ai-folder-stuff before or after the washing machine skills PR #6127 or the memory/retro skill PR #6138

Adds Node-based validators under .ai/scripts/ and wires them into yarn lint and the Lint GitHub Actions workflow so AI-related docs and config stay consistent with the rules we expect agents to follow.

Orchestrator

.ai/scripts/validate.js: Runs four checks in sequence, prints per-section pass/fail (errors fail the process; warnings are reported only).

Checks

validate-story-tags.js
a. Scans 2nd-gen *.stories.ts for Storybook tags
b. values must be in the "allowed set" that is aligned with .ai/rules/stories-format.md
c. each file must include migrated somewhere in a tags declaration.
validate-agents-paths.js
a. Finds AGENTS.md files under the repo (skipping common build/vendor dirs)
b. verifies relative markdown links resolve to existing paths so agent bootstrap links do not silently rot.
validate-config-schema.js
a. Validates .ai/config.json structure (branch/Jira/text-formatting rules, etc.) and that regex patterns compile.
validate-symlinks.js
a. Checks that .cursor/rules/*.mdc per-file symlinks exist, are symlinks (not real files), and point to the correct ../../.ai/rules/<name>.md target
b. detects stale .mdc symlinks with no matching .ai/rules/*.md source
c. checks that the three directory symlinks (.cursor/skills, .claude/rules, .claude/skills) exist and point to the correct .ai/ targets

Integration

package.json: extends lint with lint:ai
.github/workflows/lint.yml has a new step to "Validate AI tooling”. It should run yarn lint:ai after install (full repo check, not diff-scoped like some other linters).

Follow-up commit: Minor grammar/wording fix in workflow-related messaging (consistent grammar in workflow steps).

Motivation and context

.ai/ rules and AGENTS.md only help if tags, links, and config stay valid. Automated checks catch drift at PR time instead of in agent sessions.

Related issue(s)

swc-1889

Screenshots (if appropriate)

Author's checklist

I have read the CONTRIBUTING and PULL_REQUESTS documents.
I have reviewed at the Accessibility Practices for this feature, see: Aria Practices
I have added automated tests to cover my changes.
I have included a well-written changeset if my change needs to be published.
I have included updated documentation if my change required it.

Reviewer's checklist

Includes a Github Issue with appropriate flag or Jira ticket number without a link
Includes thoughtfully written changeset if changes suggested include patch, minor, or major features
Automated tests cover all use cases and follow best practices for writing
Validated on all supported browsers
All VRTs are approved before the author can update Golden Hash

Manual review test cases

Locally: yarn lint:ai (or node .ai/scripts/validate.js) passes on this branch
- Intentionally break a story tag or an AGENTS.md link and confirm the relevant validator reports an error.
- Intentionally break a symlink (e.g. rm .claude/rules && mkdir .claude/rules) and confirm the validator reports it as "exists but is not a symlink"
- Make sure to restore any changes afterward!!
Confirm allowed story tags match .ai/rules/stories-format.md and project conventions (including exclusion tags like !test).
Confirm AGENTS.md discovery matches intent (no false positives/negatives for skipped directories).
CI: Lint workflow runs Validate AI tooling and fails the job when validators exit non-zero.

NOTE: AGENTS.md paths will fail until we merge #6137

Device review

Did it pass in Desktop?
Did it pass in (emulated) Mobile?
Did it pass in (emulated) iPad?

Accessibility testing checklist

Required: Complete each applicable item and document your testing steps (replace the placeholders with your component-specific instructions).

Keyboard
_ Adds Node scripts and CI only._
Screen reader
Story-tag validation indirectly supports consistent a11y-tagged stories in 2nd-gen docs.

changeset-bot · 2026-04-03T16:17:14Z

⚠️ No Changeset found

Latest commit: a93e15a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-04-03T16:52:00Z

📚 Branch Preview Links

🔍 First Generation Visual Regression Test Results

When a visual regression test fails (or has previously failed while working on this branch), its results can be found in the following URLs:

Deployed to Azure Blob Storage: pr-6136

If the changes are expected, update the current_golden_images_cache hash in the circleci config to accept the new images. Instructions are included in that file.
If the changes are unexpected, you can investigate the cause of the differences and update the code accordingly.

…skill - The a11y migration analysis doc is a full authoring playbook, not a linting rule. Moves it from .ai/rules/ to .ai/skills/ with a SKILL.md that mirrors the component-migration-analysis skill structure. - Also fixes a stale .cursor/ path reference in component-migration-analysis/SKILL.md.

- Delete .cursor/README.md and .cursor/config.json - Delete all skill files under .cursor/skills/

- Add validate.js entry point running 3 checks, exits with code 1 on errors - Add validate-agents-paths.js — checks relative links in AGENTS.md files - Add validate-config-schema.js — validates .ai/config.json structure - Add validate-story-tags.js — validates tags in 2nd-gen stories files - Add lint:ai script to package.json; include in overall lint command - Add AI tooling validation step to .github/workflows/lint.yml Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

caseyisonit

approving known #6137 will resolve the failing linting CI

marissahuysentruyt force-pushed the marissahuysentruyt/feat-ai-validation branch from 47a03cc to fa10bc7 Compare April 3, 2026 16:37

marissahuysentruyt changed the base branch from marissahuysentruyt/feat-agnostic-folder-ai-stuff to marissahuysentruyt/feat-agent-md-files April 3, 2026 16:38

marissahuysentruyt self-assigned this Apr 3, 2026

marissahuysentruyt added Status:WIP PR is a work in progress or draft do-not-merge NO MERGE-Y! labels Apr 3, 2026

marissahuysentruyt force-pushed the marissahuysentruyt/feat-ai-validation branch from 90c1390 to fdbda95 Compare April 3, 2026 16:55

marissahuysentruyt changed the base branch from marissahuysentruyt/feat-agent-md-files to marissahuysentruyt/feat-agent-memory-retro-skill April 3, 2026 16:55

marissahuysentruyt force-pushed the marissahuysentruyt/feat-agent-memory-retro-skill branch from d51a80a to 86ad45b Compare April 6, 2026 13:15

marissahuysentruyt changed the title ~~Marissahuysentruyt/feat ai validation~~ feat(.ai): ai validation in CI Apr 6, 2026

marissahuysentruyt changed the base branch from marissahuysentruyt/feat-agent-memory-retro-skill to marissahuysentruyt/feat-agnostic-folder-ai-stuff April 6, 2026 13:26

marissahuysentruyt force-pushed the marissahuysentruyt/feat-ai-validation branch from fdbda95 to a06247e Compare April 6, 2026 13:26

marissahuysentruyt force-pushed the marissahuysentruyt/feat-ai-validation branch from a06247e to d5bfa08 Compare April 6, 2026 17:05

marissahuysentruyt added the do-not-merge NO MERGE-Y! label Apr 6, 2026

marissahuysentruyt added 10 commits April 8, 2026 13:21

feat(.ai): .ai/ dir for tool-agnostic support

c856887

docs(.ai): updates readme with .ai references and new skills

ad25fb7

chore(.cursor): remove files relocated to .ai/

e1f184a

- Delete .cursor/README.md and .cursor/config.json - Delete all skill files under .cursor/skills/

feat(.ai): adds .cursor symlinks

0395c95

feat(.ai): add .claude symlinks

d63c149

fix(.ai): updates path references in rules and scripts to .ai

03fc71c

feat: adds new directories to gitignore

5aee790

feat: repo level AGENTS.md

35a8e58

docs(.ai): tweaks to readme language

ee07e57

marissahuysentruyt added 9 commits April 8, 2026 13:21

docs(.ai): overview and reusable prompts

78b3dc8

docs(.ai): fix cursor reference in .ai/rule

d9b6b08

fix(.ai): cursor skills symlink

3d87dbe

feat(.ai): adds missing frontmatter to storybook rules

bcc6554

fix(.ai): contributor docs nav scripts stay out of .ai

373dc21

fix(.ai): handoff references .ai instead of .cursor

095103b

docs(.ai): clarify symlink usage in readme

15b52b3

fix(.ai): missing instructions in a11y migration skill

5419ec0

docs(.ai): update readme with content from agnostic overview file

117b2a4

marissahuysentruyt force-pushed the marissahuysentruyt/feat-agnostic-folder-ai-stuff branch from 7bdadee to 117b2a4 Compare April 8, 2026 17:21

marissahuysentruyt and others added 4 commits April 8, 2026 13:24

fix: uses consistent grammar in workflow steps

7e69e97

feat(.ai): script to validate symlinks

ce0adec

docs(.ai): ci and symlinking in readme

74a0e0d

marissahuysentruyt force-pushed the marissahuysentruyt/feat-ai-validation branch from d5bfa08 to 74a0e0d Compare April 8, 2026 17:24

marissahuysentruyt mentioned this pull request Apr 8, 2026

feat(.ai): washing machine workflow skills #6127

Open

19 tasks

Base automatically changed from marissahuysentruyt/feat-agnostic-folder-ai-stuff to main April 13, 2026 19:25

caseyisonit marked this pull request as ready for review April 13, 2026 19:39

caseyisonit requested a review from a team as a code owner April 13, 2026 19:39

chore: merge main

a93e15a

caseyisonit added Status:Ready for review PR ready for review or re-review. and removed do-not-merge NO MERGE-Y! labels Apr 13, 2026

caseyisonit approved these changes Apr 13, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(.ai): ai validation in CI#6136

feat(.ai): ai validation in CI#6136
marissahuysentruyt wants to merge 24 commits intomainfrom
marissahuysentruyt/feat-ai-validation

marissahuysentruyt commented Apr 3, 2026 •

edited

Loading

Uh oh!

changeset-bot bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

caseyisonit left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

marissahuysentruyt commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and context

Related issue(s)

Screenshots (if appropriate)

Author's checklist

Reviewer's checklist

Manual review test cases

Device review

Accessibility testing checklist

Uh oh!

changeset-bot bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📚 Branch Preview Links

🔍 First Generation Visual Regression Test Results

Uh oh!

caseyisonit left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

marissahuysentruyt commented Apr 3, 2026 •

edited

Loading

changeset-bot bot commented Apr 3, 2026 •

edited

Loading

github-actions bot commented Apr 3, 2026 •

edited

Loading