-
Notifications
You must be signed in to change notification settings - Fork 2
Factor out shared build, test, and release infrastructure #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
d1661c8
Import OAS infrastructure files verbatim
handrews 3f941a3
Add shared OAI build infrastructure package
handrews a07cc36
Centralize spec test dependencies
handrews 65eb52c
Add shared release lifecycle commands
handrews 58e6828
Add npm dependency update groups
handrews 23047ca
Document shared infrastructure maintenance
handrews 661c7bb
Add self-contained release command tests
handrews e3c4283
Clarify build-infra lockfile behavior
handrews 7c57c8f
Fix npm ci optional dependency lockfile
handrews a9f98cf
Resolve hoisted consumer tool binaries
handrews fdf1c57
Fix schema publishing in source mode
handrews 3921b2f
Expand build-infra fixture coverage
handrews 06e9195
Generalize specification HTML builds
handrews db50a5f
Track GitHub Actions dependency updates
handrews File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| version: 2 | ||
| updates: | ||
| - package-ecosystem: github-actions | ||
| directory: "/" | ||
| schedule: | ||
| interval: daily | ||
| open-pull-requests-limit: 10 | ||
|
|
||
| - package-ecosystem: npm | ||
| directory: "/" | ||
| schedule: | ||
| interval: daily | ||
| open-pull-requests-limit: 10 | ||
| groups: | ||
| vitest: | ||
| patterns: | ||
| - "*vitest*" | ||
| hyperjump: | ||
| patterns: | ||
| - "@hyperjump/*" | ||
| markdown: | ||
| patterns: | ||
| - "markdown-*" | ||
| - "markdownlint-*" | ||
| publishing: | ||
| patterns: | ||
| - "respec" | ||
| - "@umbrelladocs/linkspector" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| name: Test | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: | ||
| - main | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| test: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Check out repository | ||
| uses: actions/checkout@v7 | ||
|
|
||
| - name: Set up Node.js | ||
| uses: actions/setup-node@v6 | ||
| with: | ||
| node-version: "24.x" | ||
| cache: npm | ||
|
|
||
| - name: Install dependencies | ||
| run: npm ci | ||
|
|
||
| - name: Run tests | ||
| run: npm test |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| node_modules/ | ||
| coverage/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,215 @@ | ||
| # Contributing To Build Infrastructure | ||
|
|
||
| The goal is to keep the specification repositories uncluttered: each spec repo | ||
| should call clear npm scripts, while this package owns the tool versions and | ||
| reusable logic. | ||
|
|
||
| ## Mental Model | ||
|
|
||
| There are four layers: | ||
|
|
||
| 1. Specification repositories contain Markdown, schemas, tests, workflows, and | ||
| `spec.config.json`. | ||
| 2. Their `package.json` scripts call commands installed by `@oai/build-infra`. | ||
| 3. This repository implements those commands and owns the JavaScript | ||
| dependencies they need. | ||
| 4. GitHub Actions in each specification repository run the same npm scripts that | ||
| maintainers run locally. | ||
|
|
||
| If something is reusable across specification repositories, put it here. If it is | ||
| specific to one repository's governance, labels, reviewers, or branch policy, | ||
| leave it in that repository. | ||
|
|
||
| ## Command Overview | ||
|
|
||
| Build and validation: | ||
|
|
||
| * `oai-spec-build` renders Markdown to HTML using Markdown-it and ReSpec. | ||
| * `oai-spec-format-markdown` applies shared Markdown formatting. | ||
| * `oai-spec-validate-markdown` runs markdownlint and linkspector. | ||
| * `oai-spec-publish-schemas` publishes dated JSON schema iterations. | ||
|
|
||
| Tests: | ||
|
|
||
| * `oai-spec-test` runs `c8` and `vitest`. | ||
| * `@oai/build-infra/test` re-exports Vitest helpers. | ||
| * `@oai/build-infra/schema/test-config` registers YAML schema loading and any | ||
| configured custom vocabulary keywords. | ||
| * `@oai/build-infra/schema/vitest` re-exports schema coverage helpers. | ||
|
|
||
| Release lifecycle: | ||
|
|
||
| * `oai-spec-start-release` starts the next development PR branch. | ||
| * `oai-spec-adjust-release-branch` prepares a release branch for merge to | ||
| `main`. | ||
|
|
||
| ## Before Changing Code | ||
|
|
||
| Read the consuming repository's `spec.config.json` and `package.json` first. | ||
| Most behavior is configured there. | ||
|
|
||
| Be especially careful with release commands. They create branches, commit files, | ||
| delete configured paths, and may push branches. Test release-command changes in a | ||
| scratch repository before asking maintainers to trust them. | ||
|
|
||
| ## Testing | ||
|
|
||
| Run the package tests: | ||
|
|
||
| ```sh | ||
| npm test | ||
| ``` | ||
|
|
||
| These tests include self-contained fixture repositories. They exercise the | ||
| public commands against temporary consumer-shaped Git repositories, so they can | ||
| run locally and in GitHub CI without another checkout. | ||
|
|
||
| For changes that affect behavior not covered by those fixtures, also test in at | ||
| least one specification repository with a temporary local dependency: | ||
|
|
||
| ```json | ||
| { | ||
| "dependencies": { | ||
| "@oai/build-infra": "file:../build-infra" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Then run the relevant consumer scripts: | ||
|
|
||
| ```sh | ||
| npm ci | ||
| npm test | ||
| npm run validate-markdown | ||
| npm run build | ||
| npm run build-src | ||
| ``` | ||
|
|
||
| Not every repository has all of those scripts. | ||
|
|
||
| Before committing consumer changes, change the dependency back to the GitHub | ||
| dependency and refresh the lockfile after the build-infra commit is available on | ||
| GitHub. | ||
|
|
||
| ## Dependency Maintenance | ||
|
|
||
| This repository owns most npm dependencies for the specification repositories. | ||
| Dependabot is configured here for npm updates. | ||
|
|
||
| When Dependabot opens a pull request: | ||
|
|
||
| 1. Read the release notes for major updates and security updates. | ||
| 2. Run `npm ci` and `npm test`. | ||
| 3. If the update touches build, markdown, schema, or test behavior, test a | ||
| consumer repository with the local `file:../build-infra` dependency. | ||
| 4. Merge the build-infra update. | ||
| 5. In each consumer repository that should pick up the change, run: | ||
|
|
||
| ```sh | ||
| npm update @oai/build-infra | ||
| ``` | ||
|
|
||
| 6. Commit the consumer repository's `package-lock.json` update. | ||
|
|
||
| The consumer `package.json` should keep requesting | ||
| `git+https://github.com/OAI/build-infra.git#main`. The consumer | ||
| `package-lock.json` records that request at the root of the lockfile, and | ||
| records the exact resolved Git commit under | ||
| `packages["node_modules/@oai/build-infra"].resolved`. That resolved commit is | ||
| intentional: it prevents CI from silently changing behavior because | ||
| `OAI/build-infra` moved forward. | ||
|
|
||
| ### Lockfile Maintenance Warning | ||
|
|
||
| Use `npm ci` as the normal install command in this repository and in consumer | ||
| specification repositories, including on macOS. Do not use `npm install` merely | ||
| to get a working local `node_modules` tree. | ||
|
|
||
| This matters because npm has sometimes produced an incomplete `package-lock.json` | ||
| for platform-specific optional dependencies on macOS. Known failure modes | ||
| include omitting optional peer-resolution entries for packages such as | ||
| `@emnapi/core` and `@emnapi/runtime`, or leaving stale transitive entries from a | ||
| previous dependency tree. The lockfile can appear to work locally but then fail | ||
| in GitHub Actions, where `npm ci` checks the lockfile strictly on Linux. | ||
|
|
||
| Only use `npm install`, `npm update`, or similar commands when you are | ||
| intentionally creating or changing a lockfile. After any dependency change in | ||
| this repository or in a consumer repository: | ||
|
|
||
| 1. Run `npm ci`. | ||
| 2. Run `npm test`. | ||
| 3. Run any repository-specific build or validation scripts. | ||
| 4. Check that GitHub Actions also passes `npm ci`. | ||
| 5. If `npm ci` says packages are missing from the lockfile, fix the lockfile and | ||
| re-run `npm ci`. Do not paper over the problem by switching CI to | ||
| `npm install`. | ||
|
|
||
| When setting up a new specification repository whose only npm dependency is | ||
| `@oai/build-infra`, the consumer `package-lock.json` should contain the | ||
| dependency tree needed by the resolved build-infra commit. If `npm ci` reports | ||
| missing or invalid transitive packages after adding build-infra, compare the | ||
| consumer lockfile with this repository's verified `package-lock.json` and make | ||
| sure the consumer lockfile includes the same transitive package entries. This is | ||
| especially important for optional dependencies, because those are where | ||
| platform-specific lockfile gaps usually appear. | ||
|
|
||
| ## Release Command Maintenance | ||
|
|
||
| The release commands are intentionally conservative. | ||
|
|
||
| `oai-spec-adjust-release-branch`: | ||
|
|
||
| * must run on a branch named `vX.Y.Z-rel`; | ||
| * requires a clean working tree; | ||
| * copies the active source Markdown to `versions/X.Y.Z.md`; | ||
| * replaces `| TBD |` with the current date; | ||
| * copies `EDITORS.md` to `versions/X.Y.Z-editors.md`, unless disabled; | ||
| * removes paths listed in `release.removeOnReleaseBranch`. | ||
|
|
||
| `oai-spec-start-release`: | ||
|
|
||
| * must run on a branch named `vX.Y-dev`; | ||
| * requires a clean working tree; | ||
| * finds the latest published version under `versions/` on the configured main | ||
| branch; | ||
| * creates `vX.Y-dev-start-X.Y.Z`; | ||
| * resets the active source Markdown history from the previous published version; | ||
| * updates the version heading and history table; | ||
| * optionally rewrites schema/test files for a new minor version; | ||
| * pushes the branch unless `--no-push` is used. | ||
|
|
||
| Use `--no-push` in scratch tests. | ||
|
|
||
| ## Common Failure Modes | ||
|
|
||
| * `npm ci` fails in a consumer repository: the package lock's resolved | ||
| build-infra commit may not be reachable from GitHub yet, `package.json` and | ||
| `package-lock.json` may disagree about the requested dependency, or the | ||
| consumer lockfile may be missing transitive entries from build-infra's | ||
| dependency tree. Verify with `npm ci` before opening the pull request. | ||
| * Release command says the working tree is dirty: commit or stash local changes | ||
| first. These commands intentionally refuse to mix release edits with unrelated | ||
| work. | ||
| * `oai-spec-start-release` cannot find published versions: check the configured | ||
| remote, main branch, and `versions/` directory. | ||
| * Generated HTML looks wrong: check `spec.config.json` first, especially `slug`, | ||
| `shortName`, `titleName`, `specSrc`, and metadata links. | ||
|
|
||
| ## What Belongs Here | ||
|
|
||
| Good candidates for this repository: | ||
|
|
||
| * shared command-line tools; | ||
| * shared JavaScript dependency versions; | ||
| * Markdown, link, schema, and ReSpec behavior; | ||
| * release lifecycle mechanics used by multiple specification repositories; | ||
| * templates for new specification repositories. | ||
|
|
||
| Keep these in individual specification repositories: | ||
|
|
||
| * contributor policy; | ||
| * branch sync policy; | ||
| * CODEOWNERS; | ||
| * issue templates; | ||
| * labels, reviewers, and pull request wording; | ||
| * one-off scripts for that repository's issue management or governance. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.