Possibility to pin version of Bikeshed/ReSpec

[tidoust]

Via gpuweb/gpuweb#3222 (comment)

Spec-prod automatically uses the latest version of Bikeshed or ReSpec. Once in a while, an update to the edition tools introduces a bug that breaks builds of a particular spec (in the above example because the tool generates invalid HTML).

Some groups like to take a "dev" approach to spec building, treating editing tools and spec-prod as dependencies. If there was a way to pin the version of the tool being used and a mechanism to detect when a new version is available that creates a PR automatically (not to end up in situations where specs use outdated versions of the tools), that would make it possible to detect that the build breaks before switching to the newer version.

[sidvishnoi]

Something like using requirements.txt and package.json? Sending dependency update PRs could then be responsibility of Dependabot.

[tidoust]

I had mentioned "dependencies" more as a model of thinking in terms of versions than in terms of using a particular dependencies mechanism. That said, if the pinned version could be specified in a requirements.txt or package.json file, we could indeed leverage Dependabot's ability to notify repo owners about new versions.

[MasterKale]

@sidvishnoi Please, please consider finally addressing this issue. Over in WAWG our spec built fine with v5, but builds started suddenly breaking once bikeshed==6.1.0 hit pypi despite no dependency changes within our spec repo. After digging into this we discovered it's due to the versionless pipx install bikeshed --quiet that this action continues to perform that always installs the latest version of bikeshed, even if it changes between CI runs.

To be clear we're not opposed to updating spec text to make bikeshed v6+ happy. But we'd rather wait till after we navigate the W3C spec promotion process and start working on L4 to make further changes. Unfortunately this action is forcing difficult decisions onto us. It's overdue imo to start using specific versions of bikeshed per release of this action.

Thank you for considering this 🙏

[sidvishnoi]

@MasterKale I'm looking into this today & tomorrow.

Plan:

• Lock bikeshed to v5 in next release: v2.12.1
• For future releases, lock runtime/dynamically installed dependencies (ReSpec, Bikeshed, validator etc.):
  • For minor/patch dependency updates, do patch updates to action.
  • For major updates, use minor updates to action. Defer the major dependency updates until stable. When we upgrade bikeshed to v6/7, there will be a v2.13.0 release of action.
  • Do major version update to action only when action API or behavior itself changes.
• The idea is, for most people, @v2 should continue to work as is, without requiring updates to workflows whenever there are major dependency updates. In case there's a breakage, individual specs/workflows can lock version of the action (using w3c/spec-prod@v2.12.1 instead of @v2).

---

Before I lock Bikeshed version in next release, I need to fix #213 first.

Things to be figured out for versioning:

• Whether to end the idea of dynamically installed dependencies and install all dependencies during action setup - this will mean the lock file will include strict versions. Will have separate lockfile for npm and pypi dependencies.
• Alternatively, keep the idea of dynamic dependencies (they're dynamic as not all of them need to be installed for every user), but create a dynamic-dependencies.json file with hardcoded versions of each dependency.
  • This will be updated somehow regularly. Being a separate format means existing tools won't do the upgrades. Maybe can use peerDependencies field and skip automatic installation, but just use lockfile for strict matching?
• Automate release process (weekly releases when there are dependency updates?)

I don't think locking version of just ReSpec/Bikeshed is enough. If locking dependencies, we should lock all and maintain a strong version control.