Skip to content

feat(wfctl): add template validation, contract testing, and compat checking#167

Merged
intel352 merged 1 commit intomainfrom
feat/template-validation-contract-testing
Feb 25, 2026
Merged

feat(wfctl): add template validation, contract testing, and compat checking#167
intel352 merged 1 commit intomainfrom
feat/template-validation-contract-testing

Conversation

@intel352
Copy link
Contributor

Summary

  • Adds wfctl template validate to validate project templates and workflow config files against the engine's known module/step types, dependency resolution, trigger types, and unknown config field detection (with --strict mode)
  • Adds wfctl contract test to generate API contract snapshots (endpoints, modules, steps, events) from configs and compare against baselines to detect breaking changes (removed endpoints, auth added to public routes)
  • Adds wfctl compat check to verify all module/step types in a config are available in the current engine version
  • Adds cmd/wfctl/type_registry.go with a complete static registry of all ~50 module types and ~40 step types extracted from all plugin packages

Test plan

  • go build ./cmd/wfctl/ succeeds
  • go test ./cmd/wfctl/... — all tests pass (9 new test files, ~60 new test cases)
  • wfctl template validate — validates all 5 project templates (api-service, event-processor, full-stack, plugin, ui-plugin)
  • wfctl contract test config.yaml — generates contract JSON
  • wfctl contract test -baseline baseline.json -output current.json config.yaml — detects breaking changes
  • wfctl compat check config.yaml — reports module/step type availability

🤖 Generated with Claude Code

…ecking

Adds three new wfctl commands to ensure templates remain valid across
engine releases and that application configs don't introduce breaking changes:

- `wfctl template validate`: validates project templates and config files
  against the engine's known module/step types, dependency resolution,
  trigger types, and config field warnings (strict mode available)

- `wfctl contract test`: generates API contract snapshots from configs
  (endpoints, modules, steps, events) and compares against baselines to
  detect breaking changes (removed endpoints, added auth requirements)

- `wfctl compat check`: verifies all module and step types in a config
  are available in the current engine version

Also adds `cmd/wfctl/type_registry.go` with a static registry of all
~50 module types and ~40 step types extracted from plugin packages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings February 25, 2026 07:15
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds comprehensive validation, contract testing, and compatibility checking capabilities to the wfctl CLI tool. The PR introduces three new command groups that enable developers to validate workflow configurations against the engine's known types, generate and compare API contracts to detect breaking changes, and verify compatibility between configs and engine versions.

Changes:

  • Adds wfctl template validate command to validate project templates and workflow configs against known module/step types with optional strict mode
  • Adds wfctl contract test command to generate API contract snapshots (endpoints, modules, steps, events) and compare against baselines to detect breaking changes
  • Adds wfctl compat check command to verify all module/step types in a config are available in the current engine version

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
cmd/wfctl/main.go Registers three new command handlers: template, contract, and compat
cmd/wfctl/type_registry.go Defines static registry of ~50 module types and ~40 step types with metadata (plugin, stateful flag, config keys)
cmd/wfctl/type_registry_test.go Comprehensive tests for type registry validation (coverage for populated types, plugin fields, stateful flags, naming conventions)
cmd/wfctl/template_validate.go Implements template validation logic: renders templates with placeholder values, validates module/step types, checks dependencies, reports warnings on unknown config fields
cmd/wfctl/template_validate_test.go Tests for template validation across all templates, config files, error cases, strict mode, and edge cases
cmd/wfctl/contract.go Generates contracts from configs (endpoints, modules, events), compares contracts to detect breaking changes (removed endpoints, auth added to public routes)
cmd/wfctl/contract_test.go Tests for contract generation, comparison logic, breaking change detection, and JSON output
cmd/wfctl/compat.go Checks config compatibility by verifying module/step types against known registry, includes custom insertion sort implementation
cmd/wfctl/compat_test.go Tests for compatibility checking with valid/invalid configs, JSON output, and edge cases

Comment on lines +410 to +412
for key := range stepCfg {
if !knownKeys[key] {
result.Warnings = append(result.Warnings, fmt.Sprintf("pipeline %q step %q (%s) config field %q not in known fields", pipelineName, stepMap["name"], stepType, key))
Copy link

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The warning message on line 412 accesses stepMap["name"] without checking if it exists or is a string. If a step doesn't have a "name" field, this will format as <nil> in the output string, which is not user-friendly.

Consider adding a type assertion:

stepName, _ := stepMap["name"].(string)
if stepName == "" {
    stepName = "(unnamed)"
}
result.Warnings = append(result.Warnings, fmt.Sprintf("pipeline %q step %q (%s) config field %q not in known fields", pipelineName, stepName, stepType, key))
Suggested change
for key := range stepCfg {
if !knownKeys[key] {
result.Warnings = append(result.Warnings, fmt.Sprintf("pipeline %q step %q (%s) config field %q not in known fields", pipelineName, stepMap["name"], stepType, key))
stepName, _ := stepMap["name"].(string)
if stepName == "" {
stepName = "(unnamed)"
}
for key := range stepCfg {
if !knownKeys[key] {
result.Warnings = append(result.Warnings, fmt.Sprintf("pipeline %q step %q (%s) config field %q not in known fields", pipelineName, stepName, stepType, key))

Copilot uses AI. Check for mistakes.
Comment on lines +186 to +193
// sortCompatItems sorts compat items by type name.
func sortCompatItems(items []compatItem) {
for i := 1; i < len(items); i++ {
for j := i; j > 0 && items[j].Type < items[j-1].Type; j-- {
items[j], items[j-1] = items[j-1], items[j]
}
}
}
Copy link

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This manual insertion sort implementation should be replaced with the standard library's sort.Slice for consistency with the rest of the codebase. In contract.go, sort.Slice is used throughout (lines 232, 321, 335, 414, 454, 496), and using the same pattern here would improve maintainability and follow established conventions.

Replace this function with:

sort.Slice(items, func(i, j int) bool {
    return items[i].Type < items[j].Type
})

Copilot uses AI. Check for mistakes.
Comment on lines +529 to +531
// templateFSReader allows reading from the embedded templateFS for validation.
// It wraps around the existing templateFS embed.FS.
var _ fs.FS = templateFS
Copy link

Copilot AI Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment on line 530 describes a wrapper around templateFS, but line 531 only contains a compile-time interface check that doesn't actually wrap or extend any functionality. Either remove the misleading comment or implement an actual wrapper if one is needed.

The compile-time check var _ fs.FS = templateFS is valid but the comment is inaccurate since templateFS is already defined elsewhere (in init.go line 15) and this is just verifying it implements fs.FS.

Suggested change
// templateFSReader allows reading from the embedded templateFS for validation.
// It wraps around the existing templateFS embed.FS.
var _ fs.FS = templateFS
// Compile-time check that templateFS implements fs.FS for use in validation helpers.
var _ fs.FS = templateFS

Copilot uses AI. Check for mistakes.
@intel352 intel352 merged commit eedc88f into main Feb 25, 2026
17 of 18 checks passed
@intel352 intel352 deleted the feat/template-validation-contract-testing branch February 25, 2026 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants