Skip to content

Add CLI workflow handler and convert wfctl to a workflow config embedded app with full pipeline primitives#277

Merged
intel352 merged 5 commits intomainfrom
copilot/evaluate-workflow-engine-cli
Mar 7, 2026
Merged

Add CLI workflow handler and convert wfctl to a workflow config embedded app with full pipeline primitives#277
intel352 merged 5 commits intomainfrom
copilot/evaluate-workflow-engine-cli

Conversation

Copy link
Contributor

Copilot AI commented Mar 7, 2026

The workflow engine had no support for CLI app workflows. This adds a cli workflow handler type, a cli trigger type, and CLI-specific pipeline steps — then rewires wfctl so every command is a first-class workflow pipeline triggered by the engine rather than a direct Go function call.

New: handlers/cli.goCLIWorkflowHandler

  • Handles the "cli" workflow type
  • CLIWorkflowConfig defines app metadata + a command list (name, description, optional handler key)
  • RegisterCommand(key, fn CLICommandFunc) wires Go implementations to config-declared commands
  • Dispatch(args []string) / DispatchContext(ctx, args) handle --help/--version flags, unknown-command errors, usage generation, and command routing; DispatchContext accepts an explicit context for cancellation and tracing propagation
  • ExecuteWorkflow threads the provided context through to the pipeline dispatcher so cancellation and tracing signals are preserved for programmatic invocations
  • ConfigureWorkflow fully reinitialises the command index on each call so that reconfigured or removed commands never linger
  • Command parsing rejects empty names and duplicate command names with clear errors
  • RegisterService errors are surfaced unless the same handler instance is already registered (idempotent re-registration)
  • Registers itself as "cliWorkflowHandler" service during ConfigureWorkflow so callers can retrieve it from the app service registry
  • Falls back to CLIPipelineDispatcher (implemented by CLITrigger) when no direct Go runner is registered — enabling fully pipeline-backed commands

New: module/cli_trigger.goCLITrigger (trigger.cli)

  • Registered in DefaultTriggers() (now 6 built-in triggers)
  • Configured via inline pipeline trigger configs with type: cli; accumulates command → pipeline:name mappings across multiple calls
  • Rejects conflicting registrations: a different workflowType for an already-mapped command returns an error; re-registering the exact same mapping is idempotent
  • RegisterService errors are surfaced unless the same *CLITrigger instance is already registered
  • DispatchCommand(ctx, cmd, args) calls engine.TriggerWorkflow synchronously, threading the caller's context through
  • Registered as "trigger.cli" service during Configure so CLIWorkflowHandler can discover it lazily from the app

New: module/cli_command_registry.goCLICommandRegistry

  • Shared service ("cliCommandRegistry") that maps CLI command names to CLICommandFunc Go implementations
  • Registered in the app before BuildFromConfig so step.cli_invoke can resolve functions at pipeline execution time

New: step.cli_print

  • Writes a template-resolved message to stdout or stderr
  • Fundamental output primitive for workflow-powered CLI applications
  • Config: message (required), newline (default true), target ("stdout" | "stderr") — unknown target values return a clear config error

New: step.cli_invoke

  • Calls a registered Go CLI function by name from CLICommandRegistry
  • Bridges the pipeline system with existing Go implementations
  • Accepts pc.Current["args"] as either []string (direct Go call) or []any (JSON/YAML round-trip), with a clear error for non-string elements
  • Both new step types registered in plugins/pipelinesteps

Engine: pipeline logger propagation (engine.go)

configurePipelines now injects the engine's *slog.Logger directly into each Pipeline it creates. This means WithLogger(discard) fully suppresses pipeline execution logs without requiring callers to mutate the global slog.Default().

config.LoadFromBytes

New helper to parse a WorkflowConfig from an in-memory []byte — the primitive needed for //go:embed configs.

setup.DefaultHandlers / setup.DefaultTriggers

CLIWorkflowHandler added to DefaultHandlers and CLITrigger added to DefaultTriggers, both available via WithAllDefaults().

cmd/wfctl — every command is now a workflow pipeline

wfctl.yaml now declares a pipelines section with 24 cmd-<name> pipelines, each wired to a cli trigger and a step.cli_invoke step. main.go builds a full engine, registers a CLICommandRegistry with all Go implementations, calls BuildFromConfig, and retrieves the CLIWorkflowHandler from the app service registry for dispatch.

main.go uses signal.NotifyContext + DispatchContext so Ctrl+C cancels the running pipeline cleanly. The engine's discard logger is scoped entirely within the engine (no slog.SetDefault mutation).

The complete dispatch chain for every command is:

wfctl validate myfile.yaml
  → CLIWorkflowHandler.DispatchContext(ctx, ["validate", "myfile.yaml"])
  → CLITrigger.DispatchCommand(ctx, "validate", ["myfile.yaml"])
  → engine.TriggerWorkflow("pipeline:cmd-validate", "", {args: [...]})
  → PipelineWorkflowHandler → Pipeline "cmd-validate"
  → step.cli_invoke → CLICommandRegistry.Get("validate") → runValidate(args)

Pre/post steps (audit logging, telemetry, feature flags) can now be added to any command pipeline in YAML without modifying Go code.

workflows:
  cli:
    name: wfctl
    version: "1.0.0"
    description: Workflow Engine CLI
    commands:
      - name: validate
        description: Validate a config file

pipelines:
  cmd-validate:
    trigger:
      type: cli
      config:
        command: validate
    steps:
      - name: run
        type: step.cli_invoke
        config:
          command: validate
// In main.go — engine-based startup
registry := module.NewCLICommandRegistry()
registry.Register("validate", runValidate)
engineInst.App().RegisterService(module.CLICommandRegistryServiceName, registry)
engineInst.AddStepType("step.cli_invoke", module.NewCLIInvokeStepFactory())
engineInst.BuildFromConfig(cfg)

var cliHandler *handlers.CLIWorkflowHandler
engineInst.App().GetService(handlers.CLIWorkflowHandlerServiceName, &cliHandler)

ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
cliHandler.DispatchContext(ctx, os.Args[1:])
Original prompt

Evaluate workflow engine for its ability to support cli apps. If it needs functionality, extend it. Then convert wfctl to a workflow config embedded app. Test, compile, and execute the new cli functionality to ensure no regression.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

- Add handlers/cli.go: CLIWorkflowHandler for 'cli' workflow type with
  command registration, config-driven dispatch, and usage generation
- Add handlers/cli_test.go: comprehensive tests for CLIWorkflowHandler
- Add config.LoadFromBytes helper for loading configs from embedded bytes
- Register CLIWorkflowHandler in setup.DefaultHandlers()
- Add cmd/wfctl/wfctl.yaml: embedded CLI config declaring all 24 commands
- Rewrite cmd/wfctl/main.go to load embedded config and use CLIWorkflowHandler
  for dispatch, removing hardcoded usage() and command routing

Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>
Copilot AI changed the title [WIP] Evaluate workflow engine for CLI app support and extend functionality Add CLI workflow handler and convert wfctl to a workflow config embedded app Mar 7, 2026
@intel352
Copy link
Contributor

intel352 commented Mar 7, 2026

@copilot convert the remaining cli commands to workflow primitives. Extend the engine where necessary for new functionality that makes sense existing in the engine.

@github-actions
Copy link

github-actions bot commented Mar 7, 2026

⏱ Benchmark Results

No significant performance regressions detected.

benchstat comparison (baseline → PR)
## benchstat: baseline → PR
baseline-bench.txt:243: parsing iteration count: invalid syntax
baseline-bench.txt:302452: parsing iteration count: invalid syntax
baseline-bench.txt:617372: parsing iteration count: invalid syntax
baseline-bench.txt:910002: parsing iteration count: invalid syntax
baseline-bench.txt:1190155: parsing iteration count: invalid syntax
baseline-bench.txt:1488518: parsing iteration count: invalid syntax
benchmark-results.txt:243: parsing iteration count: invalid syntax
benchmark-results.txt:304813: parsing iteration count: invalid syntax
benchmark-results.txt:594377: parsing iteration count: invalid syntax
benchmark-results.txt:889758: parsing iteration count: invalid syntax
benchmark-results.txt:1190269: parsing iteration count: invalid syntax
benchmark-results.txt:1493603: parsing iteration count: invalid syntax
goos: linux
goarch: amd64
pkg: github.com/GoCodeAlone/workflow/dynamic
cpu: AMD EPYC 7763 64-Core Processor                
                            │ baseline-bench.txt │       benchmark-results.txt        │
                            │       sec/op       │    sec/op     vs base              │
InterpreterCreation-4               6.046m ± 67%   6.000m ± 63%       ~ (p=0.937 n=6)
ComponentLoad-4                     3.519m ±  1%   3.485m ±  1%       ~ (p=0.132 n=6)
ComponentExecute-4                  1.999µ ±  2%   1.979µ ±  2%       ~ (p=0.065 n=6)
PoolContention/workers-1-4          1.087µ ±  6%   1.083µ ±  5%       ~ (p=0.838 n=6)
PoolContention/workers-2-4          1.092µ ±  1%   1.084µ ±  0%       ~ (p=0.102 n=6)
PoolContention/workers-4-4          1.093µ ±  1%   1.087µ ±  1%       ~ (p=0.063 n=6)
PoolContention/workers-8-4          1.099µ ±  1%   1.093µ ±  1%       ~ (p=0.558 n=6)
PoolContention/workers-16-4         1.097µ ±  1%   1.094µ ±  1%       ~ (p=0.234 n=6)
ComponentLifecycle-4                3.564m ±  4%   3.499m ±  2%  -1.83% (p=0.015 n=6)
SourceValidation-4                  2.317µ ±  1%   2.285µ ±  2%       ~ (p=0.065 n=6)
RegistryConcurrent-4                763.4n ±  2%   807.4n ±  4%  +5.78% (p=0.002 n=6)
LoaderLoadFromString-4              3.534m ±  2%   3.513m ±  1%       ~ (p=0.132 n=6)
geomean                             18.37µ         18.32µ        -0.28%

                            │ baseline-bench.txt │        benchmark-results.txt         │
                            │        B/op        │     B/op      vs base                │
InterpreterCreation-4               1.944Mi ± 0%   1.944Mi ± 0%       ~ (p=0.937 n=6)
ComponentLoad-4                     2.097Mi ± 0%   2.097Mi ± 0%       ~ (p=0.394 n=6)
ComponentExecute-4                  1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-1-4          1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-2-4          1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-4-4          1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-8-4          1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-16-4         1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
ComponentLifecycle-4                2.099Mi ± 0%   2.099Mi ± 0%       ~ (p=0.792 n=6)
SourceValidation-4                  1.984Ki ± 0%   1.984Ki ± 0%       ~ (p=1.000 n=6) ¹
RegistryConcurrent-4                1.133Ki ± 0%   1.133Ki ± 0%       ~ (p=1.000 n=6) ¹
LoaderLoadFromString-4              2.099Mi ± 0%   2.099Mi ± 0%       ~ (p=0.727 n=6)
geomean                             15.05Ki        15.05Ki       +0.00%
¹ all samples are equal

                            │ baseline-bench.txt │        benchmark-results.txt        │
                            │     allocs/op      │  allocs/op   vs base                │
InterpreterCreation-4                15.09k ± 0%   15.09k ± 0%       ~ (p=1.000 n=6) ¹
ComponentLoad-4                      17.43k ± 0%   17.43k ± 0%       ~ (p=1.000 n=6)
ComponentExecute-4                    25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-1-4            25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-2-4            25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-4-4            25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-8-4            25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-16-4           25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
ComponentLifecycle-4                 17.48k ± 0%   17.48k ± 0%       ~ (p=1.000 n=6) ¹
SourceValidation-4                    32.00 ± 0%    32.00 ± 0%       ~ (p=1.000 n=6) ¹
RegistryConcurrent-4                  2.000 ± 0%    2.000 ± 0%       ~ (p=1.000 n=6) ¹
LoaderLoadFromString-4               17.47k ± 0%   17.47k ± 0%       ~ (p=1.000 n=6) ¹
geomean                               181.2         181.2       +0.00%
¹ all samples are equal

pkg: github.com/GoCodeAlone/workflow/middleware
                                  │ baseline-bench.txt │       benchmark-results.txt       │
                                  │       sec/op       │   sec/op     vs base              │
CircuitBreakerDetection-4                  287.4n ± 5%   286.8n ± 5%       ~ (p=0.699 n=6)
CircuitBreakerExecution_Success-4          22.57n ± 9%   22.55n ± 1%       ~ (p=0.803 n=6)
CircuitBreakerExecution_Failure-4          66.29n ± 1%   66.30n ± 0%       ~ (p=0.981 n=6)
geomean                                    75.47n        75.40n       -0.09%

                                  │ baseline-bench.txt │       benchmark-results.txt        │
                                  │        B/op        │    B/op     vs base                │
CircuitBreakerDetection-4                 144.0 ± 0%     144.0 ± 0%       ~ (p=1.000 n=6) ¹
CircuitBreakerExecution_Success-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
CircuitBreakerExecution_Failure-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                              ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                  │ baseline-bench.txt │       benchmark-results.txt        │
                                  │     allocs/op      │ allocs/op   vs base                │
CircuitBreakerDetection-4                 1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
CircuitBreakerExecution_Success-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
CircuitBreakerExecution_Failure-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                              ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/module
                                 │ baseline-bench.txt │       benchmark-results.txt        │
                                 │       sec/op       │    sec/op     vs base              │
JQTransform_Simple-4                     900.5n ± 25%   893.2n ± 25%       ~ (p=0.589 n=6)
JQTransform_ObjectConstruction-4         1.489µ ±  3%   1.484µ ±  2%       ~ (p=0.457 n=6)
JQTransform_ArraySelect-4                3.345µ ±  1%   3.462µ ±  2%  +3.50% (p=0.002 n=6)
JQTransform_Complex-4                    38.18µ ±  4%   38.83µ ±  2%       ~ (p=0.065 n=6)
JQTransform_Throughput-4                 1.797µ ±  0%   1.844µ ±  1%  +2.62% (p=0.002 n=6)
SSEPublishDelivery-4                     65.72n ±  3%   66.44n ±  1%       ~ (p=0.065 n=6)
geomean                                  1.650µ         1.672µ        +1.29%

                                 │ baseline-bench.txt │        benchmark-results.txt         │
                                 │        B/op        │     B/op      vs base                │
JQTransform_Simple-4                   1.273Ki ± 0%     1.273Ki ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_ObjectConstruction-4       1.773Ki ± 0%     1.773Ki ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_ArraySelect-4              2.625Ki ± 0%     2.625Ki ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_Complex-4                  16.22Ki ± 0%     16.22Ki ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_Throughput-4               1.984Ki ± 0%     1.984Ki ± 0%       ~ (p=1.000 n=6) ¹
SSEPublishDelivery-4                     0.000 ± 0%       0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                             ²                 +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                 │ baseline-bench.txt │       benchmark-results.txt        │
                                 │     allocs/op      │ allocs/op   vs base                │
JQTransform_Simple-4                     10.00 ± 0%     10.00 ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_ObjectConstruction-4         15.00 ± 0%     15.00 ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_ArraySelect-4                30.00 ± 0%     30.00 ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_Complex-4                    324.0 ± 0%     324.0 ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_Throughput-4                 17.00 ± 0%     17.00 ± 0%       ~ (p=1.000 n=6) ¹
SSEPublishDelivery-4                     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                             ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/schema
                                    │ baseline-bench.txt │       benchmark-results.txt       │
                                    │       sec/op       │   sec/op     vs base              │
SchemaValidation_Simple-4                    1.118µ ± 4%   1.110µ ± 1%       ~ (p=0.195 n=6)
SchemaValidation_AllFields-4                 1.678µ ± 6%   1.680µ ± 7%       ~ (p=0.725 n=6)
SchemaValidation_FormatValidation-4          1.606µ ± 3%   1.606µ ± 2%       ~ (p=0.729 n=6)
SchemaValidation_ManySchemas-4               1.801µ ± 5%   1.829µ ± 3%       ~ (p=0.699 n=6)
geomean                                      1.526µ        1.530µ       +0.24%

                                    │ baseline-bench.txt │       benchmark-results.txt        │
                                    │        B/op        │    B/op     vs base                │
SchemaValidation_Simple-4                   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_AllFields-4                0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_FormatValidation-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_ManySchemas-4              0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                                ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                    │ baseline-bench.txt │       benchmark-results.txt        │
                                    │     allocs/op      │ allocs/op   vs base                │
SchemaValidation_Simple-4                   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_AllFields-4                0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_FormatValidation-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_ManySchemas-4              0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                                ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/store
                                   │ baseline-bench.txt │       benchmark-results.txt        │
                                   │       sec/op       │    sec/op     vs base              │
EventStoreAppend_InMemory-4                1.236µ ± 28%   1.195µ ± 14%       ~ (p=0.310 n=6)
EventStoreAppend_SQLite-4                  1.352m ±  6%   1.486m ± 11%       ~ (p=0.065 n=6)
GetTimeline_InMemory/events-10-4           14.46µ ±  3%   13.79µ ±  3%  -4.64% (p=0.006 n=6)
GetTimeline_InMemory/events-50-4           70.29µ ± 12%   62.53µ ± 29%       ~ (p=0.394 n=6)
GetTimeline_InMemory/events-100-4          126.0µ ±  2%   124.1µ ±  1%       ~ (p=0.065 n=6)
GetTimeline_InMemory/events-500-4          639.6µ ±  2%   639.6µ ±  2%       ~ (p=0.937 n=6)
GetTimeline_InMemory/events-1000-4         1.305m ±  1%   1.306m ±  1%       ~ (p=0.937 n=6)
GetTimeline_SQLite/events-10-4             109.0µ ±  1%   108.8µ ±  3%       ~ (p=0.589 n=6)
GetTimeline_SQLite/events-50-4             250.1µ ±  0%   251.5µ ±  2%       ~ (p=0.065 n=6)
GetTimeline_SQLite/events-100-4            423.9µ ±  2%   423.4µ ±  1%       ~ (p=0.394 n=6)
GetTimeline_SQLite/events-500-4            1.801m ±  1%   1.795m ±  0%  -0.31% (p=0.026 n=6)
GetTimeline_SQLite/events-1000-4           3.517m ±  4%   3.512m ±  1%       ~ (p=0.589 n=6)
geomean                                    220.3µ         218.1µ        -1.00%

                                   │ baseline-bench.txt │        benchmark-results.txt         │
                                   │        B/op        │     B/op      vs base                │
EventStoreAppend_InMemory-4                  790.5 ± 6%     806.5 ± 6%       ~ (p=0.331 n=6)
EventStoreAppend_SQLite-4                  1.983Ki ± 2%   1.984Ki ± 1%       ~ (p=0.846 n=6)
GetTimeline_InMemory/events-10-4           7.953Ki ± 0%   7.953Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-50-4           46.62Ki ± 0%   46.62Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-100-4          94.48Ki ± 0%   94.48Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-500-4          472.8Ki ± 0%   472.8Ki ± 0%       ~ (p=0.182 n=6)
GetTimeline_InMemory/events-1000-4         944.3Ki ± 0%   944.3Ki ± 0%       ~ (p=0.076 n=6)
GetTimeline_SQLite/events-10-4             16.74Ki ± 0%   16.74Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-50-4             87.14Ki ± 0%   87.14Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-100-4            175.4Ki ± 0%   175.4Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-500-4            846.1Ki ± 0%   846.1Ki ± 0%       ~ (p=0.303 n=6)
GetTimeline_SQLite/events-1000-4           1.639Mi ± 0%   1.639Mi ± 0%       ~ (p=1.000 n=6)
geomean                                    67.35Ki        67.46Ki       +0.17%
¹ all samples are equal

                                   │ baseline-bench.txt │        benchmark-results.txt        │
                                   │     allocs/op      │  allocs/op   vs base                │
EventStoreAppend_InMemory-4                  7.000 ± 0%    7.000 ± 0%       ~ (p=1.000 n=6) ¹
EventStoreAppend_SQLite-4                    53.00 ± 0%    53.00 ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-10-4             125.0 ± 0%    125.0 ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-50-4             653.0 ± 0%    653.0 ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-100-4           1.306k ± 0%   1.306k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-500-4           6.514k ± 0%   6.514k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-1000-4          13.02k ± 0%   13.02k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-10-4               382.0 ± 0%    382.0 ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-50-4              1.852k ± 0%   1.852k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-100-4             3.681k ± 0%   3.681k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-500-4             18.54k ± 0%   18.54k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-1000-4            37.29k ± 0%   37.29k ± 0%       ~ (p=1.000 n=6) ¹
geomean                                     1.162k        1.162k       +0.00%
¹ all samples are equal

Benchmarks run with go test -bench=. -benchmem -count=6.
Regressions ≥ 20% are flagged. Results compared via benchstat.

…oke, step.cli_print

Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 7, 2026

@copilot convert the remaining cli commands to workflow primitives. Extend the engine where necessary for new functionality that makes sense existing in the engine.

Done in commit 4e00330. Here's what was added:

New engine primitives:

  • module.CLITrigger (trigger.cli) — registered in DefaultTriggers(), maps command → pipeline:cmd-<name> via inline pipeline trigger config. The engine's fallback name matching (trigger.cli for type cli) means no explicit RegisterTriggerType call is needed.
  • module.CLICommandRegistry — service registered in the app before BuildFromConfig; step.cli_invoke resolves functions from it at execution time.
  • step.cli_print — template-resolved stdout/stderr output for CLI workflow apps.
  • step.cli_invoke — calls a named Go function from CLICommandRegistry; bridges pipelines with Go implementations.

wfctl is now fully pipeline-driven: wfctl.yaml declares 24 cmd-<name> pipelines, each with a cli trigger and a step.cli_invoke step. Every command flows through:

wfctl validate myfile.yaml
  → CLIWorkflowHandler.Dispatch
  → CLITrigger.DispatchCommand
  → engine.TriggerWorkflow("pipeline:cmd-validate", …)
  → step.cli_invoke → CLICommandRegistry → runValidate(args)

Pre/post steps (audit, telemetry, feature flags) can now be added to any command pipeline in YAML without touching Go code. Verified: all 24 commands work, all tests pass, zero lint issues.

Copilot AI changed the title Add CLI workflow handler and convert wfctl to a workflow config embedded app Add CLI workflow handler and convert wfctl to a workflow config embedded app with full pipeline primitives Mar 7, 2026
@intel352 intel352 marked this pull request as ready for review March 7, 2026 04:33
Copilot AI review requested due to automatic review settings March 7, 2026 04:33
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class CLI workflows to the workflow engine so CLI commands can be configured and executed as pipelines, and rewires wfctl to dispatch through the engine using an embedded YAML config.

Changes:

  • Introduces handlers/cli.go (CLIWorkflowHandler) and module/cli_trigger.go (trigger.cli) to support workflows.cli and CLI-triggered pipelines.
  • Adds CLI pipeline primitives: step.cli_print and step.cli_invoke, plus a shared CLICommandRegistry service for Go-backed command execution.
  • Converts wfctl to an embedded-config, workflow-driven CLI where each command is a pipeline triggered via type: cli.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
setup/setup.go Registers the new CLI workflow handler and CLI trigger in defaults.
setup/setup_test.go Updates expected default handler/trigger counts.
plugins/pipelinesteps/plugin.go Registers step.cli_print and step.cli_invoke step types/factories.
plugins/pipelinesteps/plugin_test.go Ensures new CLI step types are included in the plugin step factory list.
module/pipeline_step_cli_print.go Implements step.cli_print (templated stdout/stderr output).
module/pipeline_step_cli_print_test.go Adds unit tests for step.cli_print.
module/pipeline_step_cli_invoke.go Implements step.cli_invoke (invokes a Go command from CLICommandRegistry).
module/pipeline_step_cli_invoke_test.go Adds unit tests for step.cli_invoke.
module/cli_trigger.go Implements trigger.cli mapping CLI commands to pipeline:<name> workflows.
module/cli_trigger_test.go Adds unit tests for CLITrigger.
module/cli_command_registry.go Adds the shared CLI command registry service used by step.cli_invoke.
handlers/cli.go Implements CLIWorkflowHandler for workflows.cli with usage/version/help + dispatch.
handlers/cli_test.go Adds unit tests covering dispatch, help/version, handler keys, and pipeline fallback.
config/config.go Adds config.LoadFromBytes([]byte) for embedded YAML configs.
cmd/wfctl/wfctl.yaml Declares workflows.cli commands and maps each to a CLI-triggered command pipeline.
cmd/wfctl/type_registry.go Exposes new CLI step types in wfctl’s type registry metadata.
cmd/wfctl/main.go Embeds wfctl.yaml, builds/configures engine, registers registry + CLI steps, dispatches via CLIWorkflowHandler.

handlers/cli.go Outdated
Comment on lines +119 to +124
func (h *CLIWorkflowHandler) ExecuteWorkflow(_ context.Context, _ string, action string, data map[string]any) (map[string]any, error) {
args, _ := data["args"].([]string)
if err := h.runCommand(action, args); err != nil {
return nil, err
}
return map[string]any{"success": true}, nil
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLIWorkflowHandler.ExecuteWorkflow ignores the provided context, and pipeline-backed commands end up being dispatched with context.Background() (see runCommand). This drops cancellation/timeouts/tracing for programmatic invocations; thread the ctx through to runCommand and into CLIPipelineDispatcher.DispatchCommand instead of discarding it.

Copilot uses AI. Check for mistakes.
Comment on lines +256 to +267
if rawCmds, ok := cfgMap["commands"].([]any); ok {
for i, rc := range rawCmds {
cmdMap, ok := rc.(map[string]any)
if !ok {
return nil, fmt.Errorf("command at index %d is not a map", i)
}
def := CLICommandDef{}
def.Name, _ = cmdMap["name"].(string)
def.Description, _ = cmdMap["description"].(string)
def.Handler, _ = cmdMap["handler"].(string)
cfg.Commands = append(cfg.Commands, def)
}
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseCLIWorkflowConfig accepts commands with an empty or missing name and appends them, which later produces a map entry under the empty string key. This makes dispatch/usage output confusing and can hide misconfigurations. Validate that each command has a non-empty name (and ideally reject duplicates) while parsing or during ConfigureWorkflow.

Copilot uses AI. Check for mistakes.
Comment on lines +178 to +183
// Fallback: pipeline dispatch via CLITrigger found in app services.
if h.app != nil {
for _, svc := range h.app.SvcRegistry() {
if d, ok := svc.(CLIPipelineDispatcher); ok {
return d.DispatchCommand(context.Background(), name, args)
}
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pipeline dispatch uses context.Background(), which prevents CLI-triggered pipeline executions from being cancellable (e.g., Ctrl+C) and drops any upstream context values. After threading ctx through ExecuteWorkflow/Dispatch, pass that ctx into DispatchCommand instead of creating a new background context here.

Copilot uses AI. Check for mistakes.
if workflowType == "" {
return fmt.Errorf("cli trigger: 'workflowType' is required in trigger config (injected by the engine)")
}

Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLITrigger.Configure overwrites an existing command→workflow mapping without warning. If multiple pipelines accidentally declare the same command, the last one wins silently. Return an error (or at least log) when a different workflowType is already registered for the command to avoid ambiguous CLI routing.

Suggested change
// Prevent ambiguous CLI routing: do not silently overwrite an existing
// command → workflowType mapping with a different workflow. Re-registering
// the same mapping is allowed for idempotency.
if existing, ok := t.commands[command]; ok && existing != workflowType {
return fmt.Errorf("cli trigger: command %q already registered for workflow %q (cannot re-register for %q)", command, existing, workflowType)
}

Copilot uses AI. Check for mistakes.
Comment on lines +315 to +326
// containsStr is a simple substring helper.
func containsStr(s, sub string) bool {
return len(sub) == 0 || (len(s) >= len(sub) && (s == sub || len(s) > 0 &&
func() bool {
for i := 0; i <= len(s)-len(sub); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
}()))
}
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The containsStr helper reimplements substring search manually, which is harder to read and maintain in tests. Prefer using strings.Contains (or bytes.Contains) for clarity and to avoid subtle edge-case bugs.

Copilot uses AI. Check for mistakes.
handlers/cli.go Outdated
h.app = app
// Register self so engine consumers can retrieve this handler by name.
if app != nil {
_ = app.RegisterService(CLIWorkflowHandlerServiceName, h)
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RegisterService errors are ignored unconditionally here. If some other service already occupies "cliWorkflowHandler", this will silently fail and callers may retrieve the wrong handler (or none). Consider checking the error and only ignoring the specific "already registered" case when the existing service is the same instance, otherwise return an error.

Suggested change
_ = app.RegisterService(CLIWorkflowHandlerServiceName, h)
if err := app.RegisterService(CLIWorkflowHandlerServiceName, h); err != nil {
return fmt.Errorf("cli workflow: register service %q: %w", CLIWorkflowHandlerServiceName, err)
}

Copilot uses AI. Check for mistakes.
Comment on lines +77 to +79
// RegisterService may return an error if the name is already taken, but
// since we re-register the *same* pointer this is idempotent and safe to ignore.
_ = app.RegisterService(t.name, t)
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLITrigger.Configure ignores the error from RegisterService. If "trigger.cli" is already registered to a different instance, this will silently keep the old service and dispatch may route to unexpected pipelines. Consider checking for an existing service and only ignoring the error when the existing value is the same *CLITrigger; otherwise return an error.

Suggested change
// RegisterService may return an error if the name is already taken, but
// since we re-register the *same* pointer this is idempotent and safe to ignore.
_ = app.RegisterService(t.name, t)
// If a service is already registered under this name, only tolerate it when
// it is the same *CLITrigger instance; otherwise surface an error so
// configuration fails instead of silently using the wrong trigger.
if err := app.RegisterService(t.name, t); err != nil {
// Check whether this instance is already present in the service registry.
var sameInstanceAlreadyRegistered bool
for _, svc := range app.SvcRegistry() {
if existing, ok := svc.(*CLITrigger); ok && existing == t {
sameInstanceAlreadyRegistered = true
break
}
}
if !sameInstanceAlreadyRegistered {
return fmt.Errorf("cli trigger: registering service %q: %w", t.name, err)
}
}

Copilot uses AI. Check for mistakes.
Comment on lines +49 to +50
if t, ok := config["target"].(string); ok && t == "stderr" {
target = stderr
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If config["target"] is set to an unrecognized value, this silently defaults to stdout. That makes misconfigurations hard to spot and can leak error output to stdout unexpectedly. Validate target is either "stdout" or "stderr" and return a config error for any other non-empty value.

Suggested change
if t, ok := config["target"].(string); ok && t == "stderr" {
target = stderr
if rawTarget, ok := config["target"]; ok {
if t, ok := rawTarget.(string); ok && t != "" {
switch t {
case "stdout":
target = stdout
case "stderr":
target = stderr
default:
return nil, fmt.Errorf("cli_print step %q: invalid 'target' %q; must be \"stdout\" or \"stderr\"", name, t)
}
}

Copilot uses AI. Check for mistakes.
Comment on lines +63 to +65
// Extract args from the pipeline context (injected by CLITrigger.DispatchCommand).
args, _ := pc.Current["args"].([]string)

Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLIInvokeStep only accepts pc.Current["args"] as a []string; if the trigger data came from JSON/YAML decoding it will often be []any, and this will silently pass nil args to the command. Consider supporting []any (convert to []string) and/or returning a clear error when args is present but not a string slice.

Copilot uses AI. Check for mistakes.
Comment on lines +77 to +80
// Suppress pipeline execution logs globally: wfctl is a CLI tool and
// internal pipeline step/run logs should not leak to the user's terminal.
// Each command creates its own logger when it needs output.
slog.SetDefault(engineLogger)
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slog.SetDefault(engineLogger) changes the process-wide default logger to discard output. This can unintentionally suppress logs from subcommands (e.g., cmd/wfctl/plugin_test_cmd.go uses slog.Default() when not in --verbose mode) and from any dependency relying on slog.Default(). Prefer keeping the discard logger scoped to the engine (WithLogger) and avoid mutating the global default, or restore the previous default after engine startup/config.

Suggested change
// Suppress pipeline execution logs globally: wfctl is a CLI tool and
// internal pipeline step/run logs should not leak to the user's terminal.
// Each command creates its own logger when it needs output.
slog.SetDefault(engineLogger)
// Suppress pipeline execution logs within the engine: wfctl is a CLI tool
// and internal pipeline step/run logs should not leak to the user's terminal.
// Each command creates its own logger when it needs output.

Copilot uses AI. Check for mistakes.
@intel352
Copy link
Contributor

intel352 commented Mar 7, 2026

@copilot apply changes based on the comments in this thread

…, engine logger propagation

Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 7, 2026

@copilot apply changes based on the comments in this thread

Applied all review feedback in commit fe50444:

  • Context threading: runCommand now accepts ctx; ExecuteWorkflow threads its ctx through; added DispatchContext(ctx, args) used in main.go via signal.NotifyContext (Ctrl+C cancellation). Dispatch delegates to DispatchContext(context.Background(), args) for backward compat.
  • ConfigureWorkflow cleanup: h.commands is fully re-initialized on every call so reconfigure doesn't leave stale commands.
  • Command validation: parseCLIWorkflowConfig now rejects empty names and duplicate command names.
  • RegisterService error handling: Both CLIWorkflowHandler.ConfigureWorkflow and CLITrigger.Configure check the error, tolerating only same-instance idempotent re-registration.
  • Duplicate command→workflow detection: CLITrigger.Configure errors when a different workflowType is registered for an already-mapped command.
  • step.cli_print target validation: Returns a clear config error for any target value other than "stdout" or "stderr".
  • step.cli_invoke []any args: Accepts both []string (direct Go call) and []any (JSON/YAML round-trip), with a clear error for non-string elements.
  • No slog.SetDefault: Fixed in engine.goconfigurePipelines now injects the engine's logger directly into each Pipeline it creates, so WithLogger(discard) properly suppresses pipeline execution logs without touching the global default.
  • gocritic exitAfterDefer: Replaced defer stop() + os.Exit(1) with an explicit stop() call before exit.

@intel352 intel352 merged commit 2a53e32 into main Mar 7, 2026
11 checks passed
@intel352 intel352 deleted the copilot/evaluate-workflow-engine-cli branch March 7, 2026 05:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants