Skip to content

feat: add multi-workflow application config and cross-workflow invocation#133

Merged
intel352 merged 3 commits intomainfrom
feat/issue-80-multi-workflow
Feb 23, 2026
Merged

feat: add multi-workflow application config and cross-workflow invocation#133
intel352 merged 3 commits intomainfrom
feat/issue-80-multi-workflow

Conversation

@intel352
Copy link
Contributor

Summary

  • Add ApplicationConfig format for referencing multiple workflow YAML files in a single application
  • Add step.workflow_call pipeline step for cross-workflow invocation (sync and async modes)
  • Add input/output mapping with template expansion for data passing between workflows
  • Add pipelineRegistry to StdEngine for runtime pipeline lookup
  • Add BuildFromApplicationConfig() with module name conflict detection
  • Add auto-detection of application configs in cmd/server startup
  • Add example multi-workflow chat platform with 3 interconnected workflows
  • 17 files changed, 1505 insertions

Test plan

  • go build ./... compiles cleanly
  • go test ./... — all tests pass (including 482 lines of new tests)
  • golangci-lint run — 0 issues
  • Backward compatible — all existing single-workflow configs work unchanged

Closes #80

🤖 Generated with Claude Code

…tion (#80)

- Add ApplicationConfig format for multi-workflow applications referencing
  separate workflow YAML files with shared module registry
- Add step.workflow_call pipeline step for cross-workflow invocation with
  sync (call-and-wait) and async (fire-and-forget) modes
- Add input/output mapping with template expansion for data passing
- Add pipelineRegistry to StdEngine for runtime pipeline lookup
- Add BuildFromApplicationConfig() with module name conflict detection
- Add auto-detection of application configs in cmd/server
- Add step.workflow_call schema to module schemas
- Add example multi-workflow app (chat platform with 3 workflows)
- Add comprehensive tests for workflow call step and multi-config loading

Closes #80

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings February 23, 2026 11:51
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds support for multi-workflow applications where multiple workflow YAML files can be composed into a single application with cross-workflow invocation capabilities. The implementation introduces:

Changes:

  • ApplicationConfig format for referencing multiple workflow files with auto-detection in cmd/server startup
  • step.workflow_call pipeline step supporting sync/async modes with input/output template mapping
  • pipelineRegistry in StdEngine for runtime pipeline lookup enabling cross-workflow calls
  • MergeApplicationConfig and BuildFromApplicationConfig methods with module name conflict detection
  • Comprehensive test coverage (482 lines of tests) and example multi-workflow chat platform demonstrating the feature

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
schema/schema.go Adds step.workflow_call to core module types list
schema/module_schema.go Defines complete schema for step.workflow_call including config fields, defaults, and documentation
plugins/pipelinesteps/plugin.go Registers step.workflow_call in plugin manifest
plugins/api/plugin.go Reformats newWorkflowRegistry constructor for consistency (no functional change)
module/pipeline_step_workflow_call.go Implements WorkflowCallStep with sync/async modes, template mapping, and timeout support
module/pipeline_step_workflow_call_test.go Comprehensive test coverage for all workflow call scenarios including edge cases
config/config.go Adds ApplicationConfig, WorkflowRef, LoadApplicationConfig, IsApplicationConfig, and MergeApplicationConfig
engine.go Adds pipelineRegistry map, registers step.workflow_call factory closure, implements BuildFromApplicationConfig
engine_multi_config_test.go Extensive integration tests for multi-workflow features and cross-pipeline invocation
cmd/server/main.go Adds config auto-detection, setupFromAppConfig, and loadConfig returns tuple
cmd/server/main_test.go Updates tests to handle new loadConfig return signature
example/multi-workflow/*.yaml Four example files demonstrating multi-workflow chat platform with cross-workflow calls
module/*_test.go Removes trailing blank lines (style cleanup)

Comment on lines +213 to +214
for k, v := range wfCfg.Pipelines {
combined.Pipelines[k] = v
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pipeline name conflicts are not detected when merging multiple workflow files. If two workflow files define pipelines with the same name, the second will silently overwrite the first (line 214 uses simple map assignment). This could lead to unexpected behavior where one workflow's pipeline is inaccessible. Consider adding conflict detection similar to the module name conflict check on lines 198-204.

Copilot uses AI. Check for mistakes.
Comment on lines +207 to +211
for k, v := range wfCfg.Workflows {
combined.Workflows[k] = v
}
for k, v := range wfCfg.Triggers {
combined.Triggers[k] = v
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trigger name conflicts are not detected when merging multiple workflow files. Similar to the pipeline name conflict issue, if two workflow files define triggers with the same name, the second will silently overwrite the first (line 211 uses simple map assignment). Consider adding conflict detection for both triggers and workflows maps to match the module conflict detection on lines 198-204.

Copilot uses AI. Check for mistakes.
Comment on lines +216 to +218
if combined.ConfigDir == "" {
combined.ConfigDir = wfCfg.ConfigDir
}
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConfigDir assignment uses only the first workflow file's directory (lines 216-218). When merging multiple workflow files from different directories, this may cause issues if module configs reference relative paths - only paths relative to the first workflow's directory will resolve correctly. Consider either: (1) using the application config's directory consistently, or (2) preserving each module's original config directory in a per-module field, or (3) documenting that all relative paths must be relative to the first workflow file's directory.

Copilot uses AI. Check for mistakes.
@intel352
Copy link
Contributor Author

@copilot open a new pull request to apply changes based on the comments in this thread

Copy link
Contributor

Copilot AI commented Feb 23, 2026

@intel352 I've opened a new pull request, #145, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot AI and others added 2 commits February 23, 2026 11:23
…fig, async context propagation, comment accuracy (#145)

* Initial plan

* fix: add conflict detection for triggers/pipelines, fix async context, clarify comment

Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>
Copilot AI review requested due to automatic review settings February 23, 2026 16:25
@intel352 intel352 merged commit 1bc3e29 into main Feb 23, 2026
9 of 12 checks passed
@intel352 intel352 deleted the feat/issue-80-multi-workflow branch February 23, 2026 16:26
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Comment on lines +382 to +383
// applies the admin config overlay, then builds the engine using
// BuildFromApplicationConfig so cross-workflow pipeline calls are wired up.
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says the engine is built using BuildFromApplicationConfig, but the implementation merges configs and then calls buildEngine(combined, ...) (which uses BuildFromConfig). Either update the comment to match the actual flow, or switch to calling engine.BuildFromApplicationConfig(appCfg) if that’s the intended entry point so the code and documentation stay aligned.

Suggested change
// applies the admin config overlay, then builds the engine using
// BuildFromApplicationConfig so cross-workflow pipeline calls are wired up.
// applies the admin config overlay, then builds the engine from the merged
// configuration so cross-workflow pipeline calls are wired up.

Copilot uses AI. Check for mistakes.
cs := &inputCapturingStep{name: "capture"}
p := &Pipeline{Name: name, Steps: []PipelineStep{cs}}
// After Execute, copy step's captured data to the output pointer.
_ = captured
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

capturingPipeline has a redundant _ = captured assignment even though captured is used later when constructing triggerDataCapture. This can be removed to reduce noise and avoid suggesting the parameter is unused.

Suggested change
_ = captured

Copilot uses AI. Check for mistakes.
Comment on lines +36 to +37
// Register the PipelineWorkflowHandler (normally done via LoadPlugin)
engine.RegisterWorkflowHandler(handlers.NewPipelineWorkflowHandler())
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newTestEngine loads the pipelinesteps plugin (which already registers PipelineWorkflowHandler via WorkflowHandlers()), but then manually registers another PipelineWorkflowHandler and the comment says this is “normally done via LoadPlugin”. This duplicates handlers and makes the test setup diverge from production wiring. Consider removing the manual RegisterWorkflowHandler(...) (or adjust the comment/setup if you intentionally need a custom handler instance).

Suggested change
// Register the PipelineWorkflowHandler (normally done via LoadPlugin)
engine.RegisterWorkflowHandler(handlers.NewPipelineWorkflowHandler())

Copilot uses AI. Check for mistakes.
Comment on lines +52 to +53
if m, ok := cfg["mode"].(string); ok && m == string(WorkflowCallModeAsync) {
mode = WorkflowCallModeAsync
Copy link

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mode is only checked for the exact string "async" and any other (including typos like "asnyc") silently falls back to sync. This makes config errors hard to detect. Consider validating mode explicitly (accept only "sync"/"async") and returning a clear factory error for unknown values.

Suggested change
if m, ok := cfg["mode"].(string); ok && m == string(WorkflowCallModeAsync) {
mode = WorkflowCallModeAsync
if rawMode, ok := cfg["mode"]; ok {
m, ok := rawMode.(string)
if !ok {
return nil, fmt.Errorf("workflow_call step %q: 'mode' must be a string ('sync' or 'async')", name)
}
switch m {
case "", string(WorkflowCallModeSync):
mode = WorkflowCallModeSync
case string(WorkflowCallModeAsync):
mode = WorkflowCallModeAsync
default:
return nil, fmt.Errorf("workflow_call step %q: invalid 'mode' %q (must be 'sync' or 'async')", name, m)
}

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multi-workflow applications: composable, reusable workflow configs that call each other

3 participants