Skip to content

fix(pipeline): fail closed on guard template errors; use reserved metadata keys for skipped output#333

Merged
intel352 merged 19 commits intomainfrom
copilot/sub-pr-332
Mar 16, 2026
Merged

fix(pipeline): fail closed on guard template errors; use reserved metadata keys for skipped output#333
intel352 merged 19 commits intomainfrom
copilot/sub-pr-332

Conversation

Copy link
Contributor

Copilot AI commented Mar 15, 2026

Three issues in SkippableStep identified during review of the skip_if/if guard implementation:

  • Silent control-flow changes on template errors — parse/exec failures in skip_if were treated as falsy (execute), and in if as falsy (skip). Both silently masked misconfigured templates.
  • Colliding output keys — skipped steps emitted skipped/reason at the top level of step output, which merges into pc.Current and can overwrite business fields.
  • Misleading comment on If field implied absent and falsy behave identically; a present if resolving to falsy should (and does) skip.

Changes

module/pipeline_step_skip.go

  • Template resolution errors in both skip_if and if now return a wrapped error from Execute (fail closed), consistent with how other step/template failures are handled.
  • Skipped step output now uses _skipped: true / _error: <reason> — the same reserved underscore-prefixed metadata keys used by ErrorStrategySkip in pipeline_executor.go.

config/pipeline.go

  • Corrected the If field comment to clearly state that a present if resolving to a falsy value skips the step.

module/pipeline_step_skip_test.go

  • Updated all assertions from "skipped"/"reason""_skipped"/"_error".
  • Added TestStepSkipIf_TemplateError_ReturnsError and TestStepIf_TemplateError_ReturnsError to verify fail-closed behavior.

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

intel352 and others added 17 commits March 15, 2026 01:55
Adds --url flag to wfctl plugin install that downloads a tar.gz archive
from a direct URL, extracts plugin.json to identify the plugin name,
installs to the plugin directory, and records the SHA-256 checksum in
the lockfile.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extends wfctl plugin init to generate cmd/workflow-plugin-<name>/main.go,
internal/provider.go, internal/steps.go, go.mod, .goreleaser.yml, CI/release
GitHub Actions workflows, Makefile, and README.md. Adds --module flag for
custom Go module paths. Preserves existing plugin.json and .go skeleton for
backward compatibility.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add AutoFetchPlugin and AutoFetchDeclaredPlugins to plugin/autofetch.go,
which shell out to wfctl to download plugins not found locally. Extend
WorkflowConfig with a new PluginsConfig / ExternalPluginDecl type so
configs can declare plugins with autoFetch: true and an optional version
constraint. StdEngine gains SetExternalPluginDir and calls
AutoFetchDeclaredPlugins in BuildFromConfig before module loading.
The server's buildEngine registers the plugin dir so auto-fetch is active
at runtime. If wfctl is absent, a warning is logged and startup continues.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Use struct conversion for staticIndexEntry → PluginSummary (staticcheck S1016)
- Remove unused updateLockfile and writePluginJSON functions
- Add nilerr annotations for intentional nil returns in integrity.go
- Add gosec annotation for exec.Command in autofetch.go
- Fix TestLoadRegistryConfigDefault to use DefaultRegistryConfig() directly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- integrity.go: fail closed when lockfile exists but is unreadable or
  unparseable, preventing integrity enforcement bypass
- autofetch.go: extract stripVersionConstraint helper; detect compound
  version constraints and fall back to latest; check both pluginName and
  workflow-plugin-<name> alternate form for installed-check; log restart
  warning when new plugins are downloaded (they require a server restart)
- autofetch_test.go: test stripVersionConstraint directly instead of
  duplicating the logic inline; add compound-constraint cases
- engine.go: clarify comment that auto-fetched plugins need a restart

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- generator.go: use plugin/external/sdk imports and types (PluginProvider,
  StepInstance, StepResult, StepTypes/CreateStep) instead of plugin/sdk
- PLUGIN_AUTHORING.md: update examples to match external SDK interfaces
- plugin_install.go: hash installed binary (not archive) for lockfile,
  add hashFileSHA256 helper, add install mode mutual exclusivity check,
  update installFromLocal to write lockfile, normalize plugin names
- plugin_lockfile.go: add registry param to updateLockfileWithChecksum,
  pass version/registry in installFromLockfile, remove dir on mismatch
- registry_source.go: validate URL in NewStaticRegistrySource
- config.go: clarify Version field forwarding semantics

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- registry_source.go: use explicit field assignment for PluginSummary
  instead of struct type conversion (clearer, avoids tag confusion)
- plugin_lockfile.go: don't pass @Version in installFromLockfile to
  prevent lockfile overwrite before checksum verification
- plugin_install.go: add verifyInstalledPlugin() call in installFromURL
  for parity with registry installs
- engine.go: add TODO to move auto-fetch before plugin discovery so
  newly fetched plugins are available without restart
- integrity_test.go: add tests for unreadable and malformed lockfile
  to verify fail-closed behavior

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- registry_source.go: add nolint:gosimple for S1016 — explicit field
  assignment preferred for clarity across different struct tags
- generator_test.go: add TestGenerateProjectStructure verifying all
  generated files exist and use correct external SDK imports/types

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…immediately

Auto-fetch was running inside BuildFromConfig, which executes after
external plugins are already discovered and loaded. Plugins downloaded
by auto-fetch required a server restart to take effect.

Move auto-fetch to buildEngine in cmd/server/main.go, before
DiscoverPlugins/LoadPlugin. Remove the now-unused externalPluginDir
field and SetExternalPluginDir from the engine.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Enables pipelines to call other pipelines by name with full context
forwarding. Supports template-resolved workflow names and stop_pipeline
option. Required for WebSocket message routing to game-specific pipelines.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- registry_source.go: switch GitHubRegistrySource to use registryHTTPClient
  (timeout-configured) for both ListPlugins and FetchManifest; update comment
- autofetch.go: scan for AutoFetch=true entries before exec.LookPath to avoid
  misleading startup warning when no plugins need auto-fetch
- autofetch.go: add internal autoFetchPlugin helper that accepts *slog.Logger
  and emits structured log entries when a logger is available; public
  AutoFetchPlugin delegates to it; AutoFetchDeclaredPlugins passes its logger
- autofetch_test.go: rename TestAutoFetchPlugin_CorrectArgs to
  TestAutoFetchPlugin_SkipsWhenExists to match what the test actually asserts
- integrity.go: replace os.ReadFile + sha256.Sum256 with streaming
  os.Open + io.Copy into sha256.New() to keep memory bounded for large binaries
- integrity_test.go: add t.Skip guard in UnreadableLockfile test when the file
  is actually readable (Windows / root environments)
- pipeline_step_workflow_call.go: use resolved workflowName (not s.workflow)
  in async return payload and sync error message for consistency
- docs/PLUGIN_AUTHORING.md: clarify the distinction between plugin.json name
  (short, used by engine) and the registry/provider manifest name
  (workflow-plugin- prefixed), and which is used for dependency resolution
- config/config.go: MergeApplicationConfig now merges Plugins.External from
  each referenced workflow file into the combined config, deduplicated by name

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- docs/PLUGIN_AUTHORING.md: close unclosed code fence after StepProvider example
- cmd/wfctl/plugin_install.go: streaming hashFileSHA256 via io.Copy + sha256.New()
- cmd/wfctl/plugin_install.go: warn on hash failure instead of silent empty checksum
- config/merge_test.go: add TestMergeApplicationConfig_PluginDedup covering first-definition-wins dedup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- pipeline_step_workflow_call.go: propagate stop_pipeline in async mode
- integrity.go: "open plugin binary" instead of "read" for os.Open error
- plugin_install.go: lowercase "warning:" for consistency
- merge_test.go: use filepath.Join and check writeFileContent errors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Steps now support optional `skip_if` and `if` Go template guards.
When `skip_if` evaluates to a truthy value the step is skipped and
the pipeline continues; `if` is the logical inverse. Skipped steps
produce `{"skipped": true, "reason": "..."}` output so downstream
steps can inspect the result.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Return errors (fail closed) instead of swallowing skip_if/if template failures
- Use _skipped/_error keys in skipped step output (align with ErrorStrategySkip convention)
- Fix misleading comment for If field in config/pipeline.go
- Update tests to reflect new key names; add 2 tests for template error behavior

Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>
Copilot AI changed the title [WIP] Add skip_if and if fields to pipeline step execution fix(pipeline): fail closed on guard template errors; use reserved metadata keys for skipped output Mar 15, 2026
Copilot AI requested a review from intel352 March 15, 2026 23:37
Base automatically changed from feat/step-skip-if to main March 16, 2026 01:01
@intel352 intel352 marked this pull request as ready for review March 16, 2026 01:39
Copilot AI review requested due to automatic review settings March 16, 2026 01:39
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens pipeline guard behavior (skip_if / if) by failing closed on template resolution errors and by emitting reserved underscore-prefixed metadata keys for skipped-step output to avoid collisions when step outputs merge into pc.Current.

Changes:

  • Add SkippableStep wrapper that evaluates skip_if / if, returns errors on template parse/exec failures, and emits "_skipped" / "_error" metadata on guarded skips.
  • Extend pipeline step config (config.PipelineStepConfig) and route inline pipeline parsing to carry skip_if / if, and wrap steps at build time when either guard is present.
  • Add unit tests validating skip/execute behavior, reserved metadata output keys, and fail-closed behavior on guard template errors.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
module/pipeline_step_skip.go Introduces the SkippableStep wrapper, truthiness rules, and reserved metadata output for guarded skips.
engine.go Plumbs skip_if / if from inline route step YAML into PipelineStepConfig and wraps steps during pipeline build.
config/pipeline.go Adds SkipIf / If fields and clarifies guard semantics in comments.
module/pipeline_step_skip_test.go Adds test coverage for guard execution, metadata output keys, and template-error fail-closed behavior.

Comment on lines +969 to +976
skipIf, _ := stepMap["skip_if"].(string)
ifExpr, _ := stepMap["if"].(string)
cfgs = append(cfgs, config.PipelineStepConfig{
Name: name,
Type: stepType,
Config: stepConfig,
SkipIf: skipIf,
If: ifExpr,
Comment on lines +21 to +26
type SkippableStep struct {
inner interfaces.PipelineStep
skipIf string // Go template; truthy result → skip
ifExpr string // Go template; falsy result → skip
tmpl *TemplateEngine
}
Comment on lines +89 to +95
func skippedResult(reason string) *interfaces.StepResult {
return &interfaces.StepResult{
Output: map[string]any{
"_skipped": true,
"_error": reason,
},
}
@intel352 intel352 merged commit 6be2a02 into main Mar 16, 2026
2 checks passed
@intel352 intel352 deleted the copilot/sub-pr-332 branch March 16, 2026 02:06
@github-actions
Copy link

⏱ Benchmark Results

No significant performance regressions detected.

benchstat comparison (baseline → PR)
## benchstat: baseline → PR
baseline-bench.txt:244: parsing iteration count: invalid syntax
baseline-bench.txt:311693: parsing iteration count: invalid syntax
baseline-bench.txt:617127: parsing iteration count: invalid syntax
baseline-bench.txt:917143: parsing iteration count: invalid syntax
baseline-bench.txt:1205051: parsing iteration count: invalid syntax
baseline-bench.txt:1501692: parsing iteration count: invalid syntax
benchmark-results.txt:244: parsing iteration count: invalid syntax
benchmark-results.txt:313907: parsing iteration count: invalid syntax
benchmark-results.txt:629517: parsing iteration count: invalid syntax
benchmark-results.txt:948832: parsing iteration count: invalid syntax
benchmark-results.txt:1267229: parsing iteration count: invalid syntax
benchmark-results.txt:1560957: parsing iteration count: invalid syntax
goos: linux
goarch: amd64
pkg: github.com/GoCodeAlone/workflow/dynamic
cpu: AMD EPYC 7763 64-Core Processor                
                            │ baseline-bench.txt │       benchmark-results.txt        │
                            │       sec/op       │    sec/op     vs base              │
InterpreterCreation-4               7.756m ± 61%   7.934m ± 62%       ~ (p=0.818 n=6)
ComponentLoad-4                     3.496m ±  6%   3.504m ±  8%       ~ (p=0.589 n=6)
ComponentExecute-4                  2.017µ ±  0%   1.999µ ±  4%       ~ (p=0.065 n=6)
PoolContention/workers-1-4          1.097µ ±  1%   1.090µ ±  1%       ~ (p=0.123 n=6)
PoolContention/workers-2-4          1.088µ ±  3%   1.082µ ±  3%       ~ (p=0.169 n=6)
PoolContention/workers-4-4          1.089µ ±  1%   1.091µ ±  1%       ~ (p=0.835 n=6)
PoolContention/workers-8-4          1.095µ ±  1%   1.096µ ±  1%       ~ (p=0.846 n=6)
PoolContention/workers-16-4         1.097µ ±  3%   1.095µ ±  6%       ~ (p=0.816 n=6)
ComponentLifecycle-4                3.523m ±  1%   3.492m ±  1%  -0.89% (p=0.026 n=6)
SourceValidation-4                  2.303µ ±  0%   2.305µ ±  2%       ~ (p=0.504 n=6)
RegistryConcurrent-4                782.5n ±  4%   786.1n ±  4%       ~ (p=0.937 n=6)
LoaderLoadFromString-4              3.582m ±  2%   3.509m ±  1%  -2.03% (p=0.002 n=6)
geomean                             18.80µ         18.76µ        -0.17%

                            │ baseline-bench.txt │        benchmark-results.txt         │
                            │        B/op        │     B/op      vs base                │
InterpreterCreation-4               1.944Mi ± 0%   1.944Mi ± 0%       ~ (p=0.937 n=6)
ComponentLoad-4                     2.097Mi ± 0%   2.097Mi ± 0%       ~ (p=0.974 n=6)
ComponentExecute-4                  1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-1-4          1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-2-4          1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-4-4          1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-8-4          1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-16-4         1.203Ki ± 0%   1.203Ki ± 0%       ~ (p=1.000 n=6) ¹
ComponentLifecycle-4                2.099Mi ± 0%   2.099Mi ± 0%  -0.00% (p=0.041 n=6)
SourceValidation-4                  1.984Ki ± 0%   1.984Ki ± 0%       ~ (p=1.000 n=6) ¹
RegistryConcurrent-4                1.133Ki ± 0%   1.133Ki ± 0%       ~ (p=1.000 n=6) ¹
LoaderLoadFromString-4              2.099Mi ± 0%   2.099Mi ± 0%       ~ (p=0.961 n=6)
geomean                             15.05Ki        15.05Ki       -0.00%
¹ all samples are equal

                            │ baseline-bench.txt │        benchmark-results.txt        │
                            │     allocs/op      │  allocs/op   vs base                │
InterpreterCreation-4                15.09k ± 0%   15.09k ± 0%       ~ (p=1.000 n=6)
ComponentLoad-4                      17.43k ± 0%   17.43k ± 0%       ~ (p=1.000 n=6)
ComponentExecute-4                    25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-1-4            25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-2-4            25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-4-4            25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-8-4            25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
PoolContention/workers-16-4           25.00 ± 0%    25.00 ± 0%       ~ (p=1.000 n=6) ¹
ComponentLifecycle-4                 17.48k ± 0%   17.48k ± 0%       ~ (p=1.000 n=6) ¹
SourceValidation-4                    32.00 ± 0%    32.00 ± 0%       ~ (p=1.000 n=6) ¹
RegistryConcurrent-4                  2.000 ± 0%    2.000 ± 0%       ~ (p=1.000 n=6) ¹
LoaderLoadFromString-4               17.47k ± 0%   17.47k ± 0%       ~ (p=1.000 n=6) ¹
geomean                               181.2         181.2       +0.00%
¹ all samples are equal

pkg: github.com/GoCodeAlone/workflow/middleware
                                  │ baseline-bench.txt │       benchmark-results.txt        │
                                  │       sec/op       │    sec/op     vs base              │
CircuitBreakerDetection-4                  288.1n ± 5%   286.3n ± 10%       ~ (p=0.084 n=6)
CircuitBreakerExecution_Success-4          22.52n ± 1%   22.49n ±  0%       ~ (p=0.673 n=6)
CircuitBreakerExecution_Failure-4          64.60n ± 1%   64.38n ±  0%  -0.34% (p=0.024 n=6)
geomean                                    74.83n        74.57n        -0.35%

                                  │ baseline-bench.txt │       benchmark-results.txt        │
                                  │        B/op        │    B/op     vs base                │
CircuitBreakerDetection-4                 144.0 ± 0%     144.0 ± 0%       ~ (p=1.000 n=6) ¹
CircuitBreakerExecution_Success-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
CircuitBreakerExecution_Failure-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                              ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                  │ baseline-bench.txt │       benchmark-results.txt        │
                                  │     allocs/op      │ allocs/op   vs base                │
CircuitBreakerDetection-4                 1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
CircuitBreakerExecution_Success-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
CircuitBreakerExecution_Failure-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                              ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/module
                                 │ baseline-bench.txt │       benchmark-results.txt        │
                                 │       sec/op       │    sec/op     vs base              │
JQTransform_Simple-4                     866.5n ± 29%   863.6n ± 25%       ~ (p=0.699 n=6)
JQTransform_ObjectConstruction-4         1.461µ ± 23%   1.449µ ±  1%       ~ (p=0.223 n=6)
JQTransform_ArraySelect-4                3.332µ ±  2%   3.317µ ±  1%       ~ (p=0.193 n=6)
JQTransform_Complex-4                    38.46µ ±  1%   37.68µ ±  0%  -2.03% (p=0.002 n=6)
JQTransform_Throughput-4                 1.770µ ±  1%   1.787µ ±  1%  +0.96% (p=0.006 n=6)
SSEPublishDelivery-4                     66.03n ±  2%   66.01n ±  0%       ~ (p=0.558 n=6)
geomean                                  1.633µ         1.625µ        -0.45%

                                 │ baseline-bench.txt │        benchmark-results.txt         │
                                 │        B/op        │     B/op      vs base                │
JQTransform_Simple-4                   1.273Ki ± 0%     1.273Ki ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_ObjectConstruction-4       1.773Ki ± 0%     1.773Ki ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_ArraySelect-4              2.625Ki ± 0%     2.625Ki ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_Complex-4                  16.22Ki ± 0%     16.22Ki ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_Throughput-4               1.984Ki ± 0%     1.984Ki ± 0%       ~ (p=1.000 n=6) ¹
SSEPublishDelivery-4                     0.000 ± 0%       0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                             ²                 +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                 │ baseline-bench.txt │       benchmark-results.txt        │
                                 │     allocs/op      │ allocs/op   vs base                │
JQTransform_Simple-4                     10.00 ± 0%     10.00 ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_ObjectConstruction-4         15.00 ± 0%     15.00 ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_ArraySelect-4                30.00 ± 0%     30.00 ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_Complex-4                    324.0 ± 0%     324.0 ± 0%       ~ (p=1.000 n=6) ¹
JQTransform_Throughput-4                 17.00 ± 0%     17.00 ± 0%       ~ (p=1.000 n=6) ¹
SSEPublishDelivery-4                     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                             ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/schema
                                    │ baseline-bench.txt │       benchmark-results.txt       │
                                    │       sec/op       │   sec/op     vs base              │
SchemaValidation_Simple-4                    1.105µ ± 7%   1.122µ ± 3%       ~ (p=0.379 n=6)
SchemaValidation_AllFields-4                 1.680µ ± 4%   1.690µ ± 5%       ~ (p=1.000 n=6)
SchemaValidation_FormatValidation-4          1.598µ ± 5%   1.597µ ± 4%       ~ (p=0.615 n=6)
SchemaValidation_ManySchemas-4               1.833µ ± 2%   1.832µ ± 2%       ~ (p=0.699 n=6)
geomean                                      1.527µ        1.534µ       +0.49%

                                    │ baseline-bench.txt │       benchmark-results.txt        │
                                    │        B/op        │    B/op     vs base                │
SchemaValidation_Simple-4                   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_AllFields-4                0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_FormatValidation-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_ManySchemas-4              0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                                ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                    │ baseline-bench.txt │       benchmark-results.txt        │
                                    │     allocs/op      │ allocs/op   vs base                │
SchemaValidation_Simple-4                   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_AllFields-4                0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_FormatValidation-4         0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
SchemaValidation_ManySchemas-4              0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                                ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/store
                                   │ baseline-bench.txt │       benchmark-results.txt        │
                                   │       sec/op       │    sec/op     vs base              │
EventStoreAppend_InMemory-4                1.255µ ± 20%   1.190µ ± 23%       ~ (p=0.221 n=6)
EventStoreAppend_SQLite-4                  1.466m ±  5%   1.378m ±  5%  -5.98% (p=0.009 n=6)
GetTimeline_InMemory/events-10-4           14.22µ ±  3%   14.56µ ±  8%  +2.41% (p=0.041 n=6)
GetTimeline_InMemory/events-50-4           70.86µ ± 15%   79.64µ ± 19%       ~ (p=0.485 n=6)
GetTimeline_InMemory/events-100-4          125.2µ ±  2%   128.4µ ±  1%  +2.54% (p=0.002 n=6)
GetTimeline_InMemory/events-500-4          649.0µ ±  2%   664.4µ ±  0%  +2.37% (p=0.002 n=6)
GetTimeline_InMemory/events-1000-4         1.325m ±  0%   1.369m ±  1%  +3.33% (p=0.002 n=6)
GetTimeline_SQLite/events-10-4             109.7µ ±  2%   111.6µ ±  1%  +1.73% (p=0.015 n=6)
GetTimeline_SQLite/events-50-4             251.4µ ±  1%   257.4µ ±  0%  +2.42% (p=0.002 n=6)
GetTimeline_SQLite/events-100-4            424.8µ ±  1%   435.1µ ±  1%  +2.43% (p=0.002 n=6)
GetTimeline_SQLite/events-500-4            1.808m ±  1%   1.846m ±  0%  +2.12% (p=0.002 n=6)
GetTimeline_SQLite/events-1000-4           3.524m ±  2%   3.586m ±  0%  +1.78% (p=0.009 n=6)
geomean                                    222.7µ         226.7µ        +1.77%

                                   │ baseline-bench.txt │        benchmark-results.txt         │
                                   │        B/op        │     B/op      vs base                │
EventStoreAppend_InMemory-4                  810.5 ± 8%     791.0 ± 5%       ~ (p=0.619 n=6)
EventStoreAppend_SQLite-4                  1.983Ki ± 3%   1.986Ki ± 1%       ~ (p=0.372 n=6)
GetTimeline_InMemory/events-10-4           7.953Ki ± 0%   7.953Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-50-4           46.62Ki ± 0%   46.62Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-100-4          94.48Ki ± 0%   94.48Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-500-4          472.8Ki ± 0%   472.8Ki ± 0%       ~ (p=0.061 n=6)
GetTimeline_InMemory/events-1000-4         944.3Ki ± 0%   944.3Ki ± 0%       ~ (p=0.130 n=6)
GetTimeline_SQLite/events-10-4             16.74Ki ± 0%   16.74Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-50-4             87.14Ki ± 0%   87.14Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-100-4            175.4Ki ± 0%   175.4Ki ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-500-4            846.1Ki ± 0%   846.1Ki ± 0%  +0.00% (p=0.002 n=6)
GetTimeline_SQLite/events-1000-4           1.639Mi ± 0%   1.639Mi ± 0%       ~ (p=0.567 n=6)
geomean                                    67.48Ki        67.36Ki       -0.19%
¹ all samples are equal

                                   │ baseline-bench.txt │        benchmark-results.txt        │
                                   │     allocs/op      │  allocs/op   vs base                │
EventStoreAppend_InMemory-4                  7.000 ± 0%    7.000 ± 0%       ~ (p=1.000 n=6) ¹
EventStoreAppend_SQLite-4                    53.00 ± 0%    53.00 ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-10-4             125.0 ± 0%    125.0 ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-50-4             653.0 ± 0%    653.0 ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-100-4           1.306k ± 0%   1.306k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-500-4           6.514k ± 0%   6.514k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_InMemory/events-1000-4          13.02k ± 0%   13.02k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-10-4               382.0 ± 0%    382.0 ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-50-4              1.852k ± 0%   1.852k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-100-4             3.681k ± 0%   3.681k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-500-4             18.54k ± 0%   18.54k ± 0%       ~ (p=1.000 n=6) ¹
GetTimeline_SQLite/events-1000-4            37.29k ± 0%   37.29k ± 0%       ~ (p=1.000 n=6) ¹
geomean                                     1.162k        1.162k       +0.00%
¹ all samples are equal

Benchmarks run with go test -bench=. -benchmem -count=6.
Regressions ≥ 20% are flagged. Results compared via benchstat.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants