Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions openspec/changes/add-matrix-execution/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Change: Matrix Task Execution

## Why

Modern workflows frequently require executing the same task logic across multiple variations of inputs or environments (e.g., building for multiple OS targets, testing against multiple Node versions). Currently, users must manually duplicate `TaskStep` configurations to achieve this, increasing boilerplate and maintenance overhead. By introducing Matrix Task Execution, the task runner can automatically generate and run parallel variations of a task based on a parameterized matrix, bringing it in line with industry standards like GitHub Actions and Nx.

## What Changes

- **Task Configuration**: Add an optional `matrix` property to the `TaskStep` interface, allowing users to define a set of parameters (arrays of values) that will multiply the task into multiple instances.
- **Workflow Executor**: Modify `TaskRunner` or `WorkflowExecutor` (and potentially `TaskGraphValidator`) to dynamically expand matrix steps into individual, parallel sub-tasks at runtime before execution.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The proposal suggests expanding matrix steps at runtime before execution. If this expansion occurs after the TaskGraphValidator.validate call in TaskRunner.execute, the validator will only check the unexpanded graph. This might miss issues like name collisions or cycles that only emerge after expansion. It is recommended to clarify if expansion should happen prior to validation or if the validator should be made 'matrix-aware'.

- **Task Identity**: Extend task naming conventions or introduce run metadata to uniquely identify matrix sub-tasks (e.g., `TaskName[node=18,os=ubuntu]`).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The suggested naming convention TaskName[node=18,os=ubuntu] uses square brackets and equals signs, which are currently sanitized to underscores by TaskRunner.sanitizeMermaidId (line 161 of src/TaskRunner.ts). This could lead to name collisions in Mermaid visualizations (e.g., Task[a=1] and Task_a_1 both becoming Task_a_1_). Consider using a naming format that avoids characters filtered by the existing Mermaid sanitizer.

- **Context Injection**: Ensure the parameters of the matrix permutation are injected into the task's context or arguments when `run` is called.

## Impact

- **Affected specs**: `task-runner`
- **Affected code**: `TaskStep.ts`, `TaskRunner.ts`, `WorkflowExecutor.ts`, and possibly `TaskGraphValidator.ts`.
- **Performance**: Will slightly increase graph validation and resolution times, but significantly boosts developer velocity and DRY configuration by abstracting fan-out patterns.
23 changes: 23 additions & 0 deletions openspec/changes/add-matrix-execution/specs/task-runner/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
## ADDED Requirements

### Requirement: Matrix Task Execution

The `TaskRunner` SHALL support parameterized task execution via a `matrix` configuration, automatically expanding a single task definition into multiple parallel instances based on permutations of the matrix values.

#### Scenario: Single dimension matrix
- **WHEN** a `TaskStep` defines a `matrix` with a single key and an array of 3 values
- **THEN** the `TaskRunner` SHALL dynamically generate and execute 3 distinct task instances
- **AND** each instance SHALL receive a specific value from the matrix during execution.

#### Scenario: Multi-dimension matrix
- **WHEN** a `TaskStep` defines a `matrix` with multiple keys (e.g., `os: ["ubuntu", "macos"]`, `node: [18, 20]`)
- **THEN** the `TaskRunner` SHALL compute the cartesian product of the matrix values
- **AND** it SHALL dynamically generate and execute a task instance for each permutation (e.g., 4 instances).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The specification should define the expected behavior when a matrix dimension is an empty array (e.g., node: []). Common patterns include either throwing a validation error or treating the entire task as skipped (generating zero instances). Explicitly defining this edge case will prevent inconsistent implementations.


#### Scenario: Unique identification of matrix tasks
- **WHEN** the `TaskRunner` expands a matrix task
- **THEN** each generated task MUST have a unique `name` derived from the base task name and its specific matrix parameters.

#### Scenario: Matrix task dependency resolution
- **WHEN** a regular task depends on a task that defines a `matrix`
- **THEN** the dependent task SHALL wait for ALL generated matrix instances to complete successfully before executing.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This requirement defines a 'fan-in' behavior where a dependent task waits for all matrix instances. It would be beneficial to clarify the behavior when the dependent task is also a matrix. For example, if Task B (matrix) depends on Task A (matrix), does every instance of B wait for every instance of A, or is there a mechanism for parallel 1:1 mapping?

10 changes: 10 additions & 0 deletions openspec/changes/add-matrix-execution/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## 1. Implementation

- [ ] 1.1 Update `TaskStep` interface in `src/TaskStep.ts` to include an optional `matrix` configuration, which maps keys to arrays of values (e.g., `Record<string, unknown[]>`).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Defining the matrix as Record<string, unknown[]> is flexible but may complicate task name generation if complex objects are used as values. Restricting matrix values to primitives (string, number, boolean) ensures that task names are deterministic and readable.

Suggested change
- [ ] 1.1 Update `TaskStep` interface in `src/TaskStep.ts` to include an optional `matrix` configuration, which maps keys to arrays of values (e.g., `Record<string, unknown[]>`).
- [ ] 1.1 Update `TaskStep` interface in `src/TaskStep.ts` to include an optional `matrix` configuration, which maps keys to arrays of primitive values (e.g., `Record<string, (string | number | boolean)[]>`).

- [ ] 1.2 Modify `TaskRunner.ts` or `WorkflowExecutor.ts` to implement a "matrix expansion" phase before executing tasks, transforming a single `TaskStep` with a matrix into multiple individual `TaskStep` objects.
- [ ] 1.3 Ensure the dynamically generated matrix sub-tasks have unique names (e.g., appending stringified parameters to the original name).
- [ ] 1.4 Update the task's execution logic to pass the specific permutation parameters to the task's `run` context.
- [ ] 1.5 Update `TaskGraphValidator.ts` to ensure it correctly resolves dependencies involving matrix tasks, including dynamically generated names.
- [ ] 1.6 Add comprehensive unit tests verifying that matrix expansion works correctly, generates the correct number of combinations, and correctly injects parameters.
- [ ] 1.7 Add integration tests verifying complex graphs that mix matrix tasks with regular tasks and dependencies.
- [ ] 1.8 Update documentation/README.md to demonstrate the usage of the new `matrix` feature.
Loading