-
Notifications
You must be signed in to change notification settings - Fork 1
docs: propose Matrix Task Execution feature #253
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Change: Matrix Task Execution | ||
|
|
||
| ## Why | ||
|
|
||
| Modern workflows frequently require executing the same task logic across multiple variations of inputs or environments (e.g., building for multiple OS targets, testing against multiple Node versions). Currently, users must manually duplicate `TaskStep` configurations to achieve this, increasing boilerplate and maintenance overhead. By introducing Matrix Task Execution, the task runner can automatically generate and run parallel variations of a task based on a parameterized matrix, bringing it in line with industry standards like GitHub Actions and Nx. | ||
|
|
||
| ## What Changes | ||
|
|
||
| - **Task Configuration**: Add an optional `matrix` property to the `TaskStep` interface, allowing users to define a set of parameters (arrays of values) that will multiply the task into multiple instances. | ||
| - **Workflow Executor**: Modify `TaskRunner` or `WorkflowExecutor` (and potentially `TaskGraphValidator`) to dynamically expand matrix steps into individual, parallel sub-tasks at runtime before execution. | ||
| - **Task Identity**: Extend task naming conventions or introduce run metadata to uniquely identify matrix sub-tasks (e.g., `TaskName[node=18,os=ubuntu]`). | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The suggested naming convention |
||
| - **Context Injection**: Ensure the parameters of the matrix permutation are injected into the task's context or arguments when `run` is called. | ||
|
|
||
| ## Impact | ||
|
|
||
| - **Affected specs**: `task-runner` | ||
| - **Affected code**: `TaskStep.ts`, `TaskRunner.ts`, `WorkflowExecutor.ts`, and possibly `TaskGraphValidator.ts`. | ||
| - **Performance**: Will slightly increase graph validation and resolution times, but significantly boosts developer velocity and DRY configuration by abstracting fan-out patterns. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| ## ADDED Requirements | ||
|
|
||
| ### Requirement: Matrix Task Execution | ||
|
|
||
| The `TaskRunner` SHALL support parameterized task execution via a `matrix` configuration, automatically expanding a single task definition into multiple parallel instances based on permutations of the matrix values. | ||
|
|
||
| #### Scenario: Single dimension matrix | ||
| - **WHEN** a `TaskStep` defines a `matrix` with a single key and an array of 3 values | ||
| - **THEN** the `TaskRunner` SHALL dynamically generate and execute 3 distinct task instances | ||
| - **AND** each instance SHALL receive a specific value from the matrix during execution. | ||
|
|
||
| #### Scenario: Multi-dimension matrix | ||
| - **WHEN** a `TaskStep` defines a `matrix` with multiple keys (e.g., `os: ["ubuntu", "macos"]`, `node: [18, 20]`) | ||
| - **THEN** the `TaskRunner` SHALL compute the cartesian product of the matrix values | ||
| - **AND** it SHALL dynamically generate and execute a task instance for each permutation (e.g., 4 instances). | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The specification should define the expected behavior when a matrix dimension is an empty array (e.g., |
||
|
|
||
| #### Scenario: Unique identification of matrix tasks | ||
| - **WHEN** the `TaskRunner` expands a matrix task | ||
| - **THEN** each generated task MUST have a unique `name` derived from the base task name and its specific matrix parameters. | ||
|
|
||
| #### Scenario: Matrix task dependency resolution | ||
| - **WHEN** a regular task depends on a task that defines a `matrix` | ||
| - **THEN** the dependent task SHALL wait for ALL generated matrix instances to complete successfully before executing. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This requirement defines a 'fan-in' behavior where a dependent task waits for all matrix instances. It would be beneficial to clarify the behavior when the dependent task is also a matrix. For example, if Task B (matrix) depends on Task A (matrix), does every instance of B wait for every instance of A, or is there a mechanism for parallel 1:1 mapping? |
||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,10 @@ | ||||||
| ## 1. Implementation | ||||||
|
|
||||||
| - [ ] 1.1 Update `TaskStep` interface in `src/TaskStep.ts` to include an optional `matrix` configuration, which maps keys to arrays of values (e.g., `Record<string, unknown[]>`). | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Defining the matrix as
Suggested change
|
||||||
| - [ ] 1.2 Modify `TaskRunner.ts` or `WorkflowExecutor.ts` to implement a "matrix expansion" phase before executing tasks, transforming a single `TaskStep` with a matrix into multiple individual `TaskStep` objects. | ||||||
| - [ ] 1.3 Ensure the dynamically generated matrix sub-tasks have unique names (e.g., appending stringified parameters to the original name). | ||||||
| - [ ] 1.4 Update the task's execution logic to pass the specific permutation parameters to the task's `run` context. | ||||||
| - [ ] 1.5 Update `TaskGraphValidator.ts` to ensure it correctly resolves dependencies involving matrix tasks, including dynamically generated names. | ||||||
| - [ ] 1.6 Add comprehensive unit tests verifying that matrix expansion works correctly, generates the correct number of combinations, and correctly injects parameters. | ||||||
| - [ ] 1.7 Add integration tests verifying complex graphs that mix matrix tasks with regular tasks and dependencies. | ||||||
| - [ ] 1.8 Update documentation/README.md to demonstrate the usage of the new `matrix` feature. | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The proposal suggests expanding matrix steps at runtime before execution. If this expansion occurs after the
TaskGraphValidator.validatecall inTaskRunner.execute, the validator will only check the unexpanded graph. This might miss issues like name collisions or cycles that only emerge after expansion. It is recommended to clarify if expansion should happen prior to validation or if the validator should be made 'matrix-aware'.