[rush] Support percentage-based weights for operationSettings in rush-project.json#5679
Conversation
| const percentValue: number = parseFloat(weight.slice(0, -1)); | ||
|
|
||
| // Use as much CPU as possible, so we round down the weight here | ||
| return Math.floor((percentValue / 100) * availableParallelism); |
There was a problem hiding this comment.
@dmichon-msft Is it okay if this rounds down to 0?
There was a problem hiding this comment.
I'd be tempted to suggest that if it rounds down to 0, you use the raw value just so it doesn't produce infinite parallelism, but passing floats to the concurrency engine isn't terribly advisable (you can accumulate error from adding and subtracting them). Probably restrict values to [1,Infinity) from percentage weights.
If we want to support fractional units we should redefine 1 concurrency unit as some power-of-two fraction of a CPU core (maybe 1/4?) and then propagate that all the way through the system.
There was a problem hiding this comment.
I changed it to:
return Math.max(1, Math.floor((percentValue / 100) * availableParallelism));dd7ca9a to
53b44c6
Compare
1. libraries/rush-lib/src/logic/operations/WeightedOperationPlugin.ts
- Added more comments to explain the percentage weight convert function.
- Use `as` syntax to cast error to Error type for better readability.
2. libraries/rush-lib/src/schemas/rush-project.schema.json
- Updated the description of the weight property to reflect that the maximum concurrency units is determined by `os.availableParallelism()`.
3. libraries/rush-lib/src/logic/operations/test/WeightedOperationPlugin.test.ts
- Fix a test name.
53b44c6 to
50b9ee3
Compare
| context: ICreateOperationsContext | ||
| ): Map<Operation, IOperationExecutionResult> { | ||
| const { projectConfigurations } = context; | ||
| const availableParallelism: number = os.availableParallelism(); |
There was a problem hiding this comment.
I'd consider it cleaner, especially for mocking, to pass this value to the plugin constructor.
| throw new Error(`Expected a percentage string like "100%".`); | ||
| } | ||
|
|
||
| const percentValue: number = parseFloat(weight.slice(0, -1)); |
There was a problem hiding this comment.
parseFloat doesn't mind the trailing %. Parsing stops as soon as it hits a character that isn't part of the float.
There was a problem hiding this comment.
I think the current code might be better; it is still correct and doesn't require the reader to have intimate knowledge of quirky API designs that were acceptable when JavaScript was first invented. :)
| const percentValue: number = parseFloat(weight.slice(0, -1)); | ||
|
|
||
| // Use as much CPU as possible, so we round down the weight here | ||
| return Math.floor((percentValue / 100) * availableParallelism); |
There was a problem hiding this comment.
I'd be tempted to suggest that if it rounds down to 0, you use the raw value just so it doesn't produce infinite parallelism, but passing floats to the concurrency engine isn't terribly advisable (you can accumulate error from adding and subtracting them). Probably restrict values to [1,Infinity) from percentage weights.
If we want to support fractional units we should redefine 1 concurrency unit as some power-of-two fraction of a CPU core (maybe 1/4?) and then propagate that all the way through the system.
| const projectConfiguration: RushProjectConfiguration | undefined = projectConfigurations.get(project); | ||
| const operationSettings: IOperationSettings | undefined = | ||
| operation.settings ?? projectConfiguration?.operationSettingsByOperationName.get(phase.name); |
There was a problem hiding this comment.
I find myself wondering why this layer is here; operation.settings should always already be defined if projectConfigurations.get(project)?.operationSettingsByOperationName.get(phase.name) returns a truthy value.
Co-authored-by: David Michon <dmichon@microsoft.com>
Co-authored-by: David Michon <dmichon@microsoft.com>
…-project.json (microsoft#5679) * feature: Support percentage-based weights for operationSettings in rush-project.json * Resolve CodeReview comments for: 1. libraries/rush-lib/src/logic/operations/WeightedOperationPlugin.ts - Added more comments to explain the percentage weight convert function. - Use `as` syntax to cast error to Error type for better readability. 2. libraries/rush-lib/src/schemas/rush-project.schema.json - Updated the description of the weight property to reflect that the maximum concurrency units is determined by `os.availableParallelism()`. 3. libraries/rush-lib/src/logic/operations/test/WeightedOperationPlugin.test.ts - Fix a test name. * Update libraries/rush-lib/src/schemas/rush-project.schema.json Co-authored-by: David Michon <dmichon@microsoft.com> * Update libraries/rush-lib/src/schemas/rush-project.schema.json Co-authored-by: David Michon <dmichon@microsoft.com> * PR feedback * PR feedback * Prepare to publish a MINOR release of Rush --------- Co-authored-by: LPegasus <lpegasus@users.noreply.github.com> Co-authored-by: David Michon <dmichon@microsoft.com> Co-authored-by: Pete Gonzalez <4673363+octogonz@users.noreply.github.com>
Summary
Add issue #5607 proposal 1 feature.
Details
weightsettings inRushProjectConfiguration.operationSettingspercentage string to integer based onos.availableParallelism().IOperationSettings.weighttype definition.weightfield json schema.How it was tested
Impacted documentation
None