JENKINS-66373: idle timeout should not fight with min spare instances setting#945
Open
sparrowt wants to merge 1 commit intojenkinsci:masterfrom
Open
JENKINS-66373: idle timeout should not fight with min spare instances setting#945sparrowt wants to merge 1 commit intojenkinsci:masterfrom
sparrowt wants to merge 1 commit intojenkinsci:masterfrom
Conversation
… setting Prior to this change, if you use "Minimum number of spare instances" setting as well as "Idle termination time", then there is a constant battle going on between: - `checkForMinimumInstances` trying to maintain the minimum spare instances as requested - the idle timeout checks in this method are killing off the 'spare' instances Make this method take account of "Minimum number of _spare_ instances" in the same way that it already accounts for the main "Minimum number of instances" setting.
Contributor
Author
|
@res0nance who would be good to have an initial review of this? |
Contributor
Could you add some unit tests for this change? |
|
@sparrowt @res0nance I'm running into something similar, any thoughts on getting this updated/reviewed? 🙏🏽 |
Contributor
Author
|
Hi folks I'm afraid I don't have time right now to add tests (much as I approve that this would be a good idea!) If others are keen for this too I'm open to contributions in a PR against my branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Currently if you use the "Minimum number of spare instances" setting as well as "Idle termination time", then there is a constant battle going on between:
MinimumInstanceChecker.checkForMinimumInstancesprovisioning new agents to try and maintain the minimum number of spare instances as configuredThis means that the spare instances are repeatedly killed & recreated as described on https://issues.jenkins.io/browse/JENKINS-66373 which is wasteful and means that, some percentage of the time, there are never enough spare instances because they are being booted (again).
Solution
This PR makes a simple change to the idle termination logic so that it takes account of "Minimum number of spare instances" in the same way that it already accounts for the main "Minimum number of instances" setting.
Testing done
None yet -
EC2RetentionStrategyTest.javaseems like the right place to add a test case for this specific scenario.Right now I've got very limited time and wanted to at least get the proposed fix up for discussion to start with.