Added a findLocks step to enumerate the available resources

[gaspardpetit]

This is a companion PR to #305 to make it easier to manipulate lockable resources in pipeline. This step adds searching into the resources using a few filters:

• no argument to get all locks owned by the current build
• maching to find locks by regex
• anyOfLabels to find locks with at least one of the specified labels
• allOfLabels to find locks with all the specified labels
• noneOfLabels to find locks that do not contain any of the specified labels
• build to specify the build owning the lock. Defaults to the current build, but supports any to search everywhere (including unlocked)

In my experience, these have been enough to get around most use cases, but you could also, of course, get the whole list and filter in groovy - these are just for convenience.

My use case is to have locks defined from SCM; we want to "sync" changes (add new locks, delete removed ones) when the lock definition changes.

Combined with #305 the script would look like this:

Map resources = readYaml(file: 'resources.yaml')

// add missing / update existing
resources.each { k, v ->
  updateLock(resource:k, createResource:true, setLabels:(v.labels))
}
// delete removed
for (r in findLocks(build:'any')) {
  if (resources.contains(resource.name) == false) {
    updateLock(resource: r, deleteResource:true)
  }
}

I have another use case where I would like to be able to update resources as offline when we detect that they are not working correctly while using them:

  updateLock(resource:myFlakyResource, addLabels:'offline')

Then in another pipeline be able to periodically scan for offline resources:

findLocks(build:'any', anyOfLabels:'offline').each{ r -> 
  lock(r.name) {
    // attempt healing here
    updateLock(resource:r.name, removeLabels:'offline')
  }
}

This PR is open for discussion obviously, if others would prefer other names, other filters, etc. I was also not sure how to correctly return the resource objects to groovy in a way that would not require to unlock methods from an administrator, so I am serializing the LockableResource to a Map - it works but I have the feeling there must be a better way to marshal objects to groovy...

To fully fulfill this use case, I will also update #110 to provide exclusion on labels so we can do something like this:

lock(label:'printer', excludeLabel:'offline')

• Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
• Ensure that the pull request title represents the desired changelog entry
• Please describe what you did
• Link to relevant issues in GitHub or Jira
• Link to relevant pull requests, esp. upstream and downstream changes
• Ensure you have provided tests - that demonstrates feature works or fixes the issue

[gaspardpetit]

Added a build filter and updated the description with it. This step would search for the current build's locks by default on a no-param findLocks() - but could still be used to search everywhere by doing findLocks(build:'any')

[jimklimov]

Hi, sorry for falling out of the loop, and thanks for your many PRs. One question to this one's description:

no argument to get all locks

vs.

This step would search for the current build's locks by default on a no-param ...

Which of these is supposed to be true? "all" or "this build's"? :)

[jimklimov]

How would this behave when findLocks() returns several values? Would Groovy resolve the .name part nicely, for each entry?

[gaspardpetit]

yes, the magic of groovy - if you .name on an array of object, it applies the .name on each entry and returns an array of the results

[jimklimov]

Thinking of the repetitive "locked or not" comment, maybe filters for lock states (not-locked - build:'none' perhaps?, reserved, stolen, ephemeral/persistent...) could be useful.

[gaspardpetit]

My initial concern was to go in a feature creep that would add complexity without fulfilling actual use cases. Do we want to search in notes? what about reservedTimestamp - and do we want to allow before / after searches, etc. I don't mind adding a few, but I think the missing ones will be discovered and added through pull requests explaining which use case the new find option is fulfilling.

For reservedBy, I think it opens an interesting case where users can reserve resources and then execute jobs that work on their reserved resources, but apart from unreserving them (ex. with the update step), it would be difficult to allow a step to lock all the resources reserved by a user unless we allow locking resources reserved by a user when the job is being triggerred by that user for example.

For stolen, ephemeral, these are easy to add, but I have the same concern about mapping these to actual use cases.

What I am hoping is that the combo of findLock/updateLock and the advanced lock filters will resolved most of the us cases by using labeling. If you really need to work on your reserved locks, findLocks on all locks, look for the ones you reserved, update the labels (ex. reserved_by_<username>) and then you can lock these specific locks by label.

I do have a much narrower view on how this plugin might be used by the rest of the community, so I'll rely on your opinion on which fields are worth adding to the findLock step

[gaspardpetit]

Hi, sorry for falling out of the loop, and thanks for your many PRs. One question to this one's description:

no argument to get all locks

vs.

This step would search for the current build's locks by default on a no-param ...

Which of these is supposed to be true? "all" or "this build's"? :)

Thanks for noticing - I changed the behaviour halfway through but had not updated the original PR description. When I started using this with real-life scenarios, it became obvious that the most common use case was getting the current locks owned by the current build, so I made findLock(). To seach all locks (owned or not by this build), we have to use findLocks(build:'any')

[DonnieKim411]

please include this PR on the next release? there are many PRs created by @gaspardpetit and it will really enrich the plugin

[mPokornyETM]

One question. Will be possible to handle labels in the same way like in nodes. So don't need make own magic.

[mPokornyETM]

will be handled in #428

[mPokornyETM]

It is done by jenkins label resolver