Drastically reduce amount of API calls when checking if something exists

[mdealer]

Description

This avoids fetching directory contents, of which there could be thousands, when looking for Jenkinsfiles. In a scenario with 500+ repos, 1000+ branches each and 1000+ files on root level this plugin otherwise does not work due to excessive API calls and throttling.

I have been running this change for months now, originally I could not use the plugin without this change. This is a pure behind the scenes API usage change.

Regarding tests, I am a bit confused about how to fix them as they seem to touch this change but also are actually testing something entirely different (like caching). I need to work on them, but someone should look over this as I don't fully understand how they are organized. There are some assumptions about the API querying directory contents on a single file stat, which is kinda convoluted.

Live test shows no issues (we use Jenkinsfile at various places, thousands of branches across hundreds of repos).

The change may help with tickets like #1459

Submitter checklist

• Link to JIRA ticket in description, if appropriate.
• Change is code complete and matches issue description
• Automated tests have been added to exercise the changes
• Reviewer's manual test instructions provided in PR description. See Reviewer's first task below.

Reviewer checklist

• Run the changes and verify that the change matches the issue description
• Reviewed the code
• Verified that the appropriate tests have been written or valid explanation given

Documentation changes

• Link to jenkins.io PR, or an explanation for why no doc changes are needed