Skip to content

fix(github): don't abort scorecard run on 422 from commit search - #5181

Open
pujitha24 wants to merge 3 commits into
ossf:mainfrom
pujitha24:auto/issue-4352
Open

fix(github): don't abort scorecard run on 422 from commit search#5181
pujitha24 wants to merge 3 commits into
ossf:mainfrom
pujitha24:auto/issue-4352

Conversation

@pujitha24

Copy link
Copy Markdown
Contributor

What kind of change does this PR introduce?

Bug fix.

What is the current behavior?

The Dependency-Update-Tool check calls GitHub's commit search API to look for
dependabot[bot] commits. Some public repositories (not yet indexed by GitHub
search) and GitHub Enterprise Server instances return 422 Validation Failed
for this otherwise-valid query. That 422 is currently propagated as a hard
error, which aborts the entire scorecard run with check runtime error: Dependency-Update-Tool: internal error: dependabot commit search: ..., even
though every other check already finished successfully.

What is the new behavior (if this is a feature change)?

searchCommitsHandler.search now detects a 422 Unprocessable Entity
response from Search.Commits and returns an empty commit list with no
error, instead of a hard error. This mirrors the existing handling of
clients.ErrUnsupportedFeature a few lines away in
checks/raw/dependency_update_tool.go, which already treats "can't search
commits" as "assume no commit-search-based tools were found" rather than
failing the whole run. Also fixed an unrelated copy-paste typo in the wrapped
error message (Search.Code -> Search.Commits, since this call is
Search.Commits).

  • Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

Validation: added TestSearchCommitsHandles422, which mocks a 422 HTTP
response and asserts search() returns no error and zero commits. Confirmed
this test fails on the pre-fix code (reproducing the reported
"Search.Code: 422 Validation Failed" error) and passes after the fix.

Commands run:

  • go build ./...
  • SKIP_GINKGO=1 go test -race ./clients/githubrepo/... ./checks/raw/... (matches CI's make unit-test)
  • golangci-lint run -c .golangci.yml --new-from-rev=HEAD ./clients/githubrepo/... (0 new issues)

This only changes the client's error handling for a specific HTTP status
code; user-visible check results for repos that don't hit this 422 are
unchanged. For repos that do hit it, Dependency-Update-Tool will now
complete (falling back to file-based detection, e.g. .github/dependabot.yml)
instead of aborting the whole scorecard run.

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

Fixed a bug where the Dependency-Update-Tool check could abort the entire
scorecard run with a "422 Validation Failed" error from GitHub's commit
search API, which occurs for some public repositories and on GitHub
Enterprise Server. The check now completes gracefully, falling back to
file-based detection.

Fixes #4352

Motivation:
The Dependency-Update-Tool check calls GitHub's commit search API to
look for dependabot[bot] commits. Some public repositories that
GitHub search hasn't indexed yet, and GitHub Enterprise Server
instances, return 422 Validation Failed for this otherwise-valid
query. That 422 was propagated as a hard error, aborting the entire
scorecard run with "check runtime error: Dependency-Update-Tool:
internal error: dependabot commit search: ..." even though every
other check had already finished successfully.

Approach:
searchCommitsHandler.search now detects a 422 Unprocessable Entity
response from Search.Commits and returns an empty commit list with
no error, instead of a hard error. This mirrors the existing handling
of clients.ErrUnsupportedFeature a few lines away in
checks/raw/dependency_update_tool.go, which already treats "can't
search commits" as "assume no commit-search-based tools were found"
rather than failing the whole run. Also fixed an unrelated
copy-paste typo in the wrapped error message (Search.Code ->
Search.Commits, since this call is Search.Commits).

This only changes error handling for this specific HTTP status code;
check results for repos that don't hit the 422 are unchanged. For
repos that do hit it, Dependency-Update-Tool now completes (falling
back to file-based detection) instead of aborting the whole run.

Validation:
Added TestSearchCommitsHandles422, which mocks a 422 HTTP response
and asserts search() returns no error and zero commits. Confirmed
this test fails on the pre-fix code (reproducing the reported
"Search.Code: 422 Validation Failed" error) and passes after the fix.

Ran:
  go build ./...
  SKIP_GINKGO=1 go test -race ./clients/githubrepo/... ./checks/raw/...
  golangci-lint run -c .golangci.yml --new-from-rev=HEAD ./clients/githubrepo/...
All passed with no new lint issues.

Report: ossf#4352
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
@pujitha24
pujitha24 requested a review from a team as a code owner August 14, 2026 08:36
@pujitha24
pujitha24 requested review from jeffmendoza and removed request for a team August 14, 2026 08:36
@dosubot dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Aug 14, 2026
@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 70.15%. Comparing base (353ed60) to head (5bf7929).
⚠️ Report is 373 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5181      +/-   ##
==========================================
+ Coverage   66.80%   70.15%   +3.34%     
==========================================
  Files         230      252      +22     
  Lines       16602    15865     -737     
==========================================
+ Hits        11091    11130      +39     
+ Misses       4808     3848     -960     
- Partials      703      887     +184     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Move the "treat GitHub commit-search 422 as no data" handling out of
searchCommitsHandler.search and into checks/raw/dependency_update_tool.go,
mirroring the existing clients.ErrUnsupportedFeature handling there.

The previous version swallowed the 422 at the generic client layer, which
also silently hid genuinely invalid search queries (e.g. a nonexistent
author) and broke e2e/searchCommits_test.go's
"Should return error as the user does not exist" case. SearchCommits now
still returns an error (wrapped in the new clients.ErrCommitSearchUnprocessable
sentinel) for 422s, and only the dependency-update-tool check - the one
issue ossf#4352 is actually about - treats that specific error as "assume no
dependabot commits found" instead of aborting.

Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
@pujitha24

Copy link
Copy Markdown
Contributor Author

Just checking in on this one — it's rebased on main and all checks are green. Happy to make any changes if something would help move review along.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S This PR changes 10-29 lines, ignoring generated files.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Public GH repo is getting 422 Validation Failed

1 participant