fix: add retry with exponential backoff to cosign keyless signing (transient OIDC failures)

[bedatty]

Affected Workflow

Security (src/security/cosign-sign/action.yml)

Workflow Version / Ref

@main (current)

Bug Description

The cosign sign --yes command in the cosign-sign composite action (line 94 of src/security/cosign-sign/action.yml) has no retry mechanism. Transient failures when fetching OIDC credentials from Fulcio cause the entire signing step — and downstream jobs — to fail.

A real-world occurrence was observed in LerianStudio/tracer run #24095706848, where cosign received an invalid OIDC response (invalid character 'u' looking for beginning of value — the token endpoint returned a non-JSON value). A manual re-run succeeded immediately, confirming the issue is transient.

Steps to Reproduce

1. Any caller workflow that uses cosign-sign action for keyless signing (e.g., go-build.yml, typescript-build.yml)
2. Fulcio/GitHub OIDC token endpoint returns a transient error or malformed response
3. cosign sign fails on the first attempt
4. Entire signing step fails, skipping downstream jobs (Helm dispatch, GitOps update)

Expected Behavior

The signing step should retry with exponential backoff (e.g., 3 attempts with delays of 5s, 15s, 45s) before failing. Transient OIDC/Fulcio issues should not cause a full pipeline failure.

Relevant Logs / Error Output

signing [docker.io/lerianstudio/tracer@sha256:8fdcc4b80d2c830a4fb6d833937200fa34f31857ec78abfeebe3d04b45e6dbbe]:
getting signer: getting key from Fulcio: fetching ambient OIDC credentials:
invalid character 'u' looking for beginning of value

Caller Workflow Configuration

# Current implementation in src/security/cosign-sign/action.yml (line 88-100)
run: |
  SIGNED=""
  while IFS= read -r ref; do
    ref=$(echo "$ref" | xargs)
    [ -z "$ref" ] && continue
    echo "Signing: $ref"
    cosign sign --yes "$ref"       # <-- no retry, single attempt
    if [ -n "$SIGNED" ]; then
      SIGNED="${SIGNED}"$'\n'"${ref}"
    else
      SIGNED="$ref"
    fi
  done <<< "$IMAGE_REFS"

Proposed Solution

Add exponential backoff around the cosign sign call. Something like:

sign_with_retry() {
  local ref="$1"
  local max_attempts=3
  local delay=5

  for attempt in $(seq 1 "$max_attempts"); do
    echo "Signing (attempt $attempt/$max_attempts): $ref"
    if cosign sign --yes "$ref"; then
      return 0
    fi
    if [ "$attempt" -lt "$max_attempts" ]; then
      echo "::warning::cosign sign failed (attempt $attempt/$max_attempts) — retrying in ${delay}s…"
      sleep "$delay"
      delay=$((delay * 3))
    fi
  done

  echo "::error::cosign sign failed after $max_attempts attempts: $ref"
  return 1
}

Runner OS

ubuntu-latest

Additional Context

• Failure link: https://github.com/LerianStudio/tracer/actions/runs/24095706848/job/70293629936b
• Re-run of the same workflow succeeded immediately — confirms the transient nature
• This affects all repos using the shared cosign-sign action (tracer, midaz, etc.)
• Consider also making max_attempts and initial_delay configurable via action inputs