Skip to content

Align Codacy coverage upload and Sonar hotspot blocking #113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Prekzursil merged 1 commit into from
Mar 9, 2026
Merged

Align Codacy coverage upload and Sonar hotspot blocking #113

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/codecov-analytics.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,3 +88,33 @@ jobs:
flags: api,worker,media-core,web,desktop-ts
fail_ci_if_error: true
verbose: true

- name: Upload coverage to Codacy
if: ${{ always() }}
env:
CODACY_API_TOKEN: ${{ secrets.CODACY_API_TOKEN }}
CODACY_ORGANIZATION_PROVIDER: gh
CODACY_USERNAME: Prekzursil
CODACY_PROJECT_NAME: ${{ github.event.repository.name }}
run: |
if [ -z "${CODACY_API_TOKEN}" ]; then
echo "Missing CODACY_API_TOKEN" >&2
exit 1
fi
Comment on lines +94 to +103
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Codacy coverage reporter authentication CODACY_PROJECT_TOKEN single repository

💡 Result:

For a single repository, Codacy Coverage Reporter authenticates using a Project (repository) API token, exposed as CODACY_PROJECT_TOKEN (or passed as --project-token). You don’t need CODACY_API_TOKEN unless you’re automating uploads for multiple repositories from the same workflow/account. [1][3]

Single-repo (recommended)

  • Create/get the Project/Repository API token in Codacy (Project/Repository settings → Integrations → Project API token), then set it in CI as:
    • Environment variable: CODACY_PROJECT_TOKEN=<token> [1]
    • Or CLI flag: codacy-coverage-reporter report --project-token <token> ... [3]

GitHub Actions example (single repo)

  • Store the token as a repo secret named CODACY_PROJECT_TOKEN, then pass it as project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}. [2]

When to use CODACY_API_TOKEN instead

  • If you’re uploading coverage for multiple repositories, use an Account API token (CODACY_API_TOKEN) plus CODACY_ORGANIZATION_PROVIDER, CODACY_USERNAME, and CODACY_PROJECT_NAME. [1][2]

Sources

  1. Codacy docs – Coverage Reporter setup/authentication (single repo vs multiple repos) [1]
  2. codacy/codacy-coverage-reporter-action README (project-token vs api-token) [2]
  3. Codacy Coverage Reporter CLI help/options (--project-token, --api-token, etc.) [3]

Use a repository-scoped Codacy token.

This workflow uploads coverage for a single repository but uses CODACY_API_TOKEN, which Codacy documents for multi-repository automation. For single-repo uploads, CODACY_PROJECT_TOKEN is the recommended approach and avoids granting unnecessary account-level access in CI.

Narrow the secret scope 
       - name: Upload coverage to Codacy
         if: ${{ always() }}
         env:
-          CODACY_API_TOKEN: ${{ secrets.CODACY_API_TOKEN }}
-          CODACY_ORGANIZATION_PROVIDER: gh
-          CODACY_USERNAME: Prekzursil
-          CODACY_PROJECT_NAME: ${{ github.event.repository.name }}
+          CODACY_PROJECT_TOKEN: ${{ secrets.CODACY_PROJECT_TOKEN }}
         run: |
-          if [ -z "${CODACY_API_TOKEN}" ]; then
-            echo "Missing CODACY_API_TOKEN" >&2
+          if [ -z "${CODACY_PROJECT_TOKEN}" ]; then
+            echo "Missing CODACY_PROJECT_TOKEN" >&2
             exit 1
           fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
env:
CODACY_API_TOKEN: ${{ secrets.CODACY_API_TOKEN }}
CODACY_ORGANIZATION_PROVIDER: gh
CODACY_USERNAME: Prekzursil
CODACY_PROJECT_NAME: ${{ github.event.repository.name }}
run: |
if [ -z "${CODACY_API_TOKEN}" ]; then
echo "Missing CODACY_API_TOKEN" >&2
exit 1
fi
env:
CODACY_PROJECT_TOKEN: ${{ secrets.CODACY_PROJECT_TOKEN }}
run: |
if [ -z "${CODACY_PROJECT_TOKEN}" ]; then
echo "Missing CODACY_PROJECT_TOKEN" >&2
exit 1
fi
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/codecov-analytics.yml around lines 94 - 103, The workflow
is using the account-scoped CODACY_API_TOKEN for a single-repo coverage upload;
replace it with the repository-scoped CODACY_PROJECT_TOKEN everywhere in this
job (env block and the runtime check) so the job exports CODACY_PROJECT_TOKEN
instead of CODACY_API_TOKEN, validates that CODACY_PROJECT_TOKEN is present, and
fails if missing; update references to CODACY_USERNAME/CODACY_PROJECT_NAME
unchanged but ensure any downstream steps that read the token consume
CODACY_PROJECT_TOKEN.


COVERAGE_REPORTS=(
"coverage/python-coverage.xml"
"apps/web/coverage/lcov.info"
"apps/desktop/coverage/lcov.info"
)
REPORT_ARGS=()
for report in "${COVERAGE_REPORTS[@]}"; do
if [ ! -f "${report}" ]; then
echo "Missing coverage report: ${report}" >&2
exit 1
fi
REPORT_ARGS+=(-r "${report}")
done

bash <(curl -Ls https://coverage.codacy.com/get.sh) report "${REPORT_ARGS[@]}" --partial
bash <(curl -Ls https://coverage.codacy.com/get.sh) final
2 changes: 1 addition & 1 deletion .github/workflows/sonar-zero.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

- name: Assert Sonar zero-open gate
- name: Assert Sonar zero-open issues and hotspots gate
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
run: |
Expand Down
62 changes: 62 additions & 0 deletions apps/api/tests/test_scripts_quality_gates.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ def test_sonar_evaluate_status_ignores_open_issues_when_flag_set():

findings = module.evaluate_status(
open_issues=12,
open_hotspots=0,
quality_gate="OK",
require_quality_gate=True,
ignore_open_issues=True,
Expand All @@ -131,9 +132,70 @@ def test_sonar_evaluate_status_still_enforces_quality_gate():

findings = module.evaluate_status(
open_issues=12,
open_hotspots=0,
quality_gate="ERROR",
require_quality_gate=True,
ignore_open_issues=True,
)

_expect(any("quality gate" in item for item in findings), "Expected quality gate finding")


def test_sonar_evaluate_status_enforces_open_hotspots():
module = _load_module("check_sonar_zero")

findings = module.evaluate_status(
open_issues=0,
open_hotspots=3,
quality_gate="OK",
require_quality_gate=False,
ignore_open_issues=True,
)

_expect(
any("security hotspots" in item for item in findings),
"Expected open hotspot finding even when open issues are ignored",
)


def test_sonar_query_status_reads_hotspots(monkeypatch):
module = _load_module("check_sonar_zero")
responses = [
{"paging": {"total": 4}},
{"projectStatus": {"status": "OK"}},
{"paging": {"total": 2}},
]
monkeypatch.setattr(module, "_request_json", lambda _url, _auth: responses.pop(0))

open_issues, quality_gate, open_hotspots = module._query_sonar_status(
api_base="https://sonarcloud.io",
auth="Basic abc",
project_key="Prekzursil_Reframe",
branch="main",
pull_request="",
)

_expect(open_issues == 4, "Expected open issues to be parsed")
_expect(quality_gate == "OK", "Expected quality gate to be parsed")
_expect(open_hotspots == 2, "Expected open hotspots to be parsed")


def test_sonar_render_md_includes_open_hotspots():
module = _load_module("check_sonar_zero")

rendered = module._render_md(
{
"status": "fail",
"project_key": "Prekzursil_Reframe",
"scope": "branch",
"branch": "main",
"pull_request": None,
"open_issues": 0,
"open_hotspots": 5,
"quality_gate": "OK",
"timestamp_utc": "2026-03-09T00:00:00+00:00",
"findings": ["Sonar reports 5 open security hotspots pending review (expected 0)."],
}
)

_expect("- Open hotspots: `5`" in rendered, "Expected hotspot count in markdown output")
38 changes: 31 additions & 7 deletions scripts/quality/check_sonar_zero.py
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@


def _parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Assert SonarCloud has zero actionable open issues.")
parser = argparse.ArgumentParser(
description="Assert SonarCloud has zero actionable open issues and zero open security hotspots."
)
parser.add_argument("--project-key", required=True, help="Sonar project key")
parser.add_argument("--token", default="", help="Sonar token (falls back to SONAR_TOKEN env)")
parser.add_argument("--branch", default="", help="Optional branch scope")
Expand Down Expand Up @@ -76,7 +78,7 @@ def _query_sonar_status(
project_key: str,
branch: str,
pull_request: str,
) -> tuple[int, str]:
) -> tuple[int, str, int]:
issues_query = {
"componentKeys": project_key,
"resolved": "false",
Expand All @@ -101,19 +103,37 @@ def _query_sonar_status(
gate_payload = _request_json(gate_url, auth)
project_status = (gate_payload.get("projectStatus") or {})
quality_gate = str(project_status.get("status") or "UNKNOWN")
return open_issues, quality_gate

hotspot_query = {
"projectKey": project_key,
"status": "TO_REVIEW",
"ps": "1",
}
if branch:
hotspot_query["branch"] = branch
if pull_request:
hotspot_query["pullRequest"] = pull_request
hotspot_url = f"{api_base}/api/hotspots/search?{urllib.parse.urlencode(hotspot_query)}"
hotspot_payload = _request_json(hotspot_url, auth)
hotspot_paging = hotspot_payload.get("paging") or {}
open_hotspots = int(hotspot_paging.get("total") or 0)

return open_issues, quality_gate, open_hotspots


def evaluate_status(
*,
open_issues: int,
open_hotspots: int,
quality_gate: str,
require_quality_gate: bool,
ignore_open_issues: bool,
) -> list[str]:
findings: list[str] = []
if not ignore_open_issues and open_issues != 0:
findings.append(f"Sonar reports {open_issues} open issues (expected 0).")
if open_hotspots != 0:
findings.append(f"Sonar reports {open_hotspots} open security hotspots pending review (expected 0).")
Comment on lines 133 to +136
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

--ignore-open-issues is now misleading.

Line 135 still fails when hotspots are present, so the CLI help on Lines 45-47 no longer matches the behavior. Please update the flag text or rename the flag so callers do not read “quality gate only” and then get a hotspot failure anyway.

📝 Suggested text update 
     parser.add_argument(
         "--ignore-open-issues",
         action="store_true",
-        help="Skip open-issue enforcement and evaluate quality gate only.",
+        help="Skip open-issue enforcement; quality-gate and hotspot checks still apply.",
     )
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@scripts/quality/check_sonar_zero.py` around lines 133 - 136, The CLI flag
--ignore-open-issues is misleading because open security hotspots
(open_hotspots) still trigger failures; update either the flag name or help text
and/or the check logic so behavior matches the description: either rename the
flag to something explicit like --ignore-open-hotspots (and update all
references) or change the help text to clarify that hotspots are treated as
quality-gate failures, and if desired modify the conditional that appends to
findings (the open_hotspots check) to respect the ignore_open_issues boolean;
reference variables/strings: --ignore-open-issues, ignore_open_issues,
open_issues, open_hotspots, and findings to locate the code to change.

if require_quality_gate and quality_gate != "OK":
findings.append(f"Sonar quality gate status is {quality_gate} (expected OK).")
return findings
Expand All @@ -130,6 +150,7 @@ def _render_md(payload: dict) -> str:
f"- Branch: `{payload.get('branch')}`",
f"- Pull request: `{payload.get('pull_request')}`",
f"- Open issues: `{payload.get('open_issues')}`",
f"- Open hotspots: `{payload.get('open_hotspots')}`",
f"- Quality gate: `{payload.get('quality_gate')}`",
f"- Timestamp (UTC): `{payload['timestamp_utc']}`",
"",
Expand Down Expand Up @@ -170,6 +191,7 @@ def main() -> int:
scope = "branch"
findings: list[str] = []
open_issues: int | None = None
open_hotspots: int | None = None
quality_gate: str | None = None

if not token:
Expand All @@ -178,7 +200,7 @@ def main() -> int:
else:
auth = _auth_header(token)
try:
open_issues, quality_gate = _query_sonar_status(
open_issues, quality_gate, open_hotspots = _query_sonar_status(
api_base=api_base,
auth=auth,
project_key=args.project_key,
Expand All @@ -187,11 +209,11 @@ def main() -> int:
)
quality_gate = quality_gate or "UNKNOWN"

if args.pull_request and open_issues != 0 and args.wait_seconds > 0:
if args.pull_request and (open_issues != 0 or open_hotspots != 0) and args.wait_seconds > 0:
deadline = time.time() + max(0, args.wait_seconds)
while open_issues != 0 and time.time() < deadline:
while (open_issues != 0 or open_hotspots != 0) and time.time() < deadline:
time.sleep(10)
open_issues, quality_gate = _query_sonar_status(
open_issues, quality_gate, open_hotspots = _query_sonar_status(
api_base=api_base,
auth=auth,
project_key=args.project_key,
Expand All @@ -203,6 +225,7 @@ def main() -> int:
findings.extend(
evaluate_status(
open_issues=open_issues,
open_hotspots=open_hotspots,
quality_gate=quality_gate,
require_quality_gate=args.require_quality_gate,
ignore_open_issues=args.ignore_open_issues,
Expand All @@ -221,6 +244,7 @@ def main() -> int:
"branch": args.branch or None,
"pull_request": args.pull_request or None,
"open_issues": open_issues,
"open_hotspots": open_hotspots,
"quality_gate": quality_gate,
"ignore_open_issues": bool(args.ignore_open_issues),
"timestamp_utc": datetime.now(timezone.utc).isoformat(),
Expand Down
Loading