Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/workflows/coverage_security_guard.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: PR Coverage & Security Guard (ELUSoC_2026)

on:
pull_request:
branches: [ main ]

jobs:
coverage_security_guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install .[dev]

- name: Run Bandit Security Scan
run: |
bandit -r agentwatch/ -ll

- name: Run Tests and Measure Coverage
run: |
pytest --cov=agentwatch/ tests/

- name: Verify Minimum Coverage Thresholds
run: |
python scripts/verify_coverage.py
Comment on lines +27 to +33

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Consider using pytest's native --cov-fail-under=70 instead of a separate script.

The coding guidelines state coverage is Maintain minimum test coverage of 70% (enforced in CI via pytest with --cov-fail-under=70). This workflow instead reimplements the threshold check via scripts/verify_coverage.py, adding a custom script, extra CI step, and test to maintain, when pytest --cov=agentwatch/ --cov-fail-under=70 tests/ would achieve the same gate natively.

If the custom script is intentional (e.g., for richer output/logging), consider ignoring this; otherwise consolidating removes duplicate logic.

As per coding guidelines, tests/**/*.py: "Maintain minimum test coverage of 70% (enforced in CI via pytest with --cov-fail-under=70)".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/coverage_security_guard.yml around lines 27 - 33, The
workflow currently checks coverage with a custom script instead of pytest’s
built-in gate. Update the test step in the coverage_security_guard workflow (the
one running pytest for agentwatch/) to include --cov-fail-under=70, and remove
the separate Verify Minimum Coverage Thresholds step that calls
scripts/verify_coverage.py. Keep the existing pytest coverage collection, but
consolidate the threshold enforcement in the pytest invocation.

Source: Coding guidelines

13 changes: 13 additions & 0 deletions docs/security_ci.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Automated CI Code Quality and Security Guard (ELUSoC_2026)

## Overview
To maintain security compliance and avoid regression, the repository implements a PR linting workflow.

## Steps Performed
1. **Security Vulnerability Scan**: Invokes `Bandit` to inspect files for security issues.
2. **Coverage Gate**: Validates that test suites execute and code coverage remains above the minimum threshold (default: 70%).

## Files Involved
- `.github/workflows/coverage_security_guard.yml` - CI runner setup.
- `scripts/verify_coverage.py` - Coverage limit checker.
- `tests/test_coverage_guard.py` - Unit test verifying threshold limits.
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ dev = [
"pytest-cov>=5.0.0",
"mypy>=1.11.0",
"ruff>=0.6.0",
"bandit>=1.7.0",
"httpx>=0.28.0",
"opentelemetry-sdk>=1.27.0",
]
Expand Down
35 changes: 35 additions & 0 deletions scripts/verify_coverage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
"""
ELUSoC_2026 - Test Coverage Verification Script.
Ensures overall code coverage does not drop below required project thresholds.
"""

from __future__ import annotations

import json
import subprocess
import sys


def main() -> None:
# Run coverage report in JSON format
subprocess.run(["coverage", "json"], check=True)

with open("coverage.json", "r") as f:
data = json.load(f)

percent = data["totals"]["percent_covered"]
required_threshold = 70.0

print(f"Current Code Coverage: {percent:.2f}%")
print(f"Required Coverage Threshold: {required_threshold}%")

if percent < required_threshold:
print("Code coverage is below the required threshold! PR blocked.")
sys.exit(1)

print("Code coverage thresholds verified successfully.")
sys.exit(0)


if __name__ == "__main__":
main()
35 changes: 35 additions & 0 deletions tests/test_coverage_guard.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
"""Unit tests for verify_coverage.py script."""

from __future__ import annotations

import json
import os
import subprocess
import sys
import pytest


def test_coverage_script_passes(tmp_path, monkeypatch):
# Mock coverage.json file
json_path = tmp_path / "coverage.json"
dummy_data = {
"totals": {
"percent_covered": 75.5
}
}
with open(json_path, "w") as f:
json.dump(dummy_data, f)

monkeypatch.chdir(tmp_path)
# Monkeypatch subprocess to mock the raw coverage call
monkeypatch.setattr(subprocess, "run", lambda *args, **kwargs: None)

# Ensure root path is in sys.path
root_path = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
if root_path not in sys.path:
sys.path.insert(0, root_path)

from scripts.verify_coverage import main
with pytest.raises(SystemExit) as excinfo:
main()
assert excinfo.value.code == 0
Comment on lines +12 to +35

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Add a test for the failing-threshold branch.

Only the passing case (coverage ≥ 70%) is tested. The sys.exit(1) branch when coverage is below threshold is never exercised, leaving a core behavior of the script unverified.

✅ Suggested addition
+def test_coverage_script_fails(tmp_path, monkeypatch):
+    json_path = tmp_path / "coverage.json"
+    dummy_data = {"totals": {"percent_covered": 50.0}}
+    with open(json_path, "w") as f:
+        json.dump(dummy_data, f)
+
+    monkeypatch.chdir(tmp_path)
+    monkeypatch.setattr(subprocess, "run", lambda *args, **kwargs: None)
+
+    root_path = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
+    if root_path not in sys.path:
+        sys.path.insert(0, root_path)
+
+    from scripts.verify_coverage import main
+    with pytest.raises(SystemExit) as excinfo:
+        main()
+    assert excinfo.value.code == 1
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def test_coverage_script_passes(tmp_path, monkeypatch):
# Mock coverage.json file
json_path = tmp_path / "coverage.json"
dummy_data = {
"totals": {
"percent_covered": 75.5
}
}
with open(json_path, "w") as f:
json.dump(dummy_data, f)
monkeypatch.chdir(tmp_path)
# Monkeypatch subprocess to mock the raw coverage call
monkeypatch.setattr(subprocess, "run", lambda *args, **kwargs: None)
# Ensure root path is in sys.path
root_path = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
if root_path not in sys.path:
sys.path.insert(0, root_path)
from scripts.verify_coverage import main
with pytest.raises(SystemExit) as excinfo:
main()
assert excinfo.value.code == 0
def test_coverage_script_passes(tmp_path, monkeypatch):
# Mock coverage.json file
json_path = tmp_path / "coverage.json"
dummy_data = {
"totals": {
"percent_covered": 75.5
}
}
with open(json_path, "w") as f:
json.dump(dummy_data, f)
monkeypatch.chdir(tmp_path)
# Monkeypatch subprocess to mock the raw coverage call
monkeypatch.setattr(subprocess, "run", lambda *args, **kwargs: None)
# Ensure root path is in sys.path
root_path = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
if root_path not in sys.path:
sys.path.insert(0, root_path)
from scripts.verify_coverage import main
with pytest.raises(SystemExit) as excinfo:
main()
assert excinfo.value.code == 0
def test_coverage_script_fails(tmp_path, monkeypatch):
json_path = tmp_path / "coverage.json"
dummy_data = {"totals": {"percent_covered": 50.0}}
with open(json_path, "w") as f:
json.dump(dummy_data, f)
monkeypatch.chdir(tmp_path)
monkeypatch.setattr(subprocess, "run", lambda *args, **kwargs: None)
root_path = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
if root_path not in sys.path:
sys.path.insert(0, root_path)
from scripts.verify_coverage import main
with pytest.raises(SystemExit) as excinfo:
main()
assert excinfo.value.code == 1
🧰 Tools
🪛 ast-grep (0.44.0)

[warning] 19-19: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(json_path, "w")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').

(open-filename-from-request)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_coverage_guard.py` around lines 12 - 35, Add a test in
test_coverage_guard.py that covers the failing-threshold path in
verify_coverage.main, alongside test_coverage_script_passes. Mock coverage.json
with percent_covered below the 70% threshold, keep subprocess.run patched as in
the passing test, and assert that main raises SystemExit with exit code 1. Use
the existing main entry point and the coverage.json setup so the new test
exercises the same script behavior.

Loading