Skip to content

🧪 QA: Add tests for HIPAA compliance monitor #325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
daggerstuff wants to merge 5 commits into
base: staging
Choose a base branch
from
+56 −0
Open

🧪 QA: Add tests for HIPAA compliance monitor #325

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
34149e7
test: add unit tests for HIPAA compliance monitor
google-labs-jules[bot] Mar 31, 2026
ee7f5cc
test: add unit tests for HIPAA compliance monitor
google-labs-jules[bot] Mar 31, 2026
d4fd3d1
test: add unit tests for HIPAA compliance monitor
google-labs-jules[bot] Mar 31, 2026
ce7247b
test: add unit tests for HIPAA compliance monitor
google-labs-jules[bot] Mar 31, 2026
18d1964
test: add unit tests for HIPAA compliance monitor
google-labs-jules[bot] Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions tests/test_compliance_monitor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
import os
import sys
import importlib.util

# Load the module dynamically since it has a hyphen in the filename
spec = importlib.util.spec_from_file_location(
"compliance_monitor",
os.path.join(os.path.dirname(__file__), "..", "security", "compliance-monitor.py")
)
compliance_monitor = importlib.util.module_from_spec(spec)
spec.loader.exec_module(compliance_monitor)

Comment on lines +1 to +12
Copy link
Copy Markdown

@charliecreates charliecreates bot Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sys is imported but never used. Also, the dynamic import block assumes spec and spec.loader are always non-null; if the file path changes or the loader can’t be constructed, this will fail with a less actionable error at import-time of the test module.

Suggestion

Remove the unused sys import and add a defensive check with a clear error before calling exec_module.

import os
import importlib.util

spec = importlib.util.spec_from_file_location(
    "compliance_monitor",
    os.path.join(os.path.dirname(__file__), "..", "security", "compliance-monitor.py"),
)
if spec is None or spec.loader is None:
    raise ImportError("Unable to load compliance-monitor.py for tests")

compliance_monitor = importlib.util.module_from_spec(spec)
spec.loader.exec_module(compliance_monitor)

Reply with "@CharlieHelps yes please" if you’d like me to add a commit with this change.

Comment on lines +1 to +12
Copy link
Copy Markdown

@charliecreates charliecreates bot Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dynamic import is brittle as written:

  • spec / spec.loader can be None (you call spec.loader.exec_module(...) unconditionally).
  • The module isn’t registered in sys.modules, which can break imports inside compliance-monitor.py (and you currently import sys but don’t use it).
  • Building the path via string .. joins works, but Path(...).resolve() is typically more robust/clear for tests.
Suggestion

Harden the loader and register the module, and remove the now-purposeful sys unused import by actually using it:

from pathlib import Path
import importlib.util
import sys

module_path = (Path(__file__).resolve().parents[1] / "security" / "compliance-monitor.py")

spec = importlib.util.spec_from_file_location("compliance_monitor", module_path)
assert spec is not None and spec.loader is not None

compliance_monitor = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = compliance_monitor
spec.loader.exec_module(compliance_monitor)

Reply with "@CharlieHelps yes please" if you’d like me to add a commit with this change.

ComplianceMonitor = compliance_monitor.ComplianceMonitor

class TestComplianceMonitorHIPAA:
def test_hipaa_compliance_all_pass(self):
monitor = ComplianceMonitor()
result = monitor.monitor_hipaa_compliance()

assert result["framework"] == "HIPAA"
assert result["compliance_score"] == 100.0
assert result["status"] == "compliant"
assert all(result["checks"].values())

Comment on lines +15 to +24
Copy link
Copy Markdown

@charliecreates charliecreates bot Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_hipaa_compliance_all_pass depends on the real implementations of the check methods returning True in the current environment. If any check consults environment/config/IO, this test becomes flaky and stops being a unit test of monitor_hipaa_compliance()’s aggregation logic.

Suggestion

Patch all check methods to deterministic values for this test (prefer pytest’s monkeypatch so patches can’t leak).

import pytest

class TestComplianceMonitorHIPAA:
    def test_hipaa_compliance_all_pass(self, monkeypatch):
        monitor = ComplianceMonitor()
        monkeypatch.setattr(monitor, "check_administrative_safeguards", lambda: True)
        monkeypatch.setattr(monitor, "check_physical_safeguards", lambda: True)
        monkeypatch.setattr(monitor, "check_technical_safeguards", lambda: True)
        monkeypatch.setattr(monitor, "check_breach_notification", lambda: True)
        monkeypatch.setattr(monitor, "check_baa", lambda: True)

        result = monitor.monitor_hipaa_compliance()
        ...

Reply with "@CharlieHelps yes please" if you’d like me to add a commit with this suggestion.

def test_hipaa_compliance_some_fail(self):
monitor = ComplianceMonitor()

# Override a few methods to simulate failure
monitor.check_administrative_safeguards = lambda: False
monitor.check_technical_safeguards = lambda: False

result = monitor.monitor_hipaa_compliance()

assert result["framework"] == "HIPAA"
assert result["compliance_score"] == 60.0
assert result["status"] == "non_compliant"
assert result["checks"]["administrative_safeguards"] is False
assert result["checks"]["technical_safeguards"] is False
assert result["checks"]["physical_safeguards"] is True

Comment on lines +25 to +40
Copy link
Copy Markdown

@charliecreates charliecreates bot Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In test_hipaa_compliance_some_fail, only two checks are overridden; any other check that returns False in the runtime environment will change the score/status and make the expected 60.0 brittle. Also, asserting a hard-coded score couples the test to the current number/weighting of checks.

Suggestion

Make the test deterministic by patching all check methods, and consider asserting the score derived from result["checks"] (or using pytest.approx) so the test focuses on aggregation correctness rather than a fragile constant.

import pytest

def test_hipaa_compliance_some_fail(self, monkeypatch):
    monitor = ComplianceMonitor()
    monkeypatch.setattr(monitor, "check_administrative_safeguards", lambda: False)
    monkeypatch.setattr(monitor, "check_technical_safeguards", lambda: False)
    monkeypatch.setattr(monitor, "check_physical_safeguards", lambda: True)
    monkeypatch.setattr(monitor, "check_breach_notification", lambda: True)
    monkeypatch.setattr(monitor, "check_baa", lambda: True)

    result = monitor.monitor_hipaa_compliance()
    expected = 100.0 * sum(result["checks"].values()) / len(result["checks"])
    assert result["compliance_score"] == pytest.approx(expected)

Reply with "@CharlieHelps yes please" if you’d like me to add a commit with this suggestion.

def test_hipaa_compliance_all_fail(self):
monitor = ComplianceMonitor()

# Override all methods to simulate failure
monitor.check_administrative_safeguards = lambda: False
monitor.check_physical_safeguards = lambda: False
monitor.check_technical_safeguards = lambda: False
monitor.check_breach_notification = lambda: False
monitor.check_baa = lambda: False

result = monitor.monitor_hipaa_compliance()

assert result["framework"] == "HIPAA"
assert result["compliance_score"] == 0.0
assert result["status"] == "non_compliant"
assert not any(result["checks"].values())
Loading