Skip to content

test: improve tests#20

Merged
olivermeyer merged 6 commits into
mainfrom
test/improve-tests
Mar 30, 2026
Merged

test: improve tests#20
olivermeyer merged 6 commits into
mainfrom
test/improve-tests

Conversation

@olivermeyer
Copy link
Copy Markdown
Collaborator

  • Implementing reset_context function
  • Updating a bunch of tests so they stop mocking own code, and making them integration tests instead

olivermeyer and others added 5 commits March 30, 2026 19:26
- Add reset_context() to foundry.py for use in test teardown
- Extend make_context() to accept version, environment, and **kwargs
- Add unit tests for reset_context() behaviour

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace monkeypatch.setattr("...foundry._context", None) with the public
reset_context() API in foundry_test.py and console_test.py autouse
fixtures.

Also adds a minimal _gui_context autouse fixture to TestGetGuiUser in
gui_test.py to fix a latent ordering bug exposed by the stricter teardown:
get_gui_user() calls load_settings(AuthSettings) -> get_context(), so a
context must be set before those tests run.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Delete the local _mk_ctx helper and the three monkeypatch.setattr
autouse fixtures that stubbed get_context. Replace them with
set_context(FoundryContext(...)) + reset_context() so the real code
path runs. Replace _mk_ctx call sites with inline FoundryContext
construction using module-level constants. Reclassify
TestSentryInitialize, TestSentrySettingsDsnValidation, and
TestSentrySettings from @pytest.mark.unit to @pytest.mark.integration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@olivermeyer olivermeyer requested a review from a team as a code owner March 30, 2026 17:28
@olivermeyer olivermeyer added the claude Trigger Claude Code automation label Mar 30, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

Files with missing lines Coverage Δ
src/aignostics_foundry_core/foundry.py 96.61% <100.00%> (+0.05%) ⬆️

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@claude
Copy link
Copy Markdown

claude Bot commented Mar 30, 2026

Claude encountered an error —— View job


Code Review Complete ✅

Reviewing PR #20: test: improve tests


Summary

This PR successfully implements reset_context() and refactors tests to eliminate internal mocking in favor of integration tests. The core changes are well-executed and follow best practices. However, there are unintentional workflow version downgrades that should be addressed.

✅ Strengths

1. Excellent reset_context() Implementation

src/aignostics_foundry_core/foundry.py:277-285

The implementation is clean, simple, and well-documented:

  • Properly uses global keyword with noqa comment
  • Clear docstring explaining purpose and use case
  • Symmetric with set_context()
  • Tests verify both functionality and idempotency (foundry_test.py:1083-1096)

2. Outstanding Test Refactoring

The conversion from unit tests with internal mocks to integration tests is exemplary:

Before (mocking internal code):

with patch("aignostics_foundry_core.api.auth.get_user", ...):
    result = await require_authenticated(request, None)

After (real setup):

request.app.state.auth_client = fake_client
result = await require_authenticated(request, None)

This approach:

  • ✅ Tests actual behavior instead of implementation details
  • ✅ Makes tests more resilient to refactoring
  • ✅ Catches integration issues that mocks would miss
  • ✅ Properly marks tests as @pytest.mark.integration

Files refactored: auth_test.py, gui_test.py, di_test.py, sentry_test.py, console_test.py

3. Improved Test Infrastructure

  • Enhanced make_context() helper with additional parameters (conftest.py:58-82)
  • Proper fixture usage with autouse=True and reset_context() cleanup
  • Consistent use of Generator[None, None, None] type hints

4. Documentation Excellence

Added critical guidance in tests/AGENTS.md:95-99:

Critical rule: @patch is only acceptable for external libraries and stdlib. Never patch symbols inside aignostics_foundry_core itself.

This prevents future regressions and educates contributors.

5. Quality Metrics

  • ✅ 100% test coverage maintained (Codecov)
  • ✅ All SonarQube rules passing
  • ✅ All test markers present (unit or integration)
  • ✅ Conventional commit messages throughout

6. Smart Dependency Injection Fix

di_test.py now mocks entry_points (stdlib) instead of discover_plugin_packages (internal):

# Before: patch.object(di, "discover_plugin_packages", return_value=(PLUGIN,))
# After: patch(DI_ENTRY_POINTS, return_value=[_mock_ep(PLUGIN)])

Extracted DI_ENTRY_POINTS constant (line 502) avoids string duplication (SonarQube S1192).

⚠️ Issues Found

Blocking: Unintentional Workflow Version Downgrades

Three workflow files contain version downgrades that appear unrelated to this PR:

  1. .github/workflows/_codeql.yml - Lines 9, 18, 22

    • github/codeql-action: v4.35.1 → v4.34.1 (downgrade)
  2. .github/workflows/_test.yml + _scheduled-test-daily.yml - Lines 47, 206

    • codecov/codecov-action: v6.0.0 → v5.5.3 (downgrade)
  3. .github/workflows/claude-code-automation-pr-review.yml - Line 60

    • anthropics/claude-code-action: v1.0.82 → v1.0.79 (downgrade)

Impact: Downgrades may reintroduce bugs or miss security fixes. These changes are not mentioned in the PR description or any commit message.

Root Cause: Likely a merge conflict resolution or rebase that reverted these to older versions from the base branch.

Recommendation: Revert these workflow changes or create a separate commit explaining why the downgrades are necessary.

Fix this →

📋 Observations (Non-blocking)

1. Consistent Fixture Pattern

All refactored test files use the same autouse fixture pattern:

@pytest.fixture(autouse=True)
def _context() -> Generator[None, None, None]:
    set_context(make_context(...))
    yield
    reset_context()

This is excellent for consistency and maintainability.

2. Integration Test Classification

The PR correctly identifies that tests calling multiple internal functions are integration tests, even without external services. This aligns with the project's testing philosophy documented in tests/AGENTS.md.

3. Backward Compatibility

The make_context() signature changes (conftest.py:58-64) are backward compatible - all new parameters have defaults.

🎯 Final Verdict

Status: ✅ Approve with requested changes

The core changes (test improvements and reset_context()) are excellent and ready to merge. However, the workflow version downgrades must be addressed first.

Action Items:

  1. ⚠️ Blocking: Revert workflow version downgrades or explain them
  2. ✅ After that, this PR is ready to merge

Excellent work on improving test quality and eliminating fragile internal mocking! This sets a strong precedent for future test development.


@sonarqubecloud
Copy link
Copy Markdown

@olivermeyer olivermeyer merged commit d1967b4 into main Mar 30, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude Trigger Claude Code automation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant