Skip to content

Broaden no_expect lint for path-loaded test modules (tokio, rstest, cfg)#191

Open
leynos wants to merge 4 commits intomainfrom
fix-tokio-test-detection-in-path-modules-qgwt7m
Open

Broaden no_expect lint for path-loaded test modules (tokio, rstest, cfg)#191
leynos wants to merge 4 commits intomainfrom
fix-tokio-test-detection-in-path-modules-qgwt7m

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented Apr 7, 2026

Summary

  • Fix no_expect lint to correctly recognise test contexts inside #[path]-loaded modules with non-standard names (e.g., service_tests) when compiling under the --test harness. This ensures Option::expect lint rules are applied appropriately in nested path-modules containing #[tokio::test] functions.

Changes

  • Core logic

    • Add has_test_module_name(name: &str) -> bool in crates/no_expect_outside_tests/src/driver/mod.rs to recognise test module naming conventions:
      • Exact matches: test, tests
      • Prefixes: test_, tests_
      • Suffixes: _test, _tests
    • Update is_test_named_module to delegate to has_test_module_name instead of hard-coded checks.
    • Expanded documentation with examples and rationale for non-standard, path-loaded module names.
  • Tests and fixtures

    • Unit tests: Extend crates/no_expect_outside_tests/src/driver/tests.rs with has_test_module_name_matches_test_conventions using rstest to cover a broad set of naming patterns, including service_tests and other non-standard forms.
    • UI/UI-regression tests: Introduced fixtures to exercise path-loaded modules with non-standard names under --test:
      • crates/no_expect_outside_tests/examples/pass_expect_in_path_module_harness.rs and support module path_module_harness_support/service_tests.rs to simulate a path-loaded test module.
      • UI tests for path-loaded tokio tests:
        • crates/no_expect_outside_tests/ui/pass_expect_in_path_module_tokio_test.rs and accompanying service_tests.module to exercise #[tokio::test] inside a path-loaded module.
        • crates/no_expect_outside_tests/ui/pass_expect_in_cfg_test_named_module.rs and related stderr fixture to verify #[cfg(test)]-guarded non-standard module names.
    • Regression UI test: Added path_module_tokio_test_compiles_under_test_harness to verify that a path-loaded tokio test compiles under the test harness (UI test runner).
  • Examples

    • Added path-loaded harness example:
      • crates/no_expect_outside_tests/examples/pass_expect_in_path_module_harness.rs
      • crates/no_expect_outside_tests/examples/path_module_harness_support/service_tests.rs

Why this matters

  • Previously, path-loaded modules with non-standard names could bypass the module-name checks, causing the no_expect lint to miss test contexts inside those modules. This change broadens the detection rules to cover common non-standard naming patterns (e.g., service_tests) that arise from #[path]-loaded tests, ensuring the lint behavior is consistent across all test structures.

How to test

  • Run unit tests for the driver:
    • cargo test -p no_expect_outside_tests
  • Run UI/regression tests per repository guidelines (these verify that path-loaded, non-standard module names containing #[tokio::test] are recognized as test contexts under --test).
  • Optionally exercise the new examples under examples/ to reproduce the path-loaded module scenario described in the regression tests.

Related issues

◳ Generated by DevBoxer


ℹ️ Tag @devboxerhub to ask questions and address PR feedback

📎 Task: https://www.devboxer.com/task/6740313c-27f0-4e46-a021-2a6a9f9aca06

📝 Closes #132

Summary by Sourcery

Broaden detection of test-only code for the no_expect_outside_tests lint, especially for rstest-based tests and non-standard, path-loaded test modules, and add regression coverage to ensure correct behavior under the --test harness.

New Features:

  • Recognise additional test module naming conventions (e.g., prefix/suffix variants like service_tests) when determining test-only contexts.
  • Detect rstest #[case]-generated companion modules so their parent functions are treated as test code.
  • Support rstest_parametrize-style attributes in the shared test-like attribute registry.

Enhancements:

  • Refine module name heuristics into a reusable has_test_module_name helper used by is_test_named_module.

Tests:

  • Extend driver and attribute unit tests to cover new module-name and rstest_parametrize patterns.
  • Add example-based regression tests for real rstest usage and non-standard tokio test modules compiled under the --test harness.
  • Introduce UI fixtures validating .expect allowance in #[tokio::test] and #[cfg(test)] modules with non-standard names.

Extend test module name detection to include common conventions like
`service_tests` and `api_test`. This allows `.expect(...)` calls inside
`#[tokio::test]` functions within `#[path]`-loaded modules that have non-standard
names, supporting the Rust `--test` harness fallback.

Add regression examples and UI tests to validate these cases, covering issue
#132. This improves the lint's accuracy in identifying test contexts beyond
standard `test`/`tests` modules.

Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 7, 2026

Overview

This PR broadens the no_expect_outside_tests lint to correctly recognise test contexts inside modules loaded via #[path] that use non-standard names (e.g., service_tests, api_test) when compiled under the --test harness. It addresses issue #132 and continues the approach of recovering test context from harness-generated descriptors.

Core logic changes

crates/no_expect_outside_tests/src/driver/mod.rs

  • Added has_test_module_name(name: &str) -> bool to recognise module-name conventions beyond exact test/tests (matches exact test/tests, prefixes test_/tests_, and suffixes _test/_tests).
  • is_test_named_module now delegates to has_test_module_name.
  • Added has_companion_test_module and related logic to detect rstest-style companion modules (sibling modules that contain harness-generated const descriptors) and to insert matching functions into harness_marked.
  • Updated documentation/comments with examples and rationale for non-standard path-loaded module names.

Test and verification

crates/no_expect_outside_tests/src/driver/tests.rs

  • New rstest unit test has_test_module_name_matches_test_conventions covering positive, negative and edge cases for has_test_module_name.
  • Extended existing tests to recognise additional rstest-related attribute patterns.

crates/no_expect_outside_tests/src/lib_ui_tests.rs

  • Replaced the single hardcoded UI test with a parameterised #[rstest] test that runs three example cases (tokio, rstest, path module). Updated panic messaging per case.

UI/regression fixtures and examples

  • Added examples demonstrating path-loaded and rstest harnesses:
    • crates/no_expect_outside_tests/examples/pass_expect_in_path_module_harness.rs (inline service_tests module with #[tokio::test])
    • crates/no_expect_outside_tests/examples/pass_expect_in_rstest_harness.rs (real rstest usage)
  • Added UI fixtures:
    • ui/pass_expect_in_path_module_tokio_test.rs and companion service_tests.module (path-loaded module with #[tokio::test])
    • ui/pass_expect_in_cfg_test_named_module.rs (#[cfg(test)] module with non-standard name check_tests containing #[tokio::test])
  • Regression UI test coverage updated to exercise these examples via the parameterised runner.

Test-like attribute recognition

common/src/attributes/mod.rs and common/src/attributes/tests.rs

  • Extended TEST_LIKE_PATHS to include rstest_parametrize patterns: ["rstest_parametrize"] and ["rstest", "rstest_parametrize"].
  • Added test cases to validate these rstest attribute paths are treated as "test-like".

Docs

  • Minor ADR formatting edits (docs/adr-002-dylint-expect-attribute-macro.md) and expanded comments in driver code describing new matching behaviour and examples.

Why this matters

Previously, path-loaded modules with non-standard names could bypass module-name checks so test contexts (e.g., functions annotated with #[tokio::test] or rstest-generated companions) were not always recognised, causing spurious diagnostics for .expect(...). This change broadens detection to cover common non-standard naming patterns and rstest companion modules while retaining prior recognition of explicit test attributes and cfg(test) modules.

How to test

Run cargo test -p no_expect_outside_tests and the repository UI/regression tests; the added unit, UI and example fixtures exercise the new detection logic.

Related

Fixes #132.

Walkthrough

Extend test-context detection for the no_expect_outside_tests lint: add rstest_parametrize attribute patterns, broaden module-name recognition (test_*, *_test, tests_*, *_tests), and add companion-module detection for rstest; include examples, UI tests and rstest-driven parameterised test cases.

Changes

Cohort / File(s) Summary
Shared attribute registry
common/src/attributes/mod.rs, common/src/attributes/tests.rs
Add ["rstest_parametrize"] and ["rstest","rstest_parametrize"] to TEST_LIKE_PATHS; extend unit tests to cover bare and qualified rstest_parametrize paths.
Test context detection
crates/no_expect_outside_tests/src/driver/mod.rs, crates/no_expect_outside_tests/src/driver/tests.rs
Introduce has_test_module_name to match test_*/*_test/tests_*/*_tests; add has_companion_test_module to detect rstest companion modules; update harness-marking control flow and add unit tests exercising name-patterns and rstest attribute patterns.
Examples
crates/no_expect_outside_tests/examples/pass_expect_in_path_module_harness.rs, crates/no_expect_outside_tests/examples/pass_expect_in_rstest_harness.rs
Add regression examples showing .expect(...) allowed in path-loaded module test harness and in real rstest-annotated tests.
UI test fixtures & runner
crates/no_expect_outside_tests/ui/pass_expect_in_cfg_test_named_module.rs, crates/no_expect_outside_tests/ui/pass_expect_in_path_module_tokio_test.rs, crates/no_expect_outside_tests/ui/pass_expect_in_path_module_tokio_test/service_tests.module, crates/no_expect_outside_tests/src/lib_ui_tests.rs
Add UI fixtures and convert single example test to parameterised #[rstest] with cases for tokio, rstest and path-module examples; update runner invocation and panic messaging per-case.
Documentation
docs/adr-002-dylint-expect-attribute-macro.md
Reflow and rewrap narrative text; no behavioural changes.

Possibly related issues

  • Recognize rstest and rstest_parametrize as test-only code in Whitaker #189 — Matches the PR addition of rstest_parametrize recognition and companion-module handling for rstest-generated code.
  • Issue #132 — Addresses the root cause by expanding attribute and module-name detection so #[tokio::test] and other async test attributes are recognised in path-loaded or non-tests modules.

Poem

🦀 Tests now wear many hats, from rstest trims to tokio flats,
modules named with prefix, suffix, or surprise — the lint now spies.
Companion patterns hum in tune, expect no false alarms at noon.

🚥 Pre-merge checks | ✅ 6 | ❌ 3

❌ Failed checks (3 warnings)

Check name Status Explanation Resolution
Testing ⚠️ Warning Test case at line 152 expects false but implementation correctly returns true; has_test_module_name properly detects the test_ prefix in test_like_utils. Correct the test expectation to true or replace with a genuinely negative example that matches no pattern, then verify all 19 test cases pass.
User-Facing Documentation ⚠️ Warning The pull request introduces new functionality for test context detection via #[path] but fails to document this behaviour change in docs/users-guide.md. Update docs/users-guide.md under the no_expect_outside_tests section with a subsection documenting that modules loaded via #[path] matching test naming conventions are recognised as test contexts.
Developer Documentation ⚠️ Warning Pull request introduces new internal APIs has_test_module_name() and has_companion_test_module() but developers guide documentation lacks explanation of these architectural helpers and broadened test context detection scope. Update the "Async test harness detection" section in docs/developers-guide.md to document both new helper functions, their recognised module naming patterns, rstest companion module detection, integration with existing harness descriptor fallback strategy, and provide clear examples.
✅ Passed checks (6 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarises the core change: broadening the no_expect lint to recognise test contexts in path-loaded modules with non-standard names (tokio, rstest, cfg).
Description check ✅ Passed The description is directly related to the changeset, providing clear rationale, implementation details, test coverage, and links to issue #132.
Linked Issues check ✅ Passed The PR comprehensively addresses #132: implements test module name pattern matching, adds rstest support, expands test-attribute recognition, and includes regression fixtures for path-loaded modules with #[tokio::test].
Out of Scope Changes check ✅ Passed All code changes directly support the stated objectives: core driver logic enhancements, unit tests, UI fixtures, regression examples, and minor documentation formatting are all aligned with issue #132 resolution.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Module-Level Documentation ✅ Passed All modified modules in the PR carry appropriate module-level docstrings that clearly explain their purpose, utility, and function, adhering to Rust documentation standards using //! conventions.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

📋 Issue Planner

Built with CodeRabbit's Coding Plans for faster development and fewer bugs.

View plan used: #132

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-tokio-test-detection-in-path-modules-qgwt7m

Comment @coderabbitai help to get the list of available commands and usage tips.

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai bot commented Apr 7, 2026

Reviewer's Guide

Broadens test-context detection in the no_expect_outside_tests lint (including rstest and #[path]-loaded modules with non-standard names) and adds regression coverage via unit, UI, and example-based tests to ensure Option::expect is only allowed in legitimate test code.

Sequence diagram for linting path-loaded tokio test modules

sequenceDiagram
    actor Developer
    participant RustcTestHarness
    participant NoExpectLintDriver
    participant Module_service_tests

    Developer->>RustcTestHarness: cargo test (with --test)
    RustcTestHarness->>NoExpectLintDriver: Run no_expect_outside_tests lint
    NoExpectLintDriver->>Module_service_tests: Visit path-loaded module service_tests
    NoExpectLintDriver->>NoExpectLintDriver: is_test_named_module("service_tests")
    NoExpectLintDriver->>NoExpectLintDriver: has_test_module_name("service_tests")
    NoExpectLintDriver-->>NoExpectLintDriver: true (matches suffix _tests)
    NoExpectLintDriver->>Module_service_tests: Treat as test context
    Module_service_tests-->>NoExpectLintDriver: #[tokio::test] fn using Option::expect
    NoExpectLintDriver-->>RustcTestHarness: Do not emit no_expect lint warning
    RustcTestHarness-->>Developer: Tests compile and run successfully
Loading

Updated class diagram for no_expect_outside_tests driver and attributes

classDiagram
    class Attributes {
        <<module>>
        +TEST_LIKE_PATHS : &[[&str]]
    }

    class NoExpectDriver {
        <<module>>
        +is_test_named_module(name: &str) bool
        +has_test_module_name(name: &str) bool
        +is_test_context(attrs: Attributes, cfg_flags: CfgFlags, in_test_harness: bool) bool
    }

    class CfgFlags {
        <<struct>>
    }

    NoExpectDriver ..> Attributes : uses
    NoExpectDriver ..> CfgFlags : reads
Loading

Flow diagram for has_test_module_name test-module detection

flowchart TD
    A[Input module name] --> B{name == test or name == tests}
    B -->|yes| G[Return true]
    B -->|no| C{name starts with test_ or tests_}
    C -->|yes| G
    C -->|no| D{name ends with _test or _tests}
    D -->|yes| G
    D -->|no| H[Return false]
Loading

File-Level Changes

Change Details Files
Generalize test-module name detection and add rstest companion-module handling in the driver.
  • Introduce has_test_module_name(name: &str) -> bool to encapsulate test module naming heuristics (exact test/tests, test_/tests_ prefixes, _test/_tests suffixes) and use it from is_test_named_module.
  • Extend collect_harness_marked_test_functions_in_group to treat functions with rstest #[case]-generated companion modules as test-only by detecting a same-named module containing harness const descriptors.
  • Add has_companion_test_module helper to implement the rstest companion-module pattern detection with minimal false positives.
  • Document the interplay between is_test_named_module, has_test_module_name, and higher-level test detection in comments for future maintainers.
crates/no_expect_outside_tests/src/driver/mod.rs
Expand attribute recognition to cover rstest_parametrize and test it.
  • Add rstest_parametrize (bare and qualified) to TEST_LIKE_PATHS so it is treated as a test-like attribute.
  • Extend attribute-path tests to assert rstest_parametrize is recognized as test-like in both bare and qualified forms.
common/src/attributes/mod.rs
common/src/attributes/tests.rs
Add focused unit tests for has_test_module_name and broaden driver tests for test attributes.
  • Add rstest-based parameterized test has_test_module_name_matches_test_conventions to cover exact, prefix, suffix, and negative name patterns, including service_tests and empty string.
  • Extend is_test_attribute_accepts_test_patterns coverage with rstest_parametrize variants.
  • Clarify test-coverage commentary in driver/tests.rs to describe where has_test_module_name is directly tested and how behavioural coverage is ensured via UI/examples.
crates/no_expect_outside_tests/src/driver/tests.rs
Add example-based regression tests for rstest and path-loaded tokio test modules under the --test harness.
  • Introduce pass_expect_in_rstest_harness example using the real rstest crate with #[rstest] and #[case] to validate end-to-end test-context detection.
  • Introduce pass_expect_in_path_module_harness example with a non-standard module name (service_tests) containing #[tokio::test] to validate has_test_module_name under --test.
  • Extend lib_ui_tests.rs with rstest_example_compiles_under_test_harness and path_module_tokio_test_compiles_under_test_harness to run the new examples through the Test::example-based harness, asserting they compile without diffs.
crates/no_expect_outside_tests/src/lib_ui_tests.rs
crates/no_expect_outside_tests/examples/pass_expect_in_rstest_harness.rs
crates/no_expect_outside_tests/examples/pass_expect_in_path_module_harness.rs
Add UI fixtures to ensure no_expect_outside_tests allows expect inside non-standard, cfg(test) and #[path]-loaded test modules with tokio::test.
  • Add pass_expect_in_cfg_test_named_module.rs to cover #[cfg(test)] modules with non-standard names (e.g., check_tests) containing #[tokio::test] where expect should be allowed.
  • Add pass_expect_in_path_module_tokio_test.rs plus its service_tests.module companion file to simulate a #[path]-loaded module with a non-standard name and verify that expect is permitted in tokio tests there.
  • Document these fixtures in driver/tests.rs comments as part of the behavioural coverage for has_test_module_name and related detection logic.
crates/no_expect_outside_tests/ui/pass_expect_in_cfg_test_named_module.rs
crates/no_expect_outside_tests/ui/pass_expect_in_path_module_tokio_test.rs
crates/no_expect_outside_tests/ui/pass_expect_in_path_module_tokio_test/service_tests.module

Assessment against linked issues

Issue Objective Addressed Explanation
#132 Treat functions annotated with async test attributes like #[tokio::test] (including those listed in additional_test_attributes) inside non-tests modules and #[path]-loaded modules as test-only code so that no_expect_outside_tests does not flag .expect(..) calls in those contexts.
#132 Ensure no_expect_outside_tests consistently recognises test contexts in non-standard test modules (e.g., service_tests) compiled under the --test harness, including #[cfg(test)] and #[path]-loaded modules, by improving module-name heuristics and adding regression tests.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

codescene-delta-analysis[bot]

This comment was marked as outdated.

…st context

- Add recognition of rstest's companion test modules generated by #[case] in the test harness.
- Include `rstest_parametrize` attribute for backwards compatibility.
- Add a regression example using the actual rstest crate, demonstrating expect usage in tests.
- Enhance documentation comment to explain the integration with rstest and test detection.
- Enhance unit tests to accept rstest_parametrize attributes.
- Add a UI test to validate end-to-end detection of rstest with #[case] attributes in test context.

Closes: #189

Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
@leynos leynos changed the title Fix no_expect lint for path-loaded tokio tests Broaden no_expect lint for path-loaded test modules (tokio, rstest, cfg) Apr 7, 2026
codescene-delta-analysis[bot]

This comment was marked as outdated.

…rd test modules

Replaced the `#[path]`-loaded external test module `service_tests` with an inline module in the `pass_expect_in_path_module_harness.rs` example. This change preserves testing of the `expect` usage inside non-standard test modules (e.g., named `service_tests`) without `#[cfg(test)]` in HIR.

The refactoring is necessary because the `Test::example()` utility copies only a single `.rs` file to the temp directory and does not preserve subdirectories, so the `#[path]` approach is not compatible.

This maintains the regression coverage for issue #132 related to harness descriptor fallback recognizing test functions in such modules.

Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos leynos marked this pull request as ready for review April 8, 2026 00:04
@leynos leynos linked an issue Apr 8, 2026 that may be closed by this pull request
Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • The has_test_module_name behavior and its tests seem inconsistent: the implementation only checks test_/tests_ prefixes and _test/_tests suffixes, but has_test_module_name_matches_test_conventions expects "test_like_utils" to be true—either adjust the predicate to cover this broader pattern or change the test/example to match the documented naming rules.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `has_test_module_name` behavior and its tests seem inconsistent: the implementation only checks `test_`/`tests_` prefixes and `_test`/`_tests` suffixes, but `has_test_module_name_matches_test_conventions` expects "test_like_utils" to be `true`—either adjust the predicate to cover this broader pattern or change the test/example to match the documented naming rules.

## Individual Comments

### Comment 1
<location path="crates/no_expect_outside_tests/src/driver/tests.rs" line_range="139-148" />
<code_context>
+
+use rstest::rstest;
+
+#[rstest]
+#[case(1)]
+#[case(42)]
</code_context>
<issue_to_address>
**suggestion (testing):** Add more edge-case names around `"test"`/`"tests"` substrings without underscores to lock in the intended `has_test_module_name` behaviour

The new `has_test_module_name_matches_test_conventions` cases cover exact, prefix, and suffix patterns well. To better pin down the boundary and prevent future regressions, consider adding a few negative cases like:

- `"mytest"` / `"mytests"` (no underscore) → `false`
- `"testx"` / `"testsx"``false`
- `"service_tests_helper"` (test-like suffix with extra tail) → `false` if that’s intended

This makes it clear that arbitrary `"test"`/`"tests"` substrings should not be treated as valid test module names.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +139 to +148
#[rstest]
#[case::exact_test("test", true)]
#[case::exact_tests("tests", true)]
#[case::prefix_test_helpers("test_helpers", true)]
#[case::prefix_tests_util("tests_util", true)]
#[case::suffix_service_tests("service_tests", true)]
#[case::suffix_api_test("api_test", true)]
#[case::suffix_integration_tests("integration_tests", true)]
#[case::suffix_unit_test("unit_test", true)]
#[case::plain_service("my_service", false)]
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Add more edge-case names around "test"/"tests" substrings without underscores to lock in the intended has_test_module_name behaviour

The new has_test_module_name_matches_test_conventions cases cover exact, prefix, and suffix patterns well. To better pin down the boundary and prevent future regressions, consider adding a few negative cases like:

  • "mytest" / "mytests" (no underscore) → false
  • "testx" / "testsx"false
  • "service_tests_helper" (test-like suffix with extra tail) → false if that’s intended

This makes it clear that arbitrary "test"/"tests" substrings should not be treated as valid test module names.

@coderabbitai coderabbitai bot added the Issue label Apr 8, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f5fae09ffa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +376 to +378
module.item_ids.iter().any(|id| {
let child = cx.tcx.hir_item(*id);
matches!(child.kind, hir::ItemKind::Const(..))
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restrict companion-module detection to real harness consts

The new has_companion_test_module path treats a function as test-only if a same-named sibling module contains any const item, which is too broad for --test builds. A non-test pair like fn foo() and mod foo { const LIMIT: usize = 3; } now gets foo inserted into harness_marked_test_functions, so .expect(...) inside foo is incorrectly allowed. This should match actual harness descriptors (similar to the stricter identity checks in is_matching_harness_test_descriptor) instead of accepting arbitrary constants.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/no_expect_outside_tests/examples/pass_expect_in_rstest_harness.rs`:
- Line 10: Update the module-level doc comment that mentions rstest_parametrize
by replacing the British spelling "recognises" with the repository standard
"recognizes" (i.e., change the verb in the line containing `rstest_parametrize`
in the module doc comment).

In `@crates/no_expect_outside_tests/src/driver/mod.rs`:
- Around line 355-381: Convert the leading explanatory line comments before
has_companion_test_module into a Rustdoc comment: replace the initial block of
`// rstest with #[case] expands...` and the following two `//` lines with a
`///` Rustdoc paragraph placed immediately above the fn
has_companion_test_module declaration so it matches the style used by
has_test_module_name and is picked up by cargo doc; leave the function signature
and body (references to cx, function_name, siblings, module, etc.) unchanged.

In `@crates/no_expect_outside_tests/src/lib_ui_tests.rs`:
- Around line 31-84: The three tests rstest_example_compiles_under_test_harness,
path_module_tokio_test_compiles_under_test_harness (and the third similar test)
duplicate the same harness logic; refactor by extracting the shared body into a
single parameterised test using #[rstest] and #[case] (e.g. a new function
test_example_compiles_under_test_harness with parameters case(example_name:
&str, case(panic_msg): &str)), call
whitaker::testing::ui::run_with_runner/run_test_runner/Test::example inside that
function using the case values, and replace the per-example panic strings with
formatted text built from the case parameters; remove the original duplicate
functions and ensure rustc_flags(["--test"]) and env_test_guard/with_vars_unset
calls remain unchanged inside the shared body.

In `@crates/no_expect_outside_tests/ui/pass_expect_in_cfg_test_named_module.rs`:
- Around line 13-17: The #[tokio::test] annotated function
tokio_expect_in_named_module is currently synchronous; update its signature to
be asynchronous by changing fn tokio_expect_in_named_module() to async fn
tokio_expect_in_named_module() so the tokio test runner can execute it properly
(leave the #[tokio::test] attribute and the body using option.expect unchanged).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 439ad4ed-cf2d-44c3-b0d5-b692de651f0e

📥 Commits

Reviewing files that changed from the base of the PR and between 502edb6 and f5fae09.

📒 Files selected for processing (12)
  • common/src/attributes/mod.rs
  • common/src/attributes/tests.rs
  • crates/no_expect_outside_tests/examples/pass_expect_in_path_module_harness.rs
  • crates/no_expect_outside_tests/examples/pass_expect_in_rstest_harness.rs
  • crates/no_expect_outside_tests/src/driver/mod.rs
  • crates/no_expect_outside_tests/src/driver/tests.rs
  • crates/no_expect_outside_tests/src/lib_ui_tests.rs
  • crates/no_expect_outside_tests/ui/pass_expect_in_cfg_test_named_module.rs
  • crates/no_expect_outside_tests/ui/pass_expect_in_cfg_test_named_module.stderr
  • crates/no_expect_outside_tests/ui/pass_expect_in_path_module_tokio_test.rs
  • crates/no_expect_outside_tests/ui/pass_expect_in_path_module_tokio_test.stderr
  • crates/no_expect_outside_tests/ui/pass_expect_in_path_module_tokio_test/service_tests.module

Comment on lines +355 to +381
// rstest with #[case] expands into a bare function plus a companion module
// of the same name containing harness const descriptors. Detect this pattern
// so the parent function is treated as test-only code.
fn has_companion_test_module<'tcx>(
cx: &LateContext<'tcx>,
function_name: Symbol,
siblings: &[&'tcx hir::Item<'tcx>],
) -> bool {
siblings.iter().any(|sibling| {
let hir::ItemKind::Mod(_, module) = sibling.kind else {
return false;
};
let Some(mod_ident) = sibling.kind.ident() else {
return false;
};
if mod_ident.name != function_name {
return false;
}
// The module must contain at least one harness const descriptor to
// avoid false positives on non-test modules that happen to share a
// function name.
module.item_ids.iter().any(|id| {
let child = cx.tcx.hir_item(*id);
matches!(child.kind, hir::ItemKind::Const(..))
})
})
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Convert the leading comment to a Rustdoc comment.

The comment at lines 355-357 describes the function's purpose but uses // instead of ///. Use Rustdoc style for consistency with has_test_module_name and to enable cargo doc generation.

Proposed fix
-// rstest with #[case] expands into a bare function plus a companion module
-// of the same name containing harness const descriptors. Detect this pattern
-// so the parent function is treated as test-only code.
+/// Detect rstest companion module pattern.
+///
+/// rstest with `#[case]` expands into a bare function plus a companion module
+/// of the same name containing harness const descriptors. This pattern allows
+/// the parent function to be treated as test-only code.
 fn has_companion_test_module<'tcx>(
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// rstest with #[case] expands into a bare function plus a companion module
// of the same name containing harness const descriptors. Detect this pattern
// so the parent function is treated as test-only code.
fn has_companion_test_module<'tcx>(
cx: &LateContext<'tcx>,
function_name: Symbol,
siblings: &[&'tcx hir::Item<'tcx>],
) -> bool {
siblings.iter().any(|sibling| {
let hir::ItemKind::Mod(_, module) = sibling.kind else {
return false;
};
let Some(mod_ident) = sibling.kind.ident() else {
return false;
};
if mod_ident.name != function_name {
return false;
}
// The module must contain at least one harness const descriptor to
// avoid false positives on non-test modules that happen to share a
// function name.
module.item_ids.iter().any(|id| {
let child = cx.tcx.hir_item(*id);
matches!(child.kind, hir::ItemKind::Const(..))
})
})
}
/// Detect rstest companion module pattern.
///
/// rstest with `#[case]` expands into a bare function plus a companion module
/// of the same name containing harness const descriptors. This pattern allows
/// the parent function to be treated as test-only code.
fn has_companion_test_module<'tcx>(
cx: &LateContext<'tcx>,
function_name: Symbol,
siblings: &[&'tcx hir::Item<'tcx>],
) -> bool {
siblings.iter().any(|sibling| {
let hir::ItemKind::Mod(_, module) = sibling.kind else {
return false;
};
let Some(mod_ident) = sibling.kind.ident() else {
return false;
};
if mod_ident.name != function_name {
return false;
}
// The module must contain at least one harness const descriptor to
// avoid false positives on non-test modules that happen to share a
// function name.
module.item_ids.iter().any(|id| {
let child = cx.tcx.hir_item(*id);
matches!(child.kind, hir::ItemKind::Const(..))
})
})
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/no_expect_outside_tests/src/driver/mod.rs` around lines 355 - 381,
Convert the leading explanatory line comments before has_companion_test_module
into a Rustdoc comment: replace the initial block of `// rstest with #[case]
expands...` and the following two `//` lines with a `///` Rustdoc paragraph
placed immediately above the fn has_companion_test_module declaration so it
matches the style used by has_test_module_name and is picked up by cargo doc;
leave the function signature and body (references to cx, function_name,
siblings, module, etc.) unchanged.

…ion; unify UI example test harnesses

- Refined logic detecting rstest companion test modules to verify matching harness const descriptors.
- Adjusted test case attributes for module name matching.
- Replaced multiple UI test functions with a single parameterized rstest in lib_ui_tests.
- Changed async test function signature in cfg test named module example.
- Minor docs and spelling fixes.

Co-authored-by: devboxerhub[bot] <devboxerhub[bot]@users.noreply.github.com>
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/no_expect_outside_tests/src/driver/mod.rs`:
- Around line 1-7: This file is too large; extract the test-related helpers and
harness collection logic into a new submodule to reduce mod.rs below 400 lines:
move functions has_test_module_name and has_companion_test_module plus the code
that gathers/filters test harnesses (the logic that inspects attributes,
cfg(test) guards, and harness descriptors / Crate::is_doctest handling) into a
new file driver/test_context.rs, expose only the minimal API (e.g., pub fn
has_test_module_name, pub fn has_companion_test_module, and a function to
collect/identify harness contexts) and update mod.rs to use that submodule.
- Around line 233-237: The doc comment starting with "Recognise common test
module naming conventions." should use en-GB-oxendict '-ize' spelling; change
"Recognise" to "Recognize" in that comment block (the doc comment that continues
with "Matches exact names (`test`, `tests`)..."). Update the comment text
accordingly so the header reads "Recognize common test module naming
conventions."

In `@crates/no_expect_outside_tests/src/driver/tests.rs`:
- Line 152: The test case labeled test_embedded_not_suffix is asserting the
wrong expectation: the input "test_like_utils" begins with "test_" so
has_test_module_name(...) will return true; update the case either by changing
the expected boolean from false to true for the case named
test_embedded_not_suffix or replace the input with a true negative (e.g. a
string that does not start with "test_") so the case name and expectation match;
locate the case attribute in crates/no_expect_outside_tests/src/driver/tests.rs
(the #[case::test_embedded_not_suffix("test_like_utils", false)] line) and make
the corresponding change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 882583c6-d6bf-4dd1-89a0-ee63a099baff

📥 Commits

Reviewing files that changed from the base of the PR and between f5fae09 and 30b9d26.

📒 Files selected for processing (6)
  • crates/no_expect_outside_tests/examples/pass_expect_in_rstest_harness.rs
  • crates/no_expect_outside_tests/src/driver/mod.rs
  • crates/no_expect_outside_tests/src/driver/tests.rs
  • crates/no_expect_outside_tests/src/lib_ui_tests.rs
  • crates/no_expect_outside_tests/ui/pass_expect_in_cfg_test_named_module.rs
  • docs/adr-002-dylint-expect-attribute-macro.md

Comment on lines 1 to +7
//! Lint crate forbidding `.expect(..)` outside test and doctest contexts.
//!
//! The lint inspects method calls named `expect`, verifies that the receiver
//! is an `Option` or `Result`, and checks the surrounding traversal context for
//! test-like attributes or `cfg(test)` guards. Doctest harnesses are skipped via
//! `Crate::is_doctest`, ensuring documentation examples remain ergonomic. When
//! no test context is present, the lint emits a denial with a note describing
//! the enclosing function and the receiver type to guide remediation. Teams can
//! extend the recognised test attributes through `dylint.toml` when bespoke
//! macros are in play.
//! The lint inspects method calls named `expect`, verifies the receiver is an
//! `Option` or `Result`, and checks the surrounding context for test-like
//! attributes, `cfg(test)` guards, or harness descriptors. Doctest harnesses
//! are skipped via `Crate::is_doctest`. Teams can extend the recognised test
//! attributes through `dylint.toml` when bespoke macros are in play.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

File exceeds 400-line limit.

At 417 lines, this file breaches the repository guideline of 400 lines maximum. Extract related helper functions (e.g., has_test_module_name, has_companion_test_module, and the harness collection logic) into a dedicated submodule such as driver/test_context.rs.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/no_expect_outside_tests/src/driver/mod.rs` around lines 1 - 7, This
file is too large; extract the test-related helpers and harness collection logic
into a new submodule to reduce mod.rs below 400 lines: move functions
has_test_module_name and has_companion_test_module plus the code that
gathers/filters test harnesses (the logic that inspects attributes, cfg(test)
guards, and harness descriptors / Crate::is_doctest handling) into a new file
driver/test_context.rs, expose only the minimal API (e.g., pub fn
has_test_module_name, pub fn has_companion_test_module, and a function to
collect/identify harness contexts) and update mod.rs to use that submodule.

Comment on lines +233 to +237
/// Recognise common test module naming conventions.
///
/// Matches exact names (`test`, `tests`) as well as modules whose name starts
/// with `test_` or `tests_`, or ends with `_test` or `_tests`. This covers
/// `#[path]`-loaded modules with non-standard names such as `service_tests`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix en-GB-oxendict spelling: use "recognize".

The coding guidelines mandate -ize spelling. Replace "Recognise" with "Recognize" on line 233.

Proposed fix
-/// Recognise common test module naming conventions.
+/// Recognize common test module naming conventions.

As per coding guidelines: "Comments and docs must follow en-GB-oxendict (-ize / -yse / -our) spelling and grammar."

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/// Recognise common test module naming conventions.
///
/// Matches exact names (`test`, `tests`) as well as modules whose name starts
/// with `test_` or `tests_`, or ends with `_test` or `_tests`. This covers
/// `#[path]`-loaded modules with non-standard names such as `service_tests`
/// Recognize common test module naming conventions.
///
/// Matches exact names (`test`, `tests`) as well as modules whose name starts
/// with `test_` or `tests_`, or ends with `_test` or `_tests`. This covers
/// `#[path]`-loaded modules with non-standard names such as `service_tests`
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/no_expect_outside_tests/src/driver/mod.rs` around lines 233 - 237, The
doc comment starting with "Recognise common test module naming conventions."
should use en-GB-oxendict '-ize' spelling; change "Recognise" to "Recognize" in
that comment block (the doc comment that continues with "Matches exact names
(`test`, `tests`)..."). Update the comment text accordingly so the header reads
"Recognize common test module naming conventions."

#[case::testing("testing", false)]
#[case::attest("attest", false)]
#[case::contest("contest", false)]
#[case::test_embedded_not_suffix("test_like_utils", false)]
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Test case expectation is incorrect.

"test_like_utils" starts with "test_", so has_test_module_name returns true. The expected value should be true, not false, or the case name and input should be changed to a genuinely negative example.

Proposed fix (option A: correct the expectation)
-#[case::test_embedded_not_suffix("test_like_utils", false)]
+#[case::test_prefix_with_suffix("test_like_utils", true)]
Proposed fix (option B: use a different negative example)
-#[case::test_embedded_not_suffix("test_like_utils", false)]
+#[case::test_embedded_not_affix("utils_testing_helpers", false)]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
#[case::test_embedded_not_suffix("test_like_utils", false)]
#[case::test_prefix_with_suffix("test_like_utils", true)]
Suggested change
#[case::test_embedded_not_suffix("test_like_utils", false)]
#[case::test_embedded_not_affix("utils_testing_helpers", false)]
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/no_expect_outside_tests/src/driver/tests.rs` at line 152, The test
case labeled test_embedded_not_suffix is asserting the wrong expectation: the
input "test_like_utils" begins with "test_" so has_test_module_name(...) will
return true; update the case either by changing the expected boolean from false
to true for the case named test_embedded_not_suffix or replace the input with a
true negative (e.g. a string that does not start with "test_") so the case name
and expectation match; locate the case attribute in
crates/no_expect_outside_tests/src/driver/tests.rs (the
#[case::test_embedded_not_suffix("test_like_utils", false)] line) and make the
corresponding change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recognize rstest and rstest_parametrize as test-only code in Whitaker no_expect_outside_tests does not recognise #[tokio::test]

1 participant