Skip to content

Fix 1097: Nested Tests#1107

Merged
isc-dchui merged 5 commits intomainfrom
fix-1097
Apr 23, 2026
Merged

Fix 1097: Nested Tests#1107
isc-dchui merged 5 commits intomainfrom
fix-1097

Conversation

@isc-dchui
Copy link
Copy Markdown
Collaborator

@isc-dchui isc-dchui commented Mar 26, 2026

Description

  • Fixes Silent CI failure from nested tests #1097
  • Fixes the suppression of test failures.
    • This stems from nested test/verify calls, e.g. in the scopes integration test, we call scope-test verify -only, so when zpm verify -only is called, we end up nesting.
    • The test resource processor wasn't built to handle nesting due to its use of a single process-private global ^||%UnitTest.Manager.LastResult to track test runs. This is killed at the start of the running of tests, which causes problems with nesting: the parent's results are killed by the child
    • The fix is to instead use a different process-private global ^||%UnitTest.Manager.AllResultsCount that collates individual instantiations.
    • There are two integration test classes that use nested tests: Scopes and FileCopy.
    • Since this bug is dependent on test ordering, i.e. will only occur if a failing test is before a nested test, it didn't occur previously.
    • Note: ^||%UnitTest.Manager.LastResult has been completely removed since IPM will no longer use it, so any custom code built on top of this will break

Testing

Ran zpm verify -only and the tests pass. Also made sure if there is a failing test, the verify phase will correctly fail. For nested tests, made sure that if the nested test fails, it will be properly reported in the summary at the end.

Checklist

  • This branch has the latest changes from the main branch rebased or merged.
  • Changelog entry added.
  • Unit (zpm test -only) and integration tests (zpm verify -only) pass.
  • Style matches the style guide in the contributing guide.
  • Documentation has been/will be updated
    • Source controlled docs, e.g. README.md, should be included in this PR and Wiki changes should be made after this PR is merged (add an extra issue for this if needed)
  • Pull request correctly renders in the "Preview" tab.

Copy link
Copy Markdown
Contributor

@isc-tleavitt isc-tleavitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to discuss this in our next IPM meeting.

I think the effect of this will be that zpm "install" of the same package will upgrade Python dependencies based on semvers in requirements.txt where previously it wouldn't unless you specifically told it to.

Other possible concern - running processes while the install runs. But that's probably an issue any time your upgrading even without explicitly deleting directory chains first - possibly something that could be addressed in a venv-ish solution.

This is tied to the challenges with Python dependency management more generally.

Especially where our target here is fixing CI problems, I'm hesitant to take on a significant user facing change as part of it. But we shouldn't need to solve the whole pseudo-venv / python dependency management problem; maybe this is a reasonable incremental step.

@isc-dchui isc-dchui added this to the v0.10.7 milestone Apr 22, 2026
@isc-dchui isc-dchui requested a review from isc-jlechtne as a code owner April 22, 2026 17:43
@isc-dchui isc-dchui changed the title Fix 1097: Nested Tests and Python Dependency Loading Fix 1097: Nested Tests Apr 22, 2026
@isc-dchui isc-dchui merged commit f522900 into main Apr 23, 2026
16 checks passed
@isc-dchui isc-dchui deleted the fix-1097 branch April 23, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Silent CI failure from nested tests

2 participants