From 87aa91c0cca6735b4ade8956673ceceb5ec7df62 Mon Sep 17 00:00:00 2001 From: Sebastian Mendel Date: Wed, 22 Jul 2026 12:47:53 +0200 Subject: [PATCH] docs(tests): document the Windows skip convention for shell-based tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI runs the suite on ubuntu, macos and windows-latest, while everything under scripts/ is POSIX Bash — so any test that invokes bash, sources a scripts/lib/*.sh helper, or runs an installer fails on the Windows leg. Thirteen test files already use the skip_on_windows marker, but the convention was undocumented and is invisible locally: the suite is green on Linux and only the Windows CI leg turns red. Came from /retro: yes Signed-off-by: Sebastian Mendel --- tests/AGENTS.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/tests/AGENTS.md b/tests/AGENTS.md index c7ae230..f13ae63 100644 --- a/tests/AGENTS.md +++ b/tests/AGENTS.md @@ -118,6 +118,30 @@ uv run python -m pytest -m unit - Test functions: `test_` - Use descriptive names: `test_config_invalid_timeout_too_low` +**Tests that shell out MUST be skipped on Windows.** CI runs the suite on +`ubuntu-latest`, `macos-latest` **and `windows-latest`**, while everything under +`scripts/` is POSIX Bash. Any test that invokes `bash`, sources a `scripts/lib/*.sh` +helper, or executes an installer fails on the Windows leg. Use the established +module-level marker (13 test files already do): + +```python +import sys +import pytest + +skip_on_windows = pytest.mark.skipif( + sys.platform == "win32", reason="Shell script tests require POSIX shell" +) + +@skip_on_windows +class TestSomethingThatRunsBash: + ... +``` + +Apply it to the shell-invoking class only — pure-Python assertions (catalog JSON +structure, parsing, schema invariants) stay platform-independent and must keep +running everywhere. Forgetting this is not caught locally: the suite is green on +Linux and only the Windows CI leg turns red. + **Test structure (Arrange-Act-Assert):** ```python def test_tool_config_defaults():