Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions docs/testing/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Testing Docs

Strongclaw keeps contributor-facing testing documentation in this directory.
This is the canonical source of truth for how tests are organized, authored,
and run inside the repository.

Testing docs do not ship in `src/clawops/assets/...` unless a packaged runtime
flow actually needs them.

## Start Here

- Read [authoring.md](authoring.md) when you are writing new tests, moving
existing coverage, or touching the pytest framework.
- Read [operations.md](operations.md) when you are running tests locally,
debugging CI, or checking governance commands.
- Read [../../tests/fixtures/README.md](../../tests/fixtures/README.md) for
fixture-package-specific guidance only.

## Canonical Rules

- Put tests under `tests/suites/{unit,integration,contracts,e2e,...}` and place
them in a subsystem-specific subdirectory when one exists.
- Treat `TestContext` as the default path for patching, environment mutation,
and temporary working-directory changes.
- Keep pytest framework rules and contributor docs together under
`docs/testing/` instead of scattering them across packaged asset trees.
93 changes: 93 additions & 0 deletions docs/testing/authoring.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Authoring Tests

## Lane Model

Strongclaw uses four primary default pytest lanes:

- `unit`: isolated behavior and small-surface regression checks
- `integration`: cross-module or service-shaped behavior
- `contracts`: repository policies, docs parity, and CI/test-governance rules
- `e2e`: black-box CLI and workflow-shaped orchestration coverage

The repository also maintains an explicit `framework` lane for pytest-framework
self-checks. Framework tests live under
`tests/suites/contracts/testing/framework/` and are excluded from default runs.
Use that lane only for pytest bootstrap and plugin-registration behavior.

Monkeypatch governance is a default contract, not a framework-only self-check.
The direct-monkeypatch contract lives under `tests/suites/contracts/testing/`
so ordinary pytest runs fail when new unmanaged `monkeypatch` usage appears in
suite code.

Capability markers are additive and remain module-local:

- `hypermemory`
- `qdrant`
- `network_local`

Structural markers are assigned from the suite path layout in
`tests/conftest.py`.

## Placement Rules

Add a new test in the suite that matches its behavior:

- `tests/suites/unit/...` for isolated behavior
- `tests/suites/integration/...` for cross-module or service-backed behavior
- `tests/suites/contracts/...` for repository and governance rules
- `tests/suites/contracts/testing/framework/...` for explicit pytest-framework
self-checks
- `tests/suites/e2e/...` for black-box CLI and workflow-shaped coverage

Prefer a dedicated subsystem directory inside each lane when one already
exists.

## Runtime Boundaries

Use `tests/plugins/infrastructure/` for structural test runtime behavior such
as `TestContext`, environment management, patch management, profile
registration, and framework-owned pytest hooks.

Use `tests/fixtures/` for domain-facing pytest fixture plugins only.

Use `tests/utils/helpers/` for builders, subsystem runtimes, AST tooling, and
other reusable support code that is not itself a fixture or pytest framework
surface.

Tests consume fixtures by name through pytest injection and should not import
from `tests.fixtures`.

## Preferred Authoring Path

Environment mutation, working-directory changes, and patching should flow
through the infrastructure runtime:

- use `test_context.patch.patch(...)` or `patch_object(...)`
- use `test_context.env.set(...)`, `remove(...)`, `update(...)`, or
`prepend_path(...)`
- use `test_context.chdir(...)` for temporary working-directory changes

Do not add new ordinary-suite tests that depend on raw `monkeypatch` unless the
test is an explicit governed exception.

## Service Resolution

Service-backed tests resolve mode with this precedence:

1. CLI: `--mock <service>`
2. Environment: `<SERVICE>_TEST_MODE`
3. Marker kwargs: `@pytest.mark.<service>(mode="real")`
4. Default mode

Current service support:

- `qdrant`

## Governance

Framework policy lives under `tests/suites/contracts/testing/`.
Add a contract test when a rule must stay true even if the implementation
changes.

Pytest-framework registration and bootstrap topology lives under
`tests/suites/contracts/testing/framework/`.
42 changes: 42 additions & 0 deletions docs/testing/operations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Testing Operations

## Canonical Commands

- Full suite: `uv run pytest -q`
- Unit lane: `uv run pytest -q -m unit`
- Integration lane: `uv run pytest -q -m integration`
- Contract lane: `uv run pytest -q -m contract`
- Framework lane only:
`uv run pytest -q -m framework tests/suites/contracts/testing/framework`
- E2E lane: `uv run pytest -q -m e2e`
- Hypermemory lane: `uv run pytest -q -m hypermemory`
- Qdrant lane: `uv run pytest -q -m "hypermemory and qdrant"`

## Service Mode Commands

- Force Qdrant mock mode:
`uv run pytest -q -m "hypermemory and qdrant" --mock qdrant`
- Force Qdrant real mode:
`QDRANT_TEST_MODE=real uv run pytest -q -m "hypermemory and qdrant"`
- Provide an existing real endpoint:
`TEST_QDRANT_URL=http://127.0.0.1:6333 uv run pytest -q -m "hypermemory and qdrant"`
- Override the managed Qdrant image:
`TEST_QDRANT_IMAGE=ghcr.io/example/qdrant:test uv run pytest -q -m "hypermemory and qdrant"`

## Governance Checks

- Testing contracts: `uv run pytest -q tests/suites/contracts/testing`
- Fixture analysis:
`uv run python -m tests.utils.scripts.analyze_fixtures --json`
- Safe timeout wrapper:
`uv run python -m tests.utils.scripts.pytest_safe --timeout 600 -q -m integration`

## Common Triage

- If the monkeypatch governance contract fails, migrate the test to
`TestContext` instead of expanding the allowlist by default.
- If docs or links move, rerun the repository docs contracts so relative-link
and layout drift is caught early.
- If a test needs reusable setup by fixture injection, add or extend a fixture.
If it needs reusable non-fixture logic, move that code into
`tests/utils/helpers/`.
111 changes: 0 additions & 111 deletions platform/docs/TESTING_FRAMEWORK.md

This file was deleted.

46 changes: 0 additions & 46 deletions platform/docs/TESTING_OPERATIONS.md

This file was deleted.

Loading
Loading