Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 33 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,13 @@ jobs:
- name: Install dependencies
run: uv sync --all-extras

- name: Run tests with coverage
if: github.ref_name != 'dev' && github.base_ref != 'dev'
run: uv run pytest --include-slow --cov-report=xml
- name: Run tests with coverage (Python 3.12)
if: matrix.python-version == '3.12' && github.ref_name != 'dev' && github.base_ref != 'dev'
run: uv run pytest --cov-report=xml

- name: Run compatibility tests (Python 3.11)
if: matrix.python-version == '3.11' && github.ref_name != 'dev' && github.base_ref != 'dev'
run: uv run pytest --no-cov

- name: Run fast unit tests (dev)
if: github.ref_name == 'dev' || github.base_ref == 'dev'
Expand All @@ -48,6 +52,32 @@ jobs:
file: ./coverage.xml
fail_ci_if_error: false

slow-comprehensive:
name: Slow comprehensive tests
runs-on: ubuntu-latest
timeout-minutes: 20
if: github.ref_name != 'dev' && github.base_ref != 'dev'

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v4
with:
enable-cache: true

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install dependencies
run: uv sync --all-extras

- name: Run slow comprehensive tests without coverage
run: uv run pytest --include-slow -m slow --no-cov --durations=20

lint:
name: Lint
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ When a phase is fully complete, collapse its tasks in `TODO.md` to a 2-3 line su

**Testing:**
- pytest with pytest-cov, pytest-asyncio, pytest-mock, pytest-benchmark
- Separate test markers: `@pytest.mark.slow` for large dataset tests (not run by default)
- Separate test markers: `@pytest.mark.slow` for large dataset/workload tests (not run by default). Run slow tests with `--no-cov` unless you are specifically profiling coverage behavior, because coverage instrumentation makes the generator workload much slower.
- Target coverage: 95%+ overall, 95%+ for core generation engine

**Format Support:**
Expand Down
21 changes: 12 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,14 @@ We expect new pull requests to include tests for any affected behavior, and, as
we follow semantic versioning, we may reserve breaking changes until the next
major version release.

Before submitting, run the full test suite (including slow tests) and confirm
all tests pass:
Before submitting, run the normal coverage-gated suite, the slow comprehensive
suite without coverage instrumentation, and lint/format checks:

```bash
uv run pytest --include-slow
uv run pytest
uv run pytest --include-slow -m slow --no-cov --durations=20
uv run ruff check .
uv run ruff format --check .
```

### Commit Messages
Expand Down Expand Up @@ -93,18 +96,18 @@ uv sync
uv run pytest

# Lint and format
uv run ruff check src/ tests/
uv run ruff format src/ tests/
uv run ruff check .
uv run ruff format --check .
```

### Test Markers

- `@pytest.mark.slow`: large dataset tests (100+ users), skipped by default
- `@pytest.mark.slow`: large dataset and workload tests, skipped by default and normally
run without coverage instrumentation

```bash
uv run pytest # Quick run (skips slow tests)
uv run pytest --include-slow # Full run (all tests, required before PRs)
uv run pytest -m slow # Only slow tests
uv run pytest # Normal coverage-gated run
uv run pytest --include-slow -m slow --no-cov --durations=20 # Slow comprehensive run
```

## Code Style
Expand Down
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,12 +244,15 @@ uv sync
# Run tests (1400+ tests)
uv run pytest

# Run slow comprehensive workload tests without coverage instrumentation
uv run pytest --include-slow -m slow --no-cov --durations=20

# Run specific test suite
uv run pytest tests/unit/test_network_visibility.py -v

# Lint and format
uv run ruff check src/ tests/
uv run ruff format src/ tests/
uv run ruff check .
uv run ruff format --check .
```

### Tech Stack
Expand Down
1 change: 1 addition & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ Replaced manual per-emitter field coordination with SecurityEvent intermediate r

## Pre-MVP: Consolidated Quality Fixes — IN PROGRESS

- [x] Split slow comprehensive tests from coverage instrumentation in CI and update contributor/agent testing guidance — normal coverage gate passed at 79.38% with slow tests skipped; slow comprehensive suite passed separately with `--no-cov` in 2m36s; Ruff checks passed.
- [x] Prepare `dev` → `main` PR — inspected `main..dev`, applied the required v0.6.2 version/changelog bump on `dev`, ran release checks, pushed, and opened the PR into `main`.
- [x] Retargeted and merged PR fixes #138-#141 into `dev`, then reworked PR #137 against current `dev` so Windows event spool hardening preserves all current emitter fixup passes before merge.
- [x] Remediate Windows singleton PID path traversal telemetry suppression — canonicalize Windows singleton paths with ntpath before seeded PID reuse and cover traversal variants with a unit test.
Expand Down
1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,6 @@ addopts = [
"--tb=short",
"--cov=evidenceforge",
"--cov-report=term-missing",
"--cov-report=html",
]
filterwarnings = [
"error",
Expand Down
Loading