Skip to content

fix: test suite clobbers live ~/.pyauto-heart state #78

Description

@Jammy2211

Overview

Running PyAutoHeart's own unit tests overwrites the developer's live ~/.pyauto-heart state: tests/test_test_run.py isolates the input (results_dir=tmp_path) but heart/checks/test_run.py resolves HEART_STATE_DIR at import and writes test_run.json unconditionally inside run(). Measured 2026-07-15: a 10,234-byte real state file was replaced by a 251-byte fixture, wiping the recorded failures and the 58-entry parked_stale list — recoverable only by luck. Heart's state dir is the input to the release gate, so this is a release-safety issue. Phase 0 of the build-chain integrity campaign (epic: see PyAutoBuild umbrella issue).

Plan

  • Remove the hidden global write: run(results_dir=...) returns the summary; only the CLI/tick entrypoint persists it (delete the trap, not just the one test).
  • Sweep heart/checks/ empirically for sibling module-level-resolved writes reachable from tests.
  • Add an autouse conftest.py fixture sandboxing HEART_STATE_DIR for the whole suite (belt-and-braces; deliberate decision per the prompt).
  • Verify: full suite with HEART_STATE_DIR unset against a throwaway HOME; diff the state dir before/after — measured, not argued.
Detailed implementation plan

Work Classification

Library (organism repo: PyAutoHeart)

Affected Repositories

  • PyAutoHeart (primary, only)

Branch Survey (2026-07-16)

Repository Current Branch Dirty?
./PyAutoHeart main clean

Suggested branch: feature/heart-state-clobber
Worktree root: ~/Code/PyAutoLabs-wt/heart-state-clobber/ (created by /start_library)

Implementation Steps

  1. heart/checks/test_run.py — move the state.atomic_write_json(HEART_STATE_DIR / "test_run.json", summary) call (currently ~line 231, unconditional inside run()) out of run(); run() returns the summary dict only. The CLI/tick entrypoint that invokes the check performs the persist.
  2. Audit every module in heart/checks/ for the same shape: HEART_STATE_DIR (or any global path) resolved at import + written from library code that tests call directly. Empirical proof: run each test file with HEART_STATE_DIR pointed at a sandbox and list what appears there.
  3. tests/conftest.py — autouse fixture setting HEART_STATE_DIR to tmp_path-derived sandbox for the whole suite (note: env var is read at import in test_run.py:36, so the fixture must account for import-time resolution — monkeypatching the module attribute, not just the env var, or moving resolution to call time as part of step 1).
  4. Update tests/test_test_run.py for the new run() contract; add a regression test asserting no test writes outside the sandbox.
  5. Verification (from the prompt's constraints): run the full suite with HEART_STATE_DIR unset against a throwaway HOME, diff the state dir before/after — must be byte-identical.

Key Files

  • heart/checks/test_run.pyHEART_STATE_DIR resolved at import (:36); unconditional write in run() (:231)
  • tests/test_test_run.py — calls tr.run(results_dir=tmp_path) at lines 48, 61, 69, 101, 110
  • tests/conftest.py — autouse sandbox fixture (new)
  • heart/state.pyatomic_write_json helper; reads of test_run.json

Constraints

  • Heart state dir is release-gate input: verify by measurement, not argument.
  • Do not run the suite against live ~/.pyauto-heart until the fix is in (that is how the bug was found).

Original Prompt

Click to expand starting prompt

PyAutoHeart's test suite writes into the user's live ~/.pyauto-heart state

Type: bug
Target: PyAutoHeart
Repos:

  • PyAutoHeart
    Difficulty: small
    Autonomy: supervised
    Priority: high
    Status: formalised

Running PyAutoHeart's own unit tests overwrites the developer's live Heart state.
tests/test_test_run.py calls tr.run(results_dir=tmp_path) (lines 48, 61, 69, 101, 110) and
never isolates HEART_STATE_DIR. heart/checks/test_run.py:36 resolves HEART_STATE_DIR at
import time (env or ~/.pyauto-heart default), and :231 unconditionally does
state.atomic_write_json(HEART_STATE_DIR / "test_run.json", summary). So tmp_path isolates the
input (results_dir) but not the output: the summary lands in the real state dir.

Measured 2026-07-15 (on main, not a feature branch). Running pytest tests/test_test_run.py
replaced a 10,234-byte real test_run.json with this 251-byte fixture — the last test to run wins:

{"cloud_url": "U", "failed": 0, "parked_stale": [], "parked_stale_count": 0,
 "passed": 0, "per_project": {}, "ready": true, "run_label": "cloud#9",
 "skipped": 0, "source": "cloud", "timeout": 0, "ts": "2026-06-25T00:00:00Z"}

Proven by pointing HEART_STATE_DIR at a sandbox and re-running that file alone: the same stub
appears there. With the variable unset — the normal case for anyone running pytest in this repo —
the target is ~/.pyauto-heart/.

Impact, measured not reasoned. Two of the three legs currently holding Heart at YELLOW read
from test_run. Re-aggregating with the stub in place:

before:  yellow score 60  — "workspace validation not passing (3 failed, 2026-07-09T09-48-30Z)"
                          — "58 stale parked script(s)"
after:   stale  score 70  — "test run stale (20d old)"

Both real reasons disappear, replaced by a bogus one. It is not a fake-GREEN today — the
fixture's hardcoded ts (2026-06-25) reads as 20d old, past TEST_STALE_DAYS=10, so the verdict
degrades to STALE and still blocks the GREEN-gated nightly. That is luck, not design: a fixture
with a fresh ts would clear the leg outright. Do not treat "it only goes STALE" as the safety
property.

The real damage is evidence destruction. The stub wipes the 3 recorded failures and the entire
58-entry parked_stale list — which is exactly the hygiene-triage input for the other open YELLOW
leg. On 2026-07-15 this was recoverable only because ~/.pyauto-heart/state.json still held a
pre-clobber copy of the block (restored from it, back to a byte-identical 10,234 bytes). Had a
tick re-aggregated between the pytest run and the restore, state.json would have been
overwritten too and the data would be gone permanently.

Fix direction (design first — do not assume)

The obvious monkeypatch.setenv("HEART_STATE_DIR", tmp_path) in the test only fixes this test
file. Prefer removing the trap over documenting it:

  • Is the write in tr.run() the actual defect? A run(results_dir=...) that also performs a
    hidden write to a global path is doing two jobs. Consider having run() return the summary and
    letting the CLI entrypoint persist it — then the tests cannot pollute anything because there is
    nothing to pollute.
  • Sweep for siblings. heart/checks/ resolves HEART_STATE_DIR at import in several modules;
    find every check whose test path can reach a module-level-resolved write. test_run was found by
    accident, so assume it is not alone. git status-style proof: run each test file with
    HEART_STATE_DIR sandboxed and list what appears.
  • An autouse fixture in tests/conftest.py pointing HEART_STATE_DIR at tmp_path for the
    whole suite is a cheap belt-and-braces backstop, but it hides the design smell rather than fixing
    it — decide deliberately whether to do both.

Constraints

  • Heart's state dir is the input to the release gate. Anything that can silently rewrite it
    with passing values is a release-safety issue, not a tidiness issue — hence Priority: high.
  • Verify any fix by running the full suite with HEART_STATE_DIR unset against a throwaway
    HOME, and diffing the state dir before/after. "No test writes it" must be measured, not argued.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions