Skip to content

Retain failing inputs for regression runs #55

Description

@twof

Context

The corpus refactor (Signal/Evaluator/Ledger → the corpus is now purely the scheduler's retained set, materialized at run-end) removed the path that used to persist failing inputs.

Previously the shrinking plugin emitted a .submitToCorpus(.failure) action: it minimized a failing input and wrote it into the corpus as a .failure-typed entry. Because the corpus is persisted and reloaded as seeds, a failing input would automatically replay on the next run (cheap regression).

That path is now gone:

  • submitToCorpus action + SubmitToCorpusAction removed.
  • CorpusEntry.entryType / CorpusEntry.failure, CorpusEntryType, and FailureInfo removed.
  • The corpus is now exclusively the scheduler's retained set (no plugin-submitted entries, no mid-run writers).

The shrinking plugin still minimizes the failing input, biases mutation toward it (selectForMutation), and records the issue — it just no longer persists it.

What's lost

Minimized failing inputs are no longer saved to disk, so they don't auto-replay as seeds on the next run. Failing inputs are still surfaced at runtime via result.failures (raw, un-minimized); they're just not persisted for regression.

What we want

A first-class mechanism to retain failing inputs for regression, decoupled from the coverage corpus:

  • Separate failures store: persist result.failures (and/or shrunk forms) to their own on-disk location, and load them as seeds on the next run. Keeps failure artifacts out of the scheduler's retained-set corpus (the point of the refactor).
  • Raw vs minimized: decide whether regression replays the raw failing input or the shrunk one (shrunk = faster repro, raw = closer to the discovered case).
  • Schedule bytes: failures found under schedule fuzzing need their schedule bytes persisted (element-0 convention) so the interleaving reproduces.

Explicitly not wanted: routing failures back through the scheduler's pool (conflates the mutation pool with failure-artifact storage; a minimized reproducer is a poor mutation parent).

Acceptance

  • A failing input found in run N is deterministically replayed at the start of run N+1 without re-discovering it.
  • Failure retention does not reintroduce coverage/entry-type bookkeeping into the corpus.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions