Skip to content

bench(lab): record the chain from an observation to a change - #277

Closed
luuuc wants to merge 1 commit into
lab/06-01-the-transcript-minerfrom
lab/06-02-findings-and-candidates
Closed

bench(lab): record the chain from an observation to a change#277
luuuc wants to merge 1 commit into
lab/06-01-the-transcript-minerfrom
lab/06-02-findings-and-candidates

Conversation

@luuuc

@luuuc luuuc commented Aug 16, 2026

Copy link
Copy Markdown
Owner

Problem

The miner produces observations. Between an observation and a shipped product change there are three steps that currently happen in someone's head and in prose: deciding an observation is a finding, forming a hypothesis about which surface is at fault, and proposing a change.

Unrecorded, that chain has two failure modes and the retired tree shows both. Findings evaporate: one campaign's miner output sits in a log file, its findings directory is empty, and the findings that did get written up live as three loose markdown files under a different campaign, connected to nothing. And the chain cannot be audited backwards: "why was this change made to Sense" should be answerable from the change, through its candidate, to its finding, to the transcripts, and today it is answerable only by someone who remembers.

Summary

lab/internal/record keeps three record types with ids pointing at each other, on disk, as files.

Changes

  • A finding cannot be recorded without a surface, because the route from a number to a fix runs through one, or without evidence, because a finding citing nothing is an opinion.
  • A candidate cannot be recorded without a finding that is on disk. A change with no observed problem behind it is taste, and a finding id naming nothing is the same thing with a reference attached. A finding may exist without a candidate, and most will — that is the honest state of most observations, and a schema pressuring every finding into a candidate would manufacture work.
  • A validation cannot predate its candidate's hypothesis. A hypothesis written afterwards is a description, and the whole point of writing it first is that it can be wrong. An unreadable hypothesis time is a refusal, not a pass.
  • A validation accepts or it rejects, and it gives a reason. There is no third state, and the corpus size travels with the regression result: a pass over four cells and a pass over forty are different claims.
  • Evidence is a path into the run tree, never a copy. A finding that embedded a transcript excerpt would drift from the transcript the moment either was touched.
  • A finding inherits the state of its evidence. Invalidating a run marks every finding that cites it. Nothing is removed from the evidence list — a finding that dropped its dead runs would read as weakly supported rather than as one whose support was withdrawn — and a finding whose support is entirely withdrawn is marked rather than deleted, because it was once observed.
  • The chain is queryable from either end, and from the middle: from a change back to the transcripts, and from a finding forward to whether anything came of it.
  • The miner's output lands without hand-editing, and re-mining the same corpus updates rather than duplicates, because ids are derived from what a record is about.
  • It is a record, not a tracker. No states, no assignment, no triage, no priority. And no database: the package doc names the condition under which SQLite would be justified rather than pretending it never will be.

Test Plan

  • make ci green: build, tests, per-file coverage floor with no new exception, zero complexity suppressions, lint clean.
  • 96% line and 100% function coverage on the new package.
  • Invalidation is checked through the findings on disk, not through a return value, with three outcomes in one setup: one finding partly invalidated and standing, one fully invalidated and marked dead but present, one citing a different run and untouched. So "marks every finding" cannot be satisfied by marking all of them.
  • Invalidating the same run twice changes nothing the second time — the append-twice bug would kill a finding that still has live evidence.
  • Every refusal has its own case: no surface, no evidence, no finding, a finding id naming nothing, no hypothesis, a validation before its hypothesis, an unreadable hypothesis time, no decision, a third decision, no reason, no candidate.
  • The chain traced from the validation, from the finding and from the candidate.
  • A finding nobody acted on traces cleanly, because most findings never become a candidate.
  • Mining twice leaves one finding, updated.
  • A refused batch leaves nothing half-written; an unwritable directory, an unreadable record and a missing finding are all errors rather than silent gaps.

The miner produces observations. Between one and a product change there
are three steps that otherwise happen in someone's head and in prose, and
the retired tree shows both failure modes: one campaign's miner output
sits in a log file with an empty findings directory beside it, and the
findings that were written up live as three loose markdown files under a
different campaign, connected to nothing. "Why was this change made to
Sense" is answerable only by someone who remembers.

Three records, each pointing at the last. A finding cannot be recorded
without a surface or without evidence. A candidate cannot be recorded
without a finding that is actually on disk, because a change with no
observed problem behind it is taste. A validation cannot be recorded
before its candidate wrote its hypothesis down, because a hypothesis
written afterwards is a description.

Evidence is a path into the run tree, never a copy: a finding that
embedded a transcript excerpt would drift from the transcript the moment
either was touched. And a finding inherits the state of its evidence.
Invalidating a run marks every finding that cites it, nothing is removed
from the evidence list, and a finding whose support is entirely withdrawn
is marked rather than deleted, because it was once observed.

Ids are derived from what a record is about, so re-mining the same corpus
updates rather than duplicates. It is a record and not a tracker: no
states, no assignment, no triage, no priority.
@luuuc
luuuc deleted the branch lab/06-01-the-transcript-miner August 17, 2026 07:18
@luuuc luuuc closed this Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant