bench(lab): record the chain from an observation to a change - #277
Closed
luuuc wants to merge 1 commit into
Closed
Conversation
The miner produces observations. Between one and a product change there are three steps that otherwise happen in someone's head and in prose, and the retired tree shows both failure modes: one campaign's miner output sits in a log file with an empty findings directory beside it, and the findings that were written up live as three loose markdown files under a different campaign, connected to nothing. "Why was this change made to Sense" is answerable only by someone who remembers. Three records, each pointing at the last. A finding cannot be recorded without a surface or without evidence. A candidate cannot be recorded without a finding that is actually on disk, because a change with no observed problem behind it is taste. A validation cannot be recorded before its candidate wrote its hypothesis down, because a hypothesis written afterwards is a description. Evidence is a path into the run tree, never a copy: a finding that embedded a transcript excerpt would drift from the transcript the moment either was touched. And a finding inherits the state of its evidence. Invalidating a run marks every finding that cites it, nothing is removed from the evidence list, and a finding whose support is entirely withdrawn is marked rather than deleted, because it was once observed. Ids are derived from what a record is about, so re-mining the same corpus updates rather than duplicates. It is a record and not a tracker: no states, no assignment, no triage, no priority.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The miner produces observations. Between an observation and a shipped product change there are three steps that currently happen in someone's head and in prose: deciding an observation is a finding, forming a hypothesis about which surface is at fault, and proposing a change.
Unrecorded, that chain has two failure modes and the retired tree shows both. Findings evaporate: one campaign's miner output sits in a log file, its findings directory is empty, and the findings that did get written up live as three loose markdown files under a different campaign, connected to nothing. And the chain cannot be audited backwards: "why was this change made to Sense" should be answerable from the change, through its candidate, to its finding, to the transcripts, and today it is answerable only by someone who remembers.
Summary
lab/internal/recordkeeps three record types with ids pointing at each other, on disk, as files.Changes
Test Plan
make cigreen: build, tests, per-file coverage floor with no new exception, zero complexity suppressions, lint clean.