fix(ledger): a position the user never sold keeps its cycle across a new declaration (closes #539, closes #536)

docs/prd-ledger.md

@@ -38,7 +38,17 @@ A fourth engine-assigned shape records a disappearance the user confirmed as a s
 
 Like `adjustment`, it stays out of holding derivation: the accompanying new anchor is what moves the book, and applying both would double-count. Its purpose is to be *readable* — `revisit.absence_exits` turns it into a real exit row (`kind: "full"`, `exit_price: null`), so the confirmation the user gave is not a written-never-read field (#429's failure class). The shares, currency, market and cost basis that row carries are copied from the recorded book as it stood immediately before the row was appended, never inferred from a price. Positions the user keeps because the new view simply did not capture them are carried into the new anchor with `"carried": true` on the position row; the flag is written only when true, because every snapshot payload is content-addressed and an unconditional `false` would rewrite every existing `snapshot_id`.
 
-The mirror-image answer rides the same anchor row (#531). A position that *appeared* in the declaration is confirmed too, and the user's rough answer to "how long have you held this" becomes `"since"` plus `"since_basis"` on that position — the conversion from months to a date happens in `book_refresh._months_before`, never in the agent. The two keys are one fact and `snapshot_adapter` validates them as a pair: `since` cannot exist without the `since_basis: "user_estimate"` stamp beside it, which is how "a reconstructed start date is never rendered as an exact date" holds at the storage layer rather than as a rule every renderer has to remember. `since_basis: "unknown"` carries no date at all and `ledger.derive_holdings` turns it into the two-segment `ticker#unknown` cycle that already exists for an undatable open. Like `carried`, both are written only when they apply, for the same content-addressing reason; and like `carried`, neither enters `portfolio_basis._normalized_anchor`, so two declarations of the same book still share one `state_version` regardless of how each row's start was learned. A later refresh copies an existing stamp forward onto the same position (`book_refresh._carry_forward_provenance`) so the answer is not spent on one review — but only while the record still traces that position to its anchor, because a position sold and bought back is a different cycle the ledger can date itself.
+The mirror-image answer rides the same anchor row (#531). A position that *appeared* in the declaration is confirmed too, and the user's rough answer to "how long have you held this" becomes `"since"` plus `"since_basis"` on that position — the conversion from months to a date happens in `book_refresh._months_before`, never in the agent. The two keys are one fact and `snapshot_adapter` validates them as a pair: `since` cannot exist without a `since_basis` stamp beside it saying where that date came from, which is how "a reconstructed start date is never rendered as an exact date" holds at the storage layer rather than as a rule every renderer has to remember. `since_basis: "unknown"` carries no date at all and `ledger.derive_holdings` turns it into the two-segment `ticker#unknown` cycle that already exists for an undatable open. Like `carried`, both are written only when they apply, for the same content-addressing reason; and like `carried`, neither enters `portfolio_basis._normalized_anchor`, so two declarations of the same book still share one `state_version` regardless of how each row's start was learned.
+
+Two further bases, `"trade_event"` and `"snapshot_anchor"`, are what make a position survive its own book being re-declared (#539/#536). A declaration says what is held and never since when, so an adopted position whose start it did not restate falls back to that declaration's `as_of` — reminting `cycle_id` and orphaning every memory keyed to it, the user's thesis first. `book_refresh.carry_recorded_starts` stamps each still-held position with the start the record already holds, and it is the single primitive both adoption lanes reach: `build_adoption` for the refresh lane, `review._carried_declaration` for the review lane, which is also the only path through which engine-assigned provenance may enter a book at all.
+
+**What is carried is the basis, not a marker saying the date was carried** (owner ruling, 2026-07-31). A single "this came from the record" value would describe how the date was transported and destroy why it is believed, collapsing an exact start the ledger watched open together with a lower bound inferred from the day a declaration first listed the position — and the loss would be permanent, because after an adoption `origin` describes the new snapshot writer rather than the original evidence. So `since_basis` is an evidence vocabulary and it travels unchanged: a position the record still traces to its anchor keeps that anchor's own stamp, whatever it says, and only a position the anchor does not describe — one carried for the first time, or one whose current cycle the ledger opened since — is labelled fresh, `trade_event` when the ledger watched it open and `snapshot_anchor` when a declaration first put it on the books. That branch order is load-bearing rather than incidental: an adopted trade-proven position reads `origin == "snapshot"` like every other row, so a basis recomputed from origin would demote it on the very next declaration.
+
+Reading the start off the *derived* book is what keeps a sold-and-rebought position honest without a further rule: its derived start *is* the rebuy, so there is no previous cycle to inherit from and no estimate can follow the user's words onto a cycle they were not about. Two guards on writing at all: a start later than the declaration's own `as_of` means the declaration predates the record, which the lane owning that refusal states plainly, and a `snapshot_anchor` start equal to `as_of` restates the default and would change the anchor's content address without changing a fact. The equality guard stops there — for the other bases the stamp carries something the default does not.
+
+No surface consumes the distinction yet; #532 owns making provenance visible and actionable. What this change owes is that the distinction is still *there* to consume, through any number of adoptions and a projection repair.
+
+`"cycle_seq"` rides with it, and only with it. A `cycle_id` is `ticker#since#seq`, so a position flattened and bought back on one day differs from the cycle it replaced in the sequence alone — carrying the date without it would give the live position the sold cycle's exact identity, and with it that cycle's thesis, standing conditions and closed status. `snapshot_adapter` refuses a sequence with no dated start beside it, and refuses one on `since_basis: "unknown"` because a two-segment `ticker#unknown` cycle has no sequence to state. Like `carried`, it is written only past the first cycle, for the same content-addressing reason. `ledger.cycle_sequence` is the single inverse of the composition in `derive_holdings`, and `_anchored_cycle_seq` degrades a bad value to 1 with a named `integrity` issue rather than refusing the whole book, the contract `_anchored_cycle_start` already keeps.
 
 `ledger.append_events` can stamp a `recorded_at` date alongside the schema version `v` (#472): *when this system learned the fact*, as distinct from the event's own `date`/`as_of` (*when the thing happened*). A trade imported weeks late still carries its true trade date, but `recorded_at` marks the later day it actually entered the ledger — the field a future rule backtest needs to replay history without look-ahead bias, since without it a late import reads, to the replay, as if it had been known on its trade date.
 

skills/fomo-kernel/engine/book_refresh.py

@@ -493,39 +493,135 @@ def _appearance_stamp(row, answer, as_of):
     return row
 
 
-def _carry_forward_provenance(row, recorded, derived):
-    """Keep an already-answered cycle start attached across later refreshes.
-
-    Without this the stamp survives exactly one refresh: the next declaration is
-    an ordinary envelope with no provenance on it, so the position's start would
-    snap back to the new anchor's date and the user's answer would have bought
-    exactly one review. The user is not asked again, and should not be — the
-    position is not appearing, it is simply still held.
-
-    Two gates, and both are about not carrying a stamp onto a different cycle.
-    The stamp is read from the last declaration's own row (the only place that
-    says a start was answered rather than assumed; a ``trades_derived``
-    restatement never stamps one, so reading the newest recorded row instead
-    would spend the user's answer the first time they imported a CSV). And it is applied only while
-    the record still traces the position back to that anchor
-    (``origin == "snapshot"``): a position sold and bought back reads
-    ``origin == "trades"``, and its real recorded open date must win over a
-    stamp describing the cycle before it.
+def _carried_start(row, recorded, derived, as_of):
+    """Attach the cycle start the record already holds for one position (#539).
+
+    A declaration says what is held, never since when. So an adopted position
+    whose start this declaration did not restate must be given the start the
+    book already had, or ``ledger._anchored_cycle_start`` falls back to the new
+    anchor's own date, ``cycle_id`` remints, and every memory keyed to that
+    cycle — the user's thesis first — reads as belonging to nothing.
+
+    What is carried is the *basis*, not a marker saying the date was carried
+    (owner ruling, 2026-07-31). A single "this came from the record" stamp would
+    describe how the date was transported and destroy why it is believed — an
+    exact start the ledger watched open and a lower bound inferred from the day
+    a declaration first listed the position would become the same value, and
+    nothing could tell them apart afterwards. Not even ``origin``: once the book
+    is adopted, that describes the new snapshot writer rather than the original
+    evidence. So the four bases are the vocabulary and they travel unchanged.
+
+    The cases, in the order they are decided.
+
+    A row that already carries a stamp is left alone: it came from
+    ``_appearance_stamp`` a few lines up, which is the position the user was
+    just asked about, and this declaration *is* its first statement.
+
+    A position the record still traces to its anchor keeps that anchor's own
+    stamp, whatever it says. The stamp is read from the last declaration's own
+    row — the only place that states a start rather than assuming one; a
+    ``trades_derived`` restatement never stamps one, so reading the newest
+    recorded row instead would spend the user's #531 answer the first time they
+    imported a CSV. This is also the branch that keeps a trade-proven start
+    trade-proven: once adopted, a position opened by a real buy sits in the
+    anchor and reads ``origin == "snapshot"`` like any other, so a basis
+    recomputed from origin here would quietly demote it to a lower bound on the
+    second declaration.
+
+    Everything else still held takes the start the *derived* book states, with
+    the basis that book's own evidence supports: ``trade_event`` for a cycle the
+    ledger watched open, ``snapshot_anchor`` for one a declaration first put on
+    the books. Reading it from the derived book is what makes this safe without
+    a further gate — the derived start of a rebought position is its rebuy date,
+    so there is no previous cycle to inherit from, and no old estimate can
+    follow the user's words onto a cycle they were not about. It is also a
+    fixpoint: carrying it again yields the same date and the same basis, so
+    replay, ``repair-projections`` and an idempotent finalize retry are stable
+    by construction.
+
+    A ticker absent from the derived book is untouched. It is appearing, and
+    #531 owns what happens to it.
+
+    Two guards on writing at all. A start later than this declaration's own
+    ``as_of`` means the declaration predates the record, which is refused a few
+    steps on by the lane that owns that refusal; carrying one here would replace
+    a plain "older than the current ledger anchor" with a validation error about
+    a field the user never supplied. And a ``snapshot_anchor`` start equal to
+    ``as_of`` restates exactly what ``_anchored_cycle_start`` already defaults
+    to, so writing it would change the anchor's content address without changing
+    a fact — which is what an idempotent replay of the same declaration is
+    checked by. The equality guard is deliberately not extended to the other
+    bases: for them the stamp carries something the default does not.
     """
-    ticker = row.get("ticker")
-    stamp = (recorded.get(ticker) or {}).get("since_basis")
-    if stamp not in lg.SINCE_BASES:
+    if row.get("since_basis"):
         return row
-    if (derived.get(ticker) or {}).get("origin") != "snapshot":
+    ticker = row.get("ticker")
+    held = derived.get(ticker) or {}
+    if not held:
         return row
     row = dict(row)
-    row["since_basis"] = stamp
     row.pop("since", None)
-    if stamp == "user_estimate":
-        row["since"] = recorded[ticker].get("since")
+    stamp = (recorded.get(ticker) or {}).get("since_basis")
+    if stamp in lg.SINCE_BASES and held.get("origin") == "snapshot":
+        if stamp == "unknown":
+            # No date to guard: the cycle is `ticker#unknown` either way, and
+            # that identity is already stable across declarations.
+            row["since_basis"] = "unknown"
+            return row
+        since, basis = recorded[ticker].get("since"), stamp
+    else:
+        since = held.get("since")
+        basis = ("trade_event" if held.get("origin") == "trades"
+                 else "snapshot_anchor")
+    # The date and the basis are decided together, deliberately: a start read off
+    # the derived book labelled `user_estimate` would attribute a ledger fact to
+    # the user, which is the same dishonesty as the reverse.
+    if not since or str(since) > str(as_of):
+        return row
+    if str(since) == str(as_of) and basis == "snapshot_anchor":
+        return row
+    row["since"], row["since_basis"] = since, basis
+    # The start alone is not the identity. Two cycles opened on one day differ
+    # only here, so carrying the date without the sequence would give the live
+    # position the id of the one that was sold (#539).
+    seq = lg.cycle_sequence((held.get("cycle_id") or ""))
+    if seq and seq > 1:
+        row["cycle_seq"] = seq
     return row
 
 
+def carry_recorded_starts(envelope, events, *, splits=None, today=None):
+    """Validate one adopted book with every continuously-held start carried (#539).
+
+    The single primitive both adoption lanes call, and the single place engine
+    provenance may enter a book. The refresh lane calls it on the envelope it
+    assembled from its own answers; the review lane calls it in ``cmd_prepare``
+    on the declaration it is about to build a plan from, before
+    ``snapshot_adapter.prepare`` derives anything, so the plan's cycle ids and
+    the ledger anchor come from one stamped object rather than two derivations.
+
+    It is one function rather than one call per lane because the composition is
+    the part that drifts: ``derived`` and ``recorded`` are assembled here, so a
+    caller cannot supply a book read through a different window (#444's rule —
+    a single reader is still two readers when its input is assembled twice).
+
+    Returns ``normalize_book``'s ``(snapshot, anchor)``. The envelope may be a
+    raw one or a previously normalized one; only the declared fields are read.
+    """
+    positions = list(envelope.get("positions") or [])
+    if positions:
+        derived = lg.holdings_as_of(events or [], envelope["as_of"], splits=splits)
+        recorded = _declared_map(lg.latest_anchor(events or [], declared_only=True) or {})
+        positions = [_carried_start(row, recorded, derived, envelope["as_of"])
+                     for row in positions]
+    stamped = {key: value for key, value in envelope.items()
+               if key in snapshot_adapter.ENVELOPE_KEYS and value is not None}
+    stamped["positions"] = positions
+    return snapshot_adapter.normalize_book(
+        stamped, today=today or dt.date.fromisoformat(envelope["as_of"]),
+        allow_engine_provenance=True)
+
+
 def _carried_row(ticker, fact):
     """One position carried forward from the record, copied and never invented."""
     row = {"ticker": ticker, "shares": fact.get("shares"),
@@ -552,10 +648,10 @@ def build_adoption(receipt, events, snapshot, anchor, answers, *, today=None, sp
     carried rows merged back in, the difference the ledger records must be the
     difference that actually happened.
 
-    This is also the only place ``allow_engine_provenance`` is turned on. The
-    envelope assembled here is not a caller's file: every ``since`` in it was
-    computed a few lines above by ``_appearance_stamp`` or copied off the
-    recorded anchor by ``_carry_forward_provenance``.
+    Validation and engine provenance both run through ``carry_recorded_starts``,
+    shared with the review lane: the envelope assembled here is not a caller's
+    file, and every ``since`` in the adopted book was either computed a few lines
+    above by ``_appearance_stamp`` or carried off the record by that primitive.
     """
     classifications = _validated_answers(receipt.get("pending_confirmations") or [], answers)
     resupply = sorted(ticker for ticker, value in classifications.items()
@@ -565,7 +661,6 @@ def build_adoption(receipt, events, snapshot, anchor, answers, *, today=None, sp
                 "tickers": resupply}
 
     derived = lg.holdings_as_of(events, snapshot["as_of"], splits=splits)
-    recorded = _declared_map(lg.latest_anchor(events, declared_only=True) or {})
     by_ticker = {row["ticker"]: row for row in receipt.get("pending_confirmations") or []}
     carried, sold, appeared = [], [], {}
     for ticker, answer in sorted(classifications.items()):
@@ -579,26 +674,24 @@ def build_adoption(receipt, events, snapshot, anchor, answers, *, today=None, sp
             fact = derived.get(ticker)
             if not fact:
                 raise RefreshError(f"{ticker} is no longer in the recorded book to carry forward")
-            carried.append(_carry_forward_provenance(
-                _carried_row(ticker, fact), recorded, derived))
+            carried.append(_carried_row(ticker, fact))
         elif answer["classification"] == "sold":
             sold.append(ticker)
 
     supplied = []
     for row in anchor.get("positions") or []:
         row = _supplied_row(row)
         answer = appeared.get(row.get("ticker"))
-        supplied.append(_appearance_stamp(row, answer, snapshot["as_of"]) if answer
-                        else _carry_forward_provenance(row, recorded, derived))
+        supplied.append(_appearance_stamp(row, answer, snapshot["as_of"]) if answer else row)
     envelope = {"as_of": snapshot["as_of"], "positions": supplied + carried}
     if snapshot.get("cash") is not None:
         envelope["cash"] = snapshot["cash"]
     if snapshot.get("fx"):
         envelope["fx"] = snapshot["fx"]
     try:
-        _adopted_snapshot, adopted_anchor = snapshot_adapter.normalize_book(
-            envelope, today=today or dt.date.fromisoformat(snapshot["as_of"]),
-            allow_engine_provenance=True)
+        _adopted_snapshot, adopted_anchor = carry_recorded_starts(
+            envelope, events, splits=splits,
+            today=today or dt.date.fromisoformat(snapshot["as_of"]))
     except snapshot_adapter.SnapshotError as exc:
         raise RefreshError(f"the adopted book failed validation: {exc}") from exc