diff --git a/.github/workflows/commercial-research-beta.yml b/.github/workflows/commercial-research-beta.yml new file mode 100644 index 000000000..262076961 --- /dev/null +++ b/.github/workflows/commercial-research-beta.yml @@ -0,0 +1,52 @@ +name: Commercial Research Beta + +on: + pull_request: + branches: [main] + +permissions: + contents: read + +jobs: + local-engineering-gate: + runs-on: ubuntu-latest + timeout-minutes: 30 + env: + PYTHONDONTWRITEBYTECODE: "1" + PR_BASE_SHA: ${{ github.event.pull_request.base.sha }} + PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }} + steps: + - name: Check out pull request revision + uses: actions/checkout@v6 + with: + fetch-depth: 0 + ref: ${{ github.event.pull_request.head.sha }} + + - name: Set up Python + uses: actions/setup-python@v6 + with: + python-version: "3.12" + + - name: Install dependencies + run: python3 -m pip install -e '.[dev]' + + - name: Install Playwright Chromium + run: python3 -m playwright install --with-deps chromium + + - name: Run full test suite + run: make test + + - name: Smoke-test dashboard startup + run: make dashboard-smoke + + - name: Render Personal Research routes + run: make research-dashboard-render-smoke + + - name: Check public wording + run: make public-wording-check + + - name: Check generated-artifact hygiene + run: make pr-range-hygiene-check BASE_SHA="$PR_BASE_SHA" HEAD_SHA="$PR_HEAD_SHA" + + - name: Check whitespace + run: git diff --check "$PR_BASE_SHA...$PR_HEAD_SHA" diff --git a/Makefile b/Makefile index bd74fc9e5..e9e479036 100644 --- a/Makefile +++ b/Makefile @@ -1,10 +1,20 @@ -.PHONY: help help-full next-stage profile-context research-change-snapshot research-change-monitor research-review-queue research-event-review-record demo demo-data-build demo-data-check demo-dashboard demo-dashboard-smoke demo-dashboard-render-smoke local-profile-seed linkedin-share-check public-performance-contract public-performance-gate public-ux-review-checklist public-ux-review-checklist-json public-ux-review-notes public-ux-review-notes-check public-ux-review-note pilot-review-feedback pilot-feedback-closeout browser-qa-evidence browser-qa-capture-plan hosted-demo-readiness pilot-readiness-check pilot-readiness-packet pilot-share-brief trusted-data-pilot trusted-data-pilot-candidates trusted-data-pilot-packet trusted-data-pilot-lane trusted-data-pilot-board trusted-data-pilot-evidence reviewed-data-proof reviewed-data-proof-record reviewed-batch-proof reviewed-batch-proof-record reviewed-batch-compare reviewed-batch-preflight auto-refresh-plan auto-refresh-daily auto-refresh-weekly auto-refresh-optional auto-refresh-runbook auto-refresh-status scheduler-activation-checklist auto-apply-gate lane-outcome-history price-reviewed-run fundamentals-batch-proof peer-batch-proof peer-mapping-source-review peer-mapping-writeback-guard public-demo-readiness-pack readiness-ops-center readiness-queue data-coverage-proof-queues coverage-frontier data-coverage-planner coverage-expansion-loop readiness-ops-evidence reviewed-batch decision-proof-queue metric-readiness metric-readiness-board benchmark-risk-review diff-hygiene diff-hygiene-summary diff-hygiene-files data-release-decision public-release-package public-release-handoff license-status session-source-preflight source-activation-guide provider-setup-checklist provider-smoke fmp-smoke alpha-vantage-smoke finnhub-smoke universe-scope fundamentals-source-ladder fundamentals-source-ladder-queue optional-context-source-ladder optional-context-source-ladder-queue fmp-stage alpha-vantage-stage finnhub-stage yfinance-stage sec-filing-share-stage staged-hygiene-check public-wording-check public-check status status-check test pipeline stock-report stock-report-md local-tickers monthly track-record validate-data data-sources-check data-sources research-health research-health-check risk-context action-queue action-queue-check project-status project-status-check verify validate-all daily dashboard dashboard-smoke sec-stage sec-validate sec-preview sec-apply imports-validate imports-preview imports-apply import-staging universe-preview universe-preview-summary universe-stage universe-apply universe-refresh universe-report universe-active coverage data-wizard unlock-ladder unlock-summary command-bundles command-bundle-details command-bundle-runbook bundle-prices bundle-fundamentals bundle-peers bundle-prices-broader bundle-fundamentals-broader bundle-peers-broader detail-prices detail-fundamentals detail-peers detail-prices-broader detail-fundamentals-broader detail-peers-broader runbook-prices runbook-fundamentals runbook-peers runbook-prices-broader runbook-fundamentals-broader runbook-peers-broader focus-price focus-fundamentals focus-peers onboarding templates price-status price-worklist fundamentals-peer-worklist optional-context-worklist sec-stage-queue peer-mapping-queue dcf-input-proof-queue dcf-input-proof-handoff dcf-input-source-review dcf-input-source-command-plan dcf-input-source-guard share-count-proof-queue price-history-proof-queue price-validate price-preview price-apply price-refresh price-refresh-loop price-normalize import-prices price-coverage dcf-readiness import-fundamentals optional-context-summary optional-context-readiness import-earnings import-analyst-estimates readiness readiness-snapshot research-decisions earnings-nowcast-pilot earnings-nowcast-walkthrough earnings-nowcast-templates earnings-nowcast-validate earnings-nowcast-preview earnings-nowcast-readiness earnings-nowcast-prospective-plan +.PHONY: help help-full next-stage profile-context research-change-snapshot research-change-monitor research-review-queue research-event-review-record demo demo-data-build demo-data-check demo-dashboard demo-dashboard-smoke demo-dashboard-render-smoke local-profile-seed linkedin-share-check public-performance-contract public-performance-gate public-ux-review-checklist public-ux-review-checklist-json public-ux-review-notes public-ux-review-notes-check public-ux-review-note pilot-review-feedback pilot-feedback-closeout browser-qa-evidence browser-qa-capture-plan hosted-demo-readiness pilot-readiness-check pilot-readiness-packet pilot-share-brief trusted-data-pilot trusted-data-pilot-candidates trusted-data-pilot-packet trusted-data-pilot-lane trusted-data-pilot-board trusted-data-pilot-evidence reviewed-data-proof reviewed-data-proof-record reviewed-batch-proof reviewed-batch-proof-record reviewed-batch-compare reviewed-batch-preflight proof-readiness-reconciliation auto-refresh-plan auto-refresh-daily auto-refresh-weekly auto-refresh-optional auto-refresh-runbook auto-refresh-status scheduler-activation-checklist auto-apply-gate lane-outcome-history price-reviewed-run fundamentals-batch-proof peer-batch-proof peer-mapping-source-review peer-mapping-writeback-guard public-demo-readiness-pack readiness-ops-center readiness-queue data-coverage-proof-queues coverage-frontier data-coverage-planner coverage-expansion-loop readiness-ops-evidence reviewed-batch decision-proof-queue metric-readiness metric-readiness-board benchmark-risk-review diff-hygiene diff-hygiene-summary diff-hygiene-files data-release-decision public-release-package public-release-handoff license-status session-source-preflight source-activation-guide provider-setup-checklist provider-smoke fmp-smoke alpha-vantage-smoke finnhub-smoke universe-scope fundamentals-source-ladder fundamentals-source-ladder-queue optional-context-source-ladder optional-context-source-ladder-queue fmp-stage alpha-vantage-stage finnhub-stage yfinance-stage sec-filing-share-stage staged-hygiene-check public-wording-check public-check status status-check test pipeline stock-report stock-report-md local-tickers monthly track-record validate-data data-sources-check data-sources research-health research-health-check risk-context action-queue action-queue-check project-status project-status-check verify validate-all daily dashboard dashboard-smoke sec-stage sec-validate sec-preview sec-apply imports-validate imports-preview imports-apply import-staging universe-preview universe-preview-summary universe-stage universe-apply universe-refresh universe-report universe-active coverage data-wizard unlock-ladder unlock-summary command-bundles command-bundle-details command-bundle-runbook bundle-prices bundle-fundamentals bundle-peers bundle-prices-broader bundle-fundamentals-broader bundle-peers-broader detail-prices detail-fundamentals detail-peers detail-prices-broader detail-fundamentals-broader detail-peers-broader runbook-prices runbook-fundamentals runbook-peers runbook-prices-broader runbook-fundamentals-broader runbook-peers-broader focus-price focus-fundamentals focus-peers onboarding templates price-status price-worklist fundamentals-peer-worklist optional-context-worklist sec-stage-queue peer-mapping-queue dcf-input-proof-queue dcf-input-proof-handoff dcf-input-source-review dcf-input-source-command-plan dcf-input-source-guard share-count-proof-queue price-history-proof-queue price-validate price-preview price-apply price-refresh price-refresh-loop price-normalize import-prices price-coverage dcf-readiness import-fundamentals optional-context-summary optional-context-readiness import-earnings import-analyst-estimates readiness readiness-preview readiness-release-review readiness-release-record readiness-release-guard readiness-materialize readiness-snapshot research-decisions earnings-nowcast-pilot earnings-nowcast-walkthrough earnings-nowcast-templates earnings-nowcast-validate earnings-nowcast-preview earnings-nowcast-readiness earnings-nowcast-prospective-plan .PHONY: thesis-journal thesis-journal-preview thesis-journal-record +.PHONY: prospective-field-proof-status prospective-field-proof-audit prospective-field-proof-preview prospective-field-proof-record +.PHONY: refresh-operations-status refresh-operations-runbook +.PHONY: earnings-nowcast-sec-actuals-stage +.PHONY: earnings-nowcast-cohort-readiness earnings-consensus-source-status earnings-consensus-source-review earnings-consensus-collection-plan earnings-consensus-collection-status earnings-consensus-collection-preview earnings-consensus-collection-record research-outcome-review research-outcome-review-preview research-outcome-review-record catalyst-evidence-timeline catalyst-evidence-preview catalyst-evidence-record +EARNINGS_NOWCAST_COHORT ?= NVDA,AMD,AVGO,MU,QCOM +.PHONY: commercial-source-rights commercial-beta-check commercial-beta-release-check +.PHONY: private-beta-readiness +.PHONY: point-in-time-universe-status point-in-time-universe-preview +.PHONY: calibration-evidence-bundle-preview DEFAULT_TRUSTED_PILOT_TICKERS := MU,CRDO,HOOD,TSLA,META,A,APLD DEFAULT_TRUSTED_PILOT_EVIDENCE_TICKERS := MU,CRDO DEFAULT_METRIC_BENCHMARKS := SPY,QQQ +NO_WRITE_GUARD = PYTHONDONTWRITEBYTECODE=1 python3 -m src.no_write_artifact_guard --project-root . -- help: @echo "Stock Research Command Center" @@ -16,10 +26,14 @@ help: @echo " make project-status-check Read current coverage, blockers, and executable next steps" @echo " make provider-setup-checklist Review optional key setup without exposing secrets" @echo " make hosted-demo-readiness Check hosted-app readiness without deploying" - @echo " make pilot-review-feedback Print the controlled 5-10 reviewer feedback capture guide" + @echo " make commercial-beta-check Run the read-only commercial-beta contract checks" + @echo " make point-in-time-universe-status MANIFEST= [REGISTRY=] Validate one immutable universe package" + @echo " make point-in-time-universe-preview MANIFEST= [REGISTRY=] [TOP_N=20] Preview one immutable universe package" + @echo " make pilot-review-feedback Print the controlled 10-20 reviewer feedback capture guide" @echo " make pilot-feedback-closeout Print the reviewer feedback closeout and fix/defer checklist" @echo " make scheduler-activation-checklist Print the safe scheduler activation checklist" @echo " make public-check / public-release-handoff Verify sharing and terminal steps" + @echo " make company-workbench-html-browser-check Verify offline research-brief bytes in a real browser" @echo "" @echo "Useful next paths:" @echo " Review one stock: make stock-report-md TICKER=NVDA" @@ -29,7 +43,7 @@ help: @echo "" @echo "For the full local command catalog, run: make help-full" -.PHONY: dashboard-render-smoke +.PHONY: dashboard-render-smoke research-dashboard-render-smoke commercial-beta-performance-contract commercial-beta-performance-gate research-accessibility-browser-check company-workbench-html-browser-check .PHONY: price-history-batch-closeout next-stage: @@ -88,6 +102,8 @@ help-full: @echo " Print one lane group's ordered proof steps and evidence summary" @echo " make reviewed-data-proof" @echo " make reviewed-batch-proof" + @echo " make proof-readiness-reconciliation TOP_N=20" + @echo " Compare historical proof outcomes with current saved readiness without writing files" @echo " make reviewed-batch-compare" @echo " make reviewed-batch-preflight" @echo " Print the durable reviewed data proof ledger" @@ -121,6 +137,12 @@ help-full: @echo " Print the copy-ready terminal handoff for verify, stage, commit, and push" @echo " make license-status" @echo " Print the read-only license/reuse gate before public sharing" + @echo " make point-in-time-universe-status MANIFEST= [REGISTRY=]" + @echo " Print deterministic validation truth for one immutable universe package" + @echo " make point-in-time-universe-preview MANIFEST= [REGISTRY=] [TOP_N=20]" + @echo " Preview reproduction digests and capped canonical exclusions without writing" + @echo " make calibration-evidence-bundle-preview BUNDLE=" + @echo " Preview aggregate calibration evidence consistency without writing" @echo " make session-source-preflight [SEC_USER_AGENT='Name email@example.com']" @echo " Check one session's SEC/yfinance/local-fundamentals path before retrying source-backed coverage work" @echo " make source-activation-guide" @@ -152,7 +174,7 @@ help-full: @echo " make public-ux-review-note" @echo " Record one local UX review note row without staging or refreshing data" @echo " make pilot-review-feedback" - @echo " Print the controlled 5-10 reviewer feedback capture guide" + @echo " Print the controlled 10-20 reviewer feedback capture guide" @echo " make pilot-feedback-closeout" @echo " Print the reviewer feedback closeout and fix/defer checklist" @echo " make readiness-ops-center" @@ -197,13 +219,16 @@ help-full: @echo " make lane-outcome-history [LEDGER=data/reviewed_data_proofs.csv] Print lane outcome history from reviewed proof rows" @echo " make reviewed-data-proof-record LANE= PROOF_ID= PROOF_DATE= FINAL_OUTCOME= Record an intentional reviewed proof row" @echo " make reviewed-batch-proof [LEDGER=data/reviewed_batch_proofs.csv] Print durable reviewed batch proof rows" + @echo " make proof-readiness-reconciliation [TOP_N=20] [TICKERS=ARCT] [JSON=1] Compare historical proof with current saved readiness" @echo " make reviewed-batch-proof-record BATCH_ID= LANE= REVIEW_DATE= FINAL_OUTCOME= Record a reviewed or auto-gated batch outcome" - @echo " make reviewed-batch-compare [BATCH_ID=] [LANE=prices] [REVIEW_DATE=] Compare prior/current readiness snapshots for proof-ledger fields" @echo " make reviewed-batch-preflight [LANE=prices] [TOP_N=100] [MAX_CANDIDATES=3500] Check snapshot, dry-run, compare, proof, and artifact gates" @echo " make auto-refresh-plan Print scheduler-ready source-backed auto-refresh lanes and auto gates" @echo " make auto-refresh-daily Print daily price, SEC filing/share-count, and fundamentals refresh commands" @echo " make auto-refresh-runbook Print compact scheduler runbook for SCHEDULE=daily|weekly|optional|all" @echo " make auto-refresh-status Print source activation, setup gaps, next command, and runbook pointer" + @echo " make refresh-operations-status Print read-only provider/retry planning status" + @echo " make refresh-operations-runbook Print the read-only refresh operations runbook" + @echo " make private-beta-readiness Print the read-only nine-area private-beta readiness contract" @echo " make scheduler-activation-checklist Print the safe scheduler activation checklist" @echo " make auto-apply-gate Evaluate deterministic auto-apply gate fields for one staged slice" @echo " make fundamentals-batch-proof [DRY_RUN=1] [TOP_N=10] [TICKERS=NVDA,MSFT] Preview or write the SEC/manual fundamentals proof packet with validate, preview, rejected-row, compare, and proof-record gates" @@ -212,7 +237,7 @@ help-full: @echo " make price-history-batch-closeout [TOP_N=10] [TICKERS=AIAI,AMAN] Print a read-only grouped still-blocked proof-record scaffold" @echo " make peer-batch-proof [DRY_RUN=1] [TOP_N=10] [TICKERS=NVDA,MSFT] Preview or write the peer mapping and mapped-peer valuation-input proof packet without inferring peers" @echo " make peer-mapping-source-review [DRY_RUN=1] [TOP_N=10] [TICKERS=NVDA,MSFT] Preview or write a fillable source-review packet before editing data/imports/peers.csv" - @echo " make peer-mapping-writeback-guard TICKER= PEER_TICKER= PEER_GROUP= SOURCE= AS_OF_DATE= REVIEWER= REVIEW_DATE= Preview one reviewed peer import row; blocks duplicates, self-peers, placeholders, and stale readiness" + @echo " make peer-mapping-writeback-guard TICKER= PEER_TICKER= PEER_GROUP= PEER_ROLE= COMPARABILITY_BASIS= VALUATION_ANCHOR_ELIGIBLE= SOURCE= AS_OF_DATE= REVIEWER= REVIEW_DATE= Preview one reviewed peer import row; blocks incomplete evidence, duplicates, self-peers, placeholders, and stale readiness" @echo " make price-reviewed-run [MAX_CANDIDATES=3500] [TOP_N=100] [PROVIDER=auto] Print reviewed capped price-run execution, diff, and rollback plan" @echo " make public-demo-readiness-pack Print the small shareable public demo proof set" @echo " make readiness-ops-center Print lane-level ready/partial/blocked/excluded operations without refreshing data" @@ -229,6 +254,7 @@ help-full: @echo " make metric-readiness-board [TICKERS=NVDA,META] [TOP_N=10] [BENCHMARKS=SPY,QQQ] [OUTPUT=outputs/metric_readiness_board.csv] Print or optionally export the combined benchmark metric-readiness board" @echo " make diff-hygiene Print a read-only staging guide that separates product files from local data changes" @echo " make diff-hygiene-summary Print a short read-only staging summary for public checks" + @echo " make pr-range-hygiene-check BASE_SHA= HEAD_SHA= Inspect committed PR paths for generated churn" @echo " make diff-hygiene-files Write local pathspec files under outputs/staging for safer reviewed staging" @echo " make data-release-decision Print read-only post-batch keep-local, reviewed-data-release, and cleanup guidance" @echo " make public-release-package Print read-only product staging, generated exclusion, final checks, commit, and push guidance" @@ -302,8 +328,16 @@ help-full: @echo " make templates Write local CSV templates for peers, earnings, estimates, and manual fallbacks" @echo " make import-staging Write header-only staging CSV files under data/imports" @echo " make validate-data Validate local CSV datasets" - @echo " make readiness-snapshot Save current ticker readiness as data/reports/ticker_readiness_report.previous.csv before a refresh" - @echo " make readiness Write central data/reports/ticker_readiness_report.csv" + @echo "" + @echo "Advanced readiness boundaries:" + @echo " make readiness-preview [TOP_N=20] In-memory preview: stable readiness impact, change causes, and promotion evidence without writing files" + @echo " make readiness-release-review [TOP_N=20] [JSON=1] Review the exact default readiness candidate without writing" + @echo " make readiness-release-record PREVIEW_RECEIPT= REVIEWER= REVIEW_DATE= TECHNICAL_DECISION= DISTRIBUTION_DECISION= CONFIRM_REVIEWED=1 Record one exact named review" + @echo " make readiness-release-guard RECORD_ID= Revalidate one record and print exact named staging paths" + @echo " make readiness-snapshot PROFILE= Required profile; write one profile-specific prior snapshot from in-memory readiness" + @echo " make reviewed-batch-compare PROFILE= [BATCH_ID=] [LANE=prices] [REVIEW_DATE=] Compare a profile-bound prior snapshot; current readiness is composed in memory and no current report is written; Required profile" + @echo " make readiness Deprecated no-write guard; exits 2" + @echo " CONFIRM_MATERIALIZE=1 make readiness-materialize PROFILE= Confirmed ignored local materialization; write one ignored full readiness package" @echo "" @echo "Price fallback:" @echo " make price-refresh-loop DRY_RUN=1 Preview the scalable capped refresh plan without changing local CSV files" @@ -316,7 +350,8 @@ help-full: @echo " make price-coverage Write data/price_coverage_report.csv with rows per universe ticker" @echo " Start with make status, then the printed price check or guided batch" @echo " make price-normalize INPUT=data/raw/prices/NVDA.csv TICKER=NVDA SOURCE=yahoo_manual" - @echo " make price-validate && make price-preview && make price-apply" + @echo " make price-normalize INPUT=data/raw/prices/NVDA.csv TICKER=NVDA SOURCE= SOURCE_REF= RETRIEVED_AT= AS_OF=" + @echo " make price-validate AS_OF= && make price-preview AS_OF= && make price-apply AS_OF=" @echo "" @echo "Preview-first fundamentals and universe imports:" @echo " export SEC_USER_AGENT='Name email@example.com'" @@ -425,6 +460,18 @@ public-performance-contract: public-performance-gate: @python3 -m src.public_performance_gate --browser --warm-runs $(or $(WARM_RUNS),5) --cold-runs $(or $(COLD_RUNS),1) --timeout-seconds $(or $(TIMEOUT_SECONDS),30) $(if $(BASE_URL),--base-url "$(BASE_URL)",) $(if $(CHROME),--chrome "$(CHROME)",) +commercial-beta-performance-contract: + @python3 -m src.public_performance_gate --workflow research --contract --root . + +commercial-beta-performance-gate: + @python3 -m src.public_performance_gate --workflow research --browser --root . --warm-runs $(or $(WARM_RUNS),5) --cold-runs $(or $(COLD_RUNS),1) --timeout-seconds $(or $(TIMEOUT_SECONDS),30) --output "$(or $(OUTPUT),/tmp/stock-command-center-commercial-beta-performance.json)" $(if $(BASE_URL),--base-url "$(BASE_URL)",) $(if $(CHROME),--chrome "$(CHROME)",) + +research-accessibility-browser-check: + @python3 -m src.research_accessibility_browser_gate --root . --timeout-seconds $(or $(TIMEOUT_SECONDS),45) $(if $(BASE_URL),--base-url "$(BASE_URL)",) $(if $(CHROME),--chrome "$(CHROME)",) + +company-workbench-html-browser-check: + @PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_company_workbench_html_browser_gate.py -q + public-ux-review-checklist: @python3 -m src.public_ux_review_checklist @@ -442,7 +489,7 @@ public-ux-review-note: pilot-review-feedback: @echo "Controlled Pilot Review Feedback" - @echo "Read-only: this target prints the 5-10 external reviewers feedback capture guide." + @echo "Read-only: this target prints the 10-20 external reviewers feedback capture guide; all counted sessions must be independent target-persona reviews." @echo "It does not refresh data, import rows, stage files, commit, push, or deploy." @echo "Research-only: pilot feedback is product clarity evidence, not data proof, source proof, data freshness proof, investment advice, broker integration, or a trade instruction." @echo "" @@ -451,15 +498,20 @@ pilot-review-feedback: @echo "Local-only working copy: cp docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv /tmp/stock-command-center-pilot-feedback.csv" @echo "" @echo "Reviewer path:" - @echo " Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History" + @echo " Research Desk -> Discover -> Company Workbench -> Monitor" @echo "" - @echo "Ask four questions:" + @echo "Ask the complete task questions:" @echo " 1. Where did you start?" @echo " 2. What could you use now?" @echo " 3. What was blocked or excluded?" - @echo " 4. What would you do next?" + @echo " 4. What evidence supports one conclusion?" + @echo " 5. What was difficult to author or update?" + @echo " 6. What would you do next?" + @echo " 7. Would you use this workflow again?" + @echo " 8. What is the most important missing workflow?" @echo "" - @echo "Capture only anonymous route, clarity, time-to-first-answer, and reproducible issue notes." + @echo "Scorecard: Task success; Time to first useful answer; Readiness comprehension; Evidence trace; Authoring friction; Misuse risk; Trust; Perceived performance; Repeat-use case; Repeat-use intent; Most important missing workflow." + @echo "Before starting, confirm voluntary participation, withdrawal rights, minimal anonymous capture, and the working-note deletion date." @echo "Closeout outcomes: clear, reproducible_ui_issue, documentation_gap, environment_limited, intentionally_deferred." @echo "Do not capture names, account details, investment opinions, price targets, trade decisions, or portfolio information." @echo "Commit a feedback log only after removing personal information and only when it is intentionally reviewed pilot evidence." @@ -502,13 +554,13 @@ linkedin-share-check: @echo "Research-only: this is portfolio/demo packaging, not investment advice, data freshness proof, broker integration, or trade instruction." @echo "" @echo "1. Open your LinkedIn profile and confirm the Featured card title matches:" - @echo " Stock Research Command Center | Readiness-First Stock Research Workflow" - @echo "2. Confirm the Featured description says the project is research-only and mentions no broker integration, auto-trading, or investment advice." - @echo "3. Use docs/assets/linkedin-public-dashboard.png when you want the curated product screenshot." + @echo " Stock Research Command Center | Evidence-First Company Research" + @echo "2. Confirm the Featured description names Research Desk -> Discover -> Company Workbench -> Monitor and keeps the project research-only with no broker integration, auto-trading, or investment advice." + @echo "3. Use docs/assets/linkedin-public-dashboard.png only as the reviewed count-safe Company Workbench answer visual." @echo "4. If LinkedIn shows GitHub's generated OpenGraph card, keep it only if that card is acceptable; otherwise remove and re-add the Featured item with the curated screenshot." - @echo "5. Use the GitHub link unless you have separately deployed and verified a hosted Streamlit app." + @echo "5. Use the stable GitHub repository link only after this reviewed feature reaches the default branch. Until then, keep the existing stable item or label any non-default review link Draft engineering preview." @echo "6. Open the GitHub link from LinkedIn and confirm the README starts with External Reviewer Start Here and names the GitHub/local-app boundary before deeper operator detail." - @echo "7. Stop before claiming screenshots prove current data freshness, coverage completion, provider-key activation, or investment advice." + @echo "7. Stop before claiming screenshots prove current data freshness, readiness counts, coverage completion, provider-key activation, hosting, market validation, or investment advice." @echo "" @echo "Next safe repo checks:" @echo " make public-check" @@ -516,15 +568,16 @@ linkedin-share-check: @echo " make diff-hygiene-summary" pilot-readiness-check: - @python3 -m src.pilot_readiness --top-n $(or $(TOP_N),10) + @python3 -m src.pilot_readiness --profile "$(or $(PROFILE),default)" --top-n $(or $(TOP_N),10) pilot-readiness-packet: - @python3 -m src.pilot_readiness --packet --top-n $(or $(TOP_N),10) --output "$(or $(OUTPUT),outputs/pilot_readiness_packet.md)" + @python3 -m src.pilot_readiness --profile "$(or $(PROFILE),default)" --packet --top-n $(or $(TOP_N),10) $(if $(OUTPUT),--output "$(OUTPUT)",) pilot-share-brief: - @python3 -m src.pilot_readiness --share-brief --top-n $(or $(TOP_N),10) --output "$(or $(OUTPUT),outputs/pilot_share_brief.md)" + @python3 -m src.pilot_readiness --profile "$(or $(PROFILE),default)" --share-brief --top-n $(or $(TOP_N),10) $(if $(OUTPUT),--output "$(OUTPUT)",) trusted-data-pilot: + @case "$(PROFILE)" in default|demo|local) ;; *) echo "PROFILE must be exactly one of: default, demo, local" >&2; exit 2;; esac @echo "Trusted Data Pilot" @echo "Read-only guide: this target prints commands only. It does not refresh prices, import rows, edit CSVs, or change readiness outputs." @echo "" @@ -544,7 +597,7 @@ trusted-data-pilot: @echo "Stop condition: if trusted source rows are unavailable, do not fill placeholders; leave the ticker visibly blocked by missing data and record the missing input." @echo "Pilot evidence packet: baseline readiness, before report, focused blocker check, lane review path, validate/preview gate, apply boundary, rejected-row check, rebuild proof, and still-blocked evidence row." @echo "One-company packet example:" - @echo " make readiness-snapshot" + @echo " make readiness-snapshot PROFILE=$(PROFILE)" @echo " make project-status-check" @echo " make trusted-data-pilot-candidates TOP_N=10 # only when project-status-check shows executable company candidates" @echo " make trusted-data-pilot-packet TICKER=" @@ -556,13 +609,13 @@ trusted-data-pilot: @echo " make imports-apply IMPORT_TICKERS= only after validation passes, preview scope is intended, rejected rows are zero, and source provenance is present." @echo " Use the broad imports-apply sequence only after every staged row is source-reviewed and intended." @echo " Check the rejected-row report printed by the packet before treating the lane as available." - @echo " Run the matching rebuild proof:" - @echo " fundamentals lane: make readiness && make dcf-readiness" - @echo " peer lane: make readiness && make peer-mapping-queue TOP_N=25" + @echo " Run the matching in-memory comparison proof:" + @echo " fundamentals lane: make dcf-readiness && make reviewed-batch-compare PROFILE=$(PROFILE) LANE=fundamentals BATCH_ID= REVIEW_DATE=" + @echo " peer lane: make reviewed-batch-compare PROFILE=$(PROFILE) LANE=peers BATCH_ID= REVIEW_DATE= && make peer-mapping-queue TOP_N=25" @echo " make stock-report-md TICKER=" @echo "" @echo "1. Save the current baseline:" - @echo " make readiness-snapshot" + @echo " make readiness-snapshot PROFILE=$(PROFILE)" @echo "" @echo "2. Confirm current blockers:" @echo " make status-check $(if $(TICKERS),TICKERS=$(TICKERS) )TOP_N=$(or $(TOP_N),10)" @@ -588,10 +641,10 @@ trusted-data-pilot: @echo " make imports-preview IMPORT_TICKERS=" @echo " make imports-apply IMPORT_TICKERS=" @echo "" - @echo "7. Prove the lane is available before reading valuation:" - @echo " make readiness" - @echo " make dcf-readiness" - @echo " make peer-mapping-queue $(if $(TICKERS),TICKERS=$(TICKERS) )TOP_N=25" + @echo "7. Prove the lane is available before reading valuation with the same profile used for the baseline:" + @echo " make reviewed-batch-compare PROFILE=$(PROFILE) LANE=prices BATCH_ID= REVIEW_DATE=" + @echo " make dcf-readiness && make reviewed-batch-compare PROFILE=$(PROFILE) LANE=fundamentals BATCH_ID= REVIEW_DATE=" + @echo " make reviewed-batch-compare PROFILE=$(PROFILE) LANE=peers BATCH_ID= REVIEW_DATE= && make peer-mapping-queue $(if $(TICKERS),TICKERS=$(TICKERS) )TOP_N=25" @echo " make stock-report-md TICKER=" @echo "" @echo "8. Keep the public branch clean:" @@ -655,28 +708,34 @@ readiness-ops-evidence: @python3 -m src.readiness_ops --root . --evidence --top-n $(or $(TOP_N),10) reviewed-batch: - @python3 -m src.reviewed_batch --root . --lane $(or $(LANE),prices) --top-n $(or $(TOP_N),10) $(if $(TICKERS),--tickers "$(TICKERS)",) --md-output $(or $(MD_OUTPUT),outputs/reviewed_batch_packet.md) --csv-output $(or $(CSV_OUTPUT),outputs/reviewed_batch_packet.csv) $(if $(DRY_RUN),--dry-run,) + @python3 -m src.reviewed_batch --root . --profile $(or $(PROFILE),default) --lane $(or $(LANE),prices) --top-n $(or $(TOP_N),10) $(if $(TICKERS),--tickers "$(TICKERS)",) --md-output $(or $(MD_OUTPUT),outputs/reviewed_batch_packet.md) --csv-output $(or $(CSV_OUTPUT),outputs/reviewed_batch_packet.csv) $(if $(DRY_RUN),--dry-run,) fundamentals-batch-proof: - @python3 -m src.reviewed_batch --root . --lane fundamentals --top-n $(or $(TOP_N),10) $(if $(TICKERS),--tickers "$(TICKERS)",) --md-output $(or $(MD_OUTPUT),outputs/fundamentals_batch_proof.md) --csv-output $(or $(CSV_OUTPUT),outputs/fundamentals_batch_proof.csv) $(if $(DRY_RUN),--dry-run,) + @python3 -m src.reviewed_batch --root . --profile $(or $(PROFILE),default) --lane fundamentals --top-n $(or $(TOP_N),10) $(if $(TICKERS),--tickers "$(TICKERS)",) --md-output $(or $(MD_OUTPUT),outputs/fundamentals_batch_proof.md) --csv-output $(or $(CSV_OUTPUT),outputs/fundamentals_batch_proof.csv) $(if $(DRY_RUN),--dry-run,) peer-batch-proof: - @python3 -m src.reviewed_batch --root . --lane peers --top-n $(or $(TOP_N),10) $(if $(TICKERS),--tickers "$(TICKERS)",) --md-output $(or $(MD_OUTPUT),outputs/peer_batch_proof.md) --csv-output $(or $(CSV_OUTPUT),outputs/peer_batch_proof.csv) $(if $(DRY_RUN),--dry-run,) + @python3 -m src.reviewed_batch --root . --profile $(or $(PROFILE),default) --lane peers --top-n $(or $(TOP_N),10) $(if $(TICKERS),--tickers "$(TICKERS)",) --md-output $(or $(MD_OUTPUT),outputs/peer_batch_proof.md) --csv-output $(or $(CSV_OUTPUT),outputs/peer_batch_proof.csv) $(if $(DRY_RUN),--dry-run,) peer-mapping-source-review: @python3 -m src.peer_mapping_source_review --root . --top-n $(or $(TOP_N),10) $(if $(TICKERS),--tickers "$(TICKERS)",) --md-output $(or $(MD_OUTPUT),outputs/peer_mapping_source_review.md) --csv-output $(or $(CSV_OUTPUT),outputs/peer_mapping_source_review.csv) $(if $(DRY_RUN),--dry-run,) peer-mapping-writeback-guard: - @python3 -m src.peer_mapping_source_review --root . --guard-writeback --ticker "$(TICKER)" --peer-ticker "$(PEER_TICKER)" --peer-group "$(PEER_GROUP)" --sector "$(SECTOR)" --industry "$(INDUSTRY)" --source "$(SOURCE)" --as-of-date "$(AS_OF_DATE)" --relationship-rationale "$(RELATIONSHIP_RATIONALE)" --reviewer "$(REVIEWER)" --review-date "$(REVIEW_DATE)" --source-proof-status "$(SOURCE_PROOF_STATUS)" --import-row-ready "$(IMPORT_ROW_READY)" + @python3 -m src.peer_mapping_source_review --root . --guard-writeback --ticker "$(TICKER)" --peer-ticker "$(PEER_TICKER)" --peer-group "$(PEER_GROUP)" --sector "$(SECTOR)" --industry "$(INDUSTRY)" --peer-role "$(PEER_ROLE)" --source "$(SOURCE)" --as-of-date "$(AS_OF_DATE)" --relationship-rationale "$(RELATIONSHIP_RATIONALE)" --comparability-basis "$(COMPARABILITY_BASIS)" --valuation-anchor-eligible "$(VALUATION_ANCHOR_ELIGIBLE)" --reviewer "$(REVIEWER)" --review-date "$(REVIEW_DATE)" --source-proof-status "$(SOURCE_PROOF_STATUS)" --import-row-ready "$(IMPORT_ROW_READY)" reviewed-batch-proof: @python3 -m src.reviewed_batch_proof --ledger $(or $(LEDGER),data/reviewed_batch_proofs.csv) +proof-readiness-reconciliation: + @python3 -m src.proof_readiness_reconciliation --root . --top-n $(or $(TOP_N),20) $(if $(TICKERS),--tickers "$(TICKERS)",) $(if $(filter 1,$(JSON)),--json,) + reviewed-batch-compare: - @python3 -m src.readiness_comparison --root . --top-n $(or $(TOP_N),25) --batch-id "$(or $(BATCH_ID),)" --lane "$(or $(LANE),prices)" --review-date "$(or $(REVIEW_DATE),)" +ifndef PROFILE + $(error PROFILE is required: default, demo, or local) +endif + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.readiness_comparison --root . --profile "$(PROFILE)" --top-n $(or $(TOP_N),25) --batch-id "$(or $(BATCH_ID),)" --lane "$(or $(LANE),prices)" --review-date "$(or $(REVIEW_DATE),)" reviewed-batch-preflight: - @python3 -m src.reviewed_batch_preflight --root . --lane $(or $(LANE),prices) --top-n $(or $(TOP_N),100) --max-candidates $(or $(MAX_CANDIDATES),3500) --provider $(or $(PROVIDER),auto) $(if $(BATCH_ID),--batch-id "$(BATCH_ID)",) $(if $(REVIEW_DATE),--review-date "$(REVIEW_DATE)",) + @python3 -m src.reviewed_batch_preflight --root . --profile $(or $(PROFILE),default) --lane $(or $(LANE),prices) --top-n $(or $(TOP_N),100) --max-candidates $(or $(MAX_CANDIDATES),3500) --provider $(or $(PROVIDER),auto) $(if $(BATCH_ID),--batch-id "$(BATCH_ID)",) $(if $(REVIEW_DATE),--review-date "$(REVIEW_DATE)",) auto-refresh-plan: @python3 -m src.auto_refresh_orchestrator --root . --schedule all @@ -696,6 +755,12 @@ auto-refresh-runbook: auto-refresh-status: @python3 -m src.auto_refresh_orchestrator --root . --schedule $(or $(SCHEDULE),daily) --status +refresh-operations-status: + @python3 -m src.auto_refresh_orchestrator --root . --schedule $(or $(SCHEDULE),daily) --status + +refresh-operations-runbook: + @python3 -m src.auto_refresh_orchestrator --root . --schedule $(or $(SCHEDULE),daily) --runbook + scheduler-activation-checklist: @echo "Scheduler Activation Checklist" @echo "Read-only: this target prints scheduling gates only; it does not refresh, import, apply, stage, commit, push, deploy, or expose secrets." @@ -749,6 +814,10 @@ diff-hygiene: diff-hygiene-summary: @python3 scripts/diff_hygiene.py --summary +.PHONY: pr-range-hygiene-check +pr-range-hygiene-check: + @python3 scripts/diff_hygiene.py --range-base "$(BASE_SHA)" --range-head "$(HEAD_SHA)" + diff-hygiene-files: @python3 scripts/diff_hygiene.py --write-files @@ -767,6 +836,24 @@ universe-scope: staged-hygiene-check: @python3 scripts/diff_hygiene.py --staged-check +point-in-time-universe-status point-in-time-universe-preview: export POINT_IN_TIME_UNIVERSE_MANIFEST := $(value MANIFEST) +point-in-time-universe-status point-in-time-universe-preview: export POINT_IN_TIME_UNIVERSE_REGISTRY := $(if $(strip $(value REGISTRY)),$(value REGISTRY),config/source_rights.yml) +point-in-time-universe-status point-in-time-universe-preview: export POINT_IN_TIME_UNIVERSE_TOP_N := $(if $(strip $(value TOP_N)),$(value TOP_N),20) + +point-in-time-universe-status: + @case "$${POINT_IN_TIME_UNIVERSE_MANIFEST}" in *[![:space:]]*) ;; *) echo "MANIFEST is required" >&2; exit 2;; esac + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.point_in_time_universe status \ + --manifest "$${POINT_IN_TIME_UNIVERSE_MANIFEST}" \ + --registry "$${POINT_IN_TIME_UNIVERSE_REGISTRY}" \ + --top-n "$${POINT_IN_TIME_UNIVERSE_TOP_N}" + +point-in-time-universe-preview: + @case "$${POINT_IN_TIME_UNIVERSE_MANIFEST}" in *[![:space:]]*) ;; *) echo "MANIFEST is required" >&2; exit 2;; esac + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.point_in_time_universe preview \ + --manifest "$${POINT_IN_TIME_UNIVERSE_MANIFEST}" \ + --registry "$${POINT_IN_TIME_UNIVERSE_REGISTRY}" \ + --top-n "$${POINT_IN_TIME_UNIVERSE_TOP_N}" + public-wording-check: @python3 scripts/public_wording_check.py @@ -797,16 +884,33 @@ public-check: @$(MAKE) --silent demo test: - python3 -m pytest tests -q + $(NO_WRITE_GUARD) python3 -m pytest tests -q + +COMMERCIAL_SOURCE_RIGHTS_REPO_ROOT := $(strip $(shell dirname "$(MAKEFILE_LIST)")) + +commercial-source-rights: export COMMERCIAL_SOURCE_RIGHTS_CONFIG := $(value CONFIG) +commercial-source-rights: export COMMERCIAL_SOURCE_RIGHTS_SOURCE := $(value SOURCE) +commercial-source-rights: export COMMERCIAL_SOURCE_RIGHTS_REPO_ROOT := $(COMMERCIAL_SOURCE_RIGHTS_REPO_ROOT) +commercial-source-rights: + @set --; \ + case "$${COMMERCIAL_SOURCE_RIGHTS_CONFIG}" in \ + *[![:space:]]*) set -- "$$@" "--config=$${COMMERCIAL_SOURCE_RIGHTS_CONFIG}" ;; \ + esac; \ + case "$${COMMERCIAL_SOURCE_RIGHTS_SOURCE}" in \ + *[![:space:]]*) set -- "$$@" "--source=$${COMMERCIAL_SOURCE_RIGHTS_SOURCE}" ;; \ + esac; \ + PYTHONDONTWRITEBYTECODE=1 \ + PYTHONPATH="$${COMMERCIAL_SOURCE_RIGHTS_REPO_ROOT}$${PYTHONPATH:+:$${PYTHONPATH}}" \ + python3 -m src.commercial_source_rights "$$@" status: - python3 -m src.project_status --refresh-artifacts --top-n $(or $(TOP_N),5) + $(NO_WRITE_GUARD) python3 -m src.project_status --check --top-n $(or $(TOP_N),5) status-check: python3 -m src.project_status --check --top-n $(or $(TOP_N),5) $(if $(TICKERS),--tickers $(TICKERS),) pipeline: - python3 -m src.report_generator + $(NO_WRITE_GUARD) python3 -m src.report_generator stock-report: ifndef TICKER @@ -875,13 +979,10 @@ license-status: @python3 -m src.license_status --root . verify: - $(MAKE) test - $(MAKE) pipeline - $(MAKE) validate-data - $(MAKE) onboarding + $(NO_WRITE_GUARD) $(MAKE) test pipeline validate-data onboarding validate-all: - scripts/validate_all.sh + $(NO_WRITE_GUARD) scripts/validate_all.sh coverage: python3 -m src.data_onboarding --coverage $(if $(TOP_N),--top-n $(TOP_N),) $(if $(TICKERS),--tickers $(TICKERS),) @@ -977,16 +1078,15 @@ endif python3 -m src.data_onboarding --peer-mapping-queue --top-n 1 --tickers $(TICKER) onboarding: - python3 -m src.manual_price_import --coverage-only --top-n $(or $(TOP_N),20) - python3 -m src.dcf_readiness --top-n $(or $(TOP_N),20) - python3 -m src.optional_context_readiness - python3 -m src.readiness_engine - python3 -m src.data_sources --write-output - python3 -m src.data_onboarding --write-output --top-n $(or $(TOP_N),20) - python3 -m src.research_health --write-output - python3 -m src.action_queue --write-output - python3 -m src.research_decisions - python3 -m src.project_status --write-output + $(NO_WRITE_GUARD) python3 -m src.manual_price_import --coverage-only --read-only --top-n $(or $(TOP_N),20) + $(NO_WRITE_GUARD) python3 -m src.dcf_readiness --read-only --top-n $(or $(TOP_N),20) + $(NO_WRITE_GUARD) python3 -m src.optional_context_readiness --read-only --top-n $(or $(TOP_N),20) + $(NO_WRITE_GUARD) python3 -m src.readiness_preview --top-n $(or $(TOP_N),20) + $(NO_WRITE_GUARD) python3 -m src.data_sources --check --top-n $(or $(TOP_N),20) + $(NO_WRITE_GUARD) python3 -m src.data_onboarding --coverage --top-n $(or $(TOP_N),20) + $(NO_WRITE_GUARD) python3 -m src.research_health --check --top-n $(or $(TOP_N),20) + $(NO_WRITE_GUARD) python3 -m src.action_queue --check --top-n $(or $(TOP_N),20) + $(NO_WRITE_GUARD) python3 -m src.project_status --check --top-n $(or $(TOP_N),20) templates: python3 -m src.data_onboarding --write-templates @@ -1043,13 +1143,13 @@ peer-mapping-queue: python3 -m src.data_onboarding --peer-mapping-queue $(if $(TOP_N),--top-n $(TOP_N),) $(if $(TICKERS),--tickers $(TICKERS),) price-validate: - STOCK_RESEARCH_DATA_PROFILE=local python3 -m src.data_update --validate-price-imports + STOCK_RESEARCH_DATA_PROFILE=local python3 -m src.data_update --validate-price-imports $(if $(AS_OF),--review-cutoff "$(AS_OF)",) price-preview: - STOCK_RESEARCH_DATA_PROFILE=local python3 -m src.data_update --preview-price-import-merge + STOCK_RESEARCH_DATA_PROFILE=local python3 -m src.data_update --preview-price-import-merge $(if $(AS_OF),--review-cutoff "$(AS_OF)",) price-apply: - STOCK_RESEARCH_DATA_PROFILE=local python3 -m src.data_update --apply-price-import-merge + STOCK_RESEARCH_DATA_PROFILE=local python3 -m src.data_update --apply-price-import-merge $(if $(AS_OF),--review-cutoff "$(AS_OF)",) price-refresh: ifdef TICKERS @@ -1065,30 +1165,28 @@ price-normalize: ifndef INPUT $(error INPUT is required, for example: make price-normalize INPUT=data/raw/prices/NVDA.csv TICKER=NVDA SOURCE=yahoo_manual) endif +ifdef RETRIEVED_AT +ifndef AS_OF + $(error AS_OF is required when RETRIEVED_AT is supplied) +endif +endif ifdef TICKER - python3 -m src.price_import_normalizer --input $(INPUT) --ticker $(TICKER) --source $(or $(SOURCE),generic_manual) + python3 -m src.price_import_normalizer --input $(INPUT) --ticker $(TICKER) --source $(or $(SOURCE),generic_manual) $(if $(SOURCE_REF),--source-ref "$(SOURCE_REF)",) $(if $(RETRIEVED_AT),--retrieved-at "$(RETRIEVED_AT)",) $(if $(AS_OF),--review-cutoff "$(AS_OF)",) else - python3 -m src.price_import_normalizer --input $(INPUT) --source $(or $(SOURCE),generic_manual) + python3 -m src.price_import_normalizer --input $(INPUT) --source $(or $(SOURCE),generic_manual) $(if $(SOURCE_REF),--source-ref "$(SOURCE_REF)",) $(if $(RETRIEVED_AT),--retrieved-at "$(RETRIEVED_AT)",) $(if $(AS_OF),--review-cutoff "$(AS_OF)",) endif daily: - $(MAKE) price-refresh - $(MAKE) pipeline - $(MAKE) monthly - $(MAKE) track-record - $(MAKE) validate-data - $(MAKE) onboarding - python3 -m src.action_queue --write-output - python3 -m src.project_status --write-output + $(NO_WRITE_GUARD) $(MAKE) pipeline validate-data onboarding status-check TOP_N=$(or $(TOP_N),5) dashboard: - streamlit run src/dashboard.py --client.toolbarMode viewer --server.headless true + PYTHONPATH="$(CURDIR):$${PYTHONPATH:-}" streamlit run src/dashboard.py --client.toolbarMode viewer --server.headless true demo-dashboard: - STOCK_RESEARCH_DATA_PROFILE=demo streamlit run src/dashboard.py --client.toolbarMode viewer --server.headless true + STOCK_RESEARCH_DATA_PROFILE=demo PYTHONPATH="$(CURDIR):$${PYTHONPATH:-}" streamlit run src/dashboard.py --client.toolbarMode viewer --server.headless true dashboard-smoke: - scripts/smoke_dashboard.sh + $(NO_WRITE_GUARD) scripts/smoke_dashboard.sh demo-dashboard-smoke: STOCK_RESEARCH_DATA_PROFILE=demo scripts/smoke_dashboard.sh @@ -1096,6 +1194,24 @@ demo-dashboard-smoke: dashboard-render-smoke: @python3 -m src.dashboard_render_smoke +research-dashboard-render-smoke: + @python3 -m src.dashboard_render_smoke --routes research + +.PHONY: sec-quarterly-cash-preview +sec-quarterly-cash-preview: +ifndef AS_OF + $(error AS_OF is required, for example: make sec-quarterly-cash-preview AS_OF=2026-07-20T23:59:59-04:00) +endif + @python3 -m src.sec_quarterly_cash_generation_preview \ + --ticker "$(or $(TICKER),NVDA)" \ + --cik "$(or $(CIK),0001045810)" \ + --fiscal-period "$(or $(FISCAL_PERIOD),2027-Q1)" \ + --period-start "$(or $(PERIOD_START),2026-01-26)" \ + --period-end "$(or $(PERIOD_END),2026-04-26)" \ + --accession "$(or $(ACCESSION),0001045810-26-000052)" \ + --primary-document "$(or $(PRIMARY_DOCUMENT),nvda-20260426.htm)" \ + --as-of "$(AS_OF)" + demo-dashboard-render-smoke: @STOCK_RESEARCH_DATA_PROFILE=demo python3 -m src.dashboard_render_smoke @@ -1111,6 +1227,31 @@ provider-setup-checklist: hosted-demo-readiness: @python3 -m src.hosted_demo_readiness --root . +private-beta-readiness: + @python3 -m src.private_beta_readiness + +commercial-beta-check: + @echo "Commercial Research Beta Check" + @echo "Read-only: checks source rights, refresh-operation safety, and private-beta readiness without fetching or applying data." + @$(MAKE) --silent commercial-source-rights + @$(MAKE) --silent refresh-operations-status + @$(MAKE) --silent private-beta-readiness + +commercial-beta-release-check: + @echo "Commercial Research Beta Release Check" + @echo "Read-only: verifies the local release candidate without fetching, importing, applying, staging, committing, pushing, or changing external services." + @$(MAKE) --silent commercial-beta-check + @python3 -m pytest tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py tests/test_launchers.py -q + @$(MAKE) --silent research-dashboard-render-smoke + @$(MAKE) --silent commercial-beta-performance-contract + @$(MAKE) --silent browser-qa-evidence + @$(MAKE) --silent public-check + @$(MAKE) --silent pilot-readiness-check TOP_N=10 + @$(MAKE) --silent diff-hygiene-summary + @git diff --check + @echo "Safe claims: reproducible local research workflow; readiness-first analysis; source-rights gates; research-only controlled beta candidate." + @echo "Unsafe claims: hosted or authenticated availability; complete coverage; calibrated predictive accuracy; investment recommendations; licensed operation without verified rights." + provider-smoke: ifndef TICKER $(error TICKER is required, for example: make provider-smoke PROVIDER=fmp TICKER=NVDA) @@ -1230,10 +1371,57 @@ import-analyst-estimates: python3 -m src.manual_optional_context_import analyst_estimates readiness: - python3 -m src.readiness_engine + @echo "Deprecated no-write guard: use make readiness-preview TOP_N=20 for inspection." >&2 + @echo "For an intentional ignored snapshot: CONFIRM_MATERIALIZE=1 make readiness-materialize PROFILE=" >&2 + @exit 2 + +readiness-preview: + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.readiness_preview --top-n $(or $(TOP_N),20) + +readiness-release-review: + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.readiness_release_review review --project-root . --top-n $(or $(TOP_N),20) $(if $(JSON),--json,) + +readiness-release-record: +ifndef PREVIEW_RECEIPT + $(error PREVIEW_RECEIPT is required) +endif +ifndef REVIEWER + $(error REVIEWER is required) +endif +ifndef REVIEW_DATE + $(error REVIEW_DATE is required) +endif +ifndef TECHNICAL_DECISION + $(error TECHNICAL_DECISION is required: approved or rejected) +endif +ifndef DISTRIBUTION_DECISION + $(error DISTRIBUTION_DECISION is required: approved, rejected, or external_review_required) +endif +ifneq ($(CONFIRM_REVIEWED),1) + $(error CONFIRM_REVIEWED=1 is required after reviewing the exact preview receipt) +endif + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.readiness_release_review record --project-root . --preview-receipt "$(PREVIEW_RECEIPT)" --reviewer "$(REVIEWER)" --review-date "$(REVIEW_DATE)" --technical-decision "$(TECHNICAL_DECISION)" --distribution-decision "$(DISTRIBUTION_DECISION)" --confirm-reviewed + +readiness-release-guard: +ifndef RECORD_ID + $(error RECORD_ID is required) +endif + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.readiness_release_review guard --project-root . --record-id "$(RECORD_ID)" + +readiness-materialize: +ifndef PROFILE + $(error PROFILE is required: default, demo, or local) +endif +ifneq ($(CONFIRM_MATERIALIZE),1) + $(error CONFIRM_MATERIALIZE=1 is required) +endif + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.readiness_materializer --project-root . --profile "$(PROFILE)" --confirm-materialize readiness-snapshot: - python3 -m src.readiness_engine --snapshot-only +ifndef PROFILE + $(error PROFILE is required: default, demo, or local) +endif + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.readiness_engine --project-root . --profile "$(PROFILE)" --snapshot-only research-decisions: python3 -m src.research_decisions @@ -1278,7 +1466,113 @@ earnings-nowcast-preview: @python3 -m src.earnings_nowcast_onboarding preview --input-dir $(or $(INPUT_DIR),data/imports/earnings_nowcast) $(if $(EXISTING_DIR),--existing-dir $(EXISTING_DIR),) $(if $(AS_OF),--cutoff $(AS_OF),) earnings-nowcast-readiness: - @python3 -m src.earnings_nowcast_onboarding readiness --input-dir $(or $(INPUT_DIR),data/imports/earnings_nowcast) --ticker $(or $(TICKER),SYN1) $(if $(AS_OF),--cutoff $(AS_OF),) + @python3 -m src.earnings_nowcast_onboarding readiness --input-dir $(if $(FIXTURE),tests/fixtures/earnings_nowcast_onboarding,$(or $(INPUT_DIR),data/imports/earnings_nowcast)) --ticker $(or $(TICKER),SYN1) $(if $(AS_OF),--cutoff $(AS_OF),) + +earnings-nowcast-cohort-readiness: + @python3 -m src.earnings_nowcast_cohort --input-dir $(if $(FIXTURE),tests/fixtures/earnings_nowcast_onboarding,$(or $(INPUT_DIR),data/imports/earnings_nowcast)) --tickers "$(or $(TICKERS),$(EARNINGS_NOWCAST_COHORT))" --as-of "$(or $(AS_OF),2026-01-31T23:59:59Z)" $(if $(JSON),--json,) + +calibration-evidence-bundle-preview: export CALIBRATION_EVIDENCE_BUNDLE := $(value BUNDLE) +calibration-evidence-bundle-preview: + @case "$${CALIBRATION_EVIDENCE_BUNDLE}" in *[![:space:]]*) ;; *) echo "BUNDLE is required" >&2; exit 2;; esac + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.calibration_evidence_bundle preview \ + --bundle "$${CALIBRATION_EVIDENCE_BUNDLE}" + +earnings-consensus-source-status: + @python3 -m src.earnings_consensus_sources $(if $(REVIEWED_CSV),--reviewed-csv "$(REVIEWED_CSV)",) $(if $(JSON),--json,) + +earnings-consensus-source-review: + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.earnings_consensus_sources --review-csv "$(INPUT)" --provider "$(PROVIDER)" --as-of "$(AS_OF)" $(if $(JSON),--json,) + +earnings-consensus-collection-plan: + @python3 -m src.earnings_consensus_collector plan --tickers "$(or $(TICKERS),$(EARNINGS_NOWCAST_COHORT))" --as-of "$(or $(AS_OF),2026-01-31T23:59:59Z)" --cadence "$(or $(CADENCE),weekly)" + +earnings-consensus-collection-status: + @python3 -m src.earnings_consensus_collector status --ledger "$(or $(LEDGER),data/imports/earnings_nowcast/prospective_consensus.csv)" + +earnings-consensus-collection-preview: + @python3 -m src.earnings_consensus_collector preview --input "$(INPUT)" --ledger "$(or $(LEDGER),data/imports/earnings_nowcast/prospective_consensus.csv)" --as-of "$(AS_OF)" + +earnings-consensus-collection-record: + @test -n "$(AS_OF)" || (echo "AS_OF is required and must match the reviewed preview cutoff" >&2; exit 2) + @test -n "$(PREVIEW_RECEIPT)" || (echo "PREVIEW_RECEIPT is required from the exact reviewed preview" >&2; exit 2) + @test "$(CONFIRM_REVIEWED)" = "1" || (echo "CONFIRM_REVIEWED=1 is required after reviewing the exact preview" >&2; exit 2) + @python3 -m src.earnings_consensus_collector record --input "$(INPUT)" --ledger "$(or $(LEDGER),data/imports/earnings_nowcast/prospective_consensus.csv)" --as-of "$(AS_OF)" --preview-receipt "$(PREVIEW_RECEIPT)" --confirm-reviewed + +prospective-field-proof-status: export FIELD_PROOF_LEDGER := $(if $(strip $(value LEDGER)),$(value LEDGER),data/prospective_field_proofs.csv) +prospective-field-proof-status: export FIELD_PROOF_JSON := $(value JSON) +prospective-field-proof-status: + @if [ "$${FIELD_PROOF_JSON}" = "1" ]; then \ + PYTHONDONTWRITEBYTECODE=1 python3 -m src.prospective_field_proof status --ledger "$${FIELD_PROOF_LEDGER}" --json; \ + else \ + PYTHONDONTWRITEBYTECODE=1 python3 -m src.prospective_field_proof status --ledger "$${FIELD_PROOF_LEDGER}"; \ + fi + +prospective-field-proof-audit: export FIELD_PROOF_LEDGER := $(if $(strip $(value LEDGER)),$(value LEDGER),data/prospective_field_proofs.csv) +prospective-field-proof-audit: export FIELD_PROOF_JSON := $(value JSON) +prospective-field-proof-audit: + @if [ "$${FIELD_PROOF_JSON}" = "1" ]; then \ + PYTHONDONTWRITEBYTECODE=1 python3 -m src.prospective_field_proof audit --ledger "$${FIELD_PROOF_LEDGER}" --json; \ + else \ + PYTHONDONTWRITEBYTECODE=1 python3 -m src.prospective_field_proof audit --ledger "$${FIELD_PROOF_LEDGER}"; \ + fi + +prospective-field-proof-preview: export FIELD_PROOF_INPUT := $(value INPUT) +prospective-field-proof-preview: export FIELD_PROOF_LEDGER := $(if $(strip $(value LEDGER)),$(value LEDGER),data/prospective_field_proofs.csv) +prospective-field-proof-preview: export FIELD_PROOF_AS_OF := $(value AS_OF) +prospective-field-proof-preview: export FIELD_PROOF_JSON := $(value JSON) +prospective-field-proof-preview: + @if [ "$${FIELD_PROOF_JSON}" = "1" ]; then \ + PYTHONDONTWRITEBYTECODE=1 python3 -m src.prospective_field_proof preview --input "$${FIELD_PROOF_INPUT}" --ledger "$${FIELD_PROOF_LEDGER}" --as-of "$${FIELD_PROOF_AS_OF}" --json; \ + else \ + PYTHONDONTWRITEBYTECODE=1 python3 -m src.prospective_field_proof preview --input "$${FIELD_PROOF_INPUT}" --ledger "$${FIELD_PROOF_LEDGER}" --as-of "$${FIELD_PROOF_AS_OF}"; \ + fi + +prospective-field-proof-record: export FIELD_PROOF_INPUT := $(value INPUT) +prospective-field-proof-record: export FIELD_PROOF_LEDGER := $(if $(strip $(value LEDGER)),$(value LEDGER),data/prospective_field_proofs.csv) +prospective-field-proof-record: export FIELD_PROOF_AS_OF := $(value AS_OF) +prospective-field-proof-record: export FIELD_PROOF_PREVIEW_RECEIPT := $(value PREVIEW_RECEIPT) +prospective-field-proof-record: export FIELD_PROOF_CONFIRM_REVIEWED := $(value CONFIRM_REVIEWED) +prospective-field-proof-record: export FIELD_PROOF_JSON := $(value JSON) +prospective-field-proof-record: + @case "$${FIELD_PROOF_INPUT}" in *[![:space:]]*) ;; *) echo "INPUT is required for the exact reviewed field proof batch" >&2; exit 2;; esac + @case "$${FIELD_PROOF_AS_OF}" in *[![:space:]]*) ;; *) echo "AS_OF is required and must match the reviewed preview cutoff" >&2; exit 2;; esac + @case "$${FIELD_PROOF_PREVIEW_RECEIPT}" in *[![:space:]]*) ;; *) echo "PREVIEW_RECEIPT is required from the exact reviewed preview" >&2; exit 2;; esac + @test "$${FIELD_PROOF_CONFIRM_REVIEWED}" = "1" || (echo "CONFIRM_REVIEWED=1 is required after reviewing the exact preview" >&2; exit 2) + @if [ "$${FIELD_PROOF_JSON}" = "1" ]; then \ + PYTHONDONTWRITEBYTECODE=1 python3 -m src.prospective_field_proof record --input "$${FIELD_PROOF_INPUT}" --ledger "$${FIELD_PROOF_LEDGER}" --as-of "$${FIELD_PROOF_AS_OF}" --preview-receipt "$${FIELD_PROOF_PREVIEW_RECEIPT}" --confirm-reviewed --json; \ + else \ + PYTHONDONTWRITEBYTECODE=1 python3 -m src.prospective_field_proof record --input "$${FIELD_PROOF_INPUT}" --ledger "$${FIELD_PROOF_LEDGER}" --as-of "$${FIELD_PROOF_AS_OF}" --preview-receipt "$${FIELD_PROOF_PREVIEW_RECEIPT}" --confirm-reviewed; \ + fi + +research-outcome-review: + @python3 -m src.research_outcome_review --ledger "$(or $(LEDGER),data/research_outcome_reviews.csv)" --profile-key "$(or $(PROFILE_KEY),default)" --ticker "$(or $(TICKER),NVDA)" + +research-outcome-review-preview: + @python3 -m src.research_outcome_review --ledger "$(or $(LEDGER),data/research_outcome_reviews.csv)" --preview-input "$(INPUT)" + +research-outcome-review-record: + @python3 -m src.research_outcome_review --ledger "$(or $(LEDGER),data/research_outcome_reviews.csv)" --record-input "$(INPUT)" $(if $(CONFIRM_REVIEWED),--confirm-reviewed,) + +catalyst-evidence-timeline: + @python3 -m src.catalyst_evidence_timeline --ledger "$(or $(LEDGER),data/catalyst_evidence.csv)" --profile-key "$(or $(PROFILE_KEY),default)" --ticker "$(or $(TICKER),NVDA)" --as-of "$(or $(AS_OF),2026-01-31T23:59:59Z)" + +catalyst-evidence-preview: + @python3 -m src.catalyst_evidence_timeline --ledger "$(or $(LEDGER),data/catalyst_evidence.csv)" --preview-input "$(INPUT)" + +catalyst-evidence-record: + @python3 -m src.catalyst_evidence_timeline --ledger "$(or $(LEDGER),data/catalyst_evidence.csv)" --record-input "$(INPUT)" $(if $(CONFIRM_REVIEWED),--confirm-reviewed,) earnings-nowcast-prospective-plan: @python3 -m src.earnings_nowcast_onboarding prospective-plan --output-dir $(or $(OUTPUT_DIR),data/imports/earnings_nowcast) + +earnings-nowcast-sec-actuals-stage: +ifndef TICKERS + $(error TICKERS is required, for example: make earnings-nowcast-sec-actuals-stage TICKERS=NVDA OUTPUT_DIR=/tmp/sec-actuals AS_OF=2026-07-16T03:59:59Z) +endif +ifndef OUTPUT_DIR + $(error OUTPUT_DIR is required; use a generated temporary/review directory) +endif +ifndef AS_OF + $(error AS_OF is required for cutoff-safe evidence staging) +endif + @python3 -m src.earnings_nowcast_sec_actuals --tickers "$(TICKERS)" --output-dir "$(OUTPUT_DIR)" --cutoff "$(AS_OF)" --max-runtime-seconds "$(or $(SEC_STAGE_MAX_RUNTIME_SECONDS),300)" diff --git a/PRODUCT_SPEC.md b/PRODUCT_SPEC.md index 51b11966e..569badc66 100644 --- a/PRODUCT_SPEC.md +++ b/PRODUCT_SPEC.md @@ -6,15 +6,31 @@ This project is a local, CSV-first stock research command center. It helps a use The product is not a trading bot. It does not place orders, connect to brokers, recommend options trades, or fabricate missing market, fundamentals, earnings, or analyst-estimate data. +The supported workflow is `Research Desk -> Discover -> Company Workbench -> Monitor`. Operator-only legacy compatibility utilities are retained for deterministic regression and historical file contracts, not as current investing capability. + +## In-App Reviewed Research Records + +Thesis, evidence, catalyst, and outcome records are all available in the collapsed Company Workbench composer. +A valid record requires an exact preview and explicit confirmation before save. +Final integrity commit `e3a090dba` ensures confirmation appends only the receipt-matched recomputed record and enforces one readable active thesis lineage: revisions must supersede the exact active entry and preserve its thesis ID. The Company Workbench locks and explains that relationship, with temporary-ledger create -> revise -> reload coverage. +Confirmation-integrity commit `5a6c55921` binds every displayed preview field, preview time, and destination label to the exact receipt. If an append raises after it may have written, confirmation returns one-shot `save_pending_reload` with the exact record ID unless the locked ledger is provably unchanged; it never invites a blind duplicate retry. +Drafts are untrusted and preview receipts are session-only. +Production tests never append repository ledgers; persistence tests use temporary ledgers. +A saved record cannot change readiness, forecasts, probabilities, recommendations, or any other ledger. + +Priority 4's local validator is frozen; its permitted real-data exit gate remains externally incomplete. +Priority 6's provider-neutral authorization contract is complete locally; hosted implementation remains environment-dependent. +If the current branch head lacks direct local matrix, branch synchronization, draft-PR update, or exact-head CI evidence, complete those release steps first; otherwise select the first incomplete safe roadmap priority. Modal modifiers and active exposure fail closed. Broad-review repairs must be evaluated only through direct current-head local and exact-head CI evidence; their presence alone establishes neither gate. + ## Target User The target user is an individual investor or research reviewer who wants a deterministic local workflow for: - maintaining a broad market universe; - narrowing that universe into an active research list; -- reviewing portfolio holdings; +- reviewing one company and its source-backed research record; - tracking readiness by analysis feature; -- generating watchlists only when the needed data exists; +- monitoring a focused cohort without ranking companies for action; - seeing exact data blockers and next import actions. ## Non-Goals @@ -99,6 +115,12 @@ The product is now organized around a few research-facing surfaces: The primary public path is: Home workflow start -> Stock Selector -> Single-Stock Report -> Data Health lane answer -> Proof History evidence. +## Legacy Compatibility Boundary + +`Monthly Picks`, `Momentum Leaders`, `Portfolio Review`, `Value / Re-rating`, and `Final Watchlist` are Operator-only compatibility utilities. Every retained page is labeled **Legacy research utility — not part of Personal Research Mode**, and its detailed output is hidden behind an explicit collapsed compatibility control. + +These utilities preserve historical algorithms, regression checks, and output filenames only. They cannot feed Research Decision Lab, cannot change readiness, and cannot produce recommendations, sizing, or transaction behavior. Public and Personal Research routes cannot open them. Retained position, cost-basis, disposition, ranked-candidate, or action-like fields are not supported product claims. + These surfaces must show supported, blocked, partial, and excluded states before showing detailed tables. Broad-universe tables, command blocks, route maps, and proof ledgers should stay filtered, row-limited, collapsed, or operator-scoped by default. ## MVP Definition diff --git a/READINESS_MODEL.md b/READINESS_MODEL.md index c9ee9a7cb..8cfb242b9 100644 --- a/READINESS_MODEL.md +++ b/READINESS_MODEL.md @@ -37,6 +37,8 @@ The feature does not apply to this ticker or asset type. | `portfolio_ready` | holdings, prices, purpose metadata | holding row plus enough relevant data for review component | holding ticker parseable, position fields numeric when present | `portfolio_review.csv` | show review state | show missing components | mark holding blocked | not applicable | | `final_decision_ready` | central readiness report plus analysis outputs | enough data for the chosen decision bucket | blocked features reduce confidence | `research_decisions.csv`, `final_watchlist.csv` | show decision bucket and confidence | show reduced confidence | classify as Blocked by Data | classify as Excluded | +`momentum_ready`, `portfolio_ready`, and `final_decision_ready` remain compatibility-only readiness rows for historical pipelines. Their retained pages are labeled **Legacy research utility — not part of Personal Research Mode** and are available only in Operator mode behind a collapsed compatibility control. They cannot feed Research Decision Lab, cannot change readiness outside their own historical calculation contracts, and cannot produce recommendations, sizing, or transaction behavior. Personal Research and Public routes do not expose these rows as current product capability. + ## General Dashboard Rules - Show readiness before conclusions. diff --git a/README.md b/README.md index ce479a013..c71e6c43c 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,19 @@ # Stock Research Command Center -A local, CSV-first research dashboard for screening stocks, reviewing portfolio names, and seeing exactly which data is ready to support analysis. +An **Evidence-First Research Workbench** for serious individual equity researchers and small research teams performing repeated company research, valuation review, forward-scenario analysis, and truthful data-readiness decisions. > Data readiness first, analysis second, research decision last. -![Dashboard preview](docs/assets/public-demo-home-real.jpg) + ## External Reviewer Start Here + +![Company Workbench answer preview](docs/assets/linkedin-public-dashboard.png) + This repository is ready to review as a controlled GitHub/LinkedIn portfolio demo. It is not currently published as a hosted Streamlit app. +**Primary product workflow:** Research Desk -> Discover -> Company Workbench -> Monitor. Run `make dashboard`, then open `http://localhost:8501/`. Personal Research has one visible top workflow navigation; the sidebar selects the workspace but does not repeat the four page choices. Broad command, profile, and readiness chrome no longer precedes the route answer. Research Desk starts with one read-only **Today's Research Brief** answering what saved work needs attention, why, how fresh the saved readiness is, and one Monitor-or-Discover action. It does not claim to be a market-complete event feed; weekly, cohort, observation, coverage, and source-change detail remains under **Advanced Evidence**. Discover keeps strict screen eligibility separate from alphabetical saved-company evidence browsing: a company can be inspectable without passing the screen, and neither path is a recommendation or expected-return ranking. Company Workbench starts with one Company Brief answering **Use now**, **Still withheld**, **What changed**, and **Next research task**; trend, valuation, scenarios, authoring, methodology, and the offline HTML brief stay closed until **Open evidence and analysis modules** is selected. Monitor uses one read-only **Follow-up Queue** for **Since last review**, **Needs verification**, **Waiting on evidence**, **Scheduled context**, and **Evidence freshness**. A zero queue appears once, does not claim that no external event exists, and returns to Discover; complete process identities and source-change evidence remain under **Advanced: Monitor evidence**. Data Health and Proof History stay under **Advanced Evidence**. See [Personal Research Mode](docs/PERSONAL_RESEARCH_MODE.md) for workflow states and truth boundaries. + +**Secondary controlled demo:** Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History. This is the shorter public-review path for visitors who do not need the complete company-research workspace. + +This is a **local Commercial Research Beta foundation**, not a hosted or commercially launched product. Operator context remains the source/proof workspace. Authentication, private workspaces, operated data rights, real beta users, and repeatable provider operations remain separate gates. A local contract pass does not prove hosting, licensed operation, or external validation. + | Question | Short answer | | --- | --- | | What should I open first? | Start with this README preview, then use `docs/PUBLIC_DEMO_WALKTHROUGH.md` for the five-page workflow. | @@ -19,21 +28,24 @@ This repository is ready to review as a controlled GitHub/LinkedIn portfolio dem | What proves current local readiness? | `make status-check TOP_N=5` remains the source for current local counts; screenshots are product evidence only. | | What should I not claim? | No hosted app yet, no open-source reuse, no investment advice, no broker integration, no auto-trading, and no screenshot-based data freshness proof. | -First review move: open Stock Selector, choose a ticker such as `NVDA`, read the Single-Stock Report answer, then open Data Health only when an input is blocked. The selected-profile trust strip follows [Data Profiles](docs/DATA_PROFILES.md) and shows source/readiness dates, identity, freshness, and matching counts. The local Change Monitor queues same-profile evidence changes as research tasks, never recommendations. The append-only Research Thesis Journal preserves reviewer-authored hypotheses, evidence, invalidation conditions, confidence history, and review dates without changing readiness. The session-local Scenario Lab varies bounded DCF assumptions without writing canonical data. The Source Freshness Timeline separates effective, retrieval, market, and report times. The operator-only Research Comparison View compares two or three selected tickers across evidence availability and reviewed journal context without a score or winner. The Peer Read-Through Map keeps candidate and trusted relationships separate and withholds contextual read-through until relationship, result, and fiscal-timing evidence are explicit. The Decision-Process Scorecard reviews documentation discipline without grading the company or measuring returns; see [Methodology](docs/METHODOLOGY.md). +First personal-research move: open Research Desk for the deterministic focused cohort and weekly summary, use Discover to choose a ticker such as `NVDA`, and open Company Workbench. Read the Company Brief first; open the detailed modules only when trend, valuation, scenarios, authoring, methodology, or raw evidence is needed. The detailed layer preserves source-backed quarterly Revenue/EPS trend, valuation, forward context, withheld inputs, conclusion, and research tools without creating a second primary task. Missing canonical quarterly rows fail closed, Q4 is never derived, and the cohort order is reviewability rather than expected return. Use Monitor for source-backed changes; open Data Health only when a missing input is the question. Supporting research tools remain secondary to that answer flow. The selected-profile trust strip follows [Data Profiles](docs/DATA_PROFILES.md). The append-only Research Thesis Journal, session-local Scenario Lab, Source Freshness Timeline, operator-only Research Comparison View, Peer Read-Through Map, and Decision-Process Scorecard preserve their existing evidence and no-ranking boundaries; see [Methodology](docs/METHODOLOGY.md). Thesis, evidence, catalyst, and outcome records are all available in the collapsed Company Workbench composer. The composer appears only after the explicit detail action. A valid record requires an exact preview and explicit confirmation before save. Drafts are untrusted and preview receipts are session-only. Production tests never append repository ledgers; persistence tests use temporary ledgers. A saved record cannot change readiness, forecasts, probabilities, recommendations, or any other ledger. Priority 4's local validator is frozen; its permitted real-data exit gate remains externally incomplete. Priority 6's provider-neutral authorization contract is complete locally; hosted implementation remains environment-dependent. If the current branch head lacks direct local matrix, branch synchronization, draft-PR update, or exact-head CI evidence, complete those release steps first; otherwise select the first incomplete safe roadmap priority. Broad-review repairs must be evaluated only through direct current-head local and exact-head CI evidence; their presence alone establishes neither gate. Final integrity commit `e3a090dba` ensures confirmation appends only the receipt-matched recomputed record and enforces one readable active thesis lineage: revisions must supersede the exact active entry and preserve its thesis ID. The Company Workbench locks and explains that relationship, with temporary-ledger create -> revise -> reload coverage. Confirmation-integrity commit `5a6c55921` binds every displayed preview field, preview time, and destination label to the exact receipt. If an append raises after it may have written, confirmation returns one-shot `save_pending_reload` with the exact record ID unless the locked ledger is provably unchanged; it never invites a blind duplicate retry. + +Older Monthly Picks, Momentum Leaders, Portfolio Review, Value / Re-rating, and Final Watchlist views are retained only behind the Operator boundary **Legacy research utility — not part of Personal Research Mode**. They are compatibility and regression aids, not supported investing features. They cannot feed Research Decision Lab, cannot change readiness, and cannot produce recommendations, sizing, or transaction behavior. Public and Personal Research deep links fail closed to their safe start pages; legacy details require an explicit collapsed Operator control. +First review move: open Stock Selector, choose a ticker such as `NVDA`, read the Single-Stock Report answer, then open Data Health only when an input is blocked. ## What It Does This project turns a broad stock universe into a readiness-first research dashboard. It checks market data before analysis, separates `Research Now`, `Monitor`, and `Blocked by Data` review states, explains missing prices, fundamentals, DCF inputs, peers, earnings, and analyst estimates, and produces Streamlit pages plus single-stock reports with At A Glance status, a plain-English Reader Guide, an Evaluation Snapshot, a Proof Checklist, Best Review Path, data-confidence cues, source readiness notes, and read-only proof steps. - ### Earnings Nowcast Pilot The repository includes a readiness-gated, deterministic synthetic-fixture workflow for quarterly Revenue/EPS ranges and consensus-relative classification. It records fiscal period, forecast cutoff, expected report date, forecast horizon, provenance, model version, input hash, freshness, metric definitions, and withheld states; peer/news signals are directional evidence only and cannot change forecast numbers. Duplicate fiscal periods cannot inflate history, unresolved revisions fail closed per metric, and Revenue/EPS definitions must match on currency, unit scale, accounting basis, share basis, operations basis, and split treatment. Run `make earnings-nowcast-walkthrough` for six clearly synthetic reviewer scenarios, or `FIXTURE=1 make earnings-nowcast-pilot TICKER=SYN1 AS_OF=2026-01-31T23:59:59Z` to inspect one offline packet. Read-only templates, validation, preview, readiness, and prospective collection planning are documented in [Earnings Nowcast Pilot](docs/EARNINGS_NOWCAST_PILOT.md); there is no automatic apply path. - -This infrastructure does not establish real-company coverage or predictive accuracy. Real semiconductor output remains blocked until append-only point-in-time consensus and quarterly actual histories are source-backed. Numerical Beat/Miss probability is withheld until at least 100 leakage-safe out-of-sample events pass calibration and benchmark gates. The pilot does not predict post-earnings price movement and remains research-only, not investment advice. +This infrastructure does not establish real-company coverage or predictive accuracy. Real semiconductor output remains blocked until append-only point-in-time consensus and quarterly actual histories are source-backed. Numerical Beat/Miss probability is withheld until at least 100 leakage-safe out-of-sample events pass calibration and benchmark gates. The pilot does not predict post-earnings price movement and remains research-only, not investment advice. The activation layer adds a five-company readiness board, prospective consensus collection, point-in-time valuation context, outcome learning, and catalyst evidence inside the existing research workflow; see the pilot and methodology docs for their fail-closed contracts. Stage A is prospective-only: `make prospective-field-proof-status`, read-only `make prospective-field-proof-audit`, `make prospective-field-proof-preview INPUT= AS_OF=`, and explicit `make prospective-field-proof-record INPUT= AS_OF= PREVIEW_RECEIPT= CONFIRM_REVIEWED=1` preserve `technical_write_eligible` and `commercial_evidence_eligible` independently. The preview receipt binds ledger, input, cutoff, commercial mode, and source-rights registry. Audit exposes append history and active-head blockers with `preview_receipt_persisted=false` and `receipt_revalidation_required=true`; it does not activate readiness, update canonical data, or activate Company Workbench. An absent ledger is a valid empty state, and legacy narrative proof is not upgraded. No sample field-proof rows are checked in. Detailed operation and locking limits are in [Local Workflow Guide](docs/OPERATOR_GUIDE.md). ```mermaid flowchart LR - Home["Home: workflow start"] --> Selector["Stock Selector: readiness-backed queue"] - Selector --> Report["Single-Stock Report: one ticker"] - Report --> Health["Data Health: missing input"] - Health --> Proof["Proof History: source-proof trail"] + Desk["Research Desk: changed evidence"] --> Discover["Discover: strict eligibility or saved evidence"] + Discover --> Workbench["Company Workbench: Company Brief first"] + Workbench --> Monitor["Monitor: unresolved research changes"] + Workbench -. advanced evidence .-> Health["Data Health and Proof History"] ``` +### Download HTML Research Brief +Company Workbench can prepare **Download HTML Research Brief** from existing saved evidence and Python scenario math already shown in the selected research session. The download is an immutable, offline, research-only review snapshot: missing, partial, stale, mismatched, or unsupported fields remain independently labelled or withheld instead of being inferred. It does not refresh data or acquire a new source, change readiness, create a recommendation, or add a second valuation engine. No repository HTML or PDF artifact is written; the browser receives deterministic UTF-8 download bytes only. Modal modifiers and active exposure fail closed. Broad-review repairs must be evaluated only through direct current-head local and exact-head CI evidence; their presence alone establishes neither gate. Local engineering evidence does not establish source rights, current-market data, readiness activation, a new or professional line-item model, hosted operation, human or screen-reader conformance, independent validation, market fit, screening alpha, or probability calibration. ## Now / Next / Not Yet This is the fastest reviewer answer: the product is shareable as a controlled demo now, deeper coverage is source-gated, and hosting/provider automation stays optional until verified. | Stage | Answer | Guardrail | @@ -41,22 +53,8 @@ This is the fastest reviewer answer: the product is shareable as a controlled de | Now | GitHub/LinkedIn portfolio demo with public workflow, screenshots, methodology, local run commands, manual gates, and a locally passed performance gate. | Use `make public-check` before sharing; keep generated churn excluded and do not treat local timing as hosted proof. | | Next | Optional controlled hosted preview and task-based external pilot review. | Hosting remains external until a URL is verified; reviewer feedback must remain anonymous workflow evidence, not investment opinion. | | Not yet | Full hosted data product, complete fundamentals/peer/optional coverage, or provider-backed automation across the universe. | Do not claim this until external hosting, provider keys, source proof, validation, preview, apply, rebuilt readiness, and proof history support it. | -## Current Next Stages -Use this table to decide what to do next without reopening exhausted proof loops or overstating the public demo. -| Stage | Current state | Next safe move | -| --- | --- | --- | -| Performance release candidate | Passed locally on the fixed demo profile | Re-run `make public-performance-gate` for current route evidence; do not mix measurements with broad data refreshes or treat local timing as hosted proof. | -| LinkedIn publish | Ready after GitHub sync | If the branch is ahead, push reviewed commits after `make public-check`; if GitHub is synced, use the GitHub link and `docs/LINKEDIN_PROJECT_BRIEF.md`; do not claim hosted app availability. | -| Hosted Streamlit demo | External account required | Run `make hosted-demo-readiness`, then follow `docs/HOSTED_DEMO_DEPLOYMENT.md`; keep GitHub as the public link until the hosted route is verified. | -| FMP provider activation | External key required | Configure `FMP_API_KEY` outside the repo, then run one reviewed ticker smoke before any broader batch. | -| Peer readiness upgrade | Source-gated | Keep candidate peers as context only until source-backed peer rows pass review. | -| Optional earnings / estimates | Locked | Use trusted provider or reviewed manual rows only; do not infer optional context. | -| Earnings Nowcast evidence | Real-data safety gates ready; real evidence blocked | Use the versioned templates and prospective plan; add no real output until comparable quarterly actuals and point-in-time consensus pass conflict, cutoff, validation, and readiness gates. | -| Broad proof queues | Do not retry now | Current queues are exhausted; reopen only after keyed provider rows, reviewed manual rows, or changed blockers exist. | -| Public UX polish | Review limited | Public checks and repeated local cold/warm route timings pass; external reviewer evidence is still required before upgrading the claim. | -| Generated artifacts | Excluded by default | Keep local CSV/report/sample-report churn unstaged unless one exact artifact is reviewed as public evidence. | ## What You Can Analyze -When trusted local data is available, the product can produce price, momentum, benchmark-relative return, drawdown, volatility, beta, Sharpe/Sortino review metrics, liquidity, market-direction context, portfolio purpose checks, thesis-review flags, DCF readiness, conservative scenario valuation, source-backed peer context, ETF/index monitor reports, and single-stock reports with reader guidance, proof checklists, blockers, read-only proof steps, and source readiness notes. Most blocked rows are not errors. They are data gaps the command center exposes instead of hiding. +When trusted local data is available, the supported workflow can produce price and benchmark context, drawdown, volatility, beta, Sharpe/Sortino review metrics, liquidity, market-direction context, thesis-review flags, DCF readiness, conservative scenario valuation, source-backed peer context, ETF/index monitor reports, and single-stock reports with reader guidance, proof checklists, blockers, read-only proof steps, and source readiness notes. Historical ranked, picks, portfolio, and final-watchlist calculations remain operator-only compatibility utilities. Most blocked rows are not errors. They are data gaps the command center exposes instead of hiding. ## How Analysis Works The report is not a black box: local data rows provide inputs, and project rules decide what can be analyzed. Price-ready rows can support setup/risk context and benchmark/risk review metrics, DCF-ready rows can support assumptions and sensitivity, and peer-ready rows can support source-backed relative context. Missing fundamentals, peer inputs, earnings, or estimates stay locked; company valuation is excluded for ETF/index/fund monitor rows, not failed. ## Current Snapshot @@ -81,7 +79,7 @@ Use this as the short GitHub/LinkedIn review path before reading operator detail | Shares outstanding proof | Use `make share-count-proof-queue TOP_N=10` when DCF is blocked specifically by `shares_outstanding`. | Share count must come from SEC/manual source proof or trusted local rows; the product does not infer it from price, market cap, or peers. | | Peers | Use `DRY_RUN=1 make peer-batch-proof TOP_N=10` and `docs/TRUSTED_PEER_PILOT_SOURCE_TEMPLATE.csv` to collect reviewed 25-50 company source rows outside the import file; use `make peer-mapping-writeback-guard ...` before copy/paste, and use the ranked pilot packet first when a peer-input lane leads. | Peer trend and peer valuation stay separate; guessed peers or file row counts do not become valuation, and candidate context stays out of trusted proof as `candidate_context_only` until source-backed proof passes. | | Earnings / estimates | Keep locked until trusted local rows exist. | Empty optional context is intentional, not a broken chart. | -Pilot packaging is read-only first: `make pilot-readiness-check TOP_N=10` checks sync, hygiene, freshness, source-proof queues, proof ledger, screenshot evidence, public-check, and guardrails; `make pilot-share-brief` writes the concise public/demo share brief at `outputs/pilot_share_brief.md`, which does not refresh data or unlock blocked inputs. Use `make pilot-readiness-packet` only when a reviewer needs the fuller packet. +Pilot packaging is read-only first, but not entirely read-only: start with `make pilot-readiness-check TOP_N=10`, which checks sync, hygiene, freshness, source-proof queues, proof ledger, screenshot evidence, public-check, and guardrails. `make pilot-share-brief` writes the concise public/demo share brief at `outputs/pilot_share_brief.md`; `make pilot-readiness-packet` writes `outputs/pilot_readiness_packet.md` for a fuller reviewer packet. `make pilot-readiness-packet` is not read-only. The generated packet does not refresh data or unlock blocked inputs. When proof queues are exhausted, use `make project-status-check` and then `make provider-setup-checklist`. Provider setup is only an activation boundary: it can activate a source, but readiness changes still require validate, preview, rejected-row review, source provenance, apply/skip decision, rebuilt readiness, and proof ledger evidence. No broad coverage batch should run from setup alone. Do not retry exhausted proof queues until new source-backed rows, keyed provider data, reviewed manual rows, or changed blockers exist. Operator runbooks live outside the first-review README: use `docs/OPERATOR_GUIDE.md` for reviewed batch execution, `docs/DATA_STRATEGY.md` for lane mechanics, and `docs/SOURCE_ACTIVATION_GUIDE.md` for provider/source setup. Before turning any refresh path into a recurring job, run `make scheduler-activation-checklist`. Scheduler maturity starts as status-only monitoring; mutating refresh or apply paths stay off until provider smoke or source proof, validation, preview, zero rejected rows, provenance, no-fabrication checks, rebuilt readiness, proof history, and proof recording pass. @@ -90,7 +88,15 @@ This is a working local research prototype with deterministic outputs, dashboard Useful with limits: price/momentum, fundamentals/DCF, peer review, and final decision buckets when trusted local data exists. Intentionally locked: broad-universe fundamentals, peer comparison, earnings, and analyst estimates until trusted rows are imported. Not built to be: a full-market data vendor, real-time recommendation service, broker/execution system, or auto-refreshing trading system. ## Product Tour -Start with the five public paths the dashboard is built around: +For repeated local use, start with the four Personal Research destinations: +| Path | Use it when | First answer | +| --- | --- | --- | +| Research Desk | You want one saved-evidence briefing before choosing where to work. | What saved work needs attention today, why, and where do I go next? | +| Discover | You want to separate strict screen eligibility from saved-company evidence access without a buy ranking. | Which company is inspectable, and which qualifies when all screen evidence is ready? | +| Company Workbench | You want one company answer spanning data usability, business trend, valuation, forward context, uncertainty, and next review work. | What can I use now, and what remains withheld? | +| Monitor | You want unresolved source-backed changes and wait conditions. | Which evidence change needs review? | + +The controlled Public workspace keeps its existing five-page path: | Path | Use it when | First place to open | | --- | --- | --- | | Home | You want the workflow question, next safe action, stop rule, and then readiness context before choosing a route. | `Home` | @@ -98,14 +104,14 @@ Start with the five public paths the dashboard is built around: | Single-Stock Report | You want a ticker-level research note with ready, blocked, excluded, and data-confidence states. | `Single-Stock Report` | | Data Health | You want to understand what trusted input is missing and which proof path should be reviewed next. | `Data Health` | | Proof History | You want one evidence answer before opening raw proof ledger details. | `Proof History` | -The dashboard starts in public visitor mode at `http://localhost:8501/?mode=public`. +The dashboard starts in Personal Research mode at `http://localhost:8501/`. Public review remains available at `http://localhost:8501/?mode=public`. - Home answers what the product is, where to start, and when to stop. - Stock Selector filters readiness-backed candidates without framing the queue as advice. - Single-Stock Report shows selected-ticker readiness, usable sections, blocked inputs, and one next step before detailed report sections. - Data Health starts with Coverage Summary / What Can I Use, one answer per lane, and advanced proof drawers collapsed. - Proof History is evidence-only before trusting a changed readiness state. -Switch off Public visitor mode only for internal Operator context, detailed boards, local proof commands, and validate / preview / apply guidance. Advanced pages remain secondary, and watchlist-style outputs stay readiness-state output, not an action list. +Choose Operator only for detailed boards, local proof commands, and validate / preview / apply guidance. Advanced pages remain secondary, and watchlist-style outputs stay readiness-state output, not an action list. ## Quick Start Run these from the repository root so `make` can find the project targets. Open the product before proof packets or report commands so reviewers see the guided workflow before operator detail. @@ -116,13 +122,13 @@ make demo # print the safe visitor path without changing local data make demo-dashboard # open the compact tracked profile at http://localhost:8501/?mode=public ``` -Optional read-only proof after the app flow is clear: `make status-check TOP_N=5`, `make pilot-readiness-check TOP_N=10 && make pilot-readiness-packet`, and `make stock-report-md TICKER=NVDA`. +Optional read-only proof after the app flow is clear starts with `make status-check TOP_N=5` and `make pilot-readiness-check TOP_N=10`; `make pilot-readiness-packet` and `make stock-report-md TICKER=NVDA` intentionally write their documented Markdown outputs. For the mutable default operator workspace, use `make dashboard`; it is intentionally separate from the public demo profile and can reflect local refresh/import work. When you want to run a controlled pilot, use the [Pilot Runbook](docs/PILOT_RUNBOOK.md). When you want to rebuild local outputs after changing data, use the deeper [Local Workflow Guide](docs/OPERATOR_GUIDE.md) for rebuild, import, refresh, and proof steps. -For 5-10 external reviewer sessions, run `make pilot-review-feedback` and use [Controlled Pilot Review Feedback](docs/PILOT_REVIEW_FEEDBACK_TEMPLATE.md) plus the structured [feedback log template](docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv). Then run `make pilot-feedback-closeout` and follow the [Pilot Feedback Closeout Checklist](docs/PILOT_FEEDBACK_CLOSEOUT_CHECKLIST.md) to classify each row as `clear`, `reproducible_ui_issue`, `documentation_gap`, `environment_limited`, or `intentionally_deferred`. Capture route clarity and reproducible UX issues only; keep the working log outside Git until it is anonymized and intentionally reviewed; feedback does not prove data freshness, source readiness, investment conclusions, or coverage completion. +For 10-20 external reviewer sessions, run `make pilot-review-feedback` and use [Controlled Pilot Review Feedback](docs/PILOT_REVIEW_FEEDBACK_TEMPLATE.md) plus the structured [feedback log template](docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv). Then run `make pilot-feedback-closeout` and follow the [Pilot Feedback Closeout Checklist](docs/PILOT_FEEDBACK_CLOSEOUT_CHECKLIST.md) to classify each row as `clear`, `reproducible_ui_issue`, `documentation_gap`, `environment_limited`, or `intentionally_deferred`. Capture route clarity and reproducible UX issues only; keep the working log outside Git until it is anonymized and intentionally reviewed; feedback does not prove data freshness, source readiness, investment conclusions, or coverage completion. ## Try This Visitor Workflow Open the product first and follow the five-page path. Use terminal commands only when you want to inspect the same proof artifacts locally. @@ -166,14 +172,8 @@ This repository is shared under a controlled portfolio-demo license. Visitors ma The stock-analysis method is implemented in this repository: readiness gates, momentum rules, DCF assumptions, relative-valuation checks, peer readiness, and report wording live under `src/`. Standard Python packages support data handling and UI; optional `yfinance` is an unofficial research-grade adapter, and configured FMP/Alpha Vantage/Finnhub keys can serve as research-grade fallback sources for price and fundamentals staging. The analysis rules, valuation gates, decision buckets, and research-only guardrails come from project code plus local CSV inputs. Fundamentals-ready means trusted company fields can be reviewed, DCF-ready means scenario math can be reviewed, and peer-ready means source-backed relative context can be reviewed. See [Research Methodology](docs/METHODOLOGY.md) for the calculation flow and [Analysis Capability Audit](docs/analysis_capability_audit.md) for what is strong today, what remains limited, and where the method lives. ## Core Outputs -The main build creates deterministic research files under `outputs/`, including purpose classification, market direction, momentum leaders, portfolio review, valuation-readiness context, final watchlist, and research decisions. `undervalued_candidates.csv` is a legacy filename for valuation-readiness and re-rating context, not automatic undervalued calls. Readiness and source-health reports live under `data/reports/`. +The main build retains deterministic historical files under `outputs/`, including purpose classification, market direction, momentum leaders, portfolio review, valuation-readiness context, final watchlist, and research decisions. Momentum, portfolio, value/re-rating, and final-watchlist files are compatibility outputs rather than current Personal Research capabilities. `undervalued_candidates.csv` is a legacy filename for valuation-readiness and re-rating context, not automatic undervalued calls. Readiness and source-health reports live under `data/reports/`. ## Research-Only Guardrails This is investment research software, not investment advice and not a trading system. It does not place orders, connect to brokers, route trades, auto-trade, recommend option trades, provide direct buy/sell instructions, or fabricate prices, fundamentals, peers, earnings, analyst estimates, valuation inputs, or recommendations. That constraint is intentional. The product is useful because it says when data is missing instead of pretending every ticker is ready. - -## Architecture -The app is organized around dashboard, readiness, decision, report, provider, local-data, and test modules. It is CSV-first and deterministic by default. Optional network-backed data stays behind provider interfaces and is labeled as research-grade when used. - -## Roadmap Snapshot -The current public workflow is intentionally guided: Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History. The local fixed-demo performance gate has passed and remains a regression check. The active evidence stage is a narrow, append-only Earnings Nowcast pilot using comparable quarterly actuals and historical point-in-time consensus snapshots; hosting and external review remain separate external stages. Fundamentals/DCF, source-backed peers, optional earnings/estimate context, and real nowcast output stay locked until trusted rows prove readiness. diff --git a/ROADMAP.md b/ROADMAP.md index 7eeae4f2d..148732f61 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,259 +1,319 @@ # Roadmap -Stock Research Command Center follows one principle: **data readiness first, analysis second, research decision last**. It is research-only software: no investment advice, broker trading, order routing, auto-trading, direct buy/sell instructions, or fabricated data. +Stock Research Command Center is an **Evidence-First Research Workbench** for serious individual equity researchers and small research teams. Its operating rule is **data readiness first, analysis second, research decision last**. -This is the active plan only. Completed delivery history lives in [Completed Milestones](docs/COMPLETED_MILESTONES.md). +It is research-only: no investment advice, recommendation, company ranking, broker integration, order routing, auto-trading, direct buy/sell instruction, allocation, position sizing, stop-loss, take-profit, or fabricated evidence. The active roadmap is this file; detailed delivery history lives in [Completed Milestones](docs/COMPLETED_MILESTONES.md), accessibility findings in [Accessibility Evidence](docs/ACCESSIBILITY_EVIDENCE.md), point-in-time-universe remediation in [Point-in-Time Universe Review History](docs/internal/POINT_IN_TIME_UNIVERSE_REVIEW_HISTORY.md), and persistent execution boundaries in the [Commercial Research Beta Continuation Contract](docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md). ## Current Truth -Use live, read-only commands instead of static counts: - - Master universe rows: use `make project-status` or `make status-check TOP_N=5`. - Active research rows: use `make project-status` or the dashboard Home page. - Lane readiness: use `make readiness-ops-center`. +- Historical proof versus current readiness: use `make proof-readiness-reconciliation TOP_N=20`. It is a current-snapshot audit; historical proof cannot promote current readiness. +- Readiness evidence impact: use `make readiness-preview TOP_N=20`. It computes future proposed states in memory, writes no files, and does not authorize staging or a readiness rebuild. - Source/provider state: use `make session-source-preflight` and `make provider-setup-checklist`. -- Package/share state: use `make pilot-readiness-check TOP_N=10` and `make public-check`. +- Package/share state: use `make pilot-readiness-check TOP_N=10`, `make public-check`, and `make browser-qa-evidence`. +- Commercial-beta state: use `make commercial-beta-check`, `make commercial-beta-performance-gate`, and `make commercial-beta-release-check`. Commit `6328c8cea` passed the category-separated 48-sample browser gate, the aggregate release check, and exact-head CI. Shell and first-useful evidence keep warm p90 separate from cold maximum at the unchanged one-second and three-second limits. These are local evidence only; they do not refresh data and do not prove market validation. +- Point-in-time universe software: only `make point-in-time-universe-status MANIFEST=` and `make point-in-time-universe-preview MANIFEST= TOP_N=20` are supported. Both are read-only. The product deliberately separates the tracked master universe, active universe, and analysis-ready subset. It must never imply that the whole tracked universe is analysis-ready. +The tracked June 7 readiness snapshot remains stale under this roadmap's declared-date policy. An excluded July 21 local generated working-data snapshot is date-current relative to the saved local source files but is not committed PR evidence. The product now reports those two states independently: `freshness_state=current` and `readiness_evidence_state=working_artifact_uncommitted`, while the continuation and reviewed-batch gates remain inspection-only. A read-only preview reported zero stable readiness changes, but that observation does not authorize staging or a readiness rebuild. Rerun the current commands instead of copying counts into documentation. + +`historical_supported_currently_blocked`, `explicit_ticker_change`, and `current_canonical_row_missing` are distinct reconciliation outcomes. Current saved readiness remains authoritative, and reconciliation does not establish the historical cause of a transition, restore canonical data, or report current readiness totals. + +The readiness continuation gate follows declared source dates, never file mtimes, and keeps date freshness independent from tracked evidence eligibility. It routes stale, incomplete, unverified, or current but untracked readiness to inspection-only. A date-current but uncommitted default-profile readiness snapshot cannot become PR evidence through local generated files. `make readiness-preview TOP_N=20` does not make saved readiness current or reviewable, does not refresh data, and is not current readiness counts or rebuild approval; provider ordering and ranking output remain planning context only until a separate intentional reviewed write runs `make readiness` and its exact artifacts are reviewed. + +Primary research flow: **Research Desk -> Discover -> Company Workbench -> Monitor**. + Public visitor flow: **Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History**. -## Completed Regression Gate +Operator source/proof work remains separate. Data Health and Proof History are Advanced Evidence in Personal Research mode. Empty ledgers stay empty; candidate context stays untrusted. -### P0: Profile Truth And Local Research Change Workflow +Data Health and Proof History stay inside Personal Research mode and retain `Return to Company Workbench`; the detour does not change readiness. The direct-open loading state preserves Selected ticker -> `Use now` -> `Still withheld` -> `Open Data Health`, a 44px primary action, at least 50px result rows, no horizontal overflow, no traceback, and the desktop four-column layout. Owner-approved responsive repair anchor `18417d90766e9aeb629eef6bb1ef4fc40805773c` has fresh zero-scroll `resolved_post_fix` evidence at `390x844`: Home `stop_bottom=843.609375`, `action_height=44`, and `visible_stops=1`; Single-Stock Report `stop_bottom=836.421875`, `action_height=44`, and `trust_gap=2.21875`; both recorded `scroll_width=390`, all four measured scroll offsets were zero, no horizontal overflow, no traceback, and zero open Advanced details. Home phone order is primary -> stop -> metrics in both source and visual order. At `1280x720`, desktop retained two Home grid tracks and four Single-Stock Report grid tracks, one breakpoint-appropriate Home stop, 44px actions, and zero scroll/overflow/runtime errors. The historical `stop_bottom=886.3125` failure and the earlier unscrolled `843.4296875` claim remain superseded evidence, not current completion proof. Phone first-action density, Advanced data health cards, Auto-refresh status, and Session Source Preflight are workflow evidence only. This repair does not prove market validation, does not change readiness, source, research, or generated-artifact state, and does not prove hosted behavior, accessibility conformance, reviewer validation, demand, or product-market fit. -**Status:** implemented and locally verified on 2026-07-15. +Stage A-G labels are continuation maturity lanes only; they do not replace the numbered Stage 0-6 exit gates. Stage B — local field-proof audit and operator hardening is complete locally. A blocked priority does not become complete; classify it once and move to the next safe executable priority. +## Now: Commercial Research Beta Foundation -Every dashboard and status surface uses one selected-profile context for source date, readiness build time, snapshot identity, freshness, and matching coverage counts. Generated comparable snapshots support deterministic filing, readiness, price-history, fundamentals/share-count, and Nowcast-consensus change events. The derived review queue prioritizes unresolved research work, while append-only review outcomes remain separate from readiness mutation. +**Current product stage:** local Commercial Research Beta release candidate and controlled portfolio/demo package, not a hosted or commercially launched product. -Use `make profile-context`, `make research-change-snapshot`, `make research-change-monitor`, and `make research-review-queue`. Generated snapshots and event previews stay unstaged. A missing baseline means no comparison is available; it never means no changes occurred. +**Positioning:** a maintained evidence-first research tool candidate, not a smaller Bloomberg, Koyfin, TIKR, Fiscal.ai, AlphaSense, Quartr, or QuantConnect. -**Boundary:** local monitoring is read-only except for the explicit reviewed-resolution append. Hosted alerts, scheduled snapshot rotation, and notification delivery remain Later and require operating evidence. +**What works locally** -### P0: Research Thesis And Evidence Journal +- Research Desk now composes one read-only **Today's Research Brief** from the saved weekly summary, source-change state, and readiness freshness. It answers what saved work needs attention, gives one traceable reason, and routes once to Monitor or Discover. Weekly, cohort, observation, coverage, and source-change detail remains under Advanced Evidence. A zero-item answer is scoped to loaded workspace evidence and explicitly does not claim that no external event exists. Discover separates the unchanged strict **Screen eligibility — when supported** contract from alphabetical **Browse saved companies** evidence access. Saved rows explain inspectability, usable evidence, and the main gap; they are not screened opportunities, rankings, expected-return claims, or recommendations. The research browse path reads focused-cohort ticker readiness and never reads legacy decision, ranking, or watchlist outputs. Company Workbench composes trends, valuation, forward scenarios, evidence, and authoring. Monitor composes one **Follow-up Queue** with Since last review, Needs verification, Waiting on evidence, Scheduled context, and Evidence freshness. It renders one fail-closed empty state and one Discover action when no saved actionable item is due; complete process identities and source-change rows remain under `Advanced: Monitor evidence`. +- Research Decision Lab supports read-only composition plus append-only thesis, counter-thesis, evidence, catalyst, invalidation, scenario-assumption, and outcome records through validate -> preview -> explicit confirm. +- SEC quarterly actuals preserve source lineage. EPS split basis remains unverified without explicit proof. Q4 actuals require an explicit SEC-filed Q4 table; Q4 is never derived. +- Historical Valuation Regime, Source Freshness Timeline - Implemented, Research Comparison View - Implemented, Peer Read-Through Map, Scenario Lab, Research Outcome Review, and Catalyst Evidence Timeline stay fail-closed when their ledgers or source inputs are empty. Historical-valuation numeric loading also rejects blank or malformed numerator/denominator evidence per row instead of coercing it to zero or discarding valid sibling rows. +- Calculation software supports price setup, drawdown, volatility, beta, Sharpe/Sortino, DCF/scenarios, valuation context, deterministic nowcast contracts, walk-forward review, and point-in-time-universe validation. +- Quant interpretation eligibility is implemented locally at `195ea18da9d1d6e06c36f8320509ccde46cdaa57`: one shared, fail-closed overlay keeps valuation, indicator, and review/risk calculations and readiness independent while classifying their interpretation as current context, historical/review-only, or withheld. +- Company Workbench preserves peer trend and peer valuation readiness independently: mapped price history may support trend context, but only `peer_valuation_comparison_ready` can unlock peer-relative valuation, premium/discount, peer DCF comparison, or route the next proof beyond the peers lane. +- Commercial source-rights, refresh-operation, provider-neutral authorization, workspace-isolation, audit-obligation, retention, monitoring, incident, and rollback contracts are locally testable. Local contracts do not prove hosted operation. +- Public/package/release gates and current-head automation verify code, wording, route rendering, hygiene, and research boundaries. Automated evidence is not independent human review. -**Status:** implemented and locally verified on 2026-07-15; retain as a research-process regression gate. +**Current truthful limitations** -The selected-profile Single-Stock Report now shows one compact, reviewer-authored thesis answer with supporting and conflicting evidence, catalysts, risks, invalidation conditions, confidence history, and review dates. `data/research_thesis_journal.csv` is append-only. Thesis revisions preserve prior entries through `supersedes_entry_id`; generated thesis text and Change Monitor tasks never write journal rows automatically. +- Real semiconductor nowcast coverage remains `awaiting_point_in_time_consensus`. +- Numerical Beat/Miss probability remains `awaiting_calibration_evidence` and withheld until every predeclared gate passes. A raw calibration count cannot establish calibrated probability. `calibrated` requires paired verified calibration status and a leakage-safe benchmark-passing `BacktestReport`. The pairing requires recalculated immutable probability observations matched one-for-one to those report events by ticker, fiscal period, and cutoff. Each retained outcome must also equal the declared strict Revenue or EPS result (`actual > consensus`); an equal actual is not a Beat. The declared metric itself must be modeled and improve its matching consensus benchmark, and each stored relative classification is rederived from its forecast interval before aggregate evidence is accepted. A canonical digest also binds the exact report package, including event chronology, ordered source IDs, model/input identity, scored fields, exclusions, failures, summaries, and benchmarks; post-assessment relabelling requires a fresh assessment. This proves internal consistency only, not external source authenticity or rights. +- A saved artifact being current relative to local files does not render its latest market observation current. Current-market interpretation requires an independent observation-recency state. +- Local market-observation recency is implemented as a read-only, independently fail-closed check of the selected local `prices.csv` path. Its exact policy is seven calendar days from the dashboard review date, not an exchange-session SLA. Permitted market-data source rights and hosted freshness remain external gates. +- Structured external provenance and exact-source rights proof for the current local quant inputs remain absent, and the historical-valuation observation ledger is absent. The daily queue therefore truthfully yields no eligible real-company candidate from current repository inputs; the shared overlay leaves local quant results historical/review-only or withheld. Neither establishes a current-market claim, hosted operation, calibration, nowcast activation, or commercial completion. +- One permitted independently reviewed real point-in-time universe package, one permitted point-in-time consensus source, and one genuinely reviewed peer relationship are not on record. +- Independent beta sessions completed: zero. The local protocol is ready, but it is not user-validation or demand evidence. +### Company Workbench HTML Research Brief +Company Workbench HTML Research Brief — Historical pre-fix evidence: Task 4 local matrix completed at `c8c313b9c`. Modal modifiers and active exposure fail closed. Broad-review repairs must be evaluated only through direct current-head local and exact-head CI evidence; their presence alone establishes neither gate. Exact-head repair evidence: commit `b69badfc80424d3a97fae5f77706aa6ed1533167` passed the 5,828-test full suite, the required dashboard, render, HTML, accessibility, public, and hygiene gates, branch/PR synchronization, and exact-head GitHub Actions run `30726301045`. The brief downloads an immutable offline view of existing saved evidence and prepared Python scenario math, preserves independent field gates and research-only wording, writes no repository artifact, and does not activate readiness or create a new calculation engine. Pilot packaging remains blocked on readiness freshness and source proof. Source rights, current data, hosted operation, human and screen-reader accessibility, independent workflow sessions, screening validation, and probability calibration remain open gates. Local engineering evidence does not establish source rights, current-market data, readiness activation, a new or professional line-item model, hosted operation, human or screen-reader conformance, independent validation, market fit, screening alpha, or probability calibration. +## Next: Ordered Maturity Work -Use `make thesis-journal TICKER=` to read, `make thesis-journal-preview ...` to validate without writing, and `CONFIRM_REVIEWED=1 make thesis-journal-record ...` only after source review. Journal entries never mutate source rows, readiness, valuation, or Review Queue outcomes. +If the current branch head lacks direct local matrix, branch synchronization, draft-PR update, or exact-head CI evidence, complete those release steps first; otherwise select the first incomplete safe roadmap priority. If its next gate needs an unavailable source, account, environment, reviewer, or elapsed event history, classify it once under **Externally blocked** and continue to the next executable priority. Passing local tests never completes an external gate. Evidence publication, snapshot, and retrieval timestamps must all be at or before the cutoff. -**Boundary:** the journal documents a research process. Confidence is not investment conviction, expected return, position size, or a transaction instruction. +Documentation and routing reconciliation is complete locally. Reopen it only when current repository evidence reproduces contract drift. -### P0: Performance Release Candidate +Current answer-first UX program: Discover truth separation, the Company Workbench primary brief, Monitor consolidation, Research Desk simplification, and shared-shell cleanup are implemented locally. Research Desk replaces the weekly summary plus four overlapping answer cards with one Today's Research Brief and keeps supporting evidence under Advanced. Workbench composes **Use now**, **Still withheld**, **What changed**, one authoritative **Next research task**, the research-only stop rule, and the ticker-bound Data Health action before secondary modules. Trend, valuation, scenarios, authoring, methodology, the Research Decision Lab, conclusion detail, and the offline HTML brief remain available only after one session-local **Open evidence and analysis modules** action; reopening cannot change canonical data or readiness. Monitor replaces three competing primary summaries with one five-panel Follow-up Queue and one fail-closed zero-state action. Personal Research now has one visible top page-navigation authority; the sidebar retains workspace selection but no longer repeats Research page choices, and the main-column Operator command/readiness header plus broad profile strip no longer precede the route answer. Public and Operator shells, direct URLs, ticker parameters, Data Health, and Proof History remain unchanged. The clean direct-browser matrix at shared-shell implementation anchor `2bc374ab8` passed all 12 six-route/two-viewport cases after replacing two weak global-label waits with route-owned answer markers. It verified one labelled navigation where applicable, exact H1/main semantics, route preservation, focus/media-preference behavior, no overflow, no browser/runtime errors, and unchanged repository state while excluding the same 18 generated paths. Release credit attaches only after full current-head gates, an intentional branch synchronization, release-evidence reconciliation, and exact-head CI. Pilot readiness remains truthfully blocked on uncommitted working readiness evidence and incomplete source proof. This remains local engineering evidence, not hosted, independent-human, screen-reader, WCAG, or market validation. These presentation changes do not alter calculations, authoring persistence, readiness, evidence identities, thresholds, or source-rights decisions. -**Status:** passed locally on the fixed demo profile on 2026-07-14; retain as a release regression gate. +The bounded observation-recency UX repair is complete locally. Research Desk, Discover, and Monitor show one profile-lane interpretation; Company Workbench shows one selected-ticker interpretation. Exact selected/profile/SPY/QQQ dates, machine states, policy, path, and excluded-date diagnostics remain inside responsive Advanced evidence. A direct four-route browser matrix at `1280x720` and `390x844` verifies one summary, four independently labelled cards, phone single-column layout, and no evidence-container horizontal overflow without writing screenshots or other artifacts. -**Goal:** keep the guided public workflow fast enough that an external reviewer does not mistake loading for a broken page. +Completed local reliability item: default-profile readiness now separates temporal freshness from tracked release-evidence eligibility. Dirty or untracked required readiness reports are labelled `working_artifact_uncommitted`, and Git comparison failures are `unverified`; project status, continuation routing, pilot readiness, and reviewed-batch preflight fail closed to the no-write preview without staging or rewriting generated artifacts. Reviewed-batch temporal routing now uses the same declared-date profile context instead of a separate mtime rule. -Use the tracked `data/demo/manifest.json` snapshot as the fixed performance dataset. Do not mix route measurements with broad data refreshes or generated local-profile churn. +Completed local item: `1. Add shared provenance and recency eligibility to valuation, indicator, and review-metric interpretation without coupling their independent readiness states.` The shared overlay is implemented at +`195ea18da9d1d6e06c36f8320509ccde46cdaa57`; this historical queue entry is not +an open task, and the independent historical-valuation numeric-integrity repair +did not substitute for it. -1. Run `make public-performance-contract` to inspect the read-only route, viewport, snapshot, and threshold contract. -2. Run `make public-performance-gate` for real-browser cold and warm evidence at desktop and phone widths. -3. Measure the visible shell, first useful answer, and full settle separately; report repeated warm results as p90 rather than selecting the fastest run. -4. Treat Stock Selector, Single-Stock Report, and Data Health as critical routes. Keep Home and Proof History regression-protected. -5. Optimize saved summaries, deferred detail, pagination, and deterministic caching in small tested slices without weakening readiness or hiding blocked states. +Completed local framework reliability item: `1. Close the same-document +Streamlit transport reliability evidence.` Exact local commit +`d68ab27bee9c07c450faeb866b08cbf13638b56f` +passed 4,381 tests, the required render/public/commercial-beta/pilot gates, and +all 12 six-route/two-viewport browser results. Every result reported zero +deprecated-component warnings, bridge iframes, bridge focusable descendants, +and bridge height; the owned bounded server-output capture also reported zero +deprecated warnings. The same 18 generated paths remained excluded. -**Exit gate:** loading feedback within 1 second, first useful answer within 3 seconds, warm full-settle p90 within 5 seconds, and cold full settle within 10 seconds on the defined local reference environment. +Those historical transport and research-state anchors were subsequently covered by the exact-head local and GitHub release matrix at `b69badfc8`; their earlier environment and synchronization blockers are no longer active routing items. +This closes only automated engineering evidence. Hosted product behavior, independent-human accessibility, source evidence, and market validation remain separate incomplete gates. -**Stop rule:** a missing browser dependency is `environment_limited`, not a pass. Keep timing JSON and screenshots generated and unstaged unless one concise artifact is intentionally reviewed. +Priority 7 accessibility remains open. Select any remaining safe automated work only through the durable release-routing rule above; true zoom, forced colors, reduced motion, screen-reader, and independent-human evidence remain external until a suitable environment exists. -## External Stages +### Priority 1 — Legacy portfolio, ranking, and action-language quarantine -### P1: Controlled Hosted Preview Verification +Priority 1 — completed locally. Legacy portfolio, ranking, position, cost-basis, picks, entry-zone, disposition, and transaction-like surfaces are Operator-only compatibility utilities labelled `Legacy research utility — not part of Personal Research Mode`. Public and Personal Research routes fail closed. Retained compatibility code cannot feed Company Workbench, Research Decision Lab, readiness, recommendations, sizing, or transaction behavior. -**Goal:** turn the deterministic `demo` profile into a verified, controlled hosted demo without exposing local refresh data or credentials. +### Priority 2 — Stage B field-proof audit and operator hardening: completed locally -Repository-side preparation is complete. The remaining deployment work requires an external host/account and a verified public URL. +Stage B — completed locally. This is prospective-only; legacy narrative proof is not upgraded, and an absent ledger is a valid empty state. `technical_write_eligible` and `commercial_evidence_eligible` remain independent. `make prospective-field-proof-audit` and preview expose append history, current/superseded state, blockers, per-row technical/commercial answers, and receipt revalidation without writes. Stage B is the second approved local priority after legacy surface quarantine. There is no readiness mapping; any activation requires a separate design. Activation remains non-active and separately designed. No sample field-proof rows are checked in. -1. Choose a Streamlit-compatible host and deploy `main` with `dashboard.py` as the entrypoint. -2. Set `STOCK_RESEARCH_DATA_PROFILE=demo` in the host environment. -3. Keep provider keys, account IDs, tokens, and broker/session files out of the repo and public app. -4. Verify the five-page workflow on the hosted URL at desktop and mobile widths. -5. Set `HOSTED_DEMO_URL` locally only after the URL opens successfully, then rerun the public gates before changing GitHub or LinkedIn copy. +`make prospective-field-proof-status`, `make prospective-field-proof-preview INPUT= AS_OF=`, and `make prospective-field-proof-record INPUT= AS_OF= PREVIEW_RECEIPT= CONFIRM_REVIEWED=1` preserve `preview_receipt_persisted=false` and `receipt_revalidation_required=true`. The preview receipt binds ledger, input, cutoff, commercial mode, and source-rights registry. Audit and preview do not activate readiness; the audit does not update canonical data, does not update proof-readiness reconciliation, and does not activate Company Workbench. -**Dependencies:** the local performance release gate, an external hosting account, a public or access-controlled preview URL, and a human browser review of the deployed route. +### Priority 3 — In-app research-record authoring: completed locally -**Stop rule:** keep GitHub as the public link until the hosted route is verified. Call the route private only when access control is actually enforced. Screenshots remain product evidence only, never data-freshness proof. +Priority 3 — completed locally after direct desktop/phone runtime review and the required automated acceptance matrix. Thesis, evidence, catalyst, and outcome records are all available in the collapsed Company Workbench composer. A valid record requires an exact preview and explicit confirmation before save. Drafts are untrusted and preview receipts are session-only. Production tests never append repository ledgers; persistence tests use temporary ledgers. automated generation cannot become reviewer-authored evidence. A saved record cannot change readiness, forecasts, probabilities, recommendations, or any other ledger. -### P1: Controlled Pilot Review +Hardening commit `07758114c` closes the confirmation race: all three append engines share one resolved-ledger cooperative lock, receipts bind resolved ledger identity, every new preview resets confirmation, and uncertain post-append teardown requires one-shot read-side reload before success. -**Goal:** validate whether an external reviewer can understand the product in under three minutes. +Final integrity commit `e3a090dba` ensures confirmation appends only the receipt-matched recomputed record and enforces one readable active thesis lineage: revisions must supersede the exact active entry and preserve its thesis ID. The Company Workbench locks and explains that relationship, with temporary-ledger create -> revise -> reload coverage. -1. Share the GitHub demo package with 5-10 reviewers. -2. Ask reviewers to follow the public visitor flow without operator instructions. -3. Record only concrete issues: where they started, what they thought was usable, what looked blocked, and what they expected to do next. -4. Prioritize reproducible first-viewport, wording, routing, or accessibility defects. Do not use pilot feedback to weaken readiness gates. +Confirmation-integrity commit `5a6c55921` binds every displayed preview field, preview time, and destination label to the exact receipt. If an append raises after it may have written, confirmation returns one-shot `save_pending_reload` with the exact record ID unless the locked ledger is provably unchanged; it never invites a blind duplicate retry. -Use [Controlled Pilot Review Feedback](docs/PILOT_REVIEW_FEEDBACK_TEMPLATE.md) to capture anonymous, reproducible workflow observations without collecting personal, portfolio, or investment-opinion data. +Priority 4's local validator is frozen; its permitted real-data exit gate remains externally incomplete. Priority 6's provider-neutral authorization contract is complete locally; hosted implementation remains environment-dependent. If the current branch head lacks direct local matrix, branch synchronization, draft-PR update, or exact-head CI evidence, complete those release steps first; otherwise select the first incomplete safe roadmap priority. -**Dependencies:** a locally passing performance release gate, a verified delivery path, external reviewers, and controlled feedback collection. +### Priority 4 — Point-in-time benchmark and universe foundation -**Stop rule:** do not call pilot feedback data proof; it only validates product clarity and workflow reliability. +The approved design is `docs/superpowers/specs/2026-07-23-point-in-time-universe-foundation-design.md`. It remains isolated from the current ticker-centric universe. -## Now +Implemented locally: read-only immutable-package status/preview with ten independent states: manifest, technical, temporal, identity, membership, corporate action, delisting, source rights, reproduction, and leakage. Synthetic fixtures remain test-only and local software evidence cannot complete Priority 4. -### P2: Scenario Lab - Implemented +Priority 4 remains open until one bounded permitted real dataset is independently reviewed, reproduces the expected membership count and digest, and passes rights, identity, corporate-action, delisting, survivorship, cutoff, partition, reproduction, and leakage gates. The controls for corporate actions, delistings, survivorship, and leakage remain explicit and independent. + +This local evidence does not change independent readiness for actuals, consensus, Revenue, EPS, valuation, catalysts, outcomes, backtesting, or calibration. -**Goal:** let a reviewer vary source-backed DCF assumptions and understand valuation sensitivity without changing canonical data or producing a recommendation. +It does not provide investment advice; numerical probability remains unavailable without calibration; Q4 evidence and EPS split-basis compatibility remain explicit; synthetic evidence stays test-only; candidate peer evidence remains candidate-context-only. -1. Start only from a company whose selected profile is DCF-ready. -2. Load the saved source-backed revenue, FCF or margin, shares, cash, debt, and price context as immutable baseline evidence. -3. Allow bounded changes to revenue growth, operating or FCF margin, discount rate, terminal growth, and forecast horizon. -4. Show baseline and scenario ranges, directional sensitivity, terminal-value contribution, and every changed assumption. -5. Keep scenarios session-local or explicitly exported as generated research artifacts; never apply them to canonical fundamentals or readiness. +Digest method: `membership_count_and_sha256_at_cutoff_v1`. Supported commands are exactly `make point-in-time-universe-status MANIFEST=` and `make point-in-time-universe-preview MANIFEST= TOP_N=20`. -**Stop rule:** blocked or excluded DCF inputs produce no valuation output. Scenario results are assumption tests, never fair-value claims, rankings, or direct actions. +Resource limits and detailed remediation evidence live in `docs/internal/POINT_IN_TIME_UNIVERSE_REVIEW_HISTORY.md`. Priority 4 remains externally incomplete. -**Implemented proof:** the detailed Valuation tab now loads source-backed defaults, enforces bounded controls, reports changed assumptions and sensitivity, and keeps provenance and scenario identity under Advanced. It is session-local and does not change canonical inputs or readiness. +Local resource budgets for one supplied package: preview sample 100 rows; manifest 1 MiB; each contract CSV 32 MiB; four contract snapshots combined 64 MiB; source-rights registry 4 MiB; declared rows 250,000 per contract; package traversal 32 entries. These local bounds do not prove scale, hosted reliability, or market readiness. No permitted independently reviewed real dataset, accepted expected count/digest, or source-rights proof is on record. -### P2: Source Freshness Timeline - Implemented +### Priority 5 — One permitted consensus source and one reviewed peer relationship -**Goal:** show a selected ticker's source chronology without confusing report time, retrieval time, market time, or financial effective date. +Review one exact-source, permitted, point-in-time consensus snapshot for one ticker/fiscal period and one independently sourced peer relationship. Technical validity, rights, Revenue, EPS, comparability, reviewer decision, append history, and valuation-anchor eligibility remain independent. -The Sources & Gaps tab now derives a deterministic same-profile timeline from the report payload, shows unknown timestamps explicitly, deduplicates exact source records, and keeps raw provenance under Advanced. It never refreshes data, changes readiness, or infers publication, cutoff, or revision dates that are not present. +Neither item unlocks probability, broad coverage, readiness in another lane, or a company conclusion. -**Stop rule:** a recent retrieval or report timestamp does not render an older source period current or unlock a blocked lane. +Set `SOURCE_INPUT=`, then run `make earnings-consensus-source-review INPUT=$SOURCE_INPUT PROVIDER= AS_OF=` before collection preview. After separate human review and explicit evidence-preserving mapping, set `COLLECTION_INPUT=` and run `make earnings-consensus-collection-preview INPUT=$COLLECTION_INPUT AS_OF=`. Only after exact preview review may `CONFIRM_REVIEWED=1 make earnings-consensus-collection-record INPUT=$COLLECTION_INPUT AS_OF= PREVIEW_RECEIPT=` create an append-only evidence record. It does not activate readiness or numerical probability. Never infer an explicit provider. -### P2: Research Comparison View - Implemented +### Priority 6 — Controlled hosted operating boundary -**Goal:** compare selected companies across usable evidence, blockers, proof freshness, and reviewed catalysts or risks without ranking them. +Provider-neutral, deny-by-default workspace authorization is complete locally. `src.hosted_access_control.evaluate_workspace_access` creates an append-only, privacy-safe audit obligation. The module has no dashboard, ledger, readiness, provider, persistence, environment, network, or generated-artifact integration. It does not prove hosted authentication, deployed isolation, audit storage, retention, monitoring, rollback, incident response, operated capacity, or market validation. -The existing operator-only selected review tray now accepts two or three tickers, preserves user order, and displays an evidence matrix for price, fundamentals, DCF, trusted peers, supported analysis, missing inputs, next proof, freshness, and profile-scoped journal context. It adds no route and writes no data. +The actual environment must prove authentication, cross-user/cross-workspace denial, least privilege, durable persistence, append-only audit, retention/deletion, health monitoring, recovery, rollback, and named operating ownership. A local contract or URL alone is insufficient. -**Stop rule:** the comparison produces no score, winner, expected return, recommendation, or action; candidate peer context remains separate from trusted-peer readiness. +**Exit gate:** the actual hosted environment directly proves every claimed control, including an observed rollback rehearsal and named owner. -### P2: Peer Read-Through Map - Implemented +### Priority 7 — Accessibility evidence beyond screenshots -**Goal:** show which peer results can be reviewed as directional business context without treating sector similarity or candidate peers as trusted evidence. +The approved narrow remediation is implemented and has reproducible local direct-browser evidence at exact tested implementation anchor `0000c97e7db17e5d4353e30e976f2b7dec6bfd46`. `make research-accessibility-browser-check` first verified a clean product/code/test/docs tree, classified and excluded exactly 18 unstaged generated CSV/output paths, and verified the rendered Stock Research Command Center `Demo` profile before attaching the commit or profile. It then passed all eight Research Desk, Discover, ticker-bound Company Workbench, and Monitor cases at `1280x720` and `390x844`: after initial focus was cleared, one physical Tab focused the sole skip link, whose box was fully inside both horizontal and vertical viewport bounds before Enter preserved the route and focused the answer; the labelled workflow navigation and every applicable route link were inside the viewport, and each link was at least `44px` high; all four actually rendered eligible Discover actions had unique ticker-bound names; focused summaries exposed a solid three-pixel outline; and rejected empty-thesis validation bound, described, and focused Thesis Id while retaining one global alert. The live Workbench regression then changed the draft, proved the bridge-owned Thesis Id node and relationships were removed, and proved the next validation associated only `effective_at is required` with Effective At on desktop and phone. The gate uses no DOM-order enumeration or programmatic skip-link focus as keyboard evidence; rejects non-loopback or mismatched servers and dirty implementation evidence without attributing a local commit/profile; fails closed without Chrome/Playwright; assumes no Discover row count; and writes no JSON, timing, report, screenshot, readiness, canonical-data, or ledger artifact. -The existing detailed Valuation tab now separates trusted relationships, candidate-only relationships, explicit business-overlap evidence, target/peer fiscal periods, source-backed Revenue/EPS actuals, and the remaining proof needed for contextual read-through. A result becomes `reviewable_context` only when relationship provenance, business overlap, actual result evidence, and both fiscal periods are explicit. +The separately reviewed framework-safe semantic-main slice is implemented and has direct local browser evidence at exact tested implementation anchor `d1328eaa4d08cf08ec2b70939e4e031ee5f907b0`. The focused gate tests returned `22 passed`; `make research-accessibility-browser-check` again verified the loopback Stock Research Command Center `Demo` identity and a clean product tree while excluding the same 18 unstaged generated paths, then passed all 12 Research Desk, Discover, ticker-bound Company Workbench, Monitor, Research Data Health, and Research Proof History cases at `1280x720` and `390x844` with no failures. Every initial DOM and every DOM after a genuine same-document Streamlit `notRunning` -> `running` -> `notRunning` script cycle had exactly one role-based main with exact `role="main"`, `id="research-main"`, `aria-label="Stock research workspace"`, one contained answer target, one level-one heading, and host bridge status `applied`; only that same-document rerun/probe phase preserved the exact route with zero top-level navigation. A hidden inert mutation probe also required the live bridge observer to restore `applied` before the probe was removed. Separately, every case deliberately navigated to an explicit different Research route and back, waited for marker/stability/exact H1, then required the full away and return URLs, including the complete query string and empty fragment with the ticker parameter where present, before repeating semantic-main, runtime, and applicable primary/secondary navigation assertions. The physical-Tab skip target was focused inside the unique initial main; all cases had no console/page error, rendered traceback, or horizontal overflow across the recorded phases. The controlled native radio event and installed Streamlit test-state transition are framework engineering evidence only, not pointer/keyboard/mobile-sidebar interaction credit or a public cross-version compatibility guarantee. The gate remained repository/data read-only and produced no repository artifact. -**Stop rule:** candidate peers never become trusted automatically. Missing relationship source, actual result, or fiscal timing withholds read-through; even reviewable context cannot alter Earnings Nowcast numbers, DCF, readiness, rankings, or actions. +This closes the five narrow reproduced defects and the stable route-level semantic-main defect for the recorded local automated browser matrices only. Commit `6328c8cea` also passed the clean-tree direct dynamic-state matrix at desktop and phone widths: loading, empty, withheld, stale, failure, and validation remained visible non-live states; the five authoring transitions used the required single polite-status or assertive-alert semantics; unchanged rerenders became non-live; and the repository snapshot remained byte-for-byte unchanged. Independent-human keyboard review, 200%/400% zoom and reflow, direct platform forced-colors and reduced-motion review, screen-reader tasks, remaining small framework controls, and material-defect retests remain incomplete. Automated DOM checks and screenshots are supporting engineering evidence only; they do not prove WCAG conformance, screen-reader usability, hosted behavior, or independent-human accessibility validation. -### P2: Decision-Process Scorecard - Implemented +The local media-preference engineering slice is implemented at exact anchor `a97a096e1caf2761176dc42798a8c9686179aecc`. `make research-accessibility-browser-check TIMEOUT_SECONDS=90` passed forced-colors and reduced-motion emulation in all 12 six-route/two-viewport results while the existing landmark, focus, navigation, authoring, dynamic-state, runtime, overflow, and repository-write checks remained green; the same 18 unstaged generated paths stayed excluded. This is automated engineering evidence only. C01, C02, M01, direct platform and human media-preference review, zoom, screen-reader, assistive-technology, hosted, independent-human, and WCAG evidence remain incomplete, so Priority 7 remains open. +The earlier K01/K03/K04/K05/K06/K09 and mobile-navigation failures remain historical evidence in `docs/ACCESSIBILITY_EVIDENCE.md`; the new direct gate supersedes those five narrow implementation findings only at its recorded product anchor and viewports. Priority 7 remains incomplete. -**Goal:** render research discipline reviewable without grading a company or measuring investment performance. +The earlier Streamlit normalization to `target="_blank"` also remains historical evidence; the route-preserving correction continues to use fragment-only `#public-page-answer` with `target="_self"`. The current gate directly verifies the resulting same-route focus transfer without treating the historical defect as current. -The Single-Stock Report now derives profile-scoped checks for readiness review, thesis documentation, recorded evidence, later review of conflicting evidence, invalidation conditions, confidence history, review-date currency, unresolved Change Monitor tasks, and visible DCF assumptions. Details stay collapsed below the Thesis Journal. +Use `docs/ACCESSIBILITY_TASK_PROTOCOL.md`; the protocol is not completion evidence. The corrected same-page target is `#public-page-answer`; incomplete direct tasks remain `blocked_environment`. No WCAG conformance claim is made. -**Stop rule:** the scorecard reports process states and next review steps only. It produces no numeric company score, expected return, performance claim, ranking, recommendation, or action; blocked and excluded analysis remain distinct from incomplete documentation. +### Priority 8 — Independent workflow validation -### P2: Earnings Nowcast Pilot Evidence +Run 10-20 independent workflow sessions with the target researcher persona through Research Desk -> Discover -> Company Workbench -> Monitor. Owner-led, automated, fixture, and screenshot sessions do not count. -**Goal:** move the implemented Earnings Nowcast pilot from synthetic infrastructure proof to a leakage-safe, source-backed semiconductor cohort. +Measure task completion, time to first useful answer, readiness comprehension, evidence tracing, authoring friction, trust, misuse risk, perceived performance, repeat-use intent, and the most important missing workflow. Use voluntary minimal-data capture, withdrawal handling, a deletion date, anonymized evidence, reproducible findings, severity decisions, and material-defect retests. -Earnings Nowcast real-data safety infrastructure is implemented for deterministic Revenue/EPS ranges, consensus-relative classification, metric-specific canonical quarterly evidence, comparability checks, evidence-only directional signals, chronological walk-forward backtesting, explicit sample-sufficiency/calibration diagnostics, and a separate probability calibration gate. Versioned read-only append-only onboarding templates, validation, preview, readiness, and prospective collection planning are implemented. The committed fixture cohort is synthetic test evidence only. +The local invitation, scorecard, log schema, runbook, and closeout checklist are ready. This does not prove demand, retention, product-market fit, or financial validity. -1. Acquire permitted append-only historical quarterly actuals and point-in-time consensus snapshots with source references, publication/retrieval timestamps, and explicit Revenue/EPS comparability definitions for a narrow semiconductor cohort. -2. Use `make earnings-nowcast-prospective-plan` for future snapshot collection, then run the implemented onboarding validate/preview/readiness gates before any real-company packet; no automatic apply path exists. -3. Keep candidate peer/news signals separate from reviewed trusted evidence; signals explain context and never mutate forecast numbers. -4. Run chronological out-of-sample evaluation against latest-consensus and prior-year benchmarks. -5. Withhold numerical Beat/Miss probability until at least 100 valid events pass Brier-score, calibration-bin, and benchmark-improvement gates. +### Priority 9 — Out-of-sample calibration cohort -Real semiconductor nowcast coverage remains `awaiting_point_in_time_consensus`; numerical probability remains `awaiting_calibration_evidence`. +Accumulate at least 100 valid leakage-safe out-of-sample events from permitted point-in-time inputs with immutable cutoffs, comparable actuals, revision lineage, exclusions, missingness, and reproducible cohort membership. -**Stop rule:** do not substitute current analyst estimates for historical point-in-time snapshots, use post-cutoff evidence, infer numeric adjustments from text, claim predictive accuracy from fixtures, or predict post-earnings price movement. +Evaluate predeclared Brier score, calibration bins, constant-benchmark improvement, sample sufficiency, missingness, stability, and temporal/cohort leakage. The local `make calibration-evidence-bundle-preview BUNDLE=` contract reads one supplied immutable bundle, reports `invalid`, `blocked`, or `contract_consistent_review_required`, and writes no artifact. It recomputes retained observations against the exact backtest package; an unrelated same-size cohort is not evidence. It does not activate readiness, establish source rights or authenticity, persist evidence, or validate real events. Actuals, consensus, Revenue, EPS, valuation, catalysts, outcomes, backtesting, and calibration remain independent; probability remains withheld unless every applicable external and calibration gate passes. -## Next +### Priority 10 — Separately approved hypothetical paper-position laboratory -### P2: FMP One-Ticker Source Smoke +Implementation is not authorized. A separate approved design must cover research-only intent, private-data policy, language, misuse analysis, and acceptance. It must prohibit recommendations, model-generated sizing, allocation, live holdings, account imports, brokers, order routing, auto-trading, stop/profit instructions, and performance claims. Live brokerage remains out of scope permanently. -**Goal:** add one controlled keyed free-tier fallback after the public pilot foundation is stable. +## Externally blocked -1. Configure `FMP_API_KEY` outside Git in the ignored local key file or host secrets. -2. Run `make project-status-check`; only continue if it identifies a reviewed candidate scope. -3. Run `make fmp-smoke TICKER=` for one ticker. -4. Run `make imports-validate IMPORT_TICKERS=` and `make imports-preview IMPORT_TICKERS=`. -5. Apply only if validation passes, preview scope is intended, rejected rows are zero, and source provenance is present. -6. Record a supported, candidate-context-only, still-blocked, skipped, or excluded outcome before any larger batch. +| Stage | Classification | Evidence checked | Exact unblock condition | +| --- | --- | --- | --- | +| Priority 4 | `point_in_time_benchmark_universe_and_rights_required` | No permitted independently reviewed real package is on record. | Supply one bounded permitted package and independent expected-count/digest review; pass rights, identity, corporate-action, delisting, survivorship, cutoff, partition, reproduction, and leakage gates. | +| Priority 5 consensus | `permitted_point_in_time_consensus_and_rights_required` | No permitted exact-period point-in-time input or approved exact-source field rights are on record. | Configure one permitted source or reviewed CSV, approve exact-source rights/field scope, then validate and preview one ticker/period without inferring a provider. | +| Priority 5 peer | `trustworthy_peer_source_and_review_required` | No genuinely reviewed relationship is on record. | Provide source/as-of evidence and reviewer capacity; record role, rationale, comparability, and an explicit valuation-anchor decision. | +| Priority 6 | `hosted_account_and_controls_required` | No verified hosted identity/persistence environment or URL is on record. | Approve the exact environment and directly verify authentication, isolation, audit, retention, monitoring, backup, rollback, incident response, and named owner capacity. | +| Priority 7 | `accessibility_manual_review_environment_required` | Automated forced-colors and reduced-motion emulation exists; true zoom, direct platform/human media-preference review, and screen-reader coverage remain incomplete. | Provide a suitable review environment and complete the task protocol plus material-defect retests. | +| Priority 8 | `independent_reviewers_required` | The protocol is locally ready; zero independent sessions are on record. | Complete 10-20 independent target-persona sessions with anonymized evidence and retest material defects. | +| Priority 9 | `calibration_cohort_required` | Valid real leakage-safe calibration events: zero. | Accumulate at least 100 permitted events and pass the predeclared calibration/benchmark gates. | -**Dependencies:** an FMP key and an executable reviewed candidate. The current source-proof queues have no unreviewed executable company candidates, so provider reachability alone does not unlock coverage. +Unavailable dependencies are recorded once and not retried until relevant external state changes. This is the Approved Next-Stage Maturity Program; blocked work must move to the next safe executable priority. -**Stop rule:** no broad batch from setup alone. Provider setup/source-boundary review must happen before `make trusted-data-pilot-candidates TOP_N=10` only after source state changes. +## Later -### P2: Price History Maintenance +### Focused price-history maintenance -Price coverage uses `PROVIDER=auto` in this fixed order: **Stooq, Yahoo**, optional IBKR read-only when explicitly configured, then keyed FMP, Alpha Vantage, and Finnhub fallbacks. This maintenance lane is finite and read-only until a separately reviewed source-backed change is eligible for the import gate. +`PROVIDER=auto` remains **Stooq, Yahoo**, optional IBKR read-only, then keyed FMP, Alpha Vantage, and Finnhub fallbacks. An explicit provider identity and commercial rights/`prices` scope are required before commercial fetch or mutation. -1. Run the default executable queue: `make price-history-proof-queue TOP_N=25`. - - `momentum-not-ready` rows describe a readiness state, not a refresh instruction. - - `unreviewed preferred-history candidates` are the only default queue rows eligible for a narrow reviewed investigation. - - `reviewed source-limited items` are excluded from the default queue because they remain wait-only. -2. Use audit mode only to inspect reviewed source-limited items: `INCLUDE_REVIEWED=1 make price-history-proof-queue TOP_N=25`. -3. When compatible reviewed evidence exists, use `make price-history-batch-closeout TOP_N=25` to produce the read-only grouped closeout scaffold. It does not record proof rows, stage files, commit, or push. +- `momentum-not-ready` is a readiness state, not a refresh instruction. +- Only `unreviewed preferred-history candidates` are default investigation candidates. +- Inspect `reviewed source-limited items` only with `INCLUDE_REVIEWED=1`. +- Use `make price-history-batch-closeout TOP_N=25` only when compatible reviewed evidence exists. +- stop on no readiness movement in reviewed scope; no identical source-limit retry unless source behavior or verified OHLCV changes; batch compatible proof evidence intentionally; never commit or push one proof row per ticker by default; pivot to the next roadmap item when no executable candidates. -**Stop rules:** stop on no readiness movement in reviewed scope; no identical source-limit retry unless source behavior or verified OHLCV changes; batch compatible proof evidence intentionally; never commit or push one proof row per ticker by default; pivot to the next roadmap item when no executable candidates. +Provider setup/source-boundary review comes before candidate loops. The current source-proof queues have no unreviewed executable company candidates. Run `make provider-setup-checklist`; run `make trusted-data-pilot-candidates TOP_N=10` only after relevant source state changes. -### P3: 25-50 Company Trusted-Peer Pilot +### Focused peer expansion -**Goal:** address the largest analytical-depth gap without inferring trusted peers across the full universe. +A 25-50 company trusted-peer pilot is governed by the Approved Next-Stage Maturity Program and begins only after the single reviewed relationship in Priority 5 proves trustworthy sourcing and repeatable review capacity. Candidate sector similarity is not peer proof. -1. Select 25-50 operating companies from a few clearly comparable industries. -2. Generate candidate peer context from SIC, industry, and product context; label it `candidate_context_only`. -3. Promote a relationship only after source-backed review captures peer source, review date, rationale, and as-of context. -4. Keep peer trend readiness separate from peer valuation readiness. -5. Require trusted peer price, fundamentals, and valuation inputs before relative valuation appears. +### Research-oriented monitoring -**Dependencies:** a licensed or otherwise trustworthy peer relationship source and reviewed mappings. +After source and workflow evidence support it, add source-linked filing/catalyst changes, thesis reminders, watchlists, and daily/weekly “what changed” recaps. Any summary must be cited to approved evidence and unable to change readiness, forecasts, or reviewed records automatically. -**Stop rule:** sector similarity is not trusted-peer proof. Do not target broad-universe peer readiness before the pilot has repeatable evidence. +### Product-direction decision -The implemented Peer Read-Through Map is the review surface for this future cohort. It does not satisfy the external trusted-relationship dependency or create broad peer coverage by itself. +Use `docs/PRODUCT_DIRECTION_DECISION.md` after hosted, independent-review, trusted-peer, data-economics, operating-capacity, and repeat-use evidence exists. Default direction remains a maintained personal/small-team research tool; an operated platform is not authorized by feature volume or test counts. -## Later +## Completed with evidence + +Detailed evidence is in `docs/COMPLETED_MILESTONES.md`; this section is an index, not duplicated chronology. + +### P0: Profile Truth And Local Research Change Workflow + +Implemented and regression-gated. Snapshot-only and source-backed changes stay distinct; missing change evidence is not a negative company signal. + +### P0: Research Thesis And Evidence Journal + +Implemented with append-only provenance, invalidation, conflicting evidence, outcome review, and no automatic readiness or forecast promotion. -### P4: Optional Earnings And Analyst Estimates +### P0: Performance Release Candidate + +Passed on commit `6328c8cea`: warm shell/first-useful p90 and cold shell/first-useful maximum are separate and independently enforced at the unchanged one-second and three-second limits. The fixed `data/demo/manifest.json` run recorded 48 successful samples with no failures; the aggregate release check, 4,474-test full suite, six-route/two-viewport accessibility browser gate, state harness, push, draft-PR exact-head check, and generated-artifact exclusion also passed. This proves a reproducible local release candidate, not hosted performance, source coverage, human validation, or market readiness. Do not retry unchanged failures, select fastest runs, or stage `/tmp` timing evidence. + +### P1 local prerequisite: Hosted operating contracts + +Local deployment and operating contracts are complete; actual hosted verification remains externally blocked under Priority 6. + +### P1 local prerequisite: Independent beta protocol + +The independent-session protocol is locally complete; actual sessions remain externally blocked under Priority 8. + +### P2: Scenario Lab - Implemented + +Scenario assumptions are explicit research context and cannot become recommendations, rankings, or deterministic forecast inputs. + +### Earnings Nowcast real-data safety infrastructure -Proceed only when a trusted provider supplies supported earnings actual/estimate fields, estimate period, source, and retrieval/as-of date. Date-only and target-price-only data remain `candidate_context_only`; optional context never unlocks DCF readiness or becomes a recommendation. +Deterministic synthetic-fixture software, point-in-time consensus contracts, actuals lineage, backtest, and calibration gates are implemented. Real-company coverage and predictive accuracy are not established. -### P4: Scheduler Maturity +### Bounded SEC cash-generation evidence -Add scheduled snapshot rotation, alerts, and source monitoring only after at least one provider pilot proves deterministic batch limits, provenance, rejection handling, and proof-ledger recording. Daily price and filing checks may be read-only; imports still require validation, preview, and source gates. The local Change Monitor is not itself a hosted alerting service. +The one-company adapter acceptance harness and bounded exact-source review started with NVIDIA Q1 FY2027; the later AMD Q1 FY2026 review established bounded two-company portability. They do **not** prove production activation, broad history, Q4 portability, or market validation. The preview does not activate Company Workbench or readiness in ordinary routes and does not prove broad company coverage. -### Later: Broader Peer Expansion +AMD Q1 FY2026 uses accession `0000002488-26-000076`. The one explicit user-flow composition can expose accepted preview evidence only through an opt-in route; it does not prove a second company at the historical NVIDIA-only milestone and never promotes production state. -Expand beyond the peer pilot only after trusted relationship sourcing, review capacity, and lane-level quality checks are repeatable. +Valuation and backtest safeguards reject non-finite valuation inputs; require a canonical real `YYYY-MM-DD` denominator period end; reject blank, malformed, and non-calendar denominator period ends; reject post-cutoff retrieval evidence; canonicalize Revenue/EPS independently through explicit `supersedes_source_ref` lineage; retain one event per ticker/period; withhold ambiguous leaves per metric so one metric does not suppress the other; and use cutoff-bounded prior-year benchmarks so post-cutoff revisions cannot leak. -### Later: Product Direction Decision +### Research Decision Lab -Use `docs/PRODUCT_DIRECTION_DECISION.md` after hosted-preview, controlled-pilot, and trusted-peer evidence exist. Choose explicitly among a portfolio-quality research prototype, maintained research tool, or operated research platform; keep the decision provisional while those dependencies remain external. +Implemented locally — Research Decision Lab. Stage 4 — Documentation and release evidence: completed locally. Read-only composition, Workbench integration, Monitor discipline review, and release evidence passed without trading or recommendation behavior. + +This does not prove source coverage, predictive accuracy, investment performance, independent adoption, hosted reliability, commercial demand, competitive superiority, or product-market fit. + +### Methodology and packaging maturity + +Current methodology maturity supports a transparent local research prototype and controlled beta candidate. It does not prove broad real-company coverage or market validation. Phone first-action density, answer-first layout, Advanced data health cards, auto-refresh status, Session Source Preflight, commercial-beta release evidence, and source-rights boundaries are locally verified product/package evidence only. ## Dependencies And Manual Gates -| Item | State | What the repo can do | What remains external | -| --- | --- | --- | --- | -| Hosted demo | repo-ready | deterministic demo profile, deployment guide, and local public checks | hosting account, verified public URL, browser review | -| FMP fallback | optional key missing | one-ticker smoke, validation, preview, provenance gate | `FMP_API_KEY` outside Git | -| Alpha Vantage / Finnhub | optional keys missing | capped fallback interfaces and source-state checks | provider keys and a reviewed use case | -| Trusted peers | source-gated | candidate/trusted state separation and proof workflow | reviewed source relationships and rationale | -| Earnings / estimates | intentionally locked | optional-context states and import gates | trusted provider/manual rows with supported fields | +| Item | Local state | Manual/external gate | +| --- | --- | --- | +| Readiness/data | fail-closed software and reviewed local artifacts | source rights, exact-source rows, validate/preview/apply, rebuild, and proof | +| Providers | provider-neutral contracts and capped adapters | explicit provider key/rights/scope and reviewed use case | +| Hosting | architecture and authorization contracts | approved account/environment plus direct operating evidence | +| Accessibility | partial direct desktop/phone evidence | remaining environment/tasks and material-defect retests | +| Reviewer validation | complete privacy-safe protocol | 10-20 independent sessions | +| Calibration | predeclared methodology | at least 100 valid permitted events | ## Success Gates -### Public Demo Gate +### Independent engineering gate -- `make demo-data-check` -- `make demo-dashboard-smoke` -- `make demo-dashboard-render-smoke` -- `make public-check` -- `make browser-qa-evidence` +- Current-head GitHub Actions result, not a previous revision. +- Full tests, dashboard startup, Personal Research route rendering, public wording, generated-artifact hygiene, and whitespace. +- Independent human review remains separate from automation. + +### Public/demo gate + +- `make dashboard-smoke` +- `make research-dashboard-render-smoke` - `make public-wording-check` +- `make public-check` +- `make commercial-beta-release-check` - `make pilot-readiness-check TOP_N=10` - `make diff-hygiene-summary` - `git diff --check` -### Source-Backed Apply Gate +### Source-backed apply gate -- A narrow, intended ticker scope. -- Source provenance and relevant as-of context. -- `make imports-validate IMPORT_TICKERS=` passes. -- `make imports-preview IMPORT_TICKERS=` is narrow and rejected rows are zero. -- Readiness and proof evidence are rebuilt after an approved apply. +A narrow intended scope, exact-source rights/provenance, validation, preview, rejected-row review, explicit apply/skip decision, rebuilt readiness, and proof-ledger evidence are all required. Candidate context, setup, screenshots, local contracts, or historical proof cannot substitute. ## Permanently Out Of Scope -- Broker execution, account actions, order routing, or auto-trading. -- Direct buy/sell instructions or investment recommendations. -- Fabricated prices, fundamentals, shares, peers, earnings, estimates, valuation inputs, or metrics. -- Promoting candidate peers, stale rows, screenshots, or provider setup into trusted readiness proof. +- Investment advice, direct buy/sell instructions, company rankings, expected-return scores, or automated stock picking. +- Broker execution, account imports, live holdings, order routing, or auto-trading. +- Model-generated sizing, allocation, stop-loss, take-profit, or post-earnings price prediction. +- Fabricated data, forecasts, probabilities, peers, events, sources, timestamps, rights, reviewers, demand, or recommendations. +- Promoting candidate context, stale rows, synthetic fixtures, screenshots, provider setup, or empty ledgers into trusted evidence. + +## Readiness Release Evidence Contract + +Run `make readiness-release-review TOP_N=20`; record the exact receipt with `make readiness-release-record PREVIEW_RECEIPT= REVIEWER= REVIEW_DATE= TECHNICAL_DECISION= DISTRIBUTION_DECISION= CONFIRM_REVIEWED=1`; then run `make readiness-release-guard RECORD_ID=`. Review and guard are read-only; record may preserve a blocked outcome. Guard requires approved decisions, matching axes/head/bytes/digests, and unstaged state before printing named paths. This workflow does not change readiness and does not grant source rights; it cannot replace independent review, external evidence, legal/distribution/accessibility approval, hosted validation, calibration, or pilot evidence. Restart review after any relevant change. diff --git a/config/source_rights.yml b/config/source_rights.yml new file mode 100644 index 000000000..f65cdbfc7 --- /dev/null +++ b/config/source_rights.yml @@ -0,0 +1,33 @@ +sources: + - source_id: sec_companyfacts + display_name: SEC Companyfacts + permitted_use: source_backed_company_facts + commercial_use: approved + redistribution: derived_data_only + storage_limits: retain normalized filing facts and source references under the applicable records policy + attribution: SEC EDGAR with filing accession or source reference + rate_limits: fair-access only; identify requests with SEC_USER_AGENT and do not automate abusive retrieval + authentication: SEC_USER_AGENT + expected_freshness: filing-driven; review after a new filing becomes available + supported_fields: + - revenue + - shares_outstanding + - filing_dates + - operating_income + - cash_from_operations + - capital_expenditures + fallback_priority: 1 + - source_id: yfinance + display_name: yfinance / Yahoo Finance + permitted_use: research_only + commercial_use: unverified + redistribution: not_permitted_without_separate_rights_review + storage_limits: local research cache only; do not retain or redistribute in commercial mode + attribution: yfinance / Yahoo Finance when used in research mode + rate_limits: follow the upstream provider terms and avoid automated bulk retrieval + authentication: none + expected_freshness: market-data dependent; not a commercial freshness guarantee + supported_fields: + - prices + - company_context + fallback_priority: 90 diff --git a/docs/ACCESSIBILITY_EVIDENCE.md b/docs/ACCESSIBILITY_EVIDENCE.md new file mode 100644 index 000000000..12b9601ee --- /dev/null +++ b/docs/ACCESSIBILITY_EVIDENCE.md @@ -0,0 +1,896 @@ +# Accessibility Evidence + +This document records direct local accessibility evidence for the supported +Personal Research workflow. It is an evidence log, not a WCAG conformance +claim. + +## 2026-07-23 partial workflow audit + +Scope: + +- Local Streamlit dashboard in Personal Research mode. +- Research Desk -> Discover -> Company Workbench -> Monitor. +- Desktop viewport: `1280x720`. +- Phone viewport: `390x844`. +- Browser: local Chromium-based browser. +- Company Workbench review ticker: `AVGO`. +- No readiness rebuild, data refresh, evidence recording, or repository + screenshot generation was performed. + +Directly observed: + +- All four routes rendered at desktop and phone widths without document-level + horizontal overflow. +- The document language was `en`. +- Visible form controls exposed labels or accessible names in the inspected + DOM. +- No duplicate element IDs were found in the inspected routes. +- A focused Data Health link showed a visible three-pixel teal outline. +- Research-only and no-account-action boundaries remained visible. +- The Monitor `Open Discover` action initially rendered white normal-size text + on Streamlit red, measured at approximately `3.3:1`. +- The local theme now gives primary link buttons a `#0b3b36` background and + border with white text. Runtime retest measured the Monitor action at + `12.4:1`. +- A route-preserving `Skip to page answer` link is now present in the + accessibility tree on all four Personal Research routes. Direct desktop and + phone retest verified the correct route, ticker, and open-state parameters + for the Company Workbench link and a focusable `tabindex="-1"` answer target. +- Primary route sections now follow each route `h1` with `h2` headings. + Direct desktop and phone DOM retest verified the complete Workbench sequence + from `What Changed` through `Advanced Evidence`; Monitor retains the nested, + non-empty `h3` for `Earnings evidence readiness`. +- The semantic correction was retested at `1280x720` and `390x844` with no + document-level horizontal overflow. New screenshots remain ephemeral under + `/tmp/stock-research-accessibility-audit-2026-07-23/` and are not repository + or release artifacts. + +Open findings: + +- The inspected routes still expose no `main` landmark. +- Current local Streamlit `1.59.2` containers expose layout and key parameters + but no semantic role or element parameter. A client-side workaround would + require unsafe JavaScript tied to framework-owned DOM selectors. That + approach is not accepted as a stable landmark implementation without a + separately reviewed design and direct runtime evidence. +- The skip link is structurally present and uses the existing visible-on-focus + CSS contract, but a complete keyboard traversal has not yet directly proved + first-focus placement, focus transfer, or return-path behavior. +- Framework help controls measured approximately `16x16` CSS pixels and + dataframe toolbar controls approximately `22x22`. Hidden native radio inputs + also measured small, but their visible labels are larger and require separate + hit-target review rather than treating the input box alone as the target. +- The inspected routes exposed no explicit live region. Dynamic update and + validation announcements still require task-level assistive-technology + testing. +- Browser automation did not prove the complete keyboard traversal order. +- Direct screen-reader tasks, 200% and 400% zoom, forced-colors behavior, + reduced-motion environment verification, and error-association tasks remain + untested. + +Classification: + +- `partial_local_accessibility_evidence` +- The contrast, route heading hierarchy, and route-preserving skip-link defects + above are fixed and directly retested within the stated scope. +- Priority 7 remains incomplete. Screenshots, DOM inspection, and one focused + contrast/semantic retest do not prove keyboard accessibility, + assisted-technology support, or WCAG conformance. + +Next safe local step: + +1. Execute `docs/ACCESSIBILITY_TASK_PROTOCOL.md` in a suitable review + environment and record direct keyboard traversal, focus, disclosure, + search, navigation, validation, and error-recovery evidence. +2. Complete zoom/reflow, forced-colors, and screen-reader tasks from the same + protocol. +3. Revisit the `main` landmark only through a stable Streamlit capability or a + separately reviewed design; do not inject an unverified DOM-mutation patch. + +The protocol is not completion evidence and no task result is inferred merely +because the protocol now exists. + +## 2026-07-26 same-page skip-link routing audit + +Scope: + +- Local Streamlit `1.59.2` dashboard in Personal Research mode. +- macOS `26.5.1` at a `1280x720` CSS viewport. +- Research Desk and ticker-bound Company Workbench route for `AVGO`. +- No readiness rebuild, source refresh, research-record save, screenshot + capture, or generated-artifact write. + +Directly reproduced: + +- Although the helper emitted no target attribute, the live Streamlit DOM + normalized `Skip to page answer` to `target="_blank"`. +- Activating that live link did not move focus to the answer target or retain + the current tab's route context. +- Existing tests asserted the constructed route URL and target element but did + not cover the live same-tab contract. + +Remediation and direct retest: + +- The skip link now uses the fragment-only destination + `#public-page-answer` plus explicit `target="_self"`. A fragment-only + destination retains every current mode, route, ticker, and disclosure query + parameter without triggering a Streamlit rerun that resets focus. +- The helper contract is covered for Public, Personal Research, and Operator + routes. +- Direct Company Workbench retest retained + `?mode=research&page=company-workbench&ticker=AVGO&open=1`, appended only + `#public-page-answer`, kept the same browser tab, scrolled the answer target + to the top of the viewport, and left `document.activeElement` on the + focusable `public-page-answer` target. + +Evidence boundary: + +- This closes the reproduced new-tab/rerun defect and proves the live + same-document target behavior in the stated environment. +- The available browser-control environment could not activate ordinary links + through its synthetic Enter input, including the unrelated `Open Discover` + control. K01 and K02 therefore remain `blocked_environment`, not + `passed_direct`; no complete keyboard-only traversal is claimed. +- Priority 7 remains incomplete. Direct keyboard, zoom, forced-colors, + reduced-motion, target-size, screen-reader, and stable-main-landmark evidence + remains required under `docs/ACCESSIBILITY_TASK_PROTOCOL.md`. + +## 2026-07-27 framework-control target-size audit + +Scope: + +- The same local Streamlit, macOS, Personal Research, and no-write boundary as + the preceding audit. +- Ticker-bound Company Workbench for `AVGO`. +- Desktop `1280x720` and phone `390x844` CSS viewports. +- The Workbench quarterly-trend disclosure was opened only to expose its + existing dataframe toolbars; no data or research state was changed. + +Directly reproduced: + +- The visible framework help target measured `16x16` CSS pixels. +- Visible dataframe `Show/hide columns`, `Download as CSV`, `Search`, and + `Fullscreen` targets measured `22x22` CSS pixels. + +Remediation and direct retest: + +- The first style correction enlarged the tooltip wrapper, not its nested + button. An expanded P02 audit caught the actual help button still at + `16x24`; this intermediate result is not completion evidence. +- The corrected shared style now reserves at least `24x24` CSS pixels for the + actual nested Streamlit tooltip/help buttons and dataframe toolbar buttons. + The phone Workbench `Open Data Health` action also retains its compact text + treatment with an explicit `24px` minimum height instead of the reproduced + `21px`. +- Final desktop retest measured the visible actual help button and all 16 + exposed dataframe toolbar buttons at exactly `24x24`. +- Final phone retest measured the visible actual help and dataframe buttons at + `24x24` and `Open Data Health` at `102x24`; document width remained equal to + the `390px` viewport width. + +Evidence boundary: + +- This directly closes the reproduced P01 size defect for the inspected + framework controls and current Streamlit markup. +- It does not prove pointer-spacing exceptions, every framework version, + complete P02 route-control coverage, keyboard order, zoom, forced colors, + reduced motion, screen-reader behavior, or WCAG conformance. +- Priority 7 remains incomplete under + `docs/ACCESSIBILITY_TASK_PROTOCOL.md`. + +## 2026-07-27 direct Chrome keyboard workflow audit + +Run metadata: + +- Run ID: `a11y-2026-07-27-chrome-keyboard-01`. +- Commit: `15c5270070238b91343a9f6eaa26f86527bfcaf2`. +- Route base: local `http://localhost:8501/`. +- Environment: macOS `26.5.1` build `25F80`; Google Chrome + `150.0.7871.182`; external display `3440x1440`; explicit CSS viewport + `1280x720`; 100% zoom; default colors and motion. +- Input: direct Chrome browser-control `Tab`, `Shift+Tab`, arrow, Enter, and + text-key input. Screen reader: `not_run`. Reviewer: + `Local supervised browser review`. +- The run used saved readiness only. It performed no refresh, readiness + rebuild, research-record save, screenshot capture, or generated-artifact + write. Browser-control input is direct functional evidence in this exact + environment; it is not independent human or assistive-technology review. + +Direct task results: + +- K01: `failed_reproducible`. After reload, the first focus sequence was + Workspace help button -> selected Workspace radio input -> selected route + radio input -> Streamlit main section -> `Skip to page answer`. Each of the + repeated shell controls therefore preceded the skip link. The inspected + controls had a visible three-pixel teal outline. +- K02: `passed_direct` in the recorded environment. Enter on the focused skip + link retained + `?mode=research&page=research-desk`, appended only + `#public-page-answer`, and moved `document.activeElement` to the + `public-page-answer` target. +- K03: `passed_direct` in the recorded environment. Arrow Down on the selected + route radio changed the URL to + `?mode=research&page=discover`, rendered the `Discover` `h1`, retained + keyboard focus, and showed the focus outline. +- K04: `failed_reproducible`. Search accepted `AVGO` and Enter, Tab, and Enter + opened the correct ticker-bound Company Workbench route without a pointer. + However, every result action exposed the same accessible text + `Open review`; neither the ticker nor company identity was present in the + link's accessible name. The action is operable but is not distinguishable + in a screen-reader links list or when encountered outside its visual row. +- K05: `failed_reproducible` at the first material focus defect. Native + disclosure `summary` elements received focus in the route sequence but had + computed `outline: none`; the complete disclosure traversal was stopped at + that first mismatch rather than promoted to a pass. +- K06-K09, Z01-Z03, C01-C02, M01, S01-S07, and remaining P02 coverage: + `not_run` in this run. + +Additional bounded observations: + +- An intentionally incomplete authoring validation was triggered with a + pointer only and saved no record. After the Streamlit rerun completed, the + `thesis_id is required` message was contained by + `data-testid="stAlertContainer"` with `role="alert"`. This DOM observation + does not pass keyboard task K06 or screen-reader task S06. +- The earlier apparent `23.2x18.7` skip-link target was measured while the link + was deliberately clipped off screen. When focused through the direct + keyboard path, the visible target measured approximately `132.6x38.2`. + No target-size defect is recorded from the clipped state. + +Root-cause and next-design boundary: + +- Streamlit renders the sidebar before the main section in DOM focus order. + The current skip link lives in the main section, so source-call ordering + cannot place it before sidebar controls. +- The shared focus selector covers links, buttons, form controls, explicit + roles, and nonnegative `tabindex`, but not native `summary`. +- Discover renders ticker-specific destinations with identical visible and + accessible `Open review` text. +- The narrow remediation should place one skip link as the first focusable + sidebar child, add ticker-specific accessible names while retaining the + concise visible action label, and include `summary:focus-visible` in the + shared focus contract. This is a proposed design, not an implemented fix. +- Priority 7 remains incomplete. Phone keyboard, complete K05-K09, zoom, + forced colors, reduced motion, screen-reader tasks, stable `main` landmark, + and remaining P02 coverage still require direct current evidence. + +## 2026-07-27 zoom-environment and phone-target follow-up + +Run metadata: + +- Run ID: `a11y-2026-07-27-reflow-targets-02`. +- Commit: `6aef0f8a9e89e8ec94d308095ca18ef33ea4177a`. +- Environment: the same macOS and Chrome versions as the preceding direct + keyboard run. Screen reader: `not_run`. Reviewer: + `Local supervised browser review`. +- The run used saved readiness and performed no refresh, readiness rebuild, + valid authoring preview, research-record save, screenshot capture, or + generated-artifact write. + +Zoom classification: + +- Chrome page-level keyboard input remained available for route tasks, but the + connected review surface did not pass browser-level zoom shortcuts through + to Chrome. Repeated `Meta`/`Control` plus zoom attempts left the CSS viewport + and device-pixel ratio unchanged. +- Z01-Z03 are therefore `blocked_environment`, not `passed_direct`. +- A separate viewport-only stress check at `640px` and `320px` found no + document-level horizontal overflow on Research Desk, Discover, Company + Workbench, or Monitor. This is bounded responsive-reflow evidence only. A + smaller viewport is not substituted for 200% or 400% browser zoom. + +Phone target and spacing follow-up: + +- At a `390x844` CSS viewport, every route was allowed to complete its + Streamlit render before measurement. +- The first overlap scan included controls retained inside closed native + disclosures and reported false-positive geometry intersections. Excluding + closed disclosure contents removed those intersections; they are not + recorded as product defects. +- Visible main-content route controls, primary links, search controls, and + disclosure headers on the four routes had no measured target below + `24px`, no measured adjacent overlap, and no document-level horizontal + overflow in the inspected collapsed state. +- Opening `Add a reviewed research record` exposed the current thesis authoring + controls without entering or saving data. The Record type control measured + approximately `265.8x38`, Summary measured `297.8x92`, and + `Validate and preview` measured `299.8x40`; no adjacent overlap or + document-level horizontal overflow was measured. +- This advances P02 only as `partial_local_evidence`. Mobile-sidebar route + choices, every record-kind field set, pointer misactivation, and controls + revealed by all Advanced disclosures were not fully exercised and are not + promoted to `passed_direct`. + +Evidence boundary: + +- The run found no new implementation defect beyond K01, K04, and K05 from the + preceding direct keyboard audit. +- Priority 7 remains incomplete. The proposed K01/K04/K05 remediation still + requires design approval; Z01-Z03 require a true browser-zoom environment; + forced colors, reduced motion, screen-reader tasks, stable `main` landmark, + phone keyboard, and remaining P02 coverage remain open. + +## 2026-07-27 Monitor keyboard and landmark continuation + +Run metadata: + +- Run ID: `a11y-2026-07-27-monitor-landmark-03`. +- Commit: `1a8727706595f87df459eeb5c7c762dafad83577`. +- Route base: local `http://localhost:8501/`. +- Environment: macOS `26.5.1` build `25F80`; Google Chrome + `150.0.7871.182`; CSS viewport `3440x1208`; device-pixel ratio `1`; + default colors and motion. +- Input: direct Chrome `Tab` and `Shift+Tab` keyboard input plus read-only + accessibility-role inspection. Screen reader: `not_run`. Reviewer: + `Local supervised browser review`. +- The run used saved readiness only. It performed no refresh, readiness + rebuild, authoring input, research-record save, screenshot capture, or + generated-artifact write. + +Direct task results: + +- K08: `passed_direct` for the recorded current Monitor state. Forward + keyboard traversal reached the Research Discipline table controls and + accessible canvas text, the truthful no-change state, the explicit + `This is a monitoring state, not a stock ranking` boundary, the + `Open Discover` next action, and the Advanced disclosures. The weekly + summary reported `0 traceable items`; no fabricated change or ranked + company was presented. +- K09: `failed_reproducible`. Forward traversal wrapped back to the first + control and fourteen direct `Shift+Tab` steps reversed through the same + route without a keyboard trap. However, each focused native `summary` + again exposed computed `outline: none`, so the expected continuously + visible reverse-focus state did not pass. This is the same focus-contract + defect recorded under K05, not a separate remediation. +- Stable route-level `main` landmark: `failed_reproducible`. Direct + accessibility-role inspection returned zero `main` landmarks on Research + Desk, Discover, ticker-bound AVGO Company Workbench, and Monitor at the + exact recorded commit. This does not promote screen-reader task S02, which + remains `not_run`. +- C01-C02 and M01: `blocked_environment`. Both connected local browser + surfaces reported forced colors and reduced motion inactive and exposed + viewport control only; neither exposed a supported media-emulation or + platform-mode control. Default-mode inspection cannot substitute for the + named forced-colors or reduced-motion tasks. + +Evidence boundary: + +- This run extends direct desktop keyboard evidence only. It is not phone + keyboard, screen-reader, forced-colors, reduced-motion, independent-human, + hosted, or WCAG-conformance evidence. +- The pending narrow remediation still covers the first-focus skip placement, + ticker-specific Discover action names, and native-summary focus visibility. + A stable semantic `main` remains a separately named open design problem. +- Priority 7 remains incomplete. K06-K07, complete K05, phone keyboard, true + zoom, forced colors, reduced motion, S01-S07, the stable `main` fix, and + remaining P02 coverage still require direct evidence or remediation. + +## 2026-07-27 keyboard authoring validation and no-write exit + +Run metadata: + +- Run ID: `a11y-2026-07-27-authoring-k06-k07-04`. +- Commit: `b1dd457b574045c06496fdf4ec6772dddfc7d751`. +- Route base: local + `http://localhost:8501/?mode=research&page=workbench&ticker=AVGO&open=1`. +- Environment: macOS `26.5.1` build `25F80`; Google Chrome + `150.0.7871.182`; default colors and motion. +- Input: direct Chrome keyboard input. Screen reader: `not_run`. Reviewer: + `Local supervised browser review`. +- The run used saved readiness only. It entered no research content, produced + no valid preview, saved no record, captured no screenshot, and wrote no + generated artifact. + +Pre-run ledger state: + +- `data/research_thesis_journal.csv` existed with one header line and SHA-256 + `0c427c359c1bdb6d8c8410aece9b3a46831ffb9ccd91d401aeb7930b5f401717`. +- `data/catalyst_evidence.csv` and + `data/research_outcome_reviews.csv` did not exist. + +Direct task results: + +- K06: `failed_reproducible`. Enter opened + `Add a reviewed research record`. Nine direct Tab steps traversed Record + type, every empty thesis field, Supersedes Entry Id, and + `Validate and preview`; every inspected control had the shared visible + three-pixel focus outline. Enter on the validation button produced + `thesis_id is required` inside one `role="alert"` and exposed no + `Confirm and save` button. However, the affected Thesis Id control had no + `aria-invalid`, `aria-describedby`, or `aria-errormessage`, and focus + remained on `Validate and preview`. The global alert is bounded positive + evidence but does not satisfy the required affected-control association or + screen-reader announcement. +- K07: `passed_direct`. Keyboard activation of the selected Company Workbench + route radio followed by Arrow Down opened Monitor, retained active focus on + the selected Monitor route, and rendered the truthful Monitor state without + a trap. After exit, the thesis journal line count and SHA-256 were unchanged, + the catalyst and outcome ledgers remained absent, and no ledger path was + dirty. + +Evidence boundary: + +- This run does not pass S06 or S07 because no screen reader was used. +- The required K06 remediation is field-level invalid-state and error + association, followed by direct keyboard and supported screen-reader + retesting. Confirmation and persistence must remain unavailable for a + rejected draft. +- Priority 7 remains incomplete. K01, K04-K06, K09, phone keyboard, true zoom, + forced colors, reduced motion, S01-S07, stable `main`, and remaining P02 + coverage still require remediation or direct evidence. + +## 2026-07-27 phone keyboard and mobile-sidebar continuation + +Run metadata: + +- Run ID: `a11y-2026-07-27-phone-keyboard-05`. +- Commit: `c748b192dd4fe27f1c17f2feba07eaccd24e4619`. +- Route base: local `http://localhost:8501/`. +- Environment: macOS `26.5.1` build `25F80`; Google Chrome + `150.0.7871.182`; explicit CSS viewport `390x844`; device-pixel ratio `1`; + default colors and motion. +- Input: direct Chrome keyboard input plus read-only target geometry. + Screen reader: `not_run`. Reviewer: `Local supervised browser review`. +- The run used saved readiness only. It entered no research content, produced + no authoring preview, saved no record, captured no screenshot, and wrote no + generated artifact. + +Direct task results: + +- K01 phone: `failed_reproducible`. After reload, the first focus sequence was + Workspace help at approximately `x=-202`, the selected Workspace radio at + `x=-271`, the selected route radio at `x=-271`, the Streamlit main section, + then the visible `Skip to page answer` link. The first three controls were + in the closed off-canvas sidebar and could not expose perceivable focus in + the `390px` viewport. +- K02 phone: `passed_direct` on ticker-bound AVGO Company Workbench. Enter on + the visible skip link retained `mode=research`, the Workbench route, + `ticker=AVGO`, and `open=1`, appended only `#public-page-answer`, and moved + focus to the answer target. +- K03 phone: `failed_reproducible`. Arrow Down on the off-canvas selected + route radio changed the URL and rendered Discover, but the focused radio + remained at approximately `x=-271`. The route was keyboard-operable but did + not satisfy visible focus. The framework sidebar-open control measured + `0x0`, was hidden, and did not occur in the recorded Tab sequence. +- K04 phone: `failed_reproducible` for the same identity defect as desktop. + Keyboard input reached the visible `334x36` search control, entered and + applied `AVGO`, and opened the correct ticker-bound Workbench from an + approximately `86x44` action. The action's accessible name remained only + `Open review`; the preceding Advanced summary also repeated the + no-visible-outline defect. +- K05/K09 phone: not rerun end to end. The directly encountered focused + Advanced summary again had computed `outline: none`; no broader pass is + inferred. + +P02 mobile-sidebar result: + +- P02: `failed_reproducible` for the mobile sidebar route choices. Research + Desk, Discover, Company Workbench, and Monitor labels measured approximately + `104x50`, but all target boxes began at `x=-279` and ended left of the + visible viewport. The hidden `0x0` sidebar-open control provided no visible + pointer or keyboard entry to those choices. +- Visible main-route controls inspected in this run retained at least `24px` + on each measured target dimension, and Company Workbench retained + `document.scrollWidth == document.clientWidth == 390`. These bounded + positives do not override the off-canvas route-choice failure. + +Evidence boundary: + +- The new defect is mobile navigation visibility, not route-state logic: + off-canvas arrow navigation changed routes deterministically, but users + cannot rely on an invisible focused control. +- Remediation requires a visible, labelled, keyboard-operable mobile route + navigation entry plus direct keyboard and pointer retesting. It must not + duplicate route state, change research readiness, or move technical + evidence into the primary answer. +- Priority 7 remains incomplete. K01, K03-K06, K09, mobile navigation P02, + true zoom, forced colors, reduced motion, S01-S07, stable `main`, and + remaining disclosure-state coverage still require remediation or direct + evidence. + +## 2026-07-28 narrow-remediation direct browser gate + +Run metadata: + +- Run ID: `a11y-2026-07-28-narrow-remediation-gate-07`. +- Product-under-test commit: + `0000c97e7db17e5d4353e30e976f2b7dec6bfd46`. +- Command: `make research-accessibility-browser-check`. +- Environment: macOS arm64; local Streamlit demo profile; Google Chrome at + `/Applications/Google Chrome.app/Contents/MacOS/Google Chrome`; Playwright + headless browser control. +- Routes: Research Desk, Discover, ticker-bound NVDA Company Workbench, and + Monitor. +- Viewports: desktop `1280x720` and phone `390x844`. +- The gate starts one local demo-profile server, reads the rendered DOM, and + returns results in memory/stdout. It does not write timing, JSON, report, + screenshot, readiness, canonical-data, or research-ledger artifacts. +- Before attributing the run, the gate verified the rendered Stock Research + Command Center identity and `Demo` profile, verified a clean + product/code/test/docs tree, and classified and excluded exactly 18 unstaged + generated CSV/output paths under the existing hygiene contract. + +Direct automated results: + +- All eight route-and-viewport cases passed with no rendered traceback and + zero horizontal overflow. +- K01/K02 retest: after initial focus was cleared, one physical Tab focused + the sole `Skip to page answer` link. No DOM-order enumeration or + programmatic link focus substitutes for that keyboard result. Its focused + box was approximately `x=8.8..141.4`, `y=8.8..47.0`, fully within both the + horizontal and vertical bounds of each viewport. Enter activation retained + the complete route query, appended only `#public-page-answer`, and focused + that target. +- K03/P02 narrow-navigation retest: every route exposed one visible + `Personal research workflow` navigation with one current route. Research + Desk, Discover, and Monitor exposed the three applicable routes; the + ticker-bound Workbench additionally exposed Company Workbench. Every + applicable route link was fully inside the horizontal viewport and at least + `44px` high. Desktop navigation began at approximately `y=24`; phone + navigation began at approximately `y=22.4`, and its links wrapped without + leaving the viewport. +- K04 retest: Discover rendered four actual eligible actions in the demo + profile at each viewport. Every action used a unique + `Open {TICKER} review` accessible name that matched its ticker-bound + destination. The gate fails when zero eligible actions render and does not + assume or fabricate a row count. +- K05/K09 retest: the directly focused native disclosure summary on every + route and viewport exposed a solid `3px` outline with color + `rgb(15, 118, 110)`. +- K06 retest: empty thesis validation on Company Workbench retained one + global `thesis_id is required` alert, applied `aria-invalid=true` and one + stable `aria-describedby` target to Thesis Id, rendered the adjacent error, + and focused the affected field. A live draft change then removed the + bridge-owned Thesis Id error node and relationships. The next validation + associated, described, and focused only Effective At for + `effective_at is required`, on both desktop and phone. AppTest additionally + verified cleanup payloads for accepted preview and non-field errors while + preserving all three ledger byte states. No confirmation action or ledger + write was performed. + +Evidence boundary: + +- This is reproducible local engineering evidence from automated direct + browser control. It is not an independent human keyboard review, + screen-reader result, hosted result, or WCAG conformance claim. +- A non-loopback or rendered-identity/profile-mismatched server, staged path, + dirty non-generated implementation path, or absent Chrome/Playwright runtime + fails the gate closed. Such a run receives no local commit/profile + attribution. The gate records no generated evidence file; terminal output + is intentionally ephemeral. +- True 200%/400% browser zoom, forced colors, reduced motion, supported + screen-reader navigation, complete loading/empty/withheld/stale/failure + states, independent human testing, and the separately designed stable + semantic `main` landmark remain incomplete. Priority 7 therefore remains + open. + +## 2026-07-28 framework-safe semantic-main browser gate + +Run metadata: + +- Run ID: `a11y-2026-07-28-semantic-main-gate-10`. +- Product-under-test commit: + `d1328eaa4d08cf08ec2b70939e4e031ee5f907b0`. +- Commands: + `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest + tests/test_research_accessibility_browser_gate.py -q` and + `make research-accessibility-browser-check`. +- Focused result: `22 passed`. +- Environment: macOS arm64; local Streamlit demo profile; Google Chrome at + `/Applications/Google Chrome.app/Contents/MacOS/Google Chrome`; Playwright + headless browser control. +- Routes: Research Desk, Discover, ticker-bound NVDA Company Workbench, + Monitor, Research Data Health, and Research Proof History. +- Viewports: desktop `1280x720` and phone `390x844`, for 12 + route-and-viewport results. +- Before attributing the browser run, the gate verified the rendered Stock + Research Command Center identity and `Demo` profile, verified a clean + product/code/test/docs tree, and classified and excluded exactly 18 + unstaged generated CSV/output paths under the existing hygiene contract. + +Direct automated results: + +- The gate verdict was `passed`, with an empty failure list and all 12 + route-and-viewport results passing. +- Every initial route DOM exposed exactly one role-based `main` with exact + `role="main"`, `id="research-main"`, and + `aria-label="Stock research workspace"`. That unique landmark contained + exactly one `#public-page-answer` and exactly one level-one heading, and the + host status was exactly + `data-research-main-bridge-status="applied"`. +- Each case activated the exact unselected `Public visitor mode` Workspace + radio through a controlled native DOM event. The installed Streamlit + `1.59.2` test-state contract recorded a complete + `notRunning` -> `running` -> `notRunning` script cycle while retaining the + same top document, recording zero top-level frame navigations, and + preserving the exact pathname and query. +- After that same-document cycle, the gate required the bridge target to be + the current connected `stMain`. It changed the host status to a probe value + and appended one hidden, `aria-hidden` inert node beneath that target; the + active bridge observer had to restore the exact `applied` status before the + probe node was removed. The gate then repeated the exact route heading, + landmark count, metadata, answer-target count, heading count, and applied + host status. Research Data Health and Research Proof History additionally + proved the intentional absence of the primary + `Personal research workflow` navigation before and after the script cycle. +- Separately from that same-document proof, each case navigated to its + explicitly mapped different Research route and then back to the exact + original route. After marker, DOM-stability, and exact route-H1 waits, the + gate required the full URL to match exactly, including the complete query + string and empty fragment, with the ticker parameter where present. It + repeated the exact semantic-main and runtime assertions on both the away + and returned DOMs, required the correct primary workflow navigation where + applicable, and explicitly required its absence on secondary evidence + routes. +- The existing primary-route assertions remained active. After initial focus + was cleared, one physical Tab focused the sole skip link; Enter preserved + the route and focused the one `#public-page-answer`, and that focused target + was inside the unique main. Primary workflow navigation geometry, summary + focus outline, Discover action names and destinations, and Workbench + field-error association and cleanup also passed where applicable. +- All 12 cases reported no browser console error, uncaught page error, + rendered traceback, or document-level horizontal overflow. Traceback and + overflow checks were repeated after the same-document Streamlit script + cycle, observer-liveness probe, away transition, and exact-route return. +- The fixed same-origin bridge accepts no research-content input and performs + no application action. The gate triggers only the controlled Workspace + widget rerun and inert observer probe before deliberately navigating to one + deterministic away route and back for transition evidence. It performs no + research-data write or persistence action and remained repository/data + read-only and in-memory/stdout-only: it wrote no screenshot, timing, JSON, + report, readiness, canonical-data, research-ledger, or generated repository + artifact. + +Evidence boundary: + +- This closes the stable route-level semantic-main defect only for the + recorded local automated DOM matrix at the exact implementation anchor. + It does not establish assistive-technology behavior beyond those DOM + assertions. +- Automated DOM verification is not WCAG conformance, screen-reader landmark + navigation or usability, hosted behavior, independent-human accessibility + validation, or complete keyboard-order evidence. +- The native DOM radio activation and Streamlit test-state transition are + controlled framework engineering evidence only. They do not prove pointer, + keyboard, or closed-mobile-sidebar operability, and the installed runtime's + test-state attribute is not a public cross-version compatibility guarantee. +- True 200%/400% browser zoom and reflow, forced colors, reduced motion, + supported screen-reader tasks, dynamic announcements, complete + loading/empty/withheld/stale/failure states, remaining small framework + controls, and independent-human testing remain incomplete. Priority 7 + therefore remains open. + +## 2026-07-29 same-document transport behavior and instrumentation + +Verified implementation evidence: + +- Product-under-test commits: + behavior-first repair `e8084099b3ea1b794ce8e2a0af00998602133084`; + exact transport-instrumentation run + `d68ab27bee9c07c450faeb866b08cbf13638b56f`. +- Supported dependency contract: `streamlit>=1.52,<2`; installed local + runtime: Streamlit `1.59.2`. +- Direct command: `make research-accessibility-browser-check`. +- Environment: macOS arm64; local `Demo` profile; Google Chrome at + `/Applications/Google Chrome.app/Contents/MacOS/Google Chrome`. +- Routes: Research Desk, Discover, ticker-bound NVDA Company Workbench, + Monitor, Research Data Health, and Research Proof History. +- Viewports: `1280x720` and `390x844`, for 12 route-and-viewport results. +- Result: `passed`, with an empty failure list. Repository hygiene classified + and excluded exactly 18 unstaged generated CSV/output paths. + +The direct run proved that the behavior-first `st.html` correction retained +one applied same-document main landmark, one contained answer, exact route H1, +skip focus and activation, exact query retention, rerun and mutation recovery, +away-and-return route recovery, Workbench Thesis Id cleanup and Effective At +rebinding, zero browser error, and zero horizontal overflow across the +recorded matrix. It supersedes the earlier iframe-topology failure only for +this exact local runtime and commit. + +The browser gate now also has fail-closed per-result fields for +`deprecated_component_warning_count`, `bridge_iframe_count`, +`bridge_focusable_count`, and `bridge_height`. Focused tests first failed +because the evaluator and live-DOM observation did not exist, then returned +`33 passed` after implementation and review hardening. Counts accept only +exact nonnegative integers; booleans, fractional values, negative values, and +strings fail closed. The normal gate-owned local server retains a bounded +in-memory stdout/stderr tail and increments the deprecation counter during +streaming against the full normalized line, before line-length truncation or +old-line eviction; lock-protected snapshots avoid reader/route races. Reader +exceptions and a reader still alive after the bounded shutdown join change +capture status to explicit failed/incomplete states, which fail the overall +verdict. An alive reader is not synchronously closed after timeout because a +shared `TextIOWrapper` lock could block the gate; normal and exception readers +that have stopped are closed. That exact count is combined with browser console and rendered +messages. Explicit `BASE_URL` mode reports +server output as `unavailable_external_base_url` and cannot receive a strict +passing transport verdict. `make research-dashboard-render-smoke` +passed all six Research routes without the former +`st.components.v1.html` deprecation warning. + +The clean exact-head run at `d68ab27bee9c07c450faeb866b08cbf13638b56f` +closed the local transport-instrumentation evidence item. All 12 +route-and-viewport results reported +`deprecated_component_warning_count=0`, `bridge_iframe_count=0`, +`bridge_focusable_count=0`, and `bridge_height=0`; each result recorded +`server_runtime_output_status=captured_local_server`. The overall bounded +server stdout/stderr evidence also passed with zero deprecated-component +warnings. The same run had an empty failure list and excluded the same 18 +generated paths. The surrounding verification returned 4,381 passing tests +and passed dashboard, Research render, public wording, public package, +commercial-beta, pilot-readiness, hygiene, and whitespace gates. + +Hosted exact-head CI remains separate and must be reverified after the +intentional push. The local result does not expand the evidence boundary below. + +Evidence boundary: + +- The bridges execute fixed local scripts only and do not read research + content, change readiness, persist a record, or perform a research action. +- No generated data, screenshot, JSON, report, or timing artifact was created + by these checks. +- This is local automated engineering evidence, not screen-reader, WCAG, + hosted, cross-major-version, independent-human, or market validation. + +## 2026-07-29 research-state implementation and environment-limited gate + +Current local implementation evidence: + +- A closed five-transition helper maps validation rejection and reload + uncertainty to assertive alerts, and preview readiness, edited drafts, and + verified reloads to polite statuses. +- Company Workbench authoring uses exact profile, ticker, record kind, receipt + or digest, and persisted record identity to deduplicate transitions. The + same state remains visible on a normal rerender without another live node. +- Focused pure and authoring tests passed for escaping, deterministic identity, + required-field association, preview-unsaved wording, edited-draft recovery, + verified reload, reload uncertainty, and temporary-ledger isolation. +- A synthetic AppTest harness now covers six ordinary static states and all + five transitions using the production renderer and TEST1-only content. +- The final surrounding full suite reported 4,424 passing tests and one existing + skip. Its only two failures were environment failures: the managed sandbox + rejected loopback socket binding before the two direct browser tests could + start. + +Evidence boundary: + +- Implementation commit `d353ed652` contains the 18-file product package. + Exact staged hygiene reported 18 product/code/docs/test files, zero generated + files, zero canonical-data files, and zero manual-review paths. +- The browser gate now runs the harness at both viewports, rejects duplicate or + hidden live nodes, verifies unchanged rerenders become visible non-live + messages, checks overflow/errors/tracebacks, and compares repository status + byte-for-byte before and after. Discover three-answer rows and Monitor + process-only semantic tables now have direct browser assertions too. Its + repository fingerprint includes Git status plus current content for every + tracked dirty and untracked path, so another write to an already-modified + generated file also fails the gate. +- The final focused browser/state/dashboard/document contract run passed 1,220 + tests; all six Research routes also passed the non-browser render smoke. +- The clean-tree gate verified product hygiene and excluded exactly 18 + unstaged generated paths. The environment then terminated headless Chrome + before any route or synthetic-harness case executed, so no desktop/phone + result is credited. +- A supported direct browser run, push, draft-PR update, and exact-head CI + remain open. +- No production research ledger or generated CSV, JSON, report, screenshot, or + timing artifact was written by the candidate tests. +- This is not screen-reader, WCAG, hosted, independent-human, or market + validation evidence. + +## 2026-07-31 clean-tree research-state accessibility matrix + +Direct current evidence at commit +`6328c8cead7c27cb901e7878cd6d7d23fa11bb0e` closes the previously +environment-limited automated state-harness matrix: + +- `make research-accessibility-browser-check TIMEOUT_SECONDS=90` returned + `verdict=passed` with an empty failure list on local Chrome. +- The gate first verified the rendered Stock Research Command Center `Demo` + identity and a clean product/code/test/docs tree while classifying and + excluding exactly 18 unstaged generated paths. +- All 12 Research Desk, Discover, ticker-bound Company Workbench, Monitor, + Research Data Health, and Research Proof History route-and-viewport results + passed at `1280x720` and `390x844`. +- The separate synthetic state harness passed at both viewports. Its six + ordinary static states—loading, empty, withheld, stale, failure, and + validation—were visible and non-live, with `aria-busy=true` only for + loading. Validation rejection and save-reload uncertainty used one assertive + alert; preview readiness, draft change, and verified reload used one polite + status. An unchanged rerender kept one visible non-live message instead of + repeating an announcement. +- Both harness cases had no traceback, horizontal overflow, console error, or + page error. The repository snapshot remained byte-for-byte unchanged before + and after the harness. Bounded local server output was captured with zero + deprecated-component warnings. + +Evidence boundary: + +- The state harness is synthetic, test-only, read-only engineering evidence. + It does not prove that a screen reader announces each transition correctly. +- It does not complete independent-human keyboard order, 200%/400% zoom and + reflow, forced colors, reduced motion, screen-reader tasks, hosted behavior, + remaining small framework controls, or material-defect retests. +- It proves no WCAG conformance, research-data readiness, source rights, + investment conclusion, market validation, or product-market fit. +- It wrote no repository CSV, JSON, report, screenshot, timing, readiness, + canonical-data, or research-ledger artifact. + +## 2026-07-31 automated media-preference matrix (historical anchor) + +Direct local engineering evidence at exact implementation anchor +`a97a096e1caf2761176dc42798a8c9686179aecc`: + +- `make research-accessibility-browser-check TIMEOUT_SECONDS=90` returned + `verdict=passed` with an empty failure list on local Chrome. +- The gate verified Research Desk, Discover, ticker-bound Company Workbench, + Monitor, Research Data Health, and Research Proof History at both + `1280x720` and `390x844`; all 12 route-and-viewport results passed. +- Every result passed the then-current automated forced-colors evaluator. The + media query was active; the first physical Tab focused the sole skip link + with a visible outline; and the H1, research-only boundary, boundary border, + overflow, and traceback checks passed. Final review found that this anchor's + route-marker measurement combined border and outline widths, so a generic + one-pixel workflow-link border could receive active-marker credit. It also + did not independently count and verify a route-specific rendered state + marker and route-appropriate next action. Do not reuse this anchor as proof + of those narrower contracts. +- Every result passed the then-current automated reduced-motion evaluator. The + media query was active; application-owned motion targets were present; + maximum animation and transition duration was `0.01ms`; maximum animation + iteration count was one; scroll behavior was `auto`; and the H1 and boundary + remained visible without overflow or traceback. This anchor did not + independently count and verify a route-specific rendered state marker and + route-appropriate next action. +- The final-review contract now observes route state markers and next actions + with exact counts and visibility in both emulated modes. Its forced-colors + active-route assertion separately requires the two-pixel active border and a + non-`none`, positive-width outline; missing, malformed, duplicate, or hidden + observations fail closed. A later exact-fix-HEAD direct-browser result is + required before expanding this historical evidence claim. +- Ordinary media preferences were restored between emulated phases. The + existing landmark, physical-Tab skip focus and activation, workflow + navigation, applicable Discover and authoring, same-document rerun, + away-and-return routing, runtime-warning, browser-error, and overflow + assertions remained green. The separate dynamic-state harness passed at + both viewports, and its repository snapshot remained byte-for-byte + unchanged. +- Repository hygiene still classified and excluded exactly the same 18 + unstaged generated CSV/output paths. The run wrote no repository JSON, + timing, screenshot, report, readiness, canonical-data, or research-ledger + artifact. + +Evidence boundary: + +- Automated media emulation is bounded engineering evidence only. It does not + pass manual protocol tasks C01, C02, or M01 and is not direct operating- + system forced-colors, platform reduced-motion, or human review evidence. +- C01, C02, M01, true 200%/400% zoom and reflow, screen-reader, + assistive-technology, independent-human, hosted, and WCAG evidence remain + incomplete. Priority 7 and + `accessibility_manual_review_environment_required` remain open. + +## 2026-08-01 Company Workbench HTML Research Brief actual-byte matrix + +Direct local browser evidence at implementation anchor +`6ad7f34310652f1b172525a0b8f00becf874c44c` exercised the production +renderer's actual UTF-8 download bytes for complete, partial, and withheld +synthetic test states at `1280x720, 390x844, and 640x900`. + +- Each of the nine state-and-viewport cases decoded and re-encoded the exact + bytes, loaded them in Chromium, and checked semantic landmarks, one H1, + captioned tables, the exact offline CSP, visible research boundaries, + blockers, provenance, and no active content or network request. +- The browser-automated Tab and Enter input verified the skip-link focus cue and target. + The matrix also checked print media, forced-colors emulation, + reduced-motion emulation, visible boundaries and provenance, no horizontal + overflow, no console or page error, and a nonempty `%PDF` in-memory PDF. +- The repository fingerprint was identical before and after the matrix. No + HTML, PDF, screenshot, JSON, timing, readiness, canonical-data, report, or + research-ledger artifact was written. + +This is automated engineering evidence only. Synthetic states are test-only; +the matrix does not prove human or screen-reader usability, true zoom/reflow, +direct operating-system media-preference behavior, hosted operation, WCAG +conformance, source rights, current data, independent review, or market fit. +Local engineering evidence does not establish source rights, current-market data, readiness activation, a new or professional line-item model, hosted operation, human or screen-reader conformance, independent validation, market fit, screening alpha, or probability calibration. diff --git a/docs/ACCESSIBILITY_TASK_PROTOCOL.md b/docs/ACCESSIBILITY_TASK_PROTOCOL.md new file mode 100644 index 000000000..da9a3b383 --- /dev/null +++ b/docs/ACCESSIBILITY_TASK_PROTOCOL.md @@ -0,0 +1,166 @@ +# Accessibility Task Protocol + +This protocol defines repeatable manual accessibility review for the supported +Personal Research workflow: + +`Research Desk -> Discover -> Company Workbench -> Monitor` + +It is an execution template, not test evidence. A completed row proves only the +named task in the recorded environment. No result may be inferred from a screenshot, +source inspection, an automated contract, or a different route. +No WCAG conformance claim may be made from a partial run. + +## Safety boundary + +- Use a local or explicitly approved private review environment. +- Use saved readiness only. Do not refresh, import, apply, or rebuild data. +- Do not save a research record. Workbench authoring may be inspected only up + to validation or a write-free preview. +- Do not enter personal, account, brokerage, portfolio, or confidential + research information. +- Do not stage screenshots, timing output, CSV, JSON, reports, sample reports, + or other generated artifacts. +- Findings must not weaken research-only, provenance, source-rights, + calibration, explicit-Q4, EPS split-basis, or fail-closed boundaries. + +## Required run metadata + +Record every field before starting: + +| Field | Required value | +| --- | --- | +| Run ID | Stable local identifier | +| Date and timezone | ISO date plus timezone | +| commit SHA | Exact checked-out revision | +| Route base URL | Local or approved private URL | +| Operating system | Name and version | +| Browser | Name and version | +| Input method | Keyboard and any assistive input | +| Screen reader | Name/version or `not_run` | +| Display | Resolution and scaling | +| Viewport | CSS width and height | +| Zoom | Browser zoom percentage | +| Color mode | Default, forced colors, or named contrast mode | +| Motion preference | Default or reduced motion | +| Reviewer | Non-sensitive identifier | + +Stop if the checked-out revision changes during the run. Start a new run ID +after any code change or environment change that can affect the result. + +## Result vocabulary + +Use exactly one state per task: + +- `passed_direct`: the reviewer directly completed the named task and observed + every expected result in the recorded environment. +- `failed_reproducible`: the task failed and the reviewer recorded exact + reproduction steps plus observed behavior. +- `not_run`: the task was not attempted. +- `blocked_environment`: the named browser, assistive technology, operating + mode, or review capability was unavailable. +- `not_applicable`: the task does not apply, with a written reason. + +Never promote `not_run`, `blocked_environment`, automated checks, screenshots, +or a result from another route to `passed_direct`. + +## Core keyboard tasks + +Run these tasks without a mouse or touch input. + +| ID | Mode | Task | Expected direct result | Failure evidence | +| --- | --- | --- | --- | --- | +| K01 | `keyboard_only` | Reload Research Desk and press Tab until the skip link or first repeated shell control receives focus. | Focus is visible; the actual order is recorded. If repeated shell controls precede the skip link, record a defect rather than inferring success from DOM presence. | Focused control sequence and first unexpected control | +| K02 | `keyboard_only` | Activate `Skip to page answer`. | The URL retains Personal Research mode and the current route parameters; focus moves to or immediately before the route answer without changing evidence state. | URL, focused element, and visible position | +| K03 | `keyboard_only` | Traverse the route rail and open Discover. | Every route choice is reachable, visibly focused, and operable without trapping focus. | Last reachable control and blocked action | +| K04 | `keyboard_only` | Use Discover search and open one readiness-backed company. | Search, result, and open action have understandable names; no ranking or recommendation language is announced. | Control name, announced text, and route | +| K05 | `keyboard_only` | Traverse the Company Workbench primary answer and each collapsed disclosure. | Reading/focus order follows the visible research answer; Advanced evidence remains optional; disclosures expose state and respond to Enter or Space. | First order mismatch or inoperable disclosure | +| K06 | `keyboard_only` | Open the authoring composer, enter a deliberately incomplete draft, and validate without previewing or saving. | Required-field errors are programmatically associated or otherwise announced at the affected control; no confirmation action appears before an exact valid preview. | Error text, focused control, and announcement | +| K07 | `keyboard_only` | Cancel or leave the composer, then open Monitor. | No research record is written; navigation remains operable and focus is not trapped. | Unexpected write, trapped focus, or lost route | +| K08 | `keyboard_only` | Traverse Monitor in its truthful empty or current saved state. | Weekly summary, Research Discipline Review, change state, and next action are understandable without implying a ranking or fabricated event. | Announced text and first misleading state | +| K09 | `keyboard_only` | Reverse through the current route with Shift+Tab. | Visible focus remains present and no keyboard trap appears. | Trap boundary and last reachable control | + +## Zoom and reflow tasks + +Run at the named browser zoom with the viewport and operating-system scaling +recorded. Local scrolling inside a genuinely two-dimensional data table may be +recorded separately; it does not excuse document-level horizontal overflow. + +| ID | Mode | Task | Expected direct result | +| --- | --- | --- | --- | +| Z01 | `zoom_200` | Complete Research Desk -> Discover -> Company Workbench -> Monitor at 200% zoom. | Primary answers, labels, controls, and research-only boundaries remain readable and operable without document-level horizontal scrolling. | +| Z02 | `zoom_400` | Repeat the core answer path at 400% zoom. | Content reflows to a single-column reading path; no primary content or action is clipped, overlapped, or hidden. | +| Z03 | `zoom_400` | Open each Workbench Advanced disclosure used in K05. | Disclosure content remains reachable; any table-local scrolling is bounded and does not move the whole document horizontally. | + +## Color, contrast, and motion tasks + +| ID | Mode | Task | Expected direct result | +| --- | --- | --- | --- | +| C01 | `forced_colors` | Traverse all four routes with the platform forced-colors mode active. | Focus, selected route, boundaries, status distinctions, and primary actions remain perceivable without color alone. | +| C02 | `forced_colors` | Inspect disabled, blocked, partial, and usable states. | Text or semantic labels preserve each distinction when authored colors are overridden. | +| M01 | `reduced_motion` | Load and navigate all four routes with reduced motion enabled. | No required information depends on animation; loading and disclosure state remain understandable. | + +## Screen-reader tasks + +Use a supported desktop screen reader and record its exact name, version, and +speech settings. Do not substitute an accessibility-tree snapshot for these +tasks. + +| ID | Mode | Task | Expected direct result | +| --- | --- | --- | --- | +| S01 | `screen_reader` | Read the page title, route heading, and research-only boundary on each route. | One route-level heading is announced and the boundary is available without reading Advanced evidence. | +| S02 | `screen_reader` | Use the landmarks and headings lists. | Available landmarks and the heading hierarchy match the current page. If no main landmark is announced, record the open defect. | +| S03 | `screen_reader` | Activate the skip link on each route, including a ticker-bound Workbench URL. | The destination and retained route/ticker parameters are announced or directly verifiable; focus transfer is understandable. | +| S04 | `screen_reader` | Operate Discover search and open a company. | Search purpose, result identity, readiness boundary, and open action are announced without recommendation language. | +| S05 | `screen_reader` | Traverse Workbench cards, disclosures, and withheld states. | Section names, status, evidence boundary, and one next research task are understandable in reading order. | +| S06 | `screen_reader` | Trigger the incomplete authoring validation used in K06. | The invalid state, error, and affected field are announced; no record is saved. | +| S07 | `screen_reader` | Observe a Streamlit rerun or validation update. | Material status changes are announced or the missing announcement is recorded as a reproducible defect. | + +## Target-size and pointer follow-up + +Measure the visible hit target, not only the hidden native input: + +| ID | Task | Expected direct result | +| --- | --- | --- | +| P01 | Measure framework help controls and dataframe toolbar controls used in the workflow. | Each material target meets the applicable target-size requirement or has a documented exception and adjacent-spacing evidence. | +| P02 | Measure route choices, primary links, disclosure headers, search controls, and authoring actions at phone width. | Targets do not overlap and can be activated without triggering an adjacent control. | + +## Finding record + +Create one record per failed task: + +```text +finding_id: +run_id: +commit_sha: +task_id: +route: +severity: +environment: +precondition: +steps: +expected: +observed: +focus_or_announcement: +artifact_location: +research_boundary_impact: +proposed_fix: +retest_run_id: +retest_state: +``` + +Use `critical`, `high`, `medium`, or `low` severity. Severity reflects blocked +task completion and misuse risk, not investment impact. + +## Completion rule + +The accessibility priority remains incomplete until: + +1. Every applicable task has `passed_direct` evidence in a suitable current + environment or a reviewed, explicitly bounded exception. +2. Every material failure has a fix and direct retest or a documented product + boundary with an owner. +3. Desktop and phone workflow evidence covers the same exact revision. +4. Automated semantic contracts and manual results agree. +5. Any public accessibility statement matches only the tested scope. + +The existence of this protocol is not completion evidence. diff --git a/docs/COMPLETED_MILESTONES.md b/docs/COMPLETED_MILESTONES.md index 964be2e1f..76cdb58e2 100644 --- a/docs/COMPLETED_MILESTONES.md +++ b/docs/COMPLETED_MILESTONES.md @@ -9,6 +9,16 @@ This document is the historical completion log for Stock Research Command Center - ETF and index-proxy exclusion from operating-company DCF, with blocked inputs withheld rather than inferred. - Master-universe, active-universe, and analysis-ready scopes separated in reports and dashboard flows. +## Personal Research And Commercial Beta Foundation + +- The primary local research flow is Research Desk -> Discover -> Company Workbench -> Monitor. Public and Operator routes remain separate, and Data Health and Proof History stay available as Advanced Evidence without changing readiness. +- Research Desk, Discover, Company Workbench, and Monitor provide answer-first, fail-closed views over saved evidence. The shared shell, responsive route matrix, observation-recency interpretation, and downloadable offline Company Workbench brief are complete as local engineering work. +- Research Decision Lab and the collapsed Company Workbench authoring flow support append-only thesis, counter-thesis, evidence, catalyst, invalidation, scenario-assumption, and outcome records through validation, exact preview, and explicit confirmation. Saving a research record cannot change readiness, forecasts, probabilities, recommendations, or another ledger. +- Priority 1 quarantines legacy portfolio, ranking, position, picks, entry-zone, and transaction-like surfaces behind an Operator-only compatibility boundary. They cannot feed Personal Research, readiness, recommendations, sizing, or transaction behavior. +- Priority 2 provides a prospective-only field-proof audit with independent technical-write and commercial-evidence eligibility, receipt revalidation, and no readiness mapping. It does not upgrade legacy narrative proof or check in sample proof rows. +- Priority 3 completes in-app research-record authoring with receipt-bound confirmation, append locking, and active-thesis lineage checks. Production verification uses temporary ledgers and does not append repository research records. +- These milestones establish a local Commercial Research Beta release candidate and controlled demo package only. They do not establish hosted operation, current-market data, source rights, independent-human or assistive-technology validation, demand, screening performance, probability calibration, or commercial launch. + ## Public Workflow And Reports - Visitor-first public workflow: Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History. diff --git a/docs/DASHBOARD_QA.md b/docs/DASHBOARD_QA.md index 63de9bd87..861f32e4c 100644 --- a/docs/DASHBOARD_QA.md +++ b/docs/DASHBOARD_QA.md @@ -2,6 +2,229 @@ This file records lightweight browser QA evidence for public-facing dashboard polish. +## 2026-08-05 Personal Research Shared Shell + +Personal Research now has one visible top workflow navigation for Research Desk, +Discover, Company Workbench, and Monitor. The sidebar still selects Personal, +Public, or Operator workspace mode and shows the selected data profile, but no +longer repeats the Research page choices. The Research main column no longer +renders the Operator command/readiness header or the broad profile strip before +the route-specific answer. Public and Operator shells, direct route URLs, ticker +parameters, and the secondary Data Health and Proof History evidence routes are +unchanged. + +At implementation anchor `2bc374ab8`, the clean read-only accessibility browser +gate passed all six routes at `1280x720` and `390x844`. It verified the labelled +top navigation on the four primary routes, exact route/H1/main identity, skip +focus, same-document rerun and away/return route preservation, forced-colors and +reduced-motion signals, route-owned primary answer markers, no horizontal +overflow, no browser or runtime errors, and unchanged repository state. The +same 18 generated data/output modifications were excluded. This is automated +local engineering evidence only, not independent-human, screen-reader, WCAG, +hosted, or market validation. + +## 2026-08-05 Research Desk Today's Brief + +Research Desk now renders one primary **Today's Research Brief** instead of a +weekly-card row plus four overlapping question cards. The brief asks **What +needs my attention today?**, gives one saved-evidence answer and reason, exposes +the saved-readiness warning, and provides one Monitor-or-Discover action. Its +stop rule says the answer is saved-workspace evidence rather than a +market-complete event feed, recommendation, or trade instruction. Weekly, +cohort, observation, coverage, and source-change detail remains available under +collapsed **Advanced Evidence**. + +At implementation anchor `5a2aaf2a44dc7ae371161bb8642ab7f0755ad019`, the +read-only accessibility browser gate passed all six Research routes at +`1280x720` and `390x844`. The Desk brief and action were visible in normal, +forced-colors, and reduced-motion observations; the route had no horizontal +overflow, browser error, rendered traceback, or repository write. The gate +classified and excluded the same 18 generated data/output modifications. This +is local automated engineering evidence only, not independent-human, +screen-reader, WCAG, hosted, or market validation. + +## 2026-08-05 Monitor Follow-up Queue + +Exact local runtime anchor `199fa94b266a8bb325bfc4a4df1742158282d47a` +passed the automated direct-browser gate at `1280x720` and `390x844` across all +six Personal Research routes. Monitor exposed five visible labels — **SINCE LAST +REVIEW**, **NEEDS VERIFICATION**, **WAITING ON EVIDENCE**, **SCHEDULED CONTEXT**, +and **EVIDENCE FRESHNESS** — in a desktop two-column grid and a phone one-column +layout. The old Evidence Monitor Brief, Research Discipline Review, and Research +change monitor no longer compete as separate primary summaries. A controlled +empty state appears once, preserves the external-event boundary, and exposes +one Open Discover action. Complete stable-order process rows, identities, and +source-change evidence remain available in **Advanced: Monitor evidence**. +Neither viewport had horizontal overflow, a rendered traceback, console/page +errors, or server runtime errors. The same 18 protected generated working-tree +paths remained unchanged and unstaged after the gate. + +This is automated local engineering evidence only. It does not establish human +or screen-reader accessibility, source rights, current-market validity, hosted +operation, independent workflow validation, probability calibration, or market +validation. + +## 2026-08-01 Company Workbench HTML Research Brief + +Open `?mode=research&page=company&ticker=` with an explicit selected +ticker. Company Workbench displays **Download HTML Research Brief** only after +preparing the same session evidence and Scenario Lab result used by the visible +research answer. The download preserves complete, partial, and withheld states; +unsupported fields remain withheld independently and the brief stays +research-only. + +Run `make company-workbench-html-browser-check` for the direct actual-byte +complete/partial/withheld browser, keyboard, responsive, print, media- +preference, PDF-in-memory, CSP, and repository-fingerprint matrix. No HTML, PDF, screenshot, JSON, timing, readiness, canonical-data, or report artifact is written. This command does not refresh data, acquire a source, activate readiness, or establish human/screen-reader accessibility. + +This historical pre-fix matrix was recorded on 2026-08-01: commits `b10ebcfbd`, `923fa850e`, `9ce1a6ce4`, `84562a7f6`, and `8218af401` passed the 101-test HTML-browser matrix, while the six modal-passive reproductions were withheld. It is not current-head evidence. Exact-head repair evidence: commit `b69badfc80424d3a97fae5f77706aa6ed1533167` passed the 5,828-test full suite, the required dashboard, render, HTML, accessibility, public, and hygiene gates, branch/PR synchronization, and exact-head GitHub Actions run `30726301045`. This remains automated engineering evidence only and does not establish human or screen-reader accessibility. + +Local engineering evidence does not establish source rights, current-market data, readiness activation, a new or professional line-item model, hosted operation, human or screen-reader conformance, independent validation, market fit, screening alpha, or probability calibration. + +## 2026-07-18 Personal Research Evidence Detour Continuity + +Read-only AppTest coverage now verifies six Personal Research surfaces: Research Desk, Discover, Company Workbench, Monitor, Research Data Health, and Research Proof History. Data Health and Proof History stay inside Personal Research mode when opened from Company Workbench Advanced Evidence, preserve the selected ticker, and show **Return to Company Workbench** before existing evidence content. + +The same-mode detour does not change readiness or evidence state, add a route, expose Operator commands, refresh data, record a review outcome, or treat Proof History as an unlock. The active continuation contract prohibited new screenshots, so this pass proves route and render continuity only; it does not prove visual spacing, focus order, contrast, assistive-technology behavior, hosted behavior, or accessibility compliance. + +## Company Workbench Authoritative Task Contract + +Company Workbench renders one overall `ONE NEXT TASK` card. The change-answer contract explicitly distinguishes `none`, `snapshot_only`, and `source_backed`: no matching change renders a neutral no-queued-change badge, snapshot-only context renders only its own badge, and eligible source-backed context renders the source-backed badge. A change can win only when its answer also carries strict source-backed eligibility. Open items keep their suggested review task; still-blocked and intentionally deferred items preserve their existing wait and monitor routing plus the recorded wait condition. Forward View keeps lane-specific guidance and is not a competing overall task. This composition does not alter readiness or evidence states, which remain independent. + +A focused Streamlit AppTest regression renders the normal AVGO Company Workbench route with zero exceptions, a scoped no-queued-change Evidence Change card without a snapshot-only badge, exactly one `ONE NEXT TASK` marker, the current peer-evidence priority `Add peer mappings`, and one `FORWARD-VIEW LANE UNBLOCK` marker. It rejects the retired uppercase `NEXT RESEARCH TASK` kicker while retaining the established title-case `Next Research Task` section heading. This is deterministic local render evidence only, not current-data, source-rights, hosted, or visual-browser evidence. + +## Readiness Continuation Gate + +When the selected profile is stale, incomplete, unverified, or current but untracked, the readiness continuation gate makes `make readiness-preview TOP_N=20` the only continuation-safe action. Project-status next steps are suppressed, while provider setup details, coverage rankings, scheduled-operation details, and Advanced Data Health cards remain planning context only. Advanced Data Health cards derive the gate from the selected profile even when their cached preflight predates the gate. `make readiness` is a separate intentional reviewed write; the UI and CLI must not imply that the preview refreshes data, makes saved readiness current or tracked, or authorizes source execution. + +## 2026-07-18 Monitor Answer-First Live Review + +The default-profile Monitor route was reviewed before and after the hierarchy +change at `1280x720` and `390x844`. Before the change, five-company Earnings +Nowcast readiness interrupted the path between the weekly summary and the +actual research-change answer. After the change, the weekly summary is followed +by `Research change monitor`, the neutral empty-queue answer, and one Open +Discover action. Five-company readiness cards and full rows remain unchanged +inside the collapsed `Advanced: five-company Earnings Nowcast readiness` +drawer. + +At both widths, no details element was expanded, the technical readiness card +was absent from visible text while Advanced was closed, and the document width +matched the viewport exactly (`1280px` and `390px`). The browser recorded no +errors. The phone review confirmed the answer, Open Discover action, and all +three Advanced drawers in reading order without horizontal overflow. Computed +styles confirmed the primary action and its nested text use a white foreground +on the existing primary-button background; the action measured approximately +`110x40px`. + +Reviewed before/after screenshots were saved outside the repository and remain +unstaged. They prove local route hierarchy, responsive reflow, and visible +control styling only; they do not prove current market data, source rights, +hosted behavior, full keyboard or assistive-technology support, or predictive +validity. + +## 2026-07-18 Research Desk Answer-First Live Review + +The live default-profile Research Desk route was reviewed at `1280x720` and +`390x844`. The workspace boundary and Discover next action were followed by the +weekly research summary, four direct research answers, and the Open Discover +action. Focused-cohort scope, concise lane coverage, full matrices, and weekly +rows remained available inside the existing collapsed `Advanced Evidence` +drawer. + +At both widths, no details element was expanded by default and the hidden cohort +cards were absent from rendered visible text. The weekly summary, `What should I +review next?`, Open Discover, and Advanced Evidence remained present. The +document width matched the viewport exactly (`1280px` and `390px`), with no +horizontal overflow, and the browser recorded no errors. + +Reviewed screenshots were saved outside the repository in the temporary review +workspace and remain unstaged. They prove local layout and route composition +only; they do not prove current data, source rights, hosted behavior, or +predictive validity. + +## 2026-07-18 Company Workbench Answer-First Live Review + +The live default-profile Company Workbench route for NVDA was reviewed at +`1280x720` and `390x844`. The route rendered the workspace boundary and one +next action, then kept selected-company lane cards closed under +`Advanced: selected-company lane coverage`. The unchanged selected-ticker +answer remained visible and the full review continued through What Changed, +Business Trend, Valuation, Forward View, What Remains Withheld, Research +Conclusion, and Next Research Task. + +At both widths, no details element was expanded by default and the hidden lane +cards were absent from rendered visible text. The document width matched the +viewport exactly (`1280px` and `390px`), with no horizontal overflow. The phone +first view retained the research-only boundary and primary next action before +the selected-company content continued below it. + +Reviewed screenshots were saved outside the repository in the temporary review +workspace and remain unstaged. They prove local layout and route composition +only; they do not prove current data, source rights, hosted behavior, or +predictive validity. + +## 2026-08-01 Personal Research Workflow Peer-Readiness Review + +The live local workflow was reviewed in order through Research Desk, Discover, +AVGO Company Workbench, and Monitor. Research Desk exposed one next action; +Discover kept the empty strict Momentum & Valuation queue separate from its +alphabetical readiness-backed review rows; Monitor preserved process timing as +non-ranking context. The first AVGO Workbench load exposed a contradictory peer +projection: broad peer trend readiness briefly unlocked peer-input wording even +though `peer_valuation_comparison_ready` was false and the settled report +withheld peer-relative valuation. + +The selected answer, next-step routing, reader guide, quick read, methodology, +and source audit now use peer-valuation readiness for valuation claims while +retaining independently ready peer-trend context. A fresh server session showed +the same fail-closed AVGO answer before and after the saved report settled: +standalone DCF reviewable, peer trend context separately available, and +peer-relative valuation still withheld. Audit screenshots remain under +`/tmp/stock-research-workflow-audit-2026-08-01` and are not repository evidence. +This local review does not prove source rights, current data, hosted behavior, +human accessibility, independent understanding, or predictive validity. + +## 2026-07-18 Discover Answer-First Live Review + +The live demo-profile Discover route was reviewed from the selected worktree at +`1280x720` and `390x844`. In both viewports, the Personal Research header and +research-only boundary are followed directly by `Which stock can I review?`, +the readiness-backed search control, and the existing Company Workbench actions. +Focused-cohort scope and lane-coverage cards now remain closed under +`Advanced: cohort readiness context` after the selection task. + +The phone viewport reported a `390px` document width at a `390px` viewport, +with no horizontal overflow and no details element expanded by default. The +selector remained limited to the same deterministic cohort, and its links +continued to target Company Workbench in Personal Research mode. This was a +presentation-order change only; no readiness, data, ranking, or route contract +changed. + +Reviewed screenshots were saved outside the repository in the temporary review +workspace and remain unstaged. They are product-layout evidence only; +they do not prove data freshness, source rights, hosted behavior, or predictive +validity. + +## 2026-07-17 Commercial Beta Research Workflow Live Review + +The live local Streamlit app was reviewed in the in-app browser at `1280x720` +and `390x844` across Research Desk, Discover, Company Workbench, and Monitor. +Each route rendered from the selected worktree, kept the research-only boundary +visible, preserved one next action, kept Advanced evidence closed, and showed no horizontal overflow at either viewport. + +The first settled Company Workbench review exposed a real `ArrowInvalid` +failure when immutable Forward View evidence reached the display table. The +display adapter now serializes that immutable nested evidence into deterministic +JSON text. A fresh server restart and repeat NVDA review completed through +Research Conclusion and Next Research Task without traceback text. Focused and +full tests plus the dashboard and public gates cover the corrected contract. + +This live review is product evidence only. It does not prove hosted behavior, +data freshness, licensed commercial source operation, external reviewer +success, or predictive validity. No screenshot from this session is committed; +generated capture output remains excluded. + ## 2026-07-11 Public Workflow Modernization The current public first-view contract is intentionally simpler than older @@ -35,6 +258,126 @@ exact current wording. Screenshot evidence is product evidence only. It does not refresh data, apply imports, unlock blocked fundamentals, peers, earnings, analyst estimates, valuation inputs, or prove current readiness counts. +## 2026-07-21 Public Desktop And Phone Workflow Review + +A fresh local review covered all five public pages—Home, Stock Selector, +Single-Stock Report, Data Health, and Proof History—at desktop and `390x844` +phone viewports. Every page passed its current review with its first answer and +primary handoff visible, Advanced/raw detail collapsed, no horizontal overflow, +and no traceback. + +Only the Single-Stock Report needed a later regression correction. Its +direct-open loading state placed three large quick-read cards before the +evidence handoff even though the completed report used the compact answer. +The loading state now renders the same compact selected-ticker answer before +provider and report work, preserving Selected ticker -> `Use now` -> `Still +withheld` -> `Open Data Health`. Browser measurement at `390x844` verified the +44px handoff fully inside the viewport with at least 50px of bottom clearance, +no horizontal overflow, and no traceback. Desktop remains a four-column layout +with the same handoff visible. + +### 2026-08-02 Complete Stop-Rule Follow-Up + +A current-head recapture found a narrower follow-up regression: the 44px +handoff still fit, but the complete two-line research-only stop rule ended +27.9px below the `390x844` first viewport. The phone action block inherited +both its desktop grid gap and stop-rule top margin, and the selected-ticker +summary still inherited its desktop top margin. Commit +`34b08ea867316ae3683a9a769d1ed37bd62d61ce` removes that duplicate phone-only +spacing without changing copy, reading order, research state, or desktop +layout. + +Fresh live phone geometry reported `stop_top=811.4609375`, +`stop_bottom=843.4296875`, `stop_clearance=0.5703125`, +`action_height=44`, `action_clearance=78.8203125`, and `scroll_width=390`. +The viewport was exactly `390x844`; Advanced details open count was zero and +no traceback was visible. +The summary began 2.2265625px after the trust strip, so the correction did not +overlap the preceding evidence. At `1280x720`, the summary retained four +computed columns (`128px 378.859px 378.859px 303.094px`), the action remained +44px high, `scroll_width=1280`, and no traceback was visible. The live screen +was inspected, but no screenshot file was created. This is local +product-layout evidence only; it does not prove data freshness, source rights, +hosted behavior, accessibility conformance, external review, demand, or market +validation. + +### 2026-08-08 Zero-Scroll Supersession + +A fresh top-of-page audit supersedes the 2026-08-02 first-viewport conclusion. +The earlier evidence did not record the scroll position, so its +`stop_bottom=843.4296875` result cannot prove that the complete stop rule was +inside the initial viewport. The new audit explicitly recorded +`window.scrollY=0`, document and body scroll offsets of zero, and +`stMain.scrollTop=0`. + +At `390x844`, both the archived evidence tree +`60492ffa5475fc00cebbc2d4c2dff0c2b140c467` and audited product head +`2e58e00bb6ddff6791299a514b562c96ce911196` rendered pixel-identical +first-answer geometry in both controlled browser surfaces. The primary action +remained fully visible with `action_top=764.0625`, `action_bottom=808.0625`, +`action_height=44`, and `action_clearance=35.9375`; the complete stop rule +reported `stop_top=854.34375`, `stop_bottom=886.3125`, and +`stop_clearance=-42.3125`. Document width remained `390` with no horizontal +overflow or traceback. The `42.8828125px` difference from the earlier recorded +stop bottom is consistent with an unrecorded scroll offset, but the prior run +did not capture that value and its exact cause is therefore not asserted. + +This proves that no later tracked product change introduced the current +failure and that the existing selector-local spacing rules still render as +implemented. It does not prove that the complete safety boundary is visible at +the top of the phone route. The phone result is `blocked_with_evidence` until +an owner-approved responsive repair is implemented and remeasured with an +explicit zero-scroll assertion. No product code, readiness, source, research, +or generated artifact changed during this reconciliation. + +### 2026-08-08 Owner-Approved Zero-Scroll Repair + +The owner-approved implementation at anchor +`18417d90766e9aeb629eef6bb1ef4fc40805773c` keeps Home's direct answer and the +Single-Stock Report direct-open loading state inside the initial phone +viewport without changing copy or the shared Public shell. The selected answer +still reads Selected ticker -> `Use now` -> `Still withheld` -> `Open Data +Health`, and both primary actions remain 44px high. + +Fresh zero-scroll `resolved_post_fix` evidence at `390x844` recorded Home +`stop_bottom=843.609375`, `action_height=44`, and `visible_stops=1`; the one +visible stop rule follows the primary action and precedes metrics in both DOM +and visual order. Single-Stock Report `stop_bottom=836.421875`, +`action_height=44`, and `trust_gap=2.21875`. Both routes recorded +`scroll_width=390`; all four measured scroll offsets were zero, Advanced open +count was zero, and there was no horizontal overflow, no traceback, console +warning, or page error. + +At `1280x720`, desktop retained two Home grid tracks and four Single-Stock +Report grid tracks. Home showed only the desktop stop across the second row; +Single Stock preserved its four-column answer; both actions remained 44px and +both routes had zero scroll offsets, matching client/document widths, and no +runtime error. The repair changed no readiness, source, research, or +generated-artifact state and wrote no screenshot, timing, report, or data +artifact. + +Data Health and Proof History remain answer/evidence destinations. This review +adds no invented calls to action, does not turn either route into a readiness +unlock, and does not change readiness, source, research, or generated-artifact +state. The earlier screenshots predate this regression fix and remain +historical presentation evidence only; their old measurements do not prove the +corrected loading state. The later result was verified through live browser +layout measurement, and no new screenshot artifact was created. Neither form +of local presentation evidence proves data freshness, source rights, hosted +behavior, accessibility compliance, external reviewer behavior, or predictive +validity. + +The review-note gate preserves the corrective history without understating the +finished audit. `resolved` is a direct successful review; +`resolved_post_fix` is a successful fresh recapture after a correction. The +current raw counts are eight `resolved` rows and two `resolved_post_fix` rows, +while the aggregate gate reports `10/10` and `share_review_ready`. Only those +two explicit labels are successful. Unknown labels—including values that merely +start with `resolved`—and deferred, environment-limited, skipped, blocked, or +pending rows remain fail closed. This is screenshot-based local workflow QA, +not WCAG, keyboard, screen-reader, hosted, external-reviewer, freshness, or +market-validation evidence. + ## V1 Public UI Replacement QA This is the current replacement-readiness browser QA contract for the public diff --git a/docs/DATA_STRATEGY.md b/docs/DATA_STRATEGY.md index b8c7d0f23..e6da66de9 100644 --- a/docs/DATA_STRATEGY.md +++ b/docs/DATA_STRATEGY.md @@ -4,6 +4,36 @@ The product is CSV-first. Local rows are the source of truth; provider-assisted Provider-assisted does not mean provider-decided. A provider can help populate local price rows, but the product still validates local CSV coverage before momentum, liquidity, DCF, peer, or decision sections appear. Fundamentals, peer relationships, earnings, and analyst estimates need trusted source review before they become analysis inputs. +## Commercial Source Rights + +Commercial Research mode is fail-closed. It may use a source only when `config/source_rights.yml` records explicitly approved commercial use alongside permitted use, redistribution, storage, attribution, rate-limit, authentication, freshness, supported-field, and fallback details. Credentials and license documents stay outside Git. + +Run `python3 -m src.commercial_source_rights` to inspect the read-only registry, or add `--source ` for one commercial-mode decision. The current registry permits SEC Companyfacts subject to SEC fair-access requirements. `yfinance`/Yahoo Finance remains research-only until separate commercial rights are verified; the commercial gate refuses it rather than treating availability or a local cache as permission. + +The exact `sec_companyfacts` record now lists `operating_income`, `cash_from_operations`, and `capital_expenditures` after the bounded SEC filing review. This is a field-scope decision under the existing SEC record, not a new legal opinion, provider entitlement, broad-coverage claim, or production activation. `make sec-quarterly-cash-preview AS_OF=` retrieves only the exact Companyfacts, submissions, and primary-filing endpoints for the declared one-company pilot, keeps all payloads in memory, and prints a human review result. Technical extraction, exact-source rights, field scope, adapter acceptance, production activation, and saved readiness remain independent. + +Price availability and price provenance are separate. `make readiness-preview TOP_N=20` audits the exact latest usable price row for every proposed DCF promotion without writing files. Commercially reviewable row lineage requires `source`, `source_ref`, and `retrieved_at`, plus exact-source approval and registered `prices` scope. A market date, `as_of_date`, local CSV label or file timestamp, configured adapter, refresh warning, or recognizable OHLCV series cannot reconstruct missing provider identity. The current canonical gap should be solved prospectively with reviewed row-level evidence; do not mass-rewrite historical price rows or assign a guessed provider. + +When one permitted reviewed price export exists, normalize only that bounded file with explicit evidence metadata: `make price-normalize INPUT= TICKER= SOURCE= SOURCE_REF= RETRIEVED_AT= AS_OF=`. Then run `make price-validate AS_OF=` and `make price-preview AS_OF=`. A `lineage_complete` result proves field preservation only; `temporal_complete` separately proves the declared retrieval is no earlier than the next UTC day after the daily observation and no later than the explicit review cutoff. Verify exact-source commercial rights and registered `prices` scope before any reviewed `make price-apply AS_OF=`. Missing metadata remains review-required without being generated automatically. + +The same validation and preview now perform that registry join directly for technically valid staged rows. They report exact-source rights and registered `prices` scope as separate states and retain deterministic distinct-source blockers. Unknown, blank, or unverified source IDs remain review-required; a rights-approved source without `prices` scope also remains scope-review-required. These read-only results do not change row validity or authorize apply. + +If `COMMERCIAL_RESEARCH_MODE` is explicitly enabled, `price-apply` now enforces lineage, retrieval timing, rights, and registered price scope before mutation. Malformed, naive, too-early, post-cutoff, or cutoff-unreviewed retrieval evidence blocks apply in every mode; missing retrieval remains a separate research-compatible lineage gap but blocks Commercial Research apply. Apply carries the one validated staged frame through the merge instead of reading the import twice, then flushes a same-directory temporary CSV and atomically replaces the canonical path. Known validation failures create no backup or canonical mutation. This is not a concurrent-writer lock or a database transaction, and passage does not validate the payload, rebuild readiness, or record proof. + +Prospective point-in-time consensus uses the same exact-source principle with metric-specific scope. Collection preview reports append-only technical validity independently from commercial rights; non-empty Revenue and EPS values require literal `revenue_consensus` and `eps_consensus` support respectively. Explicit Commercial Research mode blocks an incomplete record before ledger or directory mutation, while ordinary research mode preserves the explicit reviewed local append path. The current registry contains no approved consensus source or either consensus scope. Do not add a provider record, infer an alias, or treat a configured key as permission without reviewed rights evidence. + +Provider source-row validation also derives this evidence directly; it no longer accepts a caller-supplied rights status. Technical schema/comparability acceptance remains independent, while only technically accepted rows receive row-level exact-source rights and populated-metric scope review. `historical_evidence_reviewable` means a supplied point-in-time row can proceed to review, not that it is licensed, collected, activated, fresh, or nowcast-ready. + +Every source-row validation call must declare its review cutoff. Only explicit `current_only` and `point_in_time` scopes are recognized, and every accepted row proves `snapshot_at <= retrieved_at <= review_cutoff`. Candidate context receives no temporal exemption. The normalized cutoff is evidence of the local review boundary only; it cannot establish when a provider published a value or authorize collection and use. + +For a reviewed multi-row consensus file, preview simulates the complete input in order against the existing ledger plus earlier technically valid proposed rows. Record uses that same batch result and does not begin appending until every technical row passes and, in Commercial Research mode, every required rights/scope state passes. A later known rejection therefore leaves the existing ledger byte-identical. The collector does not reorder revisions or infer missing lineage. One append handle narrows the mutation boundary but does not provide process locking, crash recovery, or a filesystem transaction guarantee. + +Saved consensus history is never trusted merely because its CSV header parses. Status, preview, and record validate every row and require a unique, linear root-to-leaf revision chain per ticker/period. Preview returns a receipt over the normalized cutoff, exact mode, full input, and full saved ledger. Record requires the same `AS_OF` and `PREVIEW_RECEIPT` and recomputes the decision before opening the destination; edits or intervening appends require a fresh preview. Source-review exports remain a separate schema and are never silently mapped into collection rows. + +Collection preview and upstream source-row validation use one immutable exact-source field-scope review for their registry decision. It preserves the caller's ordered, non-empty, unique required fields and reports rights approval, missing registered fields, and their conjunction without generating blocker copy or changing consumer-specific technical/write rules. This shared metadata decision does not prove the estimate payload, timestamps, comparability, reviewer intent, collection, activation, readiness, backtesting, or calibration. Price, DCF, fundamentals, and cash-generation reviews retain their separate domain requirements. + +When a reviewed upstream consensus export becomes available, use `SOURCE_INPUT=` and start with the read-only source gate: `make earnings-consensus-source-review INPUT=$SOURCE_INPUT PROVIDER= AS_OF=`. It requires an explicit provider and cutoff, validates CSV shape, and prints technical rejections plus independent rights and Revenue/EPS scope evidence without writing. The upstream source export and the prospective ledger row are distinct input contracts: the source export declares `history_scope`, while `COLLECTION_INPUT=` follows the checked-in prospective schema and declares `review_state`. After separate human review and explicit mapping that preserves the accepted evidence, run `make earnings-consensus-collection-preview INPUT=$COLLECTION_INPUT AS_OF=`. No command infers scope, transforms one file into the other, or writes a mapped file. Neither gate supplies permission, collection, activation, readiness, backtesting, or calibration; the later confirmed record command remains a separate mutation boundary. + ## Public Visitor FAQ The public workflow is Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History. Use this data strategy as the explanation layer behind that path, not as a second navigation model. @@ -103,7 +133,9 @@ Before choosing a company, run `make universe-scope TICKERS=NVDA,META TOP_N=10` If you want to choose the next pilot from current local blockers, run `make project-status-check` first. Only run `make trusted-data-pilot-candidates TOP_N=10` when project-status-check shows executable company candidates; if it says current source-proof queues are exhausted, run `make provider-setup-checklist` instead. The candidate command ranks operating-company candidates and excludes ETF/index monitor examples from the company DCF pilot list. The default candidate output is compact so visitors can see the shortlist, quick path, compact review board, and safe loop without reading every row-level proof detail; use `make trusted-data-pilot-candidates TOP_N=10 VERBOSE=1` when you need full per-candidate file status, decision gates, rejected-row paths, and evidence expectations. If you want a batch operating view before drilling into individual names, run `make trusted-data-pilot-board TICKERS=MU,CRDO,HOOD,TSLA,META,A,APLD`; it prints the lane mix, outcome mix, lane-group workflows, per-ticker blocker rows, and suggested lane-group next step without writing a CSV. Lane-group workflows show candidate count, tickers, shared blocker theme, review command pattern, trusted row target, rejected-row report, proof command pattern, stop condition, and whether the lane is review-only, locked, or safe for dry-run batching. If you already know the lane group, run `make trusted-data-pilot-lane LANE=fundamentals_dcf`, `make trusted-data-pilot-lane LANE=peer_mapping`, `make trusted-data-pilot-lane LANE=peer_valuation_inputs`, `make trusted-data-pilot-lane LANE=optional_context_locked`, or `make trusted-data-pilot-lane LANE=price_coverage`; the command prints ordered lane-specific steps plus the evidence summary for what proves the lane, which rows/files are needed, which rejected-row reports matter, which readiness command confirms a change, and what remains blocked. Price coverage stays separate as a dry-run-first planning lane; fundamentals, peers, earnings, and analyst estimates remain review-required. If you want a one-company packet after choosing a lane group, run `make trusted-data-pilot-packet TICKER=CRDO`; it prints the before report, focused blocker check, lane review path, validate/preview gate, apply boundary, rejected-row report path, rebuild proof, and evidence row to record. If you want a CSV ledger for selected before-state proof paths, run `make trusted-data-pilot-evidence TICKERS=MU,CRDO`; it writes current modes, lane blockers, proof commands, trusted-row targets, rejected-row checks, and still-blocked reasons without changing source rows or readiness. If you want the copyable read-only checklist after choosing several names, run `make trusted-data-pilot TICKERS=NVDA,CRDO,META TOP_N=10`. The broader checklist remains available as `make trusted-data-pilot TOP_N=10`. Each command does not refresh, import, or edit local CSV files. -Before calling a run pilot-ready, use `make pilot-readiness-check TOP_N=10`. It checks GitHub sync, dirty-tree classification, generated CSV/report churn, readiness freshness, source-proof queues, reviewed proof-ledger status, the explicit `make public-check` gate, and research-only guardrails. Use `make pilot-share-brief` to write `outputs/pilot_share_brief.md` as the concise public/demo handoff; it does not refresh data or unlock blocked inputs. Then use `make pilot-readiness-packet` to write `outputs/pilot_readiness_packet.md` as the fuller reviewer packet. A `pilot-ready with manual gates` verdict is acceptable for a research pilot when generated churn is excluded, public-check still needs to be run, and source-proof lanes remain visibly blocked. A `blocked` verdict means fix sync, freshness, or uncommitted product files before sharing the pilot state. +Before calling a run pilot-ready, use `make pilot-readiness-check TOP_N=10`. It checks GitHub sync, dirty-tree classification, generated CSV/report churn, readiness freshness, tracked release-evidence origin, source-proof queues, reviewed proof-ledger status, the explicit `make public-check` gate, and research-only guardrails. Freshness and release-evidence origin are independent: readiness can be current for the saved source files while still remaining `working_artifact_uncommitted` and therefore unavailable as tracked PR evidence. Freshness uses both file mtimes and declared source dates: if a source row declares a later date than the saved readiness build, the pilot and reviewed-batch gates stay stale even when checkout or restore mtimes appear current. The check is read-only; it does not rebuild readiness or create CSV/JSON artifacts. When freshness is stale, or when current working readiness is not tracked release evidence, run `make readiness-preview TOP_N=20` to compute the same proposed stable readiness states in memory and compare them with the saved snapshot. A zero-change preview can confirm technical equivalence, but it does not make working artifacts tracked evidence, authorize staging, or promote readiness. The preview explains added/removed rows and newly ready, partial, or excluded feature transitions, including named behavior-equivalent DCF scope reasons; transition counts may overlap and are not current readiness totals. It also reviews false-to-true fundamentals and DCF changes separately for the exact canonical source value, source/as-of/durable-reference provenance, checked-in commercial-rights status, and registered support for required fundamentals fields. Composite or unregistered source values fail closed as exact identifiers; they are not split or granted the rights of one component. The preview prints to stdout only, creates no CSV, JSON, report, sample-report, screenshot, timing, directory, or bytecode artifact, and does not make saved readiness current. Technical promotion, method fit, provenance completeness, commercial permission, field scope, and complete DCF price provenance remain independent. It is inspection evidence only: an intentional reviewed `make readiness` run remains the separate unblock step and requires source and generated-artifact review. Use `make pilot-share-brief` to write `outputs/pilot_share_brief.md` as the concise public/demo handoff; it does not refresh data or unlock blocked inputs. Then use `make pilot-readiness-packet` to write `outputs/pilot_readiness_packet.md` as the fuller reviewer packet. A `pilot-ready with manual gates` verdict is acceptable for a research pilot when generated churn is excluded, public-check still needs to be run, and source-proof lanes remain visibly blocked. A `blocked` verdict means fix sync, freshness, uncommitted readiness release evidence, or uncommitted product files before sharing the pilot state. + +The readiness continuation gate applies that boundary consistently to project status, Session Source Preflight, provider setup, the coverage frontier, Auto-Refresh Status, its runbook, and Advanced Data Health cards. It covers stale, incomplete, unverified, or current but untracked readiness evidence. In any of those states, `make readiness-preview TOP_N=20` is the only continuation-safe command. Source availability, provider availability, scheduled operations, and ranked opportunities remain planning context only; their source, refresh, apply, proof, and rebuild commands are not executable routing. `make readiness` is a separate intentional reviewed write after source review and generated-artifact review. In Operator Data Health, start with the Pilot Reviewer Walkthrough before opening detailed tables. It compresses the pilot gate, leading manual gate, source-proof focus, packet command, and public-check boundary into one first-screen path so a reviewer can understand what is green, what remains manual, and why blocked source inputs stay blocked. @@ -297,3 +329,17 @@ Operator mode also shows a compact SPY/QQQ metric-readiness queue in Data Health - Broad generated CSV churn in the public branch. The product should get more useful by improving trusted coverage and proof paths, not by filling missing fields with guesses. + +## Readiness Release Evidence Contract + +Technical readiness and release eligibility are reviewed separately: + +```bash +make readiness-release-review TOP_N=20 +make readiness-release-record PREVIEW_RECEIPT= REVIEWER= REVIEW_DATE= TECHNICAL_DECISION= DISTRIBUTION_DECISION= CONFIRM_REVIEWED=1 +make readiness-release-guard RECORD_ID= +``` + +Review compares tracked Head, the exact working candidate, and fresh in-memory readiness. It keeps provenance, commercial rights, registered field scope, DCF price lineage, historical proof binding, distribution review, and staging hygiene as independent axes. A record can preserve a rejected or `external_review_required` decision without promoting the candidate. Guard stops on any receipt, digest, Git head, path, axis, decision, candidate-byte, or staged-state mismatch. + +This workflow does not change readiness and does not grant source rights. SEC commercial approval does not expand registered fields, a supported field does not create commercial permission, and composite source strings never borrow component rights. It cannot replace independent review, source-owner or legal approval, current external data, accessibility evidence, hosted validation, or calibration. diff --git a/docs/EARNINGS_NOWCAST_PILOT.md b/docs/EARNINGS_NOWCAST_PILOT.md index 9c6185fe7..d4b7489d5 100644 --- a/docs/EARNINGS_NOWCAST_PILOT.md +++ b/docs/EARNINGS_NOWCAST_PILOT.md @@ -38,6 +38,40 @@ make earnings-nowcast-prospective-plan OUTPUT_DIR=data/imports/earnings_nowcast Actuals and consensus files are required; reviewed signals are optional. The versioned schemas require source names, direct source references, fiscal periods, publication/retrieval timestamps, comparability definitions, and at least one supported metric. Exact duplicates, new rows, explicit append-only revisions, and unresolved conflicts are reported separately; a conflict makes the preview not packet-ready. Post-cutoff evidence is rejected. The prospective plan is scheduler-ready but does not fetch data or create its output directory. These commands never apply, overwrite, stage, commit, or push rows; there is intentionally no Earnings Nowcast apply command. +## SEC Quarterly Actuals Staging + +The read-only SEC staging command is `make earnings-nowcast-sec-actuals-stage TICKERS= OUTPUT_DIR= AS_OF=`. `OUTPUT_DIR` must be a new temporary/review directory or an existing directory marked by this generated stage; canonical `data/` and `data/imports/` paths and existing non-generated evidence directories are rejected. The command reports `automatic_apply=false`; it neither changes canonical evidence nor provides an apply path. The five-company pilot scope is NVDA, AMD, AVGO, MU, and QCOM, but a successful staging run is not real Nowcast coverage. + +Q1-Q3 lineage accepts only source-backed SEC Companyfacts duration facts with a 60-120 day quarter duration and a one-to-one fiscal identity/period-end mapping. Cumulative facts, ambiguous concepts, conflicting fiscal identities, and post-cutoff rows are rejected. Companyfacts EPS uses `companyfacts_split_basis_unverified`, so it cannot become EPS-ready without separate source-backed comparability. That sentinel is also withheld from Business Trend values, cohort EPS usability, and backtest target/benchmark math; Revenue remains independent. Q4 requires an explicit fiscal-Q4 result table and one period-end date in its selected value column; filing timestamps and submission report dates are not substitutes. Annual-minus-nine-month arithmetic, derived Q4 disclosures, guidance, and cross-exhibit metric combinations are prohibited. + +Historical actual evidence is append-only and cutoff-aware: later presentations remain separate revisions and never overwrite an earlier source-backed row. The stage summary surfaces accepted rows, rejected rows, missing Q4, direct source references, and detected fiscal-quarter continuity gaps; it never fabricates a missing period or metric. Revenue and EPS remain independent. EPS is withheld when its split-adjustment, share, operations, currency, or accounting basis cannot be kept within one source-backed comparable basis. + +Real pilot output remains `awaiting_point_in_time_consensus` until permitted historical consensus snapshots are available at each forecast cutoff. Numerical Beat/Miss probability remains `awaiting_calibration_evidence` until at least 100 valid out-of-sample events pass the calibration gates. + +## Cohort Readiness And Prospective Collection + +Use `make earnings-nowcast-cohort-readiness AS_OF=` to review NVDA, AMD, AVGO, MU, and QCOM as one evidence board. The board reports the latest actual period, next forecast period, Revenue/EPS history, explicit Q4 evidence, split-basis readiness, exact-period consensus count, backtest count, calibration count, state, blocker, and next action. It does not create a forecast. + +Use `make earnings-consensus-source-status` to inspect Alpha Vantage, FMP, Finnhub, and reviewed-CSV activation in deterministic order. A configured key is only `configured_unverified`; it is not evidence that historical snapshots, use rights, or comparable definitions are available. Current-only estimates remain `candidate_context_only` and cannot be used as reconstructed historical snapshots. + +Use `make earnings-consensus-collection-plan AS_OF=` for a weekly or pre-earnings plan and `make earnings-consensus-collection-status` for the append-only ledger status. The prospective record preserves snapshot identity, source reference, publication/retrieval cutoff, metric definitions, explicit revisions, and review state. Duplicate snapshots are rejected, cooldown is explicit, prior snapshots are never overwritten, and no automatic apply or readiness promotion exists. + +For prospective collection, start from `docs/templates/earnings_nowcast/prospective_consensus.csv` and set `COLLECTION_INPUT=`. Preview with `make earnings-consensus-collection-preview INPUT=$COLLECTION_INPUT AS_OF=` and retain the printed `preview_receipt`. Recording requires that exact unchanged input, ledger, cutoff, and mode: `CONFIRM_REVIEWED=1 make earnings-consensus-collection-record INPUT=$COLLECTION_INPUT AS_OF= PREVIEW_RECEIPT=`. The record command revalidates the complete saved ledger and proposed batch before mutation; a changed cutoff, input, ledger, or mode invalidates the receipt. It appends evidence only and does not create a forecast or promote readiness automatically. + +Preview keeps the append-only technical decision separate from commercial evidence. Each exact source is checked without aliases or composite-source inference; a populated Revenue value requires registered `revenue_consensus` scope and a populated EPS value independently requires `eps_consensus` scope. In ordinary research mode, the existing explicit reviewed append path remains available. In explicit Commercial Research mode, unapproved rights or any required missing scope blocks before the ledger or its parent directory is changed. The checked-in registry currently approves no prospective-consensus source or scope, so this guard does not unlock a real-company packet. + +Every saved ticker/period lineage must be one append-ordered root-to-leaf chain. Invalid rows, duplicate snapshot IDs or evidence identities, missing or cross-period parents, multiple roots, forks, cycles, reversed append order, and non-increasing revision timestamps fail closed with row or scope context. A proposed revision may supersede only the current leaf. This deterministic integrity check is not a concurrent-writer lock, database transaction, or crash-recovery guarantee. + +The upstream source-row validator uses the same evidence separation before a row reaches collection review. Callers cannot supply a rights label: the exact provider ID is joined to the checked-in registry, and each technically accepted row reports only the scopes required by its populated Revenue and EPS values. Technically invalid rows stay rejected without entering commercial-ready counts; current-only rows remain candidate context, and valid point-in-time rows are `historical_evidence_reviewable`, not activated evidence. This review result cannot collect, append, approve a payload, establish historical depth, or change nowcast, backtest, or calibration readiness. + +Both consensus paths now derive that exact-source rights-and-scope decision from one immutable helper. Ordered required and missing metric fields stay inspectable, blank or duplicate requirements fail locally, and composite source IDs remain unknown rather than being expanded. Each consumer still owns its technical status, blocker wording, and write decision. The shared result is registry metadata, not evidence that a payload, timestamp, comparability definition, reviewer decision, historical snapshot, or readiness gate is valid. + +Before collection preview, set `SOURCE_INPUT=` and review the upstream export with `make earnings-consensus-source-review INPUT=$SOURCE_INPUT PROVIDER= AS_OF=`. This read-only command requires the explicit provider and cutoff, rejects missing/duplicate headers and extra row values, preserves one-based row evidence, and renders technical, temporal, rights, and populated Revenue/EPS scope states in human or JSON form. Every result retains `auto_apply=false`; even a historical-reviewable, commercially complete row is not collected or activated. `SOURCE_INPUT` and `COLLECTION_INPUT` are distinct input contracts: the former requires explicit `history_scope`; the latter uses the existing prospective collection schema with `review_state`. Only after separate human payload/evidence review and explicit evidence-preserving mapping should collection preview run. The product performs no automatic mapping or file write, and recording remains a later explicitly confirmed mutation. + +That source-row review also requires an explicit UTC cutoff and a literal `current_only` or `point_in_time` scope. Both candidate and historical rows must satisfy `snapshot_at <= retrieved_at <= review_cutoff`; unknown scope, reversed timestamps, or post-cutoff evidence are rejected before rights/scope review. Cutoff passage proves only timestamp ordering against the supplied boundary. It does not prove publication availability, payload correctness, source permission, freshness, collection, activation, backtesting, or calibration. + +Multi-row preview and record use the same ordered virtual-ledger preflight. Each technically reviewable row becomes visible to the next proposed row in input order, so a valid explicit revision can reference an earlier row in the reviewed file while duplicates, reversed revisions, and same-period conflicts fail before record. Any deterministic technical blocker rejects the whole batch in every mode; any commercial-evidence blocker rejects the whole batch in explicit Commercial Research mode. Record then uses one append handle only after the applicable batch gate passes. This prevents known later rejections from causing partial input writes, but it is not concurrent-writer locking or crash-safe filesystem transactionality. + ## Signals Company news, industry indicators, macro evidence, and trusted peer earnings may provide directional explanation. Candidate peers remain `candidate_context_only`. Trusted signals require reviewed source evidence and can move the lane to `signal_context_ready`; they cannot mutate Revenue/EPS ranges or create a numeric adjustment. @@ -54,9 +88,12 @@ Run: ```bash FIXTURE=1 make earnings-nowcast-pilot TICKER=SYN1 AS_OF=2026-01-31T23:59:59Z +FIXTURE=1 make earnings-nowcast-readiness TICKER=SYN1 AS_OF=2026-01-31T23:59:59Z make earnings-nowcast-walkthrough AS_OF=2026-01-31T23:59:59Z ``` The `SYN1`-`SYN5` cohort and `SYN5-BACKTEST` walkthrough label are synthetic test evidence only. The walkthrough demonstrates baseline ready, Revenue ready with EPS withheld, candidate-peer-only context, post-cutoff blocking, non-company exclusion, and backtest-insufficient/un-calibrated behavior. It proves deterministic contracts, readiness, ranges, signal separation, CLI output, and withholding behavior. It does not prove real-company coverage, current data, predictive accuracy, or investability. +The fixture readiness command uses a schema-valid synthetic onboarding fixture and should report `baseline_ready`. The same readiness command without `FIXTURE=1` reads local real-data onboarding rows and must remain blocked until source-backed point-in-time consensus and actuals pass validation. + Real semiconductor coverage is `awaiting_point_in_time_consensus`. The next legitimate data step is a narrow append-only cohort with licensed or otherwise permitted historical consensus snapshots and source-backed actuals. Generated packets remain local and unstaged unless an exact artifact is intentionally reviewed. diff --git a/docs/HOSTED_DEMO_DEPLOYMENT.md b/docs/HOSTED_DEMO_DEPLOYMENT.md index b6a9804e7..ba0717b3e 100644 --- a/docs/HOSTED_DEMO_DEPLOYMENT.md +++ b/docs/HOSTED_DEMO_DEPLOYMENT.md @@ -17,6 +17,12 @@ make hosted-demo-readiness This command is read-only. It checks the root Streamlit entrypoint, runtime dependency manifest, hosted URL boundary, provider-secret boundary, and public verification commands. It does not deploy, open accounts, print secrets, refresh data, stage files, commit, or push. +## Private Beta Is A Separate External Gate + +The public demo deployment path does not provide private-beta authentication, private workspaces, per-user data separation, entitlements, audit retention, monitoring, or health checks. These remain `external_account_required` until an external hosting, identity, persistence, and monitoring setup has been configured and independently verified. + +Use [Private Beta Architecture](PRIVATE_BETA_ARCHITECTURE.md) for the required external setup sequence and the read-only local classification. A declared setup becomes `manual_verification_required`; it is not evidence that accounts, hosting, or user isolation are live. Keep all real credentials outside Git and do not claim private-beta availability before supervised verification with real test accounts. + ## Hosted URL Marker The repository includes `config/hosted_demo.env.example` as a blank handoff template for a future hosted app URL. Keep the real marker local and untracked: @@ -98,6 +104,16 @@ If the hosted deployment changes app startup, route behavior, or copy, also revi Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History +For a controlled Commercial Research Beta, also verify: + +Research Desk -> Discover -> Company Workbench -> Monitor + +Run the same research performance contract against the verified base URL: + +```bash +make commercial-beta-performance-gate BASE_URL= +``` + Stop before publishing the hosted link if the first public view shows tracebacks, raw tables before the answer, command-heavy copy, missing research-only boundaries, or unavailable-provider claims. ## Post-Deploy Smoke Checklist @@ -111,6 +127,19 @@ Run this before replacing the GitHub link in README, LinkedIn, or portfolio copy 5. Open `/?mode=public&page=proof-history` and confirm proof history is evidence-only before raw ledger details. 6. Keep the hosted link private if any route shows a traceback, operator mode by default, stale data-freshness claims, or a missing research-only stop rule. +## Health Check And Rollback + +Before admitting reviewers, verify `/_stcore/health`, open every public and +research route above at desktop and phone widths, and confirm the host uses the +tracked `demo` profile. Keep the previous verified revision available as the +rollback target. If startup, route rendering, access control, secrets handling, +or timing fails, remove reviewer access, restore the previous verified +revision, and keep GitHub as the share link while the issue is investigated. + +Without a real host, the correct status is `external_account_required`. Do not claim private or authenticated access until a real host enforces and has been +tested for that boundary. A healthy URL alone does not prove authentication, +per-user isolation, data rights, monitoring, or reviewer success. + ## Link Decision Ladder Use this ladder before changing README, LinkedIn, or portfolio copy: diff --git a/docs/LINKEDIN_PROJECT_BRIEF.md b/docs/LINKEDIN_PROJECT_BRIEF.md index 99615a3a1..e5bd453f8 100644 --- a/docs/LINKEDIN_PROJECT_BRIEF.md +++ b/docs/LINKEDIN_PROJECT_BRIEF.md @@ -2,34 +2,35 @@ ## Short Version -I built a local, CSV-first stock research command center that checks whether a ticker has enough trusted data before showing deeper analysis. The main idea is simple: data readiness first, analysis second, research decision last. +I built a local Python and Streamlit portfolio beta for evidence-first company research. Its primary workflow is Research Desk -> Discover -> Company Workbench -> Monitor: choose a reviewable company, see what evidence can be used now, keep unsupported conclusions withheld, and monitor source-backed changes. -The dashboard and single-stock reports show what can be reviewed now, what is blocked by missing data, what is excluded because the method does not apply, and which trusted local input would unlock the next layer. +The product separates the main research answer from technical evidence. Company Workbench brings business trend, valuation, forward context, uncertainty, conclusion, and the next research task into one review surface; Data Health and Proof History remain available when the missing input or provenance is the question. -Best demos: `NVDA` for DCF-ready company review, `META` for valuation still gated by trusted fundamentals, `QQQ` for ETF/index monitor context, `MU` for standalone DCF with peer valuation still locked, and `CRDO` for a fundamentals-gated proof workflow. +The main idea remains simple: data readiness first, analysis second, research decision last. Empty or unverified valuation, catalyst, outcome, consensus, and calibration lanes stay withheld rather than displaying fabricated content. -Best first click: open the dashboard preview, then follow Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History. +Best first click for the complete product: run the local app and follow Research Desk -> Discover -> Company Workbench -> Monitor. The shorter controlled Public demo remains Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History. Two-minute external review path: -- GitHub-only review: start with the preview image, the five-page workflow map, and `docs/PUBLIC_DEMO_WALKTHROUGH.md`. +- GitHub-only review: start with the Workbench answer preview, the four-step Personal Research workflow, and `README.md`. - Controlled reviewer handoff: send `docs/PILOT_REVIEW_INVITATION.md` for one under-three-minute workflow review before sharing the detailed feedback template. -- Live dashboard review: run `make dashboard`, open `http://localhost:8501/?mode=public`, then follow Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History. +- Live dashboard review: run `make dashboard`, open `http://localhost:8501/`, then follow Research Desk -> Discover -> Company Workbench -> Monitor. Use `http://localhost:8501/?mode=public` for the five-page controlled demo. - Hosted app status: no public hosted Streamlit URL is configured yet; share the GitHub project, curated screenshots, and local run instructions unless you separately deploy the app and complete `docs/HOSTED_DEMO_DEPLOYMENT.md`. -- Evidence boundary: `docs/assets/linkedin-public-dashboard.png` and screenshots show product UI only; `make status-check TOP_N=5` remains the source for current local counts. +- Evidence boundary: `docs/assets/linkedin-public-dashboard.png` is a real Workbench answer-first screenshot; screenshots are product evidence only. Do not publish data-readiness claims from the image. - Share boundary: controlled portfolio/demo evidence only, not open-source reuse, investment advice, broker integration, or data-freshness proof. - Coverage boundary: price coverage is broad, but fundamentals, share count, peer mapping, earnings, and analyst estimates still have blocked or locked areas. The product shows those gaps instead of treating them as complete. - Provider boundary: FMP, Alpha Vantage, and Finnhub are optional local provider fallbacks and are not configured by default. Do not claim full automated fundamentals, estimates, or provider-backed coverage unless a reviewed source path proves it. +- Link boundary: use the stable GitHub repository link only after this reviewed feature reaches the default branch. Until then, keep an existing stable Featured item or label any non-default review link `Draft engineering preview`. -Each public page now opens with one question, one short answer, one primary next action, and one stop rule. The public workflow is checked at desktop and mobile widths. The current-page shortcut is visible so visitors know where they are. +Each research destination opens with a usable answer, a withheld boundary, or a truthful wait state before Advanced Evidence. The four-step workflow and the five-page Public demo are checked at desktop and mobile widths. -Current review status is `review_limited`: deterministic public checks and repeated local cold/warm route timings pass, while external reviewer evidence and hosted-route verification are still unavailable. This is a truthful controlled-demo status, not a hosted-product or data-freshness claim. +Current review status is `share_review_ready` for the local GitHub/demo workflow: deterministic public checks, browser evidence, and repeated local cold/warm route timings pass. External reviewer evidence and hosted-route verification are still unavailable, so this remains a controlled-demo status, not a hosted-product or data-freshness claim. -The first story is the public workflow, not operator automation. Keep reviewed batch packets, provider setup, and validate / preview / apply mechanics as operator detail after the visitor understands the product. +The first story is company research, not operator automation. Keep reviewed batch packets, provider setup, validate / preview / apply mechanics, and provider-key activation as operator detail after the visitor understands the product. Operator details stay collapsed until someone intentionally leaves the public path. -Use the refreshed `docs/assets/linkedin-public-dashboard.png` thumbnail for the LinkedIn Featured card. +Use the reviewed `docs/assets/linkedin-public-dashboard.png` Workbench thumbnail for the LinkedIn Featured card only after the feature is present at the link target. ## Now / Next / Not Yet @@ -45,26 +46,24 @@ If someone asks what to do next, run `make next-stage` before opening operator p ## Suggested LinkedIn Post -I built a local Python and Streamlit stock research command center around one principle: +I built a local Python and Streamlit Stock Research Command Center around one principle: Data readiness first. Analysis second. Research decision last. -Instead of jumping straight to rankings, the project checks whether each ticker has enough trusted local data for the analysis being shown: price, momentum, liquidity, fundamentals, DCF inputs, peer context, earnings, and analyst estimates. +The core workflow is Research Desk -> Discover -> Company Workbench -> Monitor. It helps me choose a reviewable company, bring business trend, valuation, forward evidence, uncertainty, and the next research task into one place, then monitor only source-backed changes. -What I like most about the product is that missing data stays visible. If a ticker is not ready for DCF, peer comparison, earnings context, or analyst-estimate context, the dashboard says why and shows the next local proof step. If a method does not apply, such as operating-company DCF for an ETF/index proxy, the report labels it as excluded instead of failed. +What matters most is what the product refuses to invent. If actuals, consensus, valuation, catalysts, outcomes, backtesting, or calibration evidence is unavailable, the relevant conclusion stays withheld and the missing proof remains visible. Technical provenance stays under Advanced Evidence unless it is needed to understand the research answer. What it includes: -- A Streamlit command center dashboard. -- Market-wide readiness checks across a broad ticker universe. -- Single-stock Markdown reports with At A Glance, Reader Guide, Evaluation Snapshot, Proof Checklist, and Best Review Path sections. -- DCF-ready, standalone DCF, price/setup-only, monitor-only, and data-needed-before-analysis report modes. -- Source readiness notes and copyable local proof commands. -- Lane-level readiness operations and reviewed batch packets for capped, proof-first data work. -- CSV-first import, validation, preview, rejected-row, and readiness workflows. +- Research Desk for a bounded weekly research queue. +- Discover for readiness-backed company selection without a buy ranking. +- Company Workbench for an answer-first one-company review. +- Monitor for source-backed evidence changes and truthful wait states. +- Advanced Data Health and Proof History for provenance and missing-input review. - Research-only guardrails: no broker integration, no order routing, no auto-trading, and no direct buy/sell instructions. -The most important design choice was refusing to present every ticker as complete. The product is useful because it refuses to overclaim: ready data can be analyzed, blocked data is explained, and missing rows are treated as the next proof step. +This is a local portfolio beta, not a hosted product or market-validated service. The project demonstrates how I approach research workflow, evidence quality, product boundaries, and fail-closed decision support. GitHub: https://github.com/YuzeJ21/Stock-Analysis @@ -72,11 +71,11 @@ GitHub: https://github.com/YuzeJ21/Stock-Analysis LinkedIn Featured title: -`Stock Research Command Center | Readiness-First Stock Research Workflow` +`Stock Research Command Center | Evidence-First Company Research` LinkedIn Featured description: -`A Python + Streamlit portfolio project that checks stock data readiness before showing analysis. It separates ready, partial, blocked, and excluded states, keeps source-proof gaps visible, and stays research-only with no broker integration, no auto-trading, and no investment advice.` +`A local Python + Streamlit portfolio project for evidence-first company research. Research Desk -> Discover -> Company Workbench -> Monitor shows what can be used now, what remains withheld, and why. Research-only; no broker integration, auto-trading, or investment advice.` GitHub About description: @@ -92,7 +91,7 @@ About-section sentence: Optional LinkedIn post: -`I have been building a Stock Research Command Center as a portfolio project. The core idea is simple: data readiness first, analysis second, research decision last. The app checks whether stock data is ready before showing analysis, separates ready / partial / blocked / excluded states, and keeps source-proof gaps visible when inputs are missing. It is built with Python and Streamlit, and stays research-only: no broker integration, no auto-trading, no order routing, and no investment advice. The public share is a GitHub demo with real product screenshots and local run instructions, not a claim that every coverage lane is complete. This project reflects the kind of data-product work I care about: turning messy coverage, quality, and workflow problems into a guided user experience.` +`I have been building a Stock Research Command Center as a local Python and Streamlit portfolio project. The core workflow is Research Desk -> Discover -> Company Workbench -> Monitor: select a reviewable company, see the evidence that can be used now, keep unsupported conclusions withheld, and monitor verified changes. The app stays research-only with no broker integration, auto-trading, order routing, or investment advice. It is a GitHub demo with real product screenshots and local run instructions, not a hosted-product, complete-coverage, or market-validation claim.` What not to claim: @@ -102,21 +101,21 @@ What not to claim: - Do not imply there is a public hosted app link until a hosted Streamlit deployment exists and has passed the public share gates. - Do not imply FMP, Alpha Vantage, or Finnhub provider fallbacks are configured unless local keys are actually set and a reviewed one-ticker smoke has passed. -Featured thumbnail: use `docs/assets/linkedin-public-dashboard.png`. It is a real product screenshot of the public visitor path; use `make status-check TOP_N=5` for current local readiness counts because screenshot counts can become stale after local refresh/import work. Keep `docs/assets/operator-data-health-metrics-real.jpg` only for deeper operator-mode discussion. The plain GitHub URL card can use GitHub's generated OpenGraph image, so use LinkedIn Featured when you want the curated product screenshot. +Featured thumbnail: use `docs/assets/linkedin-public-dashboard.png`. It is a real Workbench answer-first screenshot showing one selected-company answer, usable and withheld evidence, the Data Health handoff, and the stop condition without publishing changing coverage figures. Keep `docs/assets/operator-data-health-metrics-real.jpg` only for deeper operator-mode discussion. The plain GitHub URL card can use GitHub's generated OpenGraph image, so use LinkedIn Featured when you want the curated product screenshot. LinkedIn can cache older GitHub preview images. If the Featured card still shows the old screenshot after GitHub is pushed, remove and re-add the Featured link, or refresh the URL through LinkedIn's post inspector before adding it again. ## Final LinkedIn Visual Checklist -Use this after GitHub is synced: +Use this after the reviewed feature is present at the link target: -1. Open your LinkedIn profile and confirm the Featured card title matches `Stock Research Command Center | Readiness-First Stock Research Workflow`. -2. Confirm the Featured card description says the project is research-only and mentions no broker integration, auto-trading, or investment advice. -3. Confirm the Featured image is `docs/assets/linkedin-public-dashboard.png` when you want the curated product screenshot. +1. Open your LinkedIn profile and confirm the Featured card title matches `Stock Research Command Center | Evidence-First Company Research`. +2. Confirm the Featured card description names Research Desk -> Discover -> Company Workbench -> Monitor and keeps the research-only, no-broker, no-auto-trading, and no-investment-advice boundaries. +3. Confirm the Featured image is the reviewed Workbench answer-first screenshot at `docs/assets/linkedin-public-dashboard.png`. 4. If LinkedIn shows a generated GitHub URL card instead, leave it only if you are okay with GitHub's OpenGraph image; otherwise remove and re-add the Featured item with the curated screenshot. -5. Confirm the link target is GitHub unless you have intentionally deployed a hosted Streamlit app. +5. Use the stable GitHub repository link only after this feature reaches the default branch. Until then, keep the stable item or label any non-default review link `Draft engineering preview`. 6. Open the GitHub link from LinkedIn and confirm the README starts with `External Reviewer Start Here` and names the GitHub/local-app boundary before deeper operator detail. -7. Stop before claiming screenshots prove current data freshness, coverage completion, provider-key activation, or investment advice. +7. Stop before claiming screenshots prove current data freshness, changing coverage figures, provider-key activation, hosting, market validation, or investment advice. ## Resume Bullet Options diff --git a/docs/METHODOLOGY.md b/docs/METHODOLOGY.md index d3d927e29..6c24f0980 100644 --- a/docs/METHODOLOGY.md +++ b/docs/METHODOLOGY.md @@ -15,6 +15,47 @@ Current public method version: **Methodology v1 - readiness-first deterministic This method is useful for a controlled research/demo product because it makes the data gate visible before analysis. It is not a complete valuation terminal, not investment advice, and not a recommendation engine. +### Quant Interpretation Eligibility + +The shared quant interpretation overlay is implemented locally at +`195ea18da9d1d6e06c36f8320509ccde46cdaa57`. It evaluates valuation, +indicator, and review/risk results without changing their calculations or saved +readiness. The overlay separately records observation recency, structured +provenance, exact-source rights, and field scope, then permits only current +context, historical/review-only context, or withholding. + +Structured external provenance and exact-source rights proof remain absent for +the current local quant inputs. Those results must therefore remain +historical/review-only or be withheld wherever the required proof is absent. +The local overlay does not establish current-market data, hosted behavior, +commercial eligibility, a forecast or nowcast, or calibration. + +### Readiness Promotion Evidence Review + +When saved readiness is stale, `make readiness-preview TOP_N=20` runs the production readiness logic in memory and keeps the proposed technical state separate from the evidence needed to review a rebuild. False-to-true fundamentals and DCF changes are checked against the exact canonical source value, source/as-of/durable-reference provenance, the checked-in commercial-rights decision, and the registered field list for Revenue, free cash flow, FCF margin, and shares outstanding. + +Composite or unregistered source values remain unknown exact identifiers; the method does not split them or borrow the rights decision of one component. Numerical completeness can therefore produce a proposed technical promotion while the evidence review remains blocked. DCF also depends on price evidence, and the current canonical price rows do not establish row-level provider provenance, so the fundamentals review never claims complete DCF commercial provenance. The preview writes nothing, does not edit source rights, does not make saved readiness current, and cannot authorize the separately reviewed `make readiness` boundary. + +The independent `DCF Price Lineage Review` inspects only false-to-true DCF promotions. For each ticker it parses canonical dates and positive closes, selects the greatest observation date, and requires exactly one row at that date. It then keeps five states independent: proposed technical DCF promotion, unique latest-row selection, complete row-level `source`/`source_ref`/`retrieved_at` lineage, exact-source commercial rights, and registered `prices` field scope. A missing usable row or duplicate latest date fails closed; the review does not choose an arbitrary duplicate. + +An observation date is not a retrieval timestamp. `as_of_date`, the `local:prices.csv` file-origin label, file modification time, adapter presence, refresh warnings, and OHLCV shape cannot establish the original provider or durable row reference. The current no-write inspection finds one unambiguous usable latest row for all 146 proposed DCF promotions, but no promoted row has complete lineage, approved exact-source rights, or registered price scope because the canonical source identifier is absent. This identifies an evidence gap; it does not invalidate local research prices, change technical readiness, or authorize a historical schema rewrite or readiness rebuild. + +For future reviewed rows, the manual normalization path accepts explicit `source_ref` and `retrieved_at` metadata and preserves them through validation, preview, and a later separately authorized apply. Validation reports row-level lineage completeness independently from technical OHLCV validity. It never substitutes normalization time for retrieval time, and an invalid retrieval timestamp remains blank. Complete preservation still does not establish commercial rights, registered `prices` support, freshness, reviewer acceptance, or apply/rebuild authorization. + +Staged validation and preview also evaluate the exact retained `source` value against the checked-in registry. The rights decision uses the existing commercial-eligibility contract, while price scope requires literal `prices` membership in the same exact record. Approved, unverified, unknown, blank, and mixed batches remain explicit; composite values are not split and aliases are not inferred. Invalid technical rows are excluded from these counts. The joined review adds evidence states without changing technical validity, lineage completeness, merge counts, source rights, or apply/readiness authorization. + +Explicit Commercial Research mode then applies a pre-mutation conjunction: every valid staged row must have complete lineage, approved exact-source rights, and registered `prices` scope. Failure returns before backup or canonical write and lists the independent blockers. Research mode does not inherit this commercial license gate, so existing local research compatibility remains intact. A passing guard is necessary for the commercial mutation path but is not payload review, freshness proof, readiness promotion, or automatic apply authorization. + +Direct provider refresh uses the same exact-source commercial metadata boundary before it can fetch or mutate. Each concrete provider has one fixed source ID; CLI aliases cannot borrow another record's rights, and an automatic ladder reviews each reachable leg independently. Commercial Research mode filters or refuses unapproved and scope-incomplete legs before provider execution, then rechecks the exact selected provider before adding fetched rows or status output. A missing or changed identity fails closed. Research mode retains its existing provider order and local refresh behavior. This enforcement does not approve any provider, add row-level lineage to canonical history, validate a payload, or make readiness current. + +Focused-cohort saved-row coverage applies field scope per lane rather than reusing one source-level approval. Technical availability, source/reference provenance, exact-source commercial rights, and registered scope remain separate for margins, free cash flow, cash, debt, shares, filing dates, earnings dates, each populated Revenue/EPS consensus metric, and every trusted-peer relationship. Cash and debt can therefore be partial independently, and Revenue-only consensus permission cannot unlock EPS. A date-only consensus row is not evidence. Candidate peers remain context-only. Commercial blockers are shown in the collapsed Advanced cohort evidence; research mode retains its source-backed local behavior. + +Adjusted price history in the same Commercial cohort additionally requires saved price readiness and at least one technically usable canonical date/positive-close row. Every retained history row must carry exact `source`, durable `source_ref`, and explicit `retrieved_at`, and its exact source must independently pass approved rights plus registered `prices` scope. One missing or unapproved row blocks the history rather than borrowing another row's permission. Current canonical history has no row lineage, so it remains local research context rather than commercially supported cohort evidence. Retrieval chronology remains a separate audit boundary. + +Canonical quarterly Revenue and EPS coverage applies the same field-specific commercial conjunction to accepted actual rows before a technical trend packet can mark the cohort lane usable. Every populated row for the metric needs source/reference/retrieval provenance, approved exact-source commercial rights, and literal `revenue` or `eps` scope. Revenue and EPS are independent: SEC Companyfacts' registered Revenue scope cannot unlock EPS. A passing commercial display review does not prove split-basis comparability, Q4 evidence, revision integrity, loader completeness, readiness, or nowcast activation; Research mode retains the existing packet-only behavior. + +The same preview summarizes newly ready, partial, and excluded feature transitions plus added or removed ticker rows. New DCF exclusions use the exact existing company-scope method and report a deterministic primary reason: non-operating asset type, acquisition/SPAC, closed-end fund, bank/bancorp, financial/insurance/mortgage, REIT, realty-trust/BDC, capital corporation, or nonpositive-revenue margin model. Reasons explain method fit; they are not company-quality labels. Transition counts can overlap for one ticker and are not current readiness totals. + Use this quick model card before relying on a page or report: | Check | What to verify | Boundary | @@ -23,7 +64,7 @@ Use this quick model card before relying on a page or report: | Freshness by lane | Latest price date, latest fundamentals filing date, peer review date, optional-context review date, and proof-ledger date. | Screenshots and sample reports are product evidence only; they do not prove current data freshness. | | Provenance | Source, as-of date, reviewed/import status, and whether the row is source-backed, candidate context, blocked, skipped, or excluded. | Metadata, candidate peers, or provider availability do not substitute for fundamentals, share-count, peer valuation, earnings, or analyst-estimate proof. | | DCF assumptions | WACC, terminal growth, forecast years, growth caps, FCF margin caps, and any normalization warning. | DCF output is scenario math, not a price target or instruction. | -| Peer context | Candidate peers are separated from trusted peer mappings and mapped-peer valuation inputs. | Candidate peers can guide review, but they are not trusted peer proof. | +| Peer context | Candidate peers are separated from trusted relationships, result read-through, and valuation-anchor inputs. | Candidate peers can guide review, but they are not trusted peer proof. | | Public share boundary | Public screenshots, walkthroughs, and QA evidence show product behavior. | They do not unlock blocked inputs or prove today's market/fundamental data. | ## Lane-Level Freshness Policy @@ -34,10 +75,18 @@ Freshness is evaluated per lane because a daily price series, an annual filing, - `review_due`: the row may remain useful as historical context, but the expected event or review cadence has arrived and the lane needs source review before a current-state claim. - `stale_or_unknown`: the as-of date, source event, or review evidence is missing or too uncertain to support a freshness claim. Required analysis stays blocked when freshness is part of its gate. +### Local Market-Observation Recency + +The four Personal Research routes receive one read-only evaluation of the selected local `prices.csv` path per dashboard run. The evaluator uses the dashboard's UTC review date, independently checks the profile price lane, selected ticker, SPY, and QQQ where shown, and fails closed for unreadable files, invalid dates, future dates, or missing usable observations. Its exact `current` threshold is seven calendar days; it is not an exchange-session SLA and does not imply intraday, real-time, provider, or hosted freshness. + +This local observation-recency check is separate from saved readiness, technical usability, source provenance, and commercial rights. A current local observation does not establish permitted market-data source rights or hosted freshness; those remain external gates. A stale observation remains historical context only and cannot support a current-market interpretation. + | Lane | Review expectation | Stale or changed behavior | | --- | --- | --- | | Price / momentum | Check the latest saved trading date when the project is opened; a daily after-close refresh is appropriate when current setup context is needed. | Show the latest date and short-history state. Do not imply intraday or real-time coverage, and do not fabricate missing OHLCV history. | | Fundamentals | Review after a verified quarterly or annual filing/source event. | Keep the filing/as-of date visible. A newer filing makes the prior row `review_due`; missing trusted fields remain blocked. | +| Quarterly business trend | Review only from a fully parseable canonical ledger of explicit, versioned quarterly actual rows available by the selected cutoff. | Any rejected canonical row blocks the complete dashboard trend/cohort packet; row-numbered reasons stay under Advanced. Revenue and EPS comparisons require compatible metric definitions and matching periods. Missing comparisons remain partial; ambiguous revisions and absent evidence remain blocked. Q4 is never derived from annual values. | +| Optional valuation, catalyst, and outcome evidence | Research-mode technical review remains separate. Commercial Research composition additionally reviews every used row against its exact source ID and literal `valuation_history`, `catalyst_evidence`, or `research_outcomes` registry scope. | One unknown, unapproved, or scope-incomplete used row blocks the supported/reviewed packet. Candidate catalyst context cannot satisfy supported evidence. Empty ledgers stay empty; blocker details stay under Advanced. | | Share count | Review after a validated capital-structure filing or explicit share-count fact. | Do not infer shares from price or market cap. Missing explicit evidence remains blocked even when metadata is current. | | DCF | Regenerate when a required fundamental, share-count, cash/debt input, price reference, or methodology version changes. | DCF inherits the weakest required input state. Stale or missing required inputs withhold current interpretation rather than silently reusing a prior conclusion. | | Trusted peers | Version and periodically review accepted relationships, and review again after a material business-model or segment change. | Missing source, rationale, reviewer, review date, or mapped-peer valuation inputs keeps the relationship candidate-only or blocked. | @@ -45,6 +94,24 @@ Freshness is evaluated per lane because a daily price series, an annual filing, A timestamp cannot turn an unsupported row into trusted evidence. Fresh metadata, screenshots, provider availability, or candidate context cannot substitute for the field-level source proof required by a readiness gate. +Quarterly business trend is descriptive and backward-looking. Sequential and year-over-year changes do not become a forecast, recommendation, or numerical adjustment to DCF or Earnings Nowcast. Revenue, EPS, operating margin, free cash flow, and FCF margin keep independent `ready`, `partial`, `blocked`, or `withheld` states. + +### Evidence-Integrity Valuation And Nowcast Backtest Method + +Historical valuation observations reject non-finite valuation inputs, require a canonical real `YYYY-MM-DD` denominator period end, reject blank, malformed, and non-calendar denominator period ends, and reject post-cutoff retrieval evidence. + +Walk-forward Nowcast target actuals and prior-year benchmarks canonicalize Revenue/EPS independently through explicit `supersedes_source_ref` lineage, retain one event per ticker/period, and withhold ambiguous leaves per metric so one metric does not suppress the other; they use cutoff-bounded prior-year benchmarks so post-cutoff revisions cannot leak. + +The quarterly cash-generation contract derives operating margin as reported operating income divided by compatible quarterly Revenue, free cash flow as **cash from operations + reported capital expenditures** while preserving the source-reported capital-expenditure sign, and FCF margin as derived free cash flow divided by compatible quarterly Revenue. Inputs must match currency, scale, accounting basis, duration basis, fiscal period, and period end. Cross-quarter comparisons use the stable accounting definition rather than treating different period-end dates as different measurement definitions. Missing, ambiguous, post-cutoff, zero-denominator, or incompatible inputs block only the affected metric. The contract is in-memory and descriptive; it does not write a source row, modify a canonical dataset, or promote DCF, Earnings Nowcast, peer, catalyst, outcome, backtest, or calibration readiness. + +The one-company adapter acceptance harness composes that derivation with the immutable commercial source-rights registry. It requires one ticker, one matching source ID, explicitly approved commercial use, explicit support for operating income, cash from operations, and capital expenditures, no unresolved cutoff/revision/component/compatibility blockers, and at least one period with all three derived metrics. Its only success state is a local review-routing state: **accepted_for_review is not production activation**. A passing candidate still requires the actual source payload, rights review, and human review before any production caller may supply it. + +The bounded SEC pilot now supplies that source-review evidence for NVIDIA Q1 FY2027 and AMD Q1 FY2026. It selects Revenue, operating income, cash from operations, and capital expenditures from one exact Companyfacts accession and three-month context, requires the matching SEC submissions `acceptanceDateTime`, and matches concept, context, magnitude, and inline-fact identity against the exact filed 10-Q. Companyfacts exposes the capex magnitude without a cash-flow sign, so the pilot records a negative capex observation only when the matching filed-table value is explicitly displayed as an outflow; that proof state is `explicit_filed_table_outflow`. Identical repeated inline disclosures may collapse to the first document fact, but conflicting Companyfacts values, contexts, accessions, primary documents, magnitudes, or timestamps block the complete preview. The read-only result remains `accepted_for_review`; it does not persist a row, activate Company Workbench, rebuild readiness, or establish another company or quarter. + +The AMD Q1 FY2026 accession `0000002488-26-000076` result proves bounded two-company portability for the same exact-filing method: one immutable identity, three official SEC endpoints, compatible quarterly components, timezone-aware acceptance, explicit filed-table capex outflow, exact rights scope, and complete withholding. It does not prove broad company coverage, and no third company is inferred from cohort membership. + +The explicit Company Workbench cash-generation preview composes that accepted result for one bounded route and fixed review cutoff. It applies **complete withholding** on any required metric, identity, rights, timing, accession, capex-sign, or compatibility failure: no partial numeric card or component value may leak into the primary answer. When accepted, the primary answer shows the three descriptive metrics with a preview state, never `ready`; **Advanced-only technical lineage** retains source references, accession, timestamps, definitions, component values, and blockers. This composition is read-only, makes no deterministic forecast change, and leaves the normal canonical Workbench route unchanged. + ## Profile Truth And Change Review Method The dashboard resolves exactly one data profile for each run: `Demo`, `Local Research`, or `Default`. Its trust strip separates: @@ -69,6 +136,28 @@ Preview validates one prospective entry without writing. Recording requires expl Confidence describes the reviewer's confidence in the documented hypothesis at that timestamp. It does not measure expected return, allocation size, or transaction direction. An empty journal is `not_started`; a thesis without a source-backed invalidation condition is `incomplete`; and a passed review date is `overdue`. +## Research Decision Lab Method + +The Research Decision Lab is a read-only composition of the selected-profile report, Thesis Journal, Decision-Process Scorecard, Research Outcome Review, and unresolved source-backed change items. Its review loop is: + +```text +Research plan -> evidence -> invalidation -> scenario -> review trigger -> learning +``` + +The six lanes are derived independently: `Plan` reflects only reviewer-authored thesis state; `Evidence` reflects only reviewed supporting or conflicting evidence; `Invalidation` requires an explicit source-backed invalidation condition; `Scenario` follows existing DCF readiness and visible-assumption gates; `Review trigger` follows unresolved evidence changes or reviewer-authored review dates; and `Learning` follows reviewed outcome evidence and its exact-source commercial-rights decision. Decision Lab lanes remain independent: a documented thesis cannot unlock Scenario, reviewed Learning cannot clear an Evidence conflict, and current readiness cannot create journal history. + +The next process step follows a deterministic documentation order: unresolved conflicting evidence, overdue review, missing thesis, missing invalidation, missing evidence, unscheduled review, DCF-ready but invisible assumptions, then continued monitoring. This order is not company attractiveness, expected return, portfolio priority, or a transaction signal. Monitor preserves the existing focused-cohort order and never reorders companies by process severity or market value. + +## Company Workbench HTML Research Brief Method + +The HTML Research Brief freezes one immutable prepared snapshot from the selected profile, ticker, saved report evidence, prepared Research Decision Lab state, and the already prepared Scenario Lab result. It is a read-only presentation of existing saved evidence and Python scenario math, not a new calculation engine, refresh, import, source collector, or readiness activator. + +The authoritative discounted explicit-cash-flow subtotal is the Scenario Lab result's `discounted_explicit_total`; the brief does not recompute that subtotal from displayed rows. Enterprise value, equity value, per-share value, sensitivity, provenance, source rights, field scope, freshness, and each research lane use independent field gates. A supported sibling may remain visible while an unsupported value is withheld. The label **Shares outstanding used by existing model** identifies the scenario input, while its split basis remains unverified unless explicit proof exists. + +The safe text/reference policy removes repository paths, local paths, secrets, control characters, unsafe or credential-bearing links, and recognized action language from portable content. Modal modifiers and active exposure fail closed. Only bounded HTTPS references without credentials, query strings, fragments, path traversal, or local paths may become links; an unsafe reference can retain only a sanitized non-link label. The standalone document applies an offline Content Security Policy with no JavaScript, image, font, form, or network access. Rendering, browser verification, print emulation, and in-memory PDF inspection follow a zero-write repository contract. Historical pre-fix evidence: Task 4 local matrix completed at `c8c313b9c`. Broad-review repairs must be evaluated only through direct current-head local and exact-head CI evidence; their presence alone establishes neither gate. + +Local engineering evidence does not establish source rights, current-market data, readiness activation, a new or professional line-item model, hosted operation, human or screen-reader conformance, independent validation, market fit, screening alpha, or probability calibration. + ## Public Workflow Boundary The public page order is a reading workflow, not an analysis shortcut. @@ -91,8 +180,20 @@ The product separates source inputs from analysis rules so the report is not a b | Earnings and analyst estimates | Trusted optional local imports. | Displays optional context only when rows pass schema and source-readiness checks. | | Report wording | Project code under `src/`. | Converts readiness, calculations, source state, and blockers into plain-language sections without inventing conclusions. | +Company Workbench answer and routing surfaces apply that peer separation directly: broad `peer_ready` or `peer_trend_comparison_ready` may support mapped-price trend context, but cannot unlock peer-relative valuation, premium/discount, or peer DCF comparison when `peer_valuation_comparison_ready` is explicitly false. An absent explicit field falls back only for legacy loaded-report compatibility; a loaded report may narrow readiness further, but it cannot widen an independently blocked peer-valuation state. + Third-party or optional provider data can supply rows, but it does not decide the research conclusion. The project code decides whether each local row is usable, which calculation is allowed, and which section must stay blocked or excluded. +## Daily Momentum And Valuation Research Queue Method + +The Discover queue is a deterministic research-prioritization filter, not a security rating. It evaluates only saved `momentum_ready=true` companies. A row passes momentum only when `close > SMA50 > SMA200`, three-month return is positive, six-month return is positive, and the existing benchmark-relative return versus SPY is positive. RSI and volume are not first-version gates. + +Valuation is company-relative rather than peer-relative or DCF-relative. The existing point-in-time Historical Valuation Regime must be `ready`, current under its 120-day policy, commercially evidence-eligible, and at or below percentile `40.0` within its latest compatible definition segment. Current denominators are never backfilled over old prices. A missing or incompatible ledger cannot be replaced with a standalone current multiple. + +Minimum value-trap safeguards require positive free cash flow, non-negative revenue growth, and debt to equity no higher than the configured quality-value threshold, currently `2.0`. Each quantitative input must be finite. Price and fundamental evidence independently require current observation dates, explicit provenance, approved exact-source commercial rights, and registered field scope. One failed gate withholds the row; no weighted score offsets another failure. + +Eligible rows are sorted alphabetically. With an explicit comparable prior queue, set comparison can derive `new_today`, `still_qualifies`, and `exited_today`; without that baseline the product shows only current eligibility and withholds change claims. The dashboard adds no writer or scheduler, reads no legacy Monthly Picks or Final Watchlist output, and cannot modify readiness, forecasts, probabilities, research ledgers, or reviewed evidence. Current repository inputs truthfully produce no eligible real-company row because required valuation history and permitted current-market price evidence are absent. + ## 1. How This Compares To Standard Research Workflows The product follows a familiar equity-research sequence, but keeps each step visible and gated: @@ -226,6 +327,7 @@ Peer analysis is separate from standalone DCF. - Missing peer mappings block the mapping proof path; mapped peers with missing price, fundamentals, market cap, or valuation fields block the peer valuation inputs proof path. - Sector or industry fallback context, if shown, must be labeled as fallback and not trusted manual peer data. - Contextual earnings read-through additionally requires a source-backed peer actual and explicit target/peer fiscal periods. Calendar proximity or business similarity is not inferred as period comparability. +- Peer valuation uses a stricter independent gate. A relationship may anchor peer medians only when it has source and as-of provenance, a reviewed `core_peer` or `secondary_peer` role, a relationship rationale, an economic comparability basis, and explicit `valuation_anchor_eligible=yes`. Aspirational, negative, excluded-close, and not-clean roles remain context-only. Legacy mappings without these fields stay visible as relationship context but cannot enter peer medians. This prevents the report from pretending that peer valuation exists when only partial peer data is available. @@ -342,7 +444,7 @@ The product uses the same readiness proof ladder in the dashboard, single-stock | --- | --- | --- | --- | | 1. Prices | Trusted local price rows. | Price/setup review, trend context, basic risk context when enough history exists. | Fundamentals, DCF, peers, earnings, and estimates. | | 2. Fundamentals / DCF inputs | Trusted company fundamentals with revenue, free cash flow or FCF margin, shares outstanding, and source metadata. | Fundamental field review and standalone DCF assumptions, scenarios, sensitivity, and fair value/share math. | Peer-relative valuation and optional earnings/estimate context. | -| 3. Source-backed peers | Trusted peer mappings first, then mapped-peer price, fundamentals, market cap, and valuation inputs. | Peer trend context first, then peer-relative valuation only when mapped-peer valuation inputs pass readiness. | Peer premium/discount or peer DCF comparison when peer mappings or mapped-peer inputs are incomplete. | +| 3. Source-backed peers | Trusted peer mappings first, then explicit peer role, economic comparability and valuation-anchor review, then mapped-peer price, fundamentals, market cap, and valuation inputs. | Peer trend context first, then peer-relative valuation only when at least two eligible anchors and their mapped-peer valuation inputs pass readiness. | Peer premium/discount or peer DCF comparison when roles, comparability, anchor decisions, mappings, or mapped-peer inputs are incomplete. | | 4. Optional context | Trusted earnings and analyst-estimate CSV rows. | Earnings timing context and analyst-estimate context. | Optional sections remain unavailable when those rows are missing. | Each step is permission to review a specific analysis layer, not permission to invent the next layer. Price-ready does not mean fundamentals-ready. Fundamentals-ready does not mean DCF-ready unless all required DCF fields pass. DCF-ready does not mean peer-ready. Peer-ready does not mean earnings or analyst estimates are available. @@ -358,17 +460,115 @@ The safe local sequence is: This ladder is why empty or partial outputs are useful: they show the first trustworthy proof step instead of hiding the gap behind a weak conclusion. +### Point-in-Time Universe Validation + +Implemented locally: read-only immutable-package status/preview with ten independent states: manifest, technical, temporal, identity, membership, corporate action, delisting, source rights, reproduction, and leakage. + +Completed review and remediation chronology is preserved in +`docs/internal/POINT_IN_TIME_UNIVERSE_REVIEW_HISTORY.md`. Priority 4 remains +externally incomplete; this methodology retains the current validation, +resource, reproduction, and evidence boundaries. + +Local resource budgets for one supplied package: preview sample 100 rows; manifest 1 MiB; each contract CSV 32 MiB; four contract snapshots combined 64 MiB; source-rights registry 4 MiB; declared rows 250,000 per contract; package traversal 32 entries. + +Every source-bearing raw evidence row is reviewed independently for exact +source rights even when technical parsing fails. Walk-forward minimum history +uses only distinct, otherwise qualifying evaluations for the same universe +strictly before the classified cutoff. Identity, membership, and event evidence +requires publication no later than retrieval. Snapshot inputs must be regular +files and are read from one descriptor with at most the declared limit plus one +byte retained for overflow detection. Deeply nested manifest JSON or +source-rights YAML fails with a stable readable input error instead of a +traceback. Duplicate JSON/YAML mapping keys and malformed contract headers +also fail nonzero, traceback-free, and write-free through the direct validator +and CLI/Make boundaries. + +These local bounds do not prove scale, hosted reliability, or market readiness. + +No permitted independently reviewed real dataset, accepted expected count/digest, or source-rights proof is on record. + +The deterministic reproduction contract is `membership_count_and_sha256_at_cutoff_v1`. It binds each declared universe and evaluation cutoff to the eligible membership count and SHA-256 digest without falling back to current constituents or ticker identity. + +The only operating entries for this method are +`make point-in-time-universe-status MANIFEST=` and +`make point-in-time-universe-preview MANIFEST= TOP_N=20`. Both are +read-only local inspection: they do not fetch a provider, write normalized or +rejected rows, apply data, rebuild readiness, or activate analysis. + +Synthetic fixtures remain test-only and local software evidence cannot complete Priority 4. + +Priority 4 remains open until one bounded permitted real dataset is independently reviewed, reproduces the expected membership count and digest, and passes rights, identity, corporate-action, delisting, survivorship, cutoff, partition, reproduction, and leakage gates. + +This local evidence does not change independent readiness for actuals, consensus, Revenue, EPS, valuation, catalysts, outcomes, backtesting, or calibration. + +It does not provide investment advice; numerical probability remains unavailable without calibration; Q4 evidence and EPS split-basis compatibility remain explicit; synthetic evidence stays test-only; candidate peer evidence remains candidate-context-only. + +The synchronized Priority 4 local validator is frozen unless a newly +reproduced Critical or Important defect requires reopening it. One bounded +permitted real dataset remains the exact external exit gate. While that +dataset and the independent Priority 5 source and peer evidence are +unavailable, Priority 6's provider-neutral authorization contract is complete +locally; hosted implementation remains environment-dependent. If the current +branch head lacks direct local matrix, branch synchronization, draft-PR update, +or exact-head CI evidence, complete those release steps first; otherwise select +the first incomplete safe roadmap priority. Local contracts cannot prove hosted +authentication, isolation, audit storage, retention, monitoring, rollback, or +operated capacity. + +If a permitted dataset or its rights evidence is unavailable, record that +external dependency and its exact unblock condition once. Do not retry an +unavailable provider or infer source access, field scope, rights, independent +review, or real-data validation from the local package result. + ## 12. Earnings Nowcast Pilot Method The Earnings Nowcast pilot is a separate readiness-gated lane. It does not reuse generic optional-context readiness and does not let an earnings date, current target price, candidate peer, or provider availability unlock a forecast. The deterministic baseline requires at least five source-backed prior quarterly actuals, the matching prior-year fiscal quarter, and an exact forecast-period point-in-time consensus snapshot available at the forecast cutoff. Fiscal-period rows are canonicalized before history is counted: exact duplicates count once, an explicit source reference can supersede a prior row, and unresolved conflicting values block only the affected metric. Revenue and EPS are evaluated independently and must use comparable currency, unit scale, accounting basis, EPS share/operations basis, and split treatment. A stable Revenue history may produce a range while incompatible or unstable EPS remains withheld. Each snapshot records the fiscal period, cutoff, expected report date, forecast horizon, model version, immutable input hash, freshness state, and source IDs. +SEC actual staging has a narrower primary-source lineage rule. Q1-Q3 accepts only 60-120 day SEC Companyfacts duration facts with a one-to-one fiscal identity/period-end mapping; cumulative facts and comparative facts without a source-backed original identity are rejected. Companyfacts EPS is explicitly marked `companyfacts_split_basis_unverified`, which cannot match an `as_reported` or split-adjusted consensus definition. Q4 is accepted only from an explicit fiscal-Q4 result table in a SEC-filed exhibit whose selected value column states one period-end date; filing timestamps and submission report dates are not substitutes. It is never derived by annual-minus-nine-month arithmetic, and guidance, ambiguous headers, or Q4 metrics spanning separate exhibits remain withheld. Evidence is append-only and cutoff-aware, so complete revision chains select the latest source-backed presentation without replacing historical evidence. The stage reports source-backed quarter-continuity gaps without inventing a missing fiscal period or value. Revenue and EPS readiness stay separate; EPS is withheld whenever split-adjustment, share, operations, accounting, currency, or scale basis is not source-backed and comparable across its usable history. The same sentinel is enforced downstream: Business Trend never displays or compares its EPS value, commercial cohort scope cannot override it, and backtesting cannot use it as a target or prior-year EPS outcome. Revenue evaluation remains independent. + +Filed-Q4 split basis has its own fail-closed proof boundary. Missing or malformed primary split language records `primary_split_basis_unverified`, not `as_reported`. Both unverified sentinels fail the shared downstream predicate, and arbitrary nonempty basis text is not treated as proof. Only a supported declared basis token or an explicitly parsed dated split statement can unlock EPS comparability; Revenue remains independent. + The model combines recent sequential behavior and same-quarter year-over-year seasonality with fixed versioned weights. It emits ranges and a `higher`, `aligned`, or `lower` consensus-relative classification. It does not accept a text-generated numeric adjustment. Peer earnings, company news, industry indicators, and macro evidence are directional context only; candidate peers remain `candidate_context_only`, while a reviewed trusted source may raise the context state without changing the numerical baseline. -Historical evaluation is chronological walk-forward only. The target actual and later consensus snapshots are excluded from model inputs and used only for scoring after the forecast. Fewer than 20 valid events is `backtest_insufficient`, not backtest-ready. Reports separate valid events from exclusions, group exclusion reasons, and include Revenue/EPS error, WAPE where valid, Revenue/EPS/joint interval coverage, directional accuracy, leakage failures, and latest-consensus/prior-year benchmarks. Numerical Beat/Miss probability stays unavailable until at least 100 valid out-of-sample observations pass finite-value, Brier-score, calibration-bin, and constant-rate benchmark gates. Every populated calibration bin reports its size, mean forecast probability, observed rate, and minimum-size status. +Historical evaluation is chronological walk-forward only. The target actual and later consensus snapshots are excluded from model inputs and used only for scoring after the forecast. A consensus row retrieved on or after the target report is a leakage failure even when its stated snapshot time is earlier. Conflicting rows at the same latest snapshot timestamp are excluded as an ambiguous revision, and a snapshot more than 90 days before the target report is excluded as stale. Fewer than 20 valid events is `backtest_insufficient`, not backtest-ready. Reports separate valid events from exclusions, group exclusion reasons, and include Revenue/EPS error, WAPE where valid, Revenue/EPS/joint interval coverage, directional accuracy, leakage failures, and latest-consensus/prior-year benchmarks. A sample that meets the event minimum but fails to improve the latest-consensus Revenue or EPS benchmark fails the validation verdict with an explicit gate. A raw calibration count cannot establish calibrated probability. `calibrated` requires paired verified calibration status and a leakage-safe benchmark-passing `BacktestReport`. The pairing requires immutable observations whose Brier score, constant-rate benchmark, calibration error, bins, and digest recompute exactly, plus one-to-one identity matching with those report events by ticker, fiscal period, and cutoff. The retained Boolean must equal the declared strict Revenue or EPS comparison (`actual > consensus`); equality is not a Beat, and identity-less or unrelated same-count evidence cannot promote probability. The declared outcome metric itself must have a forecast and beat its corresponding consensus benchmark; a report validated only for the other metric cannot authorize it. Stored relative classifications are rederived from their forecast intervals and consensus before directional aggregates are trusted. A separate canonical digest binds the complete supplied report package: normalized event chronology, ordered source IDs, model version and input snapshot hash, scored fields, exclusions, failure tuples, summary metrics, and benchmarks. Any post-assessment change requires a fresh pairing. This digest proves only that the supplied in-memory objects remain internally identical; it does not authenticate a provider, source row, rights grant, timestamp, or external provenance. Every populated calibration bin reports its size, mean forecast probability, observed rate, and minimum-size status. + +The committed `SYN1`-`SYN5` cohort is synthetic test evidence only. The intended five-company SEC staging scope is NVDA, AMD, AVGO, MU, and QCOM, but real pilot output remains `awaiting_point_in_time_consensus` until exact historical consensus evidence is present. This proves neither real semiconductor coverage, predictive accuracy, nor data freshness. The pilot does not predict post-earnings price movement and does not provide investment advice. + +The cohort readiness board reports actual-history, Q4, split-basis, exact-period consensus, backtest, and calibration gates independently. A raw calibration count cannot establish calibrated probability. `calibrated` requires the exact recalculated observation cohort and the exact leakage-safe benchmark-passing `BacktestReport` event cohort to agree on identity and declared outcome. Prospective collection preserves immutable snapshots and revisions; it does not reconstruct historical consensus from a current estimate. Provider configuration is an access state, not an evidence or rights state. + +Prospective collection preview also evaluates the exact declared source against the checked-in rights registry without splitting composite identifiers or inferring aliases. Technical `write_allowed` remains the append-only review decision. Commercial evidence separately requires approved rights plus literal `revenue_consensus` support when Revenue is populated and literal `eps_consensus` support when EPS is populated. Explicit Commercial Research mode conjuncts these states before a write and returns before filesystem mutation when either gate is incomplete; research mode retains its explicit reviewed local path. Passing this guard does not prove the estimate payload, point-in-time history, freshness, comparability, reviewer acceptance, calibration, or nowcast readiness. + +The earlier source-row validation boundary follows the same independence rule. Its exact normalized provider is resolved through the checked-in registry rather than a caller-declared rights label. Schema, fiscal-period, timestamp, value, and comparability failures determine technical rejection; only technically accepted rows enter commercial counts. Each accepted row requires `revenue_consensus` and `eps_consensus` scope only for the metrics it actually populates. Composite identifiers remain one unknown exact source, and `historical_evidence_reviewable` is a routing state rather than evidence activation or a readiness promotion. + +Collection preview and source-row validation share one pure commercial field-scope function. For one exact source ID it combines the existing rights decision with the literal ordered fields required by populated metrics; empty or duplicate field names are invalid, and unknown or composite IDs receive no alias expansion. This centralization changes no technical acceptance, blocker vocabulary, collection rule, or readiness state. Registry metadata cannot establish payload correctness, timestamp integrity, metric comparability, reviewer approval, collection, activation, backtesting, or calibration, and non-consensus evidence domains keep their own contracts. + +The supported source-review command is the read-only operating boundary before collection preview: `make earnings-consensus-source-review INPUT= PROVIDER= AS_OF=`. It reads the supplied upstream export without field enrichment, requires explicit provider identity and cutoff, and routes its ordered row mappings through the same validator. Human and JSON output expose rejected rows, candidate versus historical scope, rights, and metric scope; a completed review returns evidence even when the state is blocked. Invocation or ambiguous CSV-shape errors fail nonzero. The source-review export and prospective collection row are distinct input contracts; an accepted source row must be separately reviewed and explicitly mapped into the prospective schema, and the product neither infers fields nor writes that mapping. Neither outcome collects a row, changes readiness, or establishes activation. + +Pull-request engineering hygiene is range-based, not working-tree-based. The hosted event supplies exact base/head SHAs, checkout fetches their history, generated-artifact classification reads `BASE...HEAD`, and whitespace checks the same range. A clean checkout is therefore not treated as evidence that the PR changes no files. This automation is read-only engineering evidence, not source review, user validation, or merge approval. + +Temporal validation is fail-closed for both source categories. Evidence publication, snapshot, and retrieval timestamps must all be at or before the cutoff. The caller supplies one explicit UTC review cutoff, each row declares exactly `current_only` or `point_in_time`, and technical acceptance requires `snapshot_at <= retrieved_at <= review_cutoff`. Equality is allowed; missing/unknown scope, reversed ordering, invalid UTC timestamps, and post-cutoff evidence are rejected before a commercial row is counted. The cutoff does not infer publication time or prove provider availability, rights, payload correctness, freshness, collection, nowcast readiness, backtesting, or calibration. + +Batch collection preserves input order and evaluates each row against an in-memory virtual ledger containing saved evidence plus earlier technically reviewable proposed rows. Technical and commercial batch decisions remain separate. An empty input or any technical blocker rejects the complete record operation; explicit Commercial Research mode additionally requires every row's commercial evidence to pass. The CLI preview exposes the same row states and ordered blockers used by record, so a deterministic later rejection cannot leave earlier proposed rows appended. No row is reordered or repaired. This contract does not claim concurrent-writer exclusion or crash-safe transactionality. + +The saved prospective ledger is itself an input to that decision and is revalidated in full before status, preview, or record. Every row must satisfy the schema, snapshot IDs and evidence identities must be unique, and each ticker/period must form exactly one append-ordered, timestamp-increasing root-to-current-leaf revision chain. Preview produces a deterministic receipt bound to the normalized cutoff, Commercial Research mode, complete proposed batch, and complete saved ledger. Record requires the same cutoff and receipt and recomputes the decision before mutation; any changed input or intervening ledger write requires a new review. The receipt is local integrity evidence only, not proof of source truth, rights, reviewer approval, process locking, or crash recovery. + +Daily price lineage uses one temporal rule across normalization, staged validation/preview/apply, and DCF promotion review. For observation date `D`, local review uses `D + 1 day 00:00 UTC` as the earliest conservative availability boundary. A declared retrieval must carry an explicit timezone offset, must not precede that boundary, and must not exceed the explicit timezone-aware review cutoff. This does not infer an exchange close or provider publication time. Missing retrieval remains an independent lineage gap; malformed, naive, too-early, post-cutoff, or cutoff-unreviewed retrieval cannot become trusted commercial evidence or enter a canonical apply. Apply writes the one previously validated in-memory staged frame through same-directory atomic replacement, without claiming concurrent locking or crash-safe transactionality. + +Historical valuation regime context is computed only from observations where the denominator was publicly available by the matching numerator timestamp. Blank or malformed numerator/denominator evidence is retained only as a rejected row; it is never coerced to zero, and one invalid row does not discard valid sibling rows. Each metric definition forms a separate segment, and fewer than eight compatible observations remains `insufficient_history`. The output is a descriptive range and percentile, never a cheap/expensive label or recommendation. + +Research outcome review is an append-only learning loop tied to an original thesis and a closed observation window. Catalyst evidence is a cutoff-safe timeline of reviewed events. Neither uses price returns, sentiment scoring, or narrative adjustments to grade the company, grade the researcher, or change a numerical model. + +### Forward View V1 + +Forward View is a deterministic composition of existing evidence, not a second forecast or valuation engine. It keeps five lanes separate: explicit quarterly Revenue/EPS trend, source-backed DCF bull/base/bear scenarios, trusted peer read-through, reviewer-authored thesis evidence, and the independently gated Earnings Outlook. + +Scenario values appear only when DCF readiness passes, source metadata retains both source and source reference, and all three bounded scenarios calculate from the same saved report. Stale profile evidence is labeled partial. Candidate peers, news context, generated narrative, and journal text cannot change Revenue, EPS, DCF assumptions, valuation scenarios, or probabilities. Numerical surprise probability remains governed by the Earnings Nowcast calibration gate. + +Every Forward View packet shows its source cutoff, saved-profile freshness, model version, withheld fields, and one next research task. It accepts the production stock-report provenance contract (`provider`, `retrieved_at`, `official`, freshness, and notes) or an explicit source/source-reference pair, and preserves that provenance under Advanced evidence. Any stale, mixed, missing, or unknown saved-profile freshness downgrades otherwise usable sections to review-due rather than presenting them as current. -The committed `SYN1`-`SYN5` cohort is synthetic test evidence only. It proves deterministic software behavior, not real semiconductor coverage, predictive accuracy, or data freshness. The pilot does not predict post-earnings price movement and does not provide investment advice. +The Earnings Outlook lane loads only a real source-backed packet for the exact fiscal period named by the selected report. It never silently chooses a different real-company forecast period, never displays synthetic fixture evidence in the workbench, and reads the independent `probability_available` calibration gate before changing probability wording. Missing or incompatible evidence fails closed. The result describes plausible assumption cases and research wait conditions; it does not predict post-earnings price direction, rank companies, or provide an investment recommendation. ## 13. Methodology Limits diff --git a/docs/NEXT_STAGE_ROADMAP.md b/docs/NEXT_STAGE_ROADMAP.md index eb956fef1..d7e2124ce 100644 --- a/docs/NEXT_STAGE_ROADMAP.md +++ b/docs/NEXT_STAGE_ROADMAP.md @@ -2,17 +2,40 @@ The active roadmap is ROADMAP.md. It is the sole current prioritization and contains the active Now/Next/Later plan, dependencies, and stop rules. -This handoff is research-only and read-only: do not refresh providers, apply imports, stage files, commit, push, deploy, expose keys, or treat screenshots as data-freshness proof. +This handoff is research-only and read-only until an approved implementation slice begins: do not refresh providers, apply imports, deploy, expose keys, or treat screenshots as data-freshness proof. -Use current command output instead of static readiness counts: +Documentation and routing reconciliation is historical/completed context. Start with current repository, PR, CI, and hygiene truth. Use `make project-status-check` and `make readiness-ops-center` only as read-only diagnostics; do not follow their generic provider or broad-coverage suggestions when they conflict with the active no-refresh boundary. + +The observation-recency UX repair is complete locally: the four primary +Research routes show one route-relevant interpretation, while exact selected/ +profile/SPY/QQQ evidence remains in responsive Advanced cards. The direct +desktop/phone browser matrix writes no screenshots or other artifacts. + +The shared quant provenance/recency interpretation overlay is implemented locally +at `195ea18da9d1d6e06c36f8320509ccde46cdaa57`. It keeps valuation, indicator, +and review/risk calculation and readiness states independent from the +interpretation decision. Structured external provenance and exact-source rights +proof remain absent for the current local quant inputs, so results remain +historical/review-only or are withheld where proof is absent. This local +implementation does not establish current-market, hosted, commercial, nowcast, +or calibration completion. + +Historical pre-fix evidence: Task 4 local matrix completed at `c8c313b9c`. If the current branch head lacks direct local matrix, branch synchronization, draft-PR update, or exact-head CI evidence, complete those release steps first; otherwise select the first incomplete safe roadmap priority. Broad-review repairs must be evaluated only through direct current-head local and exact-head CI evidence; their presence alone establishes neither gate. Prior accessibility and Discover work is historical/completed context. + +Priority 7 remains open. Preserve its external manual-accessibility boundary: point-in-time data, consensus, peer review, hosted controls, independent sessions, assistive-technology review, and calibration remain incomplete and must be classified once rather than retried without changed evidence. + +The observation-recency UX repair and framework reliability slices must remain write-free with respect to readiness, canonical data, research ledgers, CSV, JSON, reports, screenshots, and timing artifacts. Preserve the product boundary: data readiness first, analysis second, research decision last; no investment advice, broker integration, order routing, auto-trading, direct buy/sell instructions, or fabricated data. + +## Readiness Release Evidence Contract + +Use this sequence for the named default-profile readiness candidate: ```bash -make project-status-check -make readiness-ops-center -make price-history-proof-queue TOP_N=25 -make price-history-batch-closeout TOP_N=25 +make readiness-release-review TOP_N=20 +make readiness-release-record PREVIEW_RECEIPT= REVIEWER= REVIEW_DATE= TECHNICAL_DECISION= DISTRIBUTION_DECISION= CONFIRM_REVIEWED=1 +make readiness-release-guard RECORD_ID= ``` -Use `INCLUDE_REVIEWED=1 make price-history-proof-queue TOP_N=25` only to audit reviewed source-limited items. The default queue shows executable unreviewed candidates; the batch closeout is read-only and does not record proof rows, stage, commit, or push. +The first and third commands are read-only. The record may truthfully preserve `rejected` or `external_review_required`; neither can pass guard. Stop and restart from review after any candidate byte, source, rights registry, proof ledger, Git head, path set, axis, decision, or staged-state change. A passing guard prints only exact named staging paths and never stages them. -Preserve the product boundary: data readiness first, analysis second, research decision last; no investment advice, broker integration, order routing, auto-trading, direct buy/sell instructions, or fabricated data. +This local contract does not change readiness and does not grant source rights. It does not substitute for independent review, external data, legal or distribution approval, hosted operation, accessibility review, calibration, or pilot evidence. The current package remains commercially withheld while those direct proofs are absent. diff --git a/docs/OPERATOR_GUIDE.md b/docs/OPERATOR_GUIDE.md index 66063e5a2..78df5812a 100644 --- a/docs/OPERATOR_GUIDE.md +++ b/docs/OPERATOR_GUIDE.md @@ -127,6 +127,53 @@ The dashboard and single-stock report use plain modes before showing detailed ta ## Data Proof Workflows +### Prospective Per-Field Proof + +The Stage A ledger is prospective-only. It preserves one exact reviewed ticker/field identity and revision chain without treating older batch narratives as structured evidence: legacy narrative proof is not upgraded. No sample field-proof rows are checked in, and an absent ledger is a valid empty state. A present empty, header-only, malformed, forked, or otherwise invalid ledger fails closed. + +Inspect status first, then preview one separately supplied reviewed CSV: + +```bash +make prospective-field-proof-status +make prospective-field-proof-audit +make prospective-field-proof-preview INPUT= AS_OF= +``` + +These inspection commands are read-only. Preview reports `technical_write_eligible` and `commercial_evidence_eligible` independently, so research-mode technical recording cannot be mistaken for commercial permission. The preview receipt binds ledger, input, cutoff, commercial mode, and source-rights registry. Review the proposed identities, timestamps, revision chain, source reference, payload digest, reviewer disposition, rights decision, registered field scope, and both blocker lists before recording. + +The Stage B audit is also read-only. It explains append order, normalized scopes, revision numbers, current versus superseded rows, reviewer dispositions, active-head blocker categories, and latest review time. It reports `preview_receipt_persisted=false` and `receipt_revalidation_required=true`: save the exact stdout receipt with the reviewed input if a later explicit record is intended. Audit does not activate readiness, does not update canonical data, and does not activate Company Workbench. It creates no report or JSON file; `JSON=1` writes structured output to stdout only. + +Record only the exact reviewed preview: + +```bash +make prospective-field-proof-record INPUT= AS_OF= PREVIEW_RECEIPT= CONFIRM_REVIEWED=1 +``` + +Record is an explicit append. It repeats validation and rejects any changed ledger, input, cutoff, commercial mode, or source-rights registry. Recording uses cooperative local locking for writers that follow this protocol. It is not crash-safe, not a database transaction, and does not protect against writers that do not cooperate or replace the path outside the protocol. + +This primitive does not activate readiness, does not update canonical data, does not update proof-readiness reconciliation, and does not activate Company Workbench, dashboards, reports, or legacy ledgers. A proof identity does not establish payload truth, source rights, commercial eligibility, freshness, or reviewer independence. Any readiness, reconciliation, canonical, or Workbench mapping requires a separate design. + +Before reusing any historical `supported`, `auto_supported`, or `human_reviewed_supported` batch outcome as evidence for current work, reconcile it with current saved readiness: + +```bash +make proof-readiness-reconciliation TOP_N=20 +make proof-readiness-reconciliation TICKERS=ARCT TOP_N=20 +``` + +`historical_supported_currently_blocked` means the older proof outcome and the current saved state disagree. Current saved readiness remains authoritative: inspect fresh source evidence before restarting that lane. `explicit_ticker_change` means the latest supporting proof explicitly names the ticker in `changed_tickers`. Scope membership alone is not ticker-level support. `current_canonical_row_missing` and other current blocker codes describe current saved inputs. Current blocker diagnosis does not establish the historical cause. The command is read-only and does not restore canonical data, rewrite proof history, promote readiness, fetch a provider, validate payload truth, or prove source rights, field scope, provenance, or commercial use. Its counts describe the current local snapshot only. + +### Calibration Evidence-Bundle Preview + +Preview one explicitly supplied immutable UTF-8 JSON bundle, no larger than 16 MiB: + +```bash +make calibration-evidence-bundle-preview BUNDLE= +``` + +The strict schema rejects missing or unknown fields, duplicate JSON keys, malformed identities, non-finite values, policy overrides, and incomplete source references. The command prints aggregate-only evidence to stdout and writes no CSV, JSON, report, readiness, ledger, or receipt file. Direct CLI consumers may request stdout JSON with `python3 -m src.calibration_evidence_bundle preview --bundle --format json`. + +`invalid` returns exit code 2. `blocked` and `contract_consistent_review_required` return exit code 0 so an operator can inspect their gates. The latter proves only that the supplied objects satisfy supported local consistency checks. It does not activate readiness, authenticate sources, establish rights, persist evidence, or authorize a numerical Beat/Miss result. External source and independent review remain required, and probability remains withheld. + Use targeted proof commands instead of broad refreshes by default: ```bash @@ -220,3 +267,19 @@ See `docs/analysis_capability_audit.md` for the deeper function-quality and prov The shipped analysis comes from project code under `src/` plus trusted local CSV inputs. Standard Python libraries support data handling, UI, and tests; optional `yfinance` is only a research-grade adapter. Support tools and libraries are not the stock-analysis rules. The shipped readiness gates, valuation gates, decision buckets, and research-only guardrails come from project code under `src/` plus trusted local CSV inputs. + +## Review A Readiness Release Candidate + +Run the exact default-profile sequence: + +```bash +make readiness-release-review TOP_N=20 +make readiness-release-record PREVIEW_RECEIPT= REVIEWER= REVIEW_DATE= TECHNICAL_DECISION= DISTRIBUTION_DECISION= CONFIRM_REVIEWED=1 +make readiness-release-guard RECORD_ID= +``` + +Review and guard are read-only. Record is the only writer and appends one exact-receipt-bound row to `data/readiness_release_reviews.csv`. Use `rejected` or `external_review_required` when that is the evidence; those decisions remain blocked. Never rerun record blindly after an uncertain write result—reload the printed record ID first. + +Stop and rerun review if any candidate byte, source input, rights registry, proof ledger, Git head, path set, review axis, decision, or staged state changes. A passing guard prints the exact named candidate and record paths; inspect those paths before staging. + +This workflow does not change readiness and does not grant source rights. It does not replace independent review, legal or distribution approval, source-owner evidence, current-market data, accessibility review, hosted validation, calibration, or pilot evidence. diff --git a/docs/PERFORMANCE_RELEASE_GATE.md b/docs/PERFORMANCE_RELEASE_GATE.md index 2d3514db0..d2a04e424 100644 --- a/docs/PERFORMANCE_RELEASE_GATE.md +++ b/docs/PERFORMANCE_RELEASE_GATE.md @@ -20,8 +20,10 @@ The tracked demo manifest supplies file hashes and row counts. Broad local CSV/r | Experience point | Limit | | --- | ---: | -| Visible Streamlit shell p90 | 1.0s | -| First useful route answer p90 | 3.0s | +| Warm visible-shell p90 | 1.0s | +| Cold visible-shell max | 1.0s | +| Warm first-useful p90 | 3.0s | +| Cold first-useful max | 3.0s | | Warm full-settle p90 | 5.0s | | Cold full settle | 10.0s | @@ -31,7 +33,7 @@ Stock Selector, Single-Stock Report, and Data Health are critical routes. Home a The release run used five warm runs and one server-cold run for every public route at both viewports: 60 recorded route samples, with zero route failures. -| Route | Viewport | Shell p90 | First useful p90 | Warm full p90 | Cold full | +| Route | Viewport | Legacy combined shell p90 | Legacy combined first useful p90 | Warm full p90 | Cold full | | --- | --- | ---: | ---: | ---: | ---: | | Home | 1280x720 | 0.250s | 1.974s | 2.487s | 2.823s | | Home | 390x844 | 0.247s | 1.987s | 2.206s | 2.843s | @@ -46,6 +48,76 @@ The release run used five warm runs and one server-cold run for every public rou Local verdict: **passed**. The result does not remove the separate hosted-preview and external-review gates. +## Commercial Beta Research Workflow Result + +The research workflow was measured on 2026-07-18 from immutable release-candidate +commit `e930bd0e1b1062c029a7633a226db8dbc03a506b` using the same tracked demo +snapshot and environment. The run covered Research Desk, Discover, Company +Workbench, and Monitor at both viewports with one cold and five warm samples: +48 recorded route samples, zero failures, and no horizontal overflow. + +| Route | Viewport | Legacy combined shell p90 | Legacy combined first useful p90 | Warm full p90 | Cold full | +| --- | --- | ---: | ---: | ---: | ---: | +| Research Desk | 1280x720 | 0.238s | 1.934s | 2.137s | 2.881s | +| Research Desk | 390x844 | 0.202s | 1.991s | 2.153s | 2.875s | +| Discover | 1280x720 | 0.211s | 1.864s | 2.444s | 2.768s | +| Discover | 390x844 | 0.208s | 1.842s | 2.474s | 2.964s | +| Company Workbench | 1280x720 | 0.224s | 1.783s | 2.856s | 3.162s | +| Company Workbench | 390x844 | 0.216s | 1.812s | 2.797s | 3.218s | +| Monitor | 1280x720 | 0.208s | 1.962s | 2.213s | 2.826s | +| Monitor | 390x844 | 0.231s | 1.811s | 2.246s | 2.679s | + +The two historical tables predate the category-correct sampling contract. Their +shell and first-useful columns each combined one cold and five warm samples; +nearest-rank p90 of six values selected the maximum of each mixed population. +Keep those values as historical evidence only. Current runs report and enforce +**Warm visible-shell p90** and **Cold visible-shell max** independently at the +same `1.0s` limit, and **Warm first-useful p90** and **Cold first-useful max** +independently at the same `3.0s` limit. + +## Current Sampling Reconciliation + +The current implementation separates warm and cold shell and first-useful +evidence before aggregation. It preserves the existing warm/cold full-settle +rules, required sample counts, raw samples, route markers, and thresholds. It +does not retry, drop outliers, select a fastest run, or omit cold evidence. + +Commit `6328c8cead7c27cb901e7878cd6d7d23fa11bb0e` passed a controlled local +Chrome run on 2026-07-31 with 48 recorded samples, zero route failures, and the +fixed demo snapshot. The aggregate Commercial Research Beta release check also +passed, including 4,474 full-suite tests. The separate accessibility browser +gate passed all six routes at both viewports plus its state harness on the same +commit. These results remain local engineering evidence only. + +| Route | Viewport | Warm shell p90 | Cold shell max | Warm first-useful p90 | Cold first-useful max | Warm full p90 | Cold full max | +| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | +| Research Desk | 1280x720 | 0.189s | 0.217s | 1.395s | 2.132s | 2.249s | 2.984s | +| Research Desk | 390x844 | 0.187s | 0.178s | 1.403s | 2.020s | 2.279s | 2.914s | +| Discover | 1280x720 | 0.184s | 0.171s | 1.385s | 1.963s | 2.419s | 2.982s | +| Discover | 390x844 | 0.191s | 0.175s | 1.389s | 1.918s | 2.423s | 2.932s | +| Company Workbench | 1280x720 | 0.184s | 0.177s | 1.413s | 1.935s | 2.751s | 3.241s | +| Company Workbench | 390x844 | 0.199s | 0.171s | 1.442s | 1.949s | 2.787s | 3.246s | +| Monitor | 1280x720 | 0.188s | 0.179s | 1.536s | 1.985s | 2.395s | 2.839s | +| Monitor | 390x844 | 0.183s | 0.174s | 1.543s | 1.978s | 2.384s | 2.824s | + +The temporary JSON remains at +`/tmp/stock-command-center-commercial-beta-performance.json` and stays out of +Git. Accept later category-specific failures without an unchanged retry loop; +only a directly measured warm or cold failure justifies route-startup +optimization. + +Reproduce the research contract and browser evidence with: + +```bash +make commercial-beta-performance-contract +make commercial-beta-performance-gate +``` + +The generated evidence path is +`/tmp/stock-command-center-commercial-beta-performance.json`. Keep it out of +Git. This local result does not prove hosted performance, external-user task +success, licensed broad data operation, or predictive accuracy. + ## Reproduce ```bash @@ -66,7 +138,7 @@ Replace the example only after a real hosted URL exists. A hosted result must be ## Stop Rules - Do not rerun broad data refreshes to improve performance numbers. -- Do not select the fastest run; use the recorded p90 summary. +- Do not select the fastest run; keep warm p90 and cold maximum independent for shell and first-useful evidence. - Do not call a missing browser dependency a pass. - Do not stage the generated JSON by default. - Do not call a preview private unless the host enforces access control. diff --git a/docs/PERSONAL_RESEARCH_MODE.md b/docs/PERSONAL_RESEARCH_MODE.md new file mode 100644 index 000000000..f20fa60e7 --- /dev/null +++ b/docs/PERSONAL_RESEARCH_MODE.md @@ -0,0 +1,106 @@ +# Personal Research Mode + +Personal Research is the default local workspace for repeated company review. It composes existing readiness, Single-Stock Report, Change Monitor, Thesis Journal, Scenario Lab, peer-context, freshness, and Earnings Outlook capabilities. It does not introduce a second calculation or data-persistence system. + +## Workflow + +Personal Research uses one top workflow navigation for its four primary pages. The sidebar retains the Personal/Public/Operator workspace selector and selected-profile caption, but it does not repeat the Research page choices. The Research main column also omits the Operator command/readiness header and broad profile strip, so the route-specific question and answer appear first. Direct URLs, ticker parameters, Public mode, and Operator mode are unchanged. Data Health and Proof History remain secondary direct-link evidence routes and intentionally do not join the primary navigation. + +1. **Research Desk** starts with one **Today's Research Brief**: what saved work needs attention, one traceable reason, the saved-readiness warning, and one action to Monitor or Discover. It deduplicates overlapping weekly and source-change counts rather than adding them. A zero-item answer applies only to evidence loaded in the workspace and never proves that no external event or research need exists. Weekly, cohort, observation, coverage, and source-change detail remains under collapsed Advanced Evidence. +2. **Discover** answers two different questions separately. **Screen eligibility — when supported** applies the unchanged strict Daily Momentum & Valuation evidence gates. **Browse saved companies** is an alphabetical, readiness-only evidence-access list that explains why each company is inspectable, what evidence is usable, and the main evidence gap before opening its Company Brief. Browsing is not screen eligibility, ranking, opportunity scoring, or a recommendation, and it does not read legacy ranking outputs. +3. **Company Workbench** starts with one Company Brief answering `Use now -> Still withheld -> What changed -> Next research task`, followed by the visible research-only stop rule and ticker-preserving Data Health action. `Open evidence and analysis modules` restores the existing Research Decision Lab, business trend, valuation, forward context, authoring, conclusion detail, methodology, and offline HTML brief for the current session. The primary task is not repeated in the detailed layer; the same task object still feeds the HTML snapshot. Technical lane coverage and Decision Lab evidence remain under collapsed Advanced sections. + +After that explicit action, the detailed sequence remains `What Changed -> Research Decision Lab -> Business Trend` before valuation, forward context, withheld evidence, and conclusion detail. +4. **Monitor** now answers one question through one **Follow-up Queue**: **Since the last review**, **Needs verification**, **Waiting on evidence**, **Scheduled context**, and **Evidence freshness**. When saved actionable evidence exists, five compact panels render once and any due process rows stay in focused-cohort order. When every actionable count is zero, Monitor renders one concise empty state, explicitly says this does not prove that no external event, risk, or research need exists, and provides one **Open Discover** action. + +The Follow-up Queue reuses the fixed seven-day summary, unresolved saved source-change rows, existing discipline precedence, and independent readiness/observation freshness. Full process rows, stable identities, and source-change evidence remain under `Advanced: Monitor evidence`. Monitor-only rows are retained there; they are not deleted, promoted, rescored, or reinterpreted. Five-company Earnings Nowcast readiness remains a separate collapsed Advanced section. Ordinary route use writes nothing. + +Company Workbench arbitrates one overall next task. Its change context is explicitly `none`, `snapshot_only`, or `source_backed`, so an empty queue receives a neutral no-queued-change badge and is never mislabeled as snapshot evidence. Only a change carrying separate explicit source-backed eligibility can win: open review keeps the event's suggested task, `still_blocked` keeps its `wait_for_evidence` condition, and `intentionally_deferred` keeps its `monitor` condition. Snapshot-only context receives no source-backed badge and cannot outrank the existing ordered Research Conclusion priority. Forward View guidance stays lane-specific and does not compete for the overall task. This presentation leaves readiness and evidence states independent and unchanged. + +Data Health and Proof History remain available through **Advanced Evidence**. Operator mode remains the place for source setup, validation, preview, proof, and maintenance commands. Public mode retains the controlled five-page demonstration. + +Data Health and Proof History stay inside Personal Research mode when opened from Company Workbench Advanced Evidence. Both preserve the selected ticker and show a direct **Return to Company Workbench** action before evidence content; a missing ticker returns to Research Desk instead of inventing a company. This navigation does not change readiness or evidence state, record a review outcome, expose Operator commands, or promote blocked inputs. + +## Research States + +Personal Research may route work only as: + +- `review_now`: verified evidence changed or a reviewed task is open. +- `monitor`: no immediate evidence task is available. +- `wait_for_evidence`: required source proof or freshness is unavailable. +- `excluded`: the analysis is not applicable. + +These are workflow states. They are not rankings, expected-return claims, investment recommendations, or transaction instructions. + +## Research Decision Lab + +Company Workbench shows exactly one compact six-lane research-process summary for Plan, Evidence, Invalidation, Scenario, Review trigger, and Learning after `What Changed`. It composes saved evidence only and cannot replace `Use now`, `Still withheld`, the Data Health handoff, Research Conclusion, or Next Research Task. Identities, evidence summaries, and technical details remain under `Advanced: Decision Lab evidence`. + +Monitor derives the same contract independently for each saved focused-cohort ticker. One invalid ticker becomes unavailable without changing another ticker, and display order remains the existing cohort order rather than process severity, market value, or expected return. Source-change evidence remains independently preserved inside `Advanced: Monitor evidence`; the primary Follow-up Queue composes it without changing its review state. + +## Truth Boundaries + +- Missing trend, valuation, peer, earnings, estimate, or nowcast inputs stay unavailable rather than inferred. +- Candidate peers and news context cannot become trusted peer proof or modify numerical forecasts. +- Source-backed peer relationships do not automatically become valuation anchors. Peer role, economic comparability, and anchor eligibility are reviewed independently; legacy or context-only rows are withheld from peer medians. +- Numerical Beat/Miss probability remains withheld without calibration evidence. +- Scenarios remain bounded, session-local assumption tests and do not change canonical data. +- The Change Monitor and Thesis Journal do not mutate readiness or source rows. +- Broad universe tracking does not imply broad analysis readiness. + +## Focused-Use Strategy + +The current saved profile deterministically selects up to 25 eligible operating companies or ADRs with price-ready evidence. Active-universe and deeper ready lanes affect review order only; they do not create a score, expected return, or recommendation. If fewer than 25 eligible companies exist, the cohort reports `awaiting_reviewed_source` and is never padded. + +## Focused Cohort Coverage + +Research Desk composes its brief and a read-only coverage matrix for every focused-cohort company. The matrix separates adjusted daily price history, quarterly Revenue, quarterly EPS, margins, free cash flow, cash/debt, shares outstanding, trusted peers, filing dates, earnings dates, and exact-period point-in-time consensus. Each lane is labeled `usable_now`, `partial`, `candidate_context_only`, `blocked`, or `excluded` from saved source evidence only. The single brief renders first; weekly cards, concise cohort cards, observation detail, the full company-by-lane matrix, and source-change rows remain under Advanced Evidence. + +Closing Advanced Evidence does not remove or combine cohort states. A DCF-ready flag does not fabricate quarterly actuals, earnings dates, or consensus. Candidate peers do not become trusted peers. Missing source provenance remains blocked, and non-company rows remain excluded rather than forced through operating-company analysis. + +In Company Workbench, the Peer Read-Through Map answers result-context and valuation-anchor questions separately. `core_peer` and `secondary_peer` are the only roles that can become anchors, and only with explicit source, as-of date, relationship rationale, comparability basis, and `valuation_anchor_eligible=yes`. Aspirational, negative, excluded-close, not-clean, candidate, and legacy-unreviewed relationships remain visible context without entering peer medians. + +Discover evaluates only saved `momentum_ready` rows through a strict, unweighted intersection: price above SMA50, SMA50 above SMA200, positive three- and six-month returns, positive SPY-relative return, a current commercial-eligible own-history valuation percentile at or below 40, positive free cash flow, non-negative revenue growth, and debt to equity no higher than the configured quality-value limit. Current-market recency, price and fundamental provenance, exact-source rights, and registered field scope must also pass. Missing, stale, non-finite, malformed, unverified, or restricted evidence withholds the row. Eligible results are alphabetical research candidates with ticker-bound Company Workbench routes, never a score or investment ranking. + +The separate saved-company browser reads focused-cohort ticker readiness only, filters to company rows, and sorts by ticker. It never imports queue rank, score, priority, or legacy decision/watchlist output. A saved row may be opened to inspect available evidence even when it does not qualify for the strict screen. Each row therefore answers **Why inspectable**, **Usable evidence**, and **Main evidence gap** and uses one `Open Company Brief` action. + +The first implementation is read-only and accepts no prior saved queue, so Discover labels current eligible rows without claiming they are newly eligible; `new_today`, `still_qualifies`, and `exited_today` remain unavailable until a separately approved operating slice supplies comparable snapshots. It writes no CSV, JSON, report, screenshot, timing, readiness, canonical-data, or ledger artifact. Current repository inputs have no historical-valuation ledger and no approved commercial price lineage, so the truthful real-data result is an empty eligible set with blockers under `Advanced: daily queue evidence`. + +Company Workbench keeps the existing selected-ticker coverage calculation under `Advanced: selected-company lane coverage`. The primary brief composes existing selected-answer, change, and next-task contracts without recalculating or promoting any lane. Detailed modules are closed by default and return only after the explicit session-local action; that action writes no file, changes no canonical evidence, and changes no readiness. Closing the lane cards does not infer coverage, combine readiness states, or unlock a report section. + +Monitor also carries one compact five-company Earnings Nowcast evidence answer. Company Workbench keeps historical valuation regime, catalyst evidence, and research outcome learning inside the existing Valuation, Forward View, and Thesis Journal sequence. Their raw rows stay under Advanced. These helpers do not add a route, refresh a provider, apply an import, or change canonical readiness. + +Historical valuation is descriptive only and requires a denominator that was public at the matching price timestamp. Research outcomes preserve reviewed learning without return attribution or skill scoring. Catalyst events require source and cutoff timestamps and cannot change forecasts, valuation inputs, readiness, or recommendations. + +Header-only templates live at `docs/templates/historical_valuation_observations.csv`, `docs/templates/research_outcome_reviews.csv`, and `docs/templates/catalyst_evidence.csv`. Outcome and catalyst inputs must be previewed before the explicit `CONFIRM_REVIEWED=1` record command. The canonical ledgers remain local working evidence unless one exact artifact is intentionally reviewed for a controlled package. + +The weekly summary is derived from deduplicated, source-backed Change Monitor events from the prior seven days plus reviewer-authored journal review dates. The Desk brief composes that saved summary with the unresolved source-change queue using the larger saved count, not their sum. It writes no data and does not turn a missing event into a no-change claim. + +Quarterly business trend is descriptive evidence, not a forecast. Revenue and EPS comparisons require explicit versioned quarterly actuals with compatible definitions. Operating margin, free cash flow, and FCF margin now have a separate in-memory evidence contract and independent states, but production values remain withheld until a **reviewed quarterly source adapter** supplies explicit compatible observations. Sequential and year-over-year changes are withheld when matching periods or definitions are unavailable. Q4 is never derived from annual results; every Q4 component requires explicit filed-quarter evidence. Primary cards show the research answer only; component values, formulas, and source references stay in the collapsed Advanced quarterly evidence table. The local acceptance harness evaluates in-memory candidates only: **no adapter file is loaded or written**, and an accepted candidate is not supplied to Company Workbench or promoted into readiness. + +## Mobile First-Action Density + +At phone width, the compact profile context keeps Data profile, Sources through, Freshness, Price-ready, and DCF-ready visible in two rows. The route card omits only its duplicate freshness row while preserving the page, selected scope or ticker, next action, and research-only boundary. Discover therefore shows its search task and first review row sooner; Monitor shows the weekly state sooner; and Company Workbench shows `Selected Company`, a collapsed `Review path`, and the first-read answer without the full sequence consuming the viewport. + +This mobile first-action density improvement does not change readiness, source dates, coverage, evidence, forecasts, valuation, or research conclusions. Desktop profile and route metadata remain unchanged. Company Workbench keeps the complete review path available through the same collapsed disclosure at desktop and phone widths. + +## SEC Quarterly Cash-Generation Pilot + +The read-only NVIDIA Q1 FY2027 pilot proves that one exact SEC accession can supply compatible Revenue, operating income, cash from operations, and explicitly signed capital expenditures for adapter review. Run `make sec-quarterly-cash-preview AS_OF=` only when `SEC_USER_AGENT` is configured. The command fetches three exact SEC endpoints in memory and writes no cache or generated artifact. + +Its success state is deliberately narrow: **accepted_for_review is not production activation**. Company Workbench continues to withhold real-company operating margin, free cash flow, and FCF margin until a separate activation review connects compatible observations. The pilot does not change saved readiness, Earnings Nowcast, consensus, valuation, catalysts, outcomes, backtesting, calibration, or another company or quarter. + +## Explicit Company Workbench Cash-Generation Preview + +The bounded route `?mode=research&page=company-workbench&ticker=NVDA&open=1&cash_preview=1` adds one explicit, read-only **Cash-generation review preview — not production evidence** below the canonical Business Trend answer. The fixed review cutoff is `2026-07-20T23:59:59-04:00`. The normal Company Workbench route does not call the preview loader and continues to show only canonical production evidence and its existing withheld states. + +AMD Q1 FY2026 accession `0000002488-26-000076` is the second exact filing available through the explicit `cash_preview=1` route. NVIDIA and AMD now provide bounded two-company portability through the same immutable loader, complete-withholding contract, preview-only cards, and Advanced lineage. Normal Company Workbench routes remain canonical and never load the preview. + +The preview is all-or-nothing: operating margin, free cash flow, and FCF margin appear only when an accepted NVIDIA Q1 FY2027 or AMD Q1 FY2026 SEC evidence packet passes the complete identity, source-rights, cutoff, accession, acceptance-time, component, capex-sign, and compatibility contract. Any required failure withholds all three preview metrics. Accession, source URL, timestamps, component values, definitions, and blockers stay in the collapsed Advanced evidence section. The route writes no cache or canonical data, creates no readiness or generated artifact, and cannot promote any product lane. + +## Repeated Review Routine + +- **Daily or after a source refresh:** open Discover and check strict screen eligibility; an empty result is valid and does not relax the gates. Independently browse an alphabetical saved company to inspect its usable evidence and gaps, then use Monitor for unresolved source tasks. +- **Company review:** use Discover, open one Company Workbench, read What Changed, Business Trend, Valuation, Forward View, and What Remains Withheld before recording a conclusion. +- **Weekly:** review the weekly summary, overdue journal reviews, and wait conditions. A no-change summary means no traceable saved event in the review window, not that the company had no real-world change. +- **Operator handoff:** use Data Health or Proof History only when source proof, blocked inputs, or event evidence is the question. diff --git a/docs/PILOT_FEEDBACK_CLOSEOUT_CHECKLIST.md b/docs/PILOT_FEEDBACK_CLOSEOUT_CHECKLIST.md index c9b54bad8..62faf3f20 100644 --- a/docs/PILOT_FEEDBACK_CLOSEOUT_CHECKLIST.md +++ b/docs/PILOT_FEEDBACK_CLOSEOUT_CHECKLIST.md @@ -1,9 +1,9 @@ # Pilot Feedback Closeout Checklist -Use this checklist after 5-10 external reviewers complete the public workflow: +Use this checklist after 10-20 independent target-persona reviewers complete the Commercial Research Beta workflow: ```text -Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History +Research Desk -> Discover -> Company Workbench -> Monitor ``` This closeout is product-workflow evidence only. It is not data proof, source proof, data freshness proof, investment advice, broker integration, or a trade instruction. @@ -16,7 +16,7 @@ Keep the working feedback log outside the repository: cp docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv /tmp/stock-command-center-pilot-feedback.csv ``` -Before any reviewed feedback artifact is committed, remove names, account details, investment opinions, price targets, trade decisions, portfolio information, and any other personal information. +Before any reviewed feedback artifact is committed, remove names, contact details, account details, investment opinions, price targets, trade decisions, portfolio information, and any other personal information. Delete rows with `consent_confirmed=no` or `consent_withdrawn=yes`. Delete remaining working rows on their `retention_delete_after` date unless they have become intentionally reviewed, anonymized aggregate evidence. ## 2. Classify Every Reviewer Row @@ -32,6 +32,12 @@ Each row should end in one of these product outcomes: Do not classify a reviewer preference as a source-proof defect unless it names a missing trusted source row, validate/preview/apply failure, or proof-ledger gap. +Before aggregating results, confirm that each included row records task success, +time to first useful answer, readiness comprehension, evidence tracing, +authoring friction, trust, misuse risk, perceived performance, repeat-use +intent, and the most important missing workflow. Missing fields remain missing; +do not infer or backfill reviewer answers. + ## 3. Fix Only Reproducible Product Issues Fix issues in this order: diff --git a/docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv b/docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv index b3ec5abc8..fcc14ca03 100644 --- a/docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv +++ b/docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv @@ -1,2 +1,2 @@ -review_label,review_path,ticker_or_example,started_at,where_started,usable_now,blocked_or_excluded,next_expected_action,time_to_first_answer_minutes,confusion_point,reviewer_signal,closeout_outcome,task_success,moderator_help_required,readiness_comprehension,misuse_risk,trust_in_evidence,perceived_performance,repeat_use_case,reproducible_issue_route,reproducible_issue_viewport,reproducible_issue_observed,reproducible_issue_expected,reproducible_issue_impact,evidence_pointer,notes -R01,GitHub-only or local demo profile,NVDA or chosen route,YYYY-MM-DD,Home / README / other page,Reviewer words only,Reviewer words only,Reviewer words only,3,Exact page label or route if any,clear / confused / blocked_by_environment / suggestion_only,clear / reproducible_ui_issue / documentation_gap / environment_limited / intentionally_deferred,completed / completed_with_help / not_completed,none / one_prompt / multiple_prompts,correct / partial / incorrect,none / uncertain / mistook_as_recommendation_or_live_terminal,high / medium / low,responsive / noticeably_slow / appeared_frozen,Research-readiness task or none,,,,,,,No names account details investment opinions price targets trade decisions or portfolio information +review_label,review_path,ticker_or_example,started_at,consent_confirmed,consent_withdrawn,retention_delete_after,where_started,usable_now,blocked_or_excluded,next_expected_action,time_to_first_answer_minutes,confusion_point,reviewer_signal,closeout_outcome,task_success,moderator_help_required,readiness_comprehension,evidence_trace_result,authoring_friction,misuse_risk,trust_in_evidence,perceived_performance,repeat_use_case,repeat_use_intent,most_important_missing_workflow,reproducible_issue_route,reproducible_issue_viewport,reproducible_issue_observed,reproducible_issue_expected,reproducible_issue_impact,evidence_pointer,notes +R01,controlled research workspace or local demo profile,NVDA or chosen route,YYYY-MM-DD,yes / no,yes / no,YYYY-MM-DD,Research Desk / other page,Reviewer words only,Reviewer words only,Reviewer words only,3,Exact page label or route if any,clear / confused / blocked_by_environment / suggestion_only,clear / reproducible_ui_issue / documentation_gap / environment_limited / intentionally_deferred,completed / completed_with_help / not_completed,none / one_prompt / multiple_prompts,correct / partial / incorrect,completed / completed_with_help / not_completed,none / low / medium / high,none / uncertain / mistook_as_recommendation_or_live_terminal,high / medium / low,responsive / noticeably_slow / appeared_frozen,Research-readiness task or none,yes / maybe / no,Reviewer words only,,,,,,,No names account details investment opinions price targets trade decisions or portfolio information diff --git a/docs/PILOT_REVIEW_FEEDBACK_TEMPLATE.md b/docs/PILOT_REVIEW_FEEDBACK_TEMPLATE.md index d88063f59..2de7e6960 100644 --- a/docs/PILOT_REVIEW_FEEDBACK_TEMPLATE.md +++ b/docs/PILOT_REVIEW_FEEDBACK_TEMPLATE.md @@ -1,23 +1,29 @@ # Controlled Pilot Review Feedback -Use this template with 5-10 external reviewers after they open the GitHub demo package or the local `demo` profile. It measures product clarity and workflow reliability only. It does not collect market data, validate a source, change readiness, or support an investment conclusion. +Use this template with 10-20 independent target-persona reviewers after they open a verified controlled delivery path or the local `demo` profile. It measures product clarity and workflow reliability only. Owner-led, automated, fixture, and screenshot sessions do not count. It does not collect market data, validate a source, change readiness, or support an investment conclusion. + +Before the session, tell the reviewer what will be observed, that participation is voluntary, that they may stop or withdraw their row, and when the working note will be deleted. Record only an anonymous session label and the minimum product feedback below. A withdrawn row must be deleted from the working log and must not appear in aggregate evidence. ## Reviewer Task -Ask the reviewer to follow the public path without operator instructions: +For the Commercial Research Beta, ask the reviewer to follow the current research path without operator instructions: ```text -Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History +Research Desk -> Discover -> Company Workbench -> Monitor ``` -Ask them to use one example ticker and answer these four questions in their own words: +Ask them to use one example ticker and answer these eight questions in their own words: 1. Where did you start? 2. What could you use now? 3. What was blocked or excluded? -4. What would you do next? +4. What evidence supports one conclusion? +5. What was difficult to author or update? +6. What would you do next? +7. Would you use this workflow again for a real research task? +8. What is the most important missing workflow? -Do not coach the answers. The purpose is to see whether the public workflow is self-explanatory in under three minutes. +Do not coach the answers. The purpose is to see whether the research workflow is self-explanatory in under three minutes. ## Optional Task-Based Pilot @@ -39,18 +45,52 @@ Record the scorecard as workflow evidence only: | Misuse risk | `none`, `uncertain`, or `mistook_as_recommendation_or_live_terminal`. | | Trust in evidence | `high`, `medium`, or `low`, with a short reason about provenance or blocker clarity. | | Perceived performance | `responsive`, `noticeably_slow`, or `appeared_frozen`, with the affected route. | +| Evidence trace | `completed`, `completed_with_help`, or `not_completed` when tracing one conclusion to its source evidence. | +| Authoring friction | `none`, `low`, `medium`, or `high`, with the affected record or step. | | Repeat-use case | Short research-readiness task they would repeat, or `none`. | +| Repeat-use intent | `yes`, `maybe`, or `no`, with an optional short reason. | +| Most important missing workflow | One reviewer-stated missing workflow, or `none`. | Do not ask whether the reviewer would buy or sell a security. Do not collect a price target, portfolio position, account information, or personal investment decision. +## Commercial Research Beta Tasks + +Use this sequence for the Commercial Research Beta after the reviewer can open +the controlled research workspace: + +```text +Research Desk -> Discover -> Company Workbench -> Monitor +``` + +1. Start at Research Desk. +2. Identify the focused cohort and freshness state. +3. Use Discover to select one reviewable company. +4. Open Company Workbench. +5. Explain what can be used now. +6. Identify one withheld input. +7. Review Business Trend, Valuation, and Forward View boundaries. +8. Use Monitor to determine whether verified evidence changed. +9. State why the product is research-only. + +Capture task success, time to first useful answer, readiness comprehension, +evidence tracing, authoring friction, misuse risk, evidence trust, +perceived performance, repeat-use case and intent, and the most important +missing workflow. Do not fabricate reviewer sessions, completion rates, quotes, or findings. +If no reviewer can access a verified delivery path, classify the +stage as `awaiting_external_review` rather than treating local test evidence as +pilot evidence. + ## Capture Sheet -Record one row per review session outside the repository unless the note contains no personal information. Use `docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv` as the copy/paste header when you want comparable rows across 5-10 reviewers; keep the working copy outside Git unless it is intentionally anonymized review evidence. +Record one row per review session outside the repository unless the note contains no personal information. Use `docs/PILOT_REVIEW_FEEDBACK_LOG_TEMPLATE.csv` as the copy/paste header when you want comparable rows across 10-20 reviewers; keep the working copy outside Git unless it is intentionally anonymized review evidence. | Field | Record | | --- | --- | | Review label | Anonymous session label such as `R01`; do not use a full name. | -| Review path | GitHub-only or local demo profile. | +| Consent confirmed | `yes` only after the reviewer agrees to the session and minimal anonymous capture. A `no` session does not begin. | +| Consent withdrawn | `yes` requires deletion of the row; do not retain it as evidence. | +| Retention/delete after | The date by which the working row will be deleted unless it becomes intentionally reviewed, anonymized aggregate evidence. | +| Review path | Controlled research workspace or local demo profile. | | Ticker/example | The example ticker or route the reviewer chose. | | Where did you start? | Their first page or artifact opened. | | What could you use now? | Their description of supported analysis or context. | @@ -60,9 +100,9 @@ Record one row per review session outside the repository unless the note contain | Confusion point | Exact page, label, or route that caused confusion, if any. | | Reviewer signal | `clear`, `confused`, `blocked_by_environment`, or `suggestion_only`. | | Closeout outcome | `clear`, `reproducible_ui_issue`, `documentation_gap`, `environment_limited`, or `intentionally_deferred`. | -| Task scorecard | Record task success, moderator help, readiness comprehension, misuse risk, trust, perceived performance, and repeat-use case only when the optional task pilot was run. | +| Task scorecard | Record task success, moderator help, readiness comprehension, evidence trace, authoring friction, misuse risk, trust, perceived performance, repeat-use case and intent, and most important missing workflow only when the task pilot was run. | -Do not record names, account details, or investment opinions. Do not ask reviewers for a stock recommendation, price target, trade decision, or personal portfolio information. +Do not record names, account details, or investment opinions. Do not record contact details. Do not ask reviewers for a stock recommendation, price target, trade decision, or personal portfolio information. Delete withdrawn rows immediately and delete non-evidence working rows by their recorded retention date. ## Structured Log Template diff --git a/docs/PILOT_REVIEW_INVITATION.md b/docs/PILOT_REVIEW_INVITATION.md index 8817458ce..5bbe1236e 100644 --- a/docs/PILOT_REVIEW_INVITATION.md +++ b/docs/PILOT_REVIEW_INVITATION.md @@ -7,7 +7,7 @@ Thanks for reviewing the Stock Research Command Center. This is a controlled por Please spend under three minutes following this path without operator guidance: ```text -Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History +Research Desk -> Discover -> Company Workbench -> Monitor ``` Use any example ticker offered by the app. As you go, answer in your own words: @@ -15,16 +15,22 @@ Use any example ticker offered by the app. As you go, answer in your own words: 1. Where did you start? 2. What could you use now? 3. What was blocked or excluded? -4. What would you do next? +4. What evidence supports one conclusion? +5. What was difficult to author or update? +6. What would you do next? +7. Would you use this workflow again? +8. What is the most important missing workflow? The product is successful only if you can see what is usable, what stays withheld, and the next safe page without treating the workflow as advice. ## What To Send Back -Reply with the four answers above, plus any exact page, label, route, or viewport that was confusing. A screenshot is useful only when it shows a reproducible product issue. +Reply with the answers above, plus any exact page, label, route, or viewport that was confusing. A screenshot is useful only when it shows a reproducible product issue. Do not send names, account details, investment opinions, price targets, trade decisions, or portfolio information. +Participation is voluntary. You may stop at any time or ask the operator to delete your anonymous row. Before starting, the operator must tell you what will be captured and the deletion date for the working note. + ## Boundaries - This feedback is product-workflow evidence only, not data proof, source proof, or data-freshness proof. diff --git a/docs/PILOT_RUNBOOK.md b/docs/PILOT_RUNBOOK.md index 2a865fc0c..c768c7052 100644 --- a/docs/PILOT_RUNBOOK.md +++ b/docs/PILOT_RUNBOOK.md @@ -114,7 +114,9 @@ make stock-report-md TICKER= Coverage gate: No broad coverage batch should run from setup alone. Provider setup is only an activation boundary: it can activate a source, but readiness changes still require validate, preview, rejected-row review, source provenance, apply/skip decision, rebuilt readiness, and proof ledger evidence. Do not retry exhausted proof queues until new source-backed rows, keyed provider data, reviewed manual rows, or changed blockers exist. -Use 5 to 10 operating companies for the controlled pilot. ETF/index examples such as QQQ and SMH are useful monitor-context demos, but they are not operating-company DCF targets. +Use 5 to 10 operating companies as the fixed research cohort and run 10 to 20 task-based reviewer sessions. ETF/index examples such as QQQ and SMH are useful monitor-context demos, but they are not operating-company DCF targets. + +Measure each session with the privacy-safe feedback template: **Task success**, **Time to first answer** (specifically, the first useful research answer), **Readiness comprehension**, **Evidence trace**, **Authoring friction**, **Misuse risk**, **Trust in evidence**, **Perceived performance**, **Repeat-use case and intent**, and **Most important missing workflow**. Before starting, confirm voluntary participation, withdrawal rights, minimal anonymous capture, and a working-note deletion date. These measures test workflow value and comprehension only; they do not validate market data, model accuracy, demand, or investment outcomes. ## 5. Refresh Data Safely @@ -189,9 +191,9 @@ make imports-validate IMPORT_TICKERS= make imports-preview IMPORT_TICKERS= ``` -For a 25-50 company peer pilot, keep the working source-review sheet outside Git until rows are reviewed. The template is a collection aid only: candidate/context rows stay `candidate_context_only`, and only source-backed relationships that pass `peer-mapping-writeback-guard`, validation, preview, rejected-row review, rebuilt readiness, and proof recording can become trusted peer mappings. +For a 25-50 company peer pilot, keep the working source-review sheet outside Git until rows are reviewed. The template is a collection aid only: candidate/context rows stay `candidate_context_only`, and only source-backed relationships with a reviewed peer role, relationship rationale, economic comparability basis, and explicit valuation-anchor decision that pass `peer-mapping-writeback-guard`, validation, preview, rejected-row review, rebuilt readiness, and proof recording can become trusted peer mappings. Only reviewed `core_peer` and `secondary_peer` rows marked `valuation_anchor_eligible=yes` may enter peer medians; all other roles remain context-only. -Use the extra template fields (`source_type`, `source_title`, `source_accessed_date`, and `source_evidence_note`) to keep reviewer evidence defensible before copy/paste into the import guard. They are review-only fields, not import columns. The guard still accepts only the trusted peer mapping fields needed for `data/imports/peers.csv`; do not bypass it by pasting the full review sheet into the import file. +Use `peer_role`, `relationship_rationale`, `comparability_basis`, and `valuation_anchor_eligible` as import fields so the reviewed decision survives into Company Workbench and valuation readiness. The extra source-document fields (`source_type`, `source_title`, `source_accessed_date`, and `source_evidence_note`) remain review-only. Do not bypass the guard by pasting the full review sheet into the import file. Earnings and analyst estimates: @@ -297,7 +299,7 @@ If `make dashboard-smoke` cannot bind a local socket in a restricted environment Exit the controlled pilot when: -- 5 to 10 selected operating-company packets have outcome states recorded as `supported`, `candidate_context_only`, `still_blocked`, `skipped`, or `excluded`. +- The fixed 5 to 10 operating-company cohort has truthful outcome states and 10 to 20 anonymous task-based reviewer sessions are completed or explicitly `awaiting_external_review`. - Every supported lane has source proof, validation, preview, rejected-row review, rebuilt readiness, regenerated report, and proof-ledger evidence. - Operators can complete the workflow from dashboard, runbook, and CLI commands without guessing the next gate. - Public/release checks pass in the target environment. diff --git a/docs/PRIVATE_BETA_ARCHITECTURE.md b/docs/PRIVATE_BETA_ARCHITECTURE.md new file mode 100644 index 000000000..3d71e69c1 --- /dev/null +++ b/docs/PRIVATE_BETA_ARCHITECTURE.md @@ -0,0 +1,74 @@ +# Private Beta Architecture + +## Boundary + +The repository contains a deterministic readiness contract for a future private beta. It does not implement runtime authentication, private workspaces, hosted persistence, user accounts, entitlements, audit storage, monitoring, health checks, incident response, rollback, owner capacity, or hosting. Real user evidence remains `awaiting_external_review`. + +The contract is available through `src.private_beta_readiness.build_private_beta_readiness()`. It accepts only declared review facts; it does not inspect environment variables, scan files, open accounts, contact a host, or read secret material. + +| Classification | Meaning | Required action | +| --- | --- | --- | +| `local_ready` | Repository-side guidance is present, while account-backed capabilities remain external. | Complete the external setup below. | +| `external_account_required` | An individual capability needs a real hosted account or service. | Do not represent the capability as available. | +| `external_operations_required` | An operating control needs a staffed hosted rehearsal, not only repository guidance. | Assign owners and rehearse the control in the actual environment before claiming it. | +| `manual_verification_required` | An external setup has been declared, but it has not been independently verified. | Verify the live behavior before any access or product claim. | +| `unsafe_secret_blocked` | A tracked or otherwise unsafe secret condition has been declared. | Remove it from tracked files, rotate it outside the repository, then repeat the review. | + +## Data Boundaries + +Private-beta data, when it exists, must be partitioned by authenticated user and workspace. Per-user watchlists, thesis journals, saved scenarios, audit events, and entitlement data belong in the external private-beta service, not in tracked repository files, public demo data, screenshots, generated reports, or logs. + +The future service must enforce workspace authorization server-side for every read and write. Client-side visibility, UI routing, or a workspace identifier alone does not establish isolation. Research evidence and readiness remain source-bound and research-only; private access does not turn incomplete data into a trusted input or an investment recommendation. + +Credentials, tokens, account identifiers, license documents, and secret values remain outside Git. The repository may carry only blank templates and variable names. Usage and error telemetry must exclude credentials and research-workspace contents unless a separately reviewed retention policy authorizes a minimal, documented record. + +## External Setup Sequence + +1. Create the chosen hosting, identity, and managed-persistence accounts outside this repository. +2. Configure an identity provider, an allowed-user invitation path, session expiry, and account recovery; verify them with non-production test accounts. +3. Create private workspaces and server-side authorization rules that scope watchlists, journals, scenarios, and uploaded evidence to the authenticated user and workspace. +4. Configure encrypted platform secret storage. Add only the required variable names, never secret values, to repository templates; rotate any credential exposed outside that store. +5. Configure audit events for sign-in, workspace membership, export, deletion, and entitlement changes. Define a retention and deletion policy before retaining beta-user data. +6. Configure role and entitlement rules, including revocation, and test that unauthorized users cannot read or modify another workspace. +7. Configure privacy-safe usage and error monitoring, alert routing, and health checks for the hosted entrypoint and backing services. +8. Define incident response severity, escalation, reviewer-access shutdown, evidence preservation, communication ownership, and recovery criteria. +9. Keep the previous verified revision available, rehearse rollback in the hosted environment, and record who can execute and verify recovery. +10. Confirm owner capacity for source failures, access incidents, reviewer support, and recovery; a named document without available coverage is not operating proof. +11. Run a supervised external verification with real test accounts. Record the outcome as `awaiting_external_review`, `manual_verification_required`, or another evidence-backed state; do not infer readiness from setup alone. + +## Local Contract + +### Provider-Neutral Authorization Policy + +`src.hosted_access_control.evaluate_workspace_access()` is a local, +provider-neutral, deny-by-default policy contract. It requires exact +authenticated-principal, active-membership, and workspace matches before an +explicit role/resource/action rule can allow a request. Thesis, evidence, +catalyst, and outcome research records remain append-only, and every allow or +deny result carries a privacy-safe audit obligation that a future approved +adapter must record. + +The evaluator performs no authentication, persistence, audit storage, +retention, monitoring, network, provider, dashboard, ledger, readiness, or +generated-artifact operation. This local contract does not prove hosted +authentication, private-workspace isolation in a deployed service, audit +storage, retention execution, monitoring, rollback, incident response, or +operated capacity. All such states remain external until directly verified in +the actual approved environment. + +The classifier reports authentication, workspaces, user data separation, secrets, audit, retention, entitlements, monitoring, and health checks independently from incident response, rollback, and owner capacity. It is intentionally read-only. + +A local runbook does not prove that an incident owner is available, rollback works on the host, or recovery can be staffed. Without a real rehearsal, those controls remain `external_operations_required`; declaring external setup moves them only to `manual_verification_required`. + +```python +from src.private_beta_readiness import build_private_beta_readiness + +readiness = build_private_beta_readiness() +assert readiness.classification == "local_ready" +``` + +Passing `external_setup_declared=True` produces `manual_verification_required`, not a hosted or authenticated claim. Passing `unsafe_secret_detected=True` produces `unsafe_secret_blocked`; pass only the declared condition, never a credential or secret value. + +## Non-Claims + +No current repository state proves that a private-beta host exists, authentication is live, user data is separated, retention works, entitlements are enforced, audit logs are stored, monitoring receives events, health checks run, incident response is staffed, rollback succeeds, or owner capacity is available. Do not present this contract as commercial launch readiness, hosting evidence, or user validation. diff --git a/docs/PRODUCT_DIRECTION_DECISION.md b/docs/PRODUCT_DIRECTION_DECISION.md index eff14b7c2..bddc02d04 100644 --- a/docs/PRODUCT_DIRECTION_DECISION.md +++ b/docs/PRODUCT_DIRECTION_DECISION.md @@ -43,7 +43,7 @@ Evidence expected: | Criterion | Evidence required | Current state | | --- | --- | --- | -| Reviewer demand | Observed task completion, repeat-use case, and return intent from 5-10 controlled reviewers. | `awaiting_external_review` | +| Reviewer demand | Observed task completion, repeat-use case, and return intent from 10-20 controlled reviewers. | `awaiting_external_review` | | Operating burden | Measured deployment, refresh, review, incident, and support work. | Local gates known; hosted burden unknown. | | Data licensing | Written rights and attribution boundaries for every operated provider/source. | Controlled repo license exists; operated data rights are not established. | | Provider reliability | Repeated source availability, deterministic limits, rejection handling, and provenance. | Optional keyed providers remain unconfigured; source-proof queues are exhausted. | @@ -55,9 +55,10 @@ Evidence expected: - Local five-route desktop/mobile performance gate: passed on the fixed demo snapshot. - Public package: pilot-ready with manual gates. -- Hosted URL: `external_account_required`. +- Independent-beta protocol: locally ready for the current four-route Commercial Research Beta workflow, with a complete privacy-safe scorecard; no independent sessions are on record. +- Hosted URL and enforced private access: `external_account_required`. - Controlled reviewer findings: `awaiting_external_review`. -- Trusted-peer expansion: `awaiting_reviewed_source`. +- Trusted-peer expansion and any commercial-use peer rights: `awaiting_reviewed_source`. - Optional earnings and estimates: intentionally locked until trusted rows exist. ## Decision Rule @@ -67,8 +68,8 @@ Do not infer demand from repository activity, screenshots, test counts, or the c Revisit the decision only after: 1. A hosted or otherwise reproducible external delivery path is verified. -2. Five to ten reviewers complete the task-based pilot. +2. Ten to twenty reviewers complete the task-based pilot. 3. Critical trust, comprehension, performance, and methodology findings are closed or intentionally deferred. 4. A 25-50 company trusted-peer pilot has actual reviewed source relationships, or is explicitly rejected as uneconomic. -Until then, the truthful label is **performance-verified controlled research prototype, preparing for external pilot**. +Until then, the truthful label is **local Commercial Research Beta foundation, preparing for controlled external validation**. diff --git a/docs/PROVENANCE_CONTRACT.md b/docs/PROVENANCE_CONTRACT.md index df09067d2..c4a12eba7 100644 --- a/docs/PROVENANCE_CONTRACT.md +++ b/docs/PROVENANCE_CONTRACT.md @@ -29,17 +29,61 @@ Every analysis-ready record or report section should expose, directly or through | Share count | Explicit filing or trusted source fact with date/context. | Per-share DCF math; shares are never inferred from price or market cap. | | DCF | Price, revenue, free cash flow or FCF margin, shares, method assumptions, and company eligibility. | Fair-value scenario math and valuation interpretation. | | Candidate peers | Industry, SIC, product, or other contextual suggestion. | Candidate context must not satisfy trusted-peer readiness. | -| Trusted peers | Source-backed relationship, review rationale, source/as-of date, and required peer inputs. | Peer-relative valuation and comparative conclusions. | +| Trusted peers | Source-backed relationship, review rationale, source/as-of date, reviewer-assigned peer role, economic comparability basis, explicit valuation-anchor decision, and required peer inputs. | Peer-relative valuation and comparative conclusions. | | Earnings and estimates | Trusted provider/import fields, fiscal period, source, and retrieval/as-of context. | Optional readiness or consensus interpretation from date-only or target-only rows. | | Earnings Nowcast | Prior quarterly actuals plus an exact-period point-in-time consensus snapshot, all timestamped no later than the forecast cutoff; model version and input hash are mandatory. | Revenue/EPS range, consensus-relative classification, and every probability output. Candidate signals never satisfy this gate. | +Trusted-peer relationship provenance and valuation-anchor eligibility are independent. A source-backed mapping remains relationship evidence, but peer medians additionally require a reviewed `core_peer` or `secondary_peer` role, explicit relationship rationale, economic comparability basis, and `valuation_anchor_eligible=yes`. Missing fields, other roles, and legacy mappings remain context-only. No role or comparability field may be inferred from sector, industry, peer group, popularity, or existing row count. + ## Earnings Nowcast Point-In-Time Contract Every forecast event must preserve `ticker`, `fiscal_period`, `as_of_timestamp`, expected report date and forecast horizon when known, source publication/retrieval timestamps, direct source references, Revenue/EPS metric definitions, `model_version`, `input_snapshot_hash`, readiness/freshness states, and source IDs. Historical consensus snapshots are append-only evidence. An exact duplicate is not re-added; a revised or currently visible estimate is retained as a separate revision and must not overwrite the snapshot that was knowable at a prior cutoff. Duplicate actual rows cannot inflate quarterly history, and conflicting actuals remain blocked unless an explicit `supersedes_source_ref` resolves the revision chain. +SEC quarterly actuals follow the same append-only cutoff boundary. Q1-Q3 requires a 60-120 day SEC Companyfacts duration fact with a one-to-one source-backed fiscal identity/period-end mapping. Companyfacts EPS is marked `companyfacts_split_basis_unverified` until separate primary evidence proves comparability. Q4 requires one explicit result table and one selected-column period-end date in a SEC-filed primary-source exhibit; filing metadata cannot supply that date, and annual-minus-nine-month derivation is forbidden. Staging may identify source-backed quarter-continuity gaps, but it must not infer the missing period, Revenue, EPS, or fiscal basis. Revenue and EPS are independently ready, and EPS is withheld if its split-adjustment, share, operations, accounting, currency, or scale basis is not comparable within the source-backed history. The unverified sentinel cannot enter Business Trend values/comparisons, cohort EPS usability, target EPS backtest outcomes, prior-year EPS benchmarks, or consensus EPS benchmarks. A Revenue-bearing backtest event can remain Revenue-evaluable, but an EPS-only sentinel target is excluded. Stage output is review-only, always declares `automatic_apply=false`, and is rejected for canonical data/import paths or existing non-generated evidence directories. + +An explicit Q4 table proves the quarter and values, not the EPS split basis by itself. If filed primary text does not contain a parseable split-basis statement, extraction records `primary_split_basis_unverified`. That sentinel and the Companyfacts sentinel are blocked by the same downstream predicate; unknown arbitrary basis strings are not proof. Explicit dated split proof remains a separate usable state. + +## Quarterly Cash-Generation Evidence Contract + +Operating income, cash from operations, and reported capital expenditures use a separate versioned observation contract with ticker, fiscal period, period end, value, currency, scale, accounting basis, duration basis, source, immutable source reference, publication time, retrieval time, and explicit revision lineage. Q4 observations must carry `explicit_filed_quarter`; annual-minus-nine-month derivation, filing-metadata period substitution, and inferred Q4 values remain forbidden. Exact duplicates may collapse, one explicit revision leaf may supersede an older source reference, and unresolved conflicting leaves block only the affected component. + +Derived operating margin, free cash flow, and FCF margin exist only in memory and retain all component source references for Advanced evidence. The implementation adds **no new data file, writer, template, or generated artifact**. Production remains withheld until a reviewed source adapter supplies compatible observations; synthetic observations remain test-only. These derived points cannot write canonical inputs, alter deterministic forecasts, or unlock valuation, consensus, catalysts, outcomes, backtesting, or calibration. + +The one-company adapter acceptance result is also in-memory. It records deterministic identity, source-rights, supported-field, cutoff, revision, component, compatibility, complete-period, and explicit-Q4 decisions without fetching or persisting a payload. Every result preserves `production_activation=false` and `readiness_promotions=()`. `accepted_for_review` proves only that a candidate batch passed the local contract; it does not change the source-rights registry, authorize storage or redistribution, establish a reviewed real-company source, or promote any product readiness state. + +The SEC quarterly pilot adds a read-only source boundary without changing that activation contract. Companyfacts supplies exact accession, three-month context, concept, and magnitude; the matching submissions row supplies the timezone-aware UTC `acceptanceDateTime`; and the primary filing supplies exact inline-XBRL fact IDs plus the filed-table capex outflow presentation. Filing-date midnight and retrieval time are never substituted for publication time. A source reference points to the exact accession document and inline fact. The live NVIDIA Q1 FY2027 and AMD Q1 FY2026 previews are accepted source-review results only: they write no cache, CSV, JSON, report, screenshot, timing, canonical row, or readiness artifact. + +AMD Q1 FY2026 accession `0000002488-26-000076` uses the same bounded two-company portability contract as NVIDIA. Both results retain `production_activation=false`, `readiness_promotions=()`, `persistence=false`, exact component references, and no canonical write. Unsupported tickers fail before fetch and expose no configured filing identity. + +The explicit Company Workbench preview carries the same immutable boundary into one user-flow composition. Every result retains `production_activation=false`, `readiness_promotions=()`, and **no canonical persistence**. A successful preview may render only the accepted in-memory values and exact lineage at the fixed review cutoff; failure withholds the complete preview. It cannot create a canonical observation, update readiness, become a forecast input, establish another company or quarter, or convert source-review acceptance into production evidence. + The target-period actual and every source published after the cutoff are evaluation evidence only and must never enter forecast inputs. Trusted peer/news signals require explicit source, publication time, excerpt hash, review state, and trusted-peer relationship evidence where applicable. They remain directional context and cannot create a numeric adjustment or numerical probability. -The synthetic fixture packet is software-test evidence only. Real output remains `awaiting_point_in_time_consensus`, and numerical probability remains `awaiting_calibration_evidence` until the documented out-of-sample gates pass. +The synthetic fixture packet is software-test evidence only. The five-company SEC staging scope (NVDA, AMD, AVGO, MU, and QCOM) is not a coverage claim. Real output remains `awaiting_point_in_time_consensus`, and numerical probability remains `awaiting_calibration_evidence` until the documented out-of-sample gates pass. + +Prospective consensus snapshots are append-only. Each row preserves a unique snapshot ID, ticker, fiscal period, snapshot and retrieval timestamps, source and durable reference, Revenue/EPS values when present, complete comparability definitions, expected report date, review state, and an explicit superseded snapshot ID for revisions. Current-only provider payloads remain candidate context and cannot be relabeled as historical point-in-time evidence. + +Consensus collection preview preserves two independent decisions. Technical validity covers schema, cutoff, immutable identity, revision lineage, cooldown, and explicit review state. Commercial evidence uses the exact source ID, explicit commercial approval, and the registered scope required by each populated metric: `revenue_consensus` for Revenue and `eps_consensus` for EPS. Unknown, composite, unverified, or scope-incomplete sources fail closed without inferred provider identity. Explicit Commercial Research mode refuses the append before directory or ledger mutation when commercial evidence is incomplete; research mode keeps its reviewed local append contract. A passing guard cannot certify payload correctness, source freshness, comparability, historical depth, calibration, or readiness. + +The prospective ledger is validated as a whole before status, preview, or record. Each ticker/period must form exactly one append-ordered, timestamp-increasing root-to-current-leaf chain with unique snapshot IDs and unique evidence identities. Missing parents, cross-scope revisions, multiple roots, forks, cycles, reversed order, and non-leaf supersession fail closed. Batch preview emits a deterministic receipt bound to the normalized review cutoff, Commercial Research mode, complete proposed input, and complete existing ledger; record recomputes that receipt immediately before mutation and refuses any mismatch. This narrows the reviewed mutation boundary but does not claim multi-process locking or crash-safe transactionality. + +Price provenance separates field presence from temporal integrity. `retrieved_at` is complete only when it has an explicit timezone and the shared daily-price validator proves `observation availability <= retrieved_at <= review cutoff`; a parseable naive value is not UTC evidence. Normalization, staged preview/apply, and DCF lineage use the same blockers. A declared invalid retrieval blocks canonical apply before backup or write, while an entirely missing retrieval remains visibly incomplete and can stay technical research context only. Successful apply uses the exact once-read validated frame and an atomic same-directory replacement. Atomic replacement does not prove source truth, publication timing, reviewer intent, locking, or recovery across backup and replace. + +Consensus source-row validation cannot receive provenance permission as a caller label. It resolves the exact provider against the same registry and returns registry-derived rights plus row-specific required and missing metric scopes only for technically accepted rows. Candidate context and point-in-time reviewability remain visible when commercial evidence is incomplete, but neither state can authorize append, activation, readiness, backtesting, or calibration. Invalid rows never become commercially ready merely because their source record is approved. + +The prospective collector and source-row validator obtain their registry-derived decision from the same immutable commercial field-scope review. The result retains the exact source ID, rights state, ordered required fields, ordered missing fields, and the combined metadata gate; it rejects blank or duplicate requirements and never infers aliases or composite membership. Consumer-specific technical states, blockers, and mutation rules remain separate. This decision cannot prove the payload, timestamps, comparability, reviewer intent, collection, activation, freshness, readiness, backtesting, or calibration. + +`make earnings-consensus-source-review INPUT= PROVIDER= AS_OF=` exposes that source validation as a read-only gate before collection preview. It accepts no default provider or cutoff, preserves original one-based accepted and rejected row positions, refuses blank/duplicate headers and undeclared extra values, and writes neither normalized rows nor a report file. This upstream contract requires explicit `history_scope`; it is distinct from the prospective collection contract's `review_state`. A human-reviewed, evidence-preserving mapping into the existing prospective schema is a separate step with no automatic transformer or writer. The returned rights and scope evidence comes only from the checked-in registry; it cannot prove payload correctness, durable-reference validity, publication availability, collection, activation, readiness, backtesting, or calibration. + +Each source-row validation result also preserves its normalized review cutoff. A row must declare an exact supported history scope and satisfy `snapshot_at <= retrieved_at <= review_cutoff` before commercial evidence is evaluated. Rejected rows retain their original one-based positions, and valid sibling rows are reviewed independently. This local cutoff guards leakage and ordering; it does not manufacture a publication timestamp, source reference, permission, freshness state, or activation decision. + +A multi-row prospective input is one ordered review batch. Preview builds a virtual ledger from saved rows and earlier technically reviewable proposed rows, preserving complete row-level technical and commercial evidence. Record consumes that same batch decision and refuses the entire mutation when any technical blocker exists or, in Commercial Research mode, any rights or populated-metric scope is incomplete. The saved ledger remains unchanged after a deterministic batch rejection. Successful record appends all reviewed rows in supplied order through one handle; it does not rewrite earlier evidence. This is validation consistency, not a guarantee against concurrent writers, process interruption, or filesystem failure. + +Historical valuation observations preserve the numerator timestamp, denominator period end, denominator availability timestamp, metric definition ID, retrieval time, and source reference. A denominator unavailable at the historical price timestamp is rejected. Definition changes create separate history segments instead of a mixed regime. + +Commercial Research historical valuation, supported catalyst events, and reviewed research outcomes add a separate exact-source eligibility boundary. Every used row must have approved rights and literal `valuation_history`, `catalyst_evidence`, or `research_outcomes` scope respectively. Permission is never inferred across sources or lanes. Any failed row withholds the scoped supported/reviewed result while preserving a blocker reason under Advanced. Research-mode technical review remains independent, candidate catalyst rows remain candidate-only, and empty ledgers remain empty. + +Research outcome and catalyst ledgers are append-only reviewed evidence. Outcomes retain the original thesis entry, observation window, review time, source, and learning; they contain no return or skill score. Catalysts retain publication, retrieval, effective time, event type, evidence state, and reviewer. Neither ledger mutates source data, readiness, forecasts, probabilities, DCF assumptions, or recommendations. ## Freshness Rules @@ -69,6 +113,38 @@ The `local` profile is an ignored mutable workspace for refreshed research data. Both profiles remain research-only: no broker execution, order routing, auto-trading, direct buy/sell instructions, or fabricated values. +## Readiness Promotion Evidence Review Contract + +The stdout-only readiness preview treats a false-to-true fundamentals or DCF flag as a proposed technical change, not as proof that its source is correct, commercially permitted, current, or complete. For each proposed promotion it recovers the exact canonical fundamentals source value, as-of date, durable source reference when supplied, exact checked-in commercial-rights decision, and registered support for `revenue`, `free_cash_flow`, `fcf_margin`, and `shares_outstanding`. + +Source identifiers are exact. A composite or unregistered value is not split, normalized to a registered component, or granted inferred rights. Missing or duplicate canonical rows, missing source/as-of/reference fields, unapproved exact sources, and missing registered field support fail closed in the evidence review without changing the technical readiness frame. The fundamentals review does not establish price-source provenance required by DCF. Even a complete review is inspection evidence only; it cannot write an artifact, mark stale readiness current, edit the rights registry, or authorize the separate reviewed make readiness rebuild. + +### DCF Price-Lineage Review + +For each false-to-true DCF promotion, the no-write preview reviews the exact latest canonical price row independently. A usable selection requires a parseable observation `date`, a positive numeric `close`, the greatest date for the ticker, and exactly one row at that date. Missing usable rows and duplicate latest-date rows fail closed rather than selecting by row order. + +The selected row must carry an exact `source`, durable `source_ref`, and parseable `retrieved_at`. The exact source must be commercially approved and its checked-in rights record must explicitly support `prices`. Technical promotion, row selection, lineage completeness, commercial rights, and registered field scope remain separate states. Missing or composite sources are not split, normalized, inferred, or granted borrowed rights. + +The market observation date, `as_of_date`, local file name or modification time, adapter/provider availability, refresh warning text, and numerical row shape are not substitutes for provider identity, retrieval time, or a durable source reference. The audit changes no technical readiness, canonical row, valuation result, source-rights record, freshness state, or reviewer decision. A complete result would still be inspection evidence, not authorization for `make readiness`; an incomplete result does not by itself invalidate the row for local research context. + +The prospective manual price path can preserve explicit `source_ref` and `retrieved_at` values alongside `source`. These values must be reviewer supplied; the normalizer generates no fallback reference or retrieval timestamp. Validation normalizes a valid retrieval timestamp to UTC and reports lineage completeness separately from technical row validity. Preview retains the summary, while apply remains a distinct reviewed mutation with the existing backup and merge boundary. Preserved lineage is necessary but not sufficient: exact-source rights, registered `prices` scope, payload review, freshness, apply intent, and rebuilt readiness remain independently required. + +Validation and preview join every exact retained source ID to the checked-in rights record without splitting composite IDs or inferring aliases. They report commercial eligibility and literal `prices` field support separately for approved, unverified, unknown, blank, and mixed batches. Technically invalid rows do not enter these counts. The result is review evidence only: it cannot edit the registry, prove the payload, authorize apply, or promote readiness. + +Explicit Commercial Research mode enforces lineage completeness, exact-source rights approval, and registered `prices` scope immediately before staged-price mutation. A failed gate returns without backup creation or canonical write and retains the preview evidence for review. Research mode preserves its separate local apply contract. Guard passage does not certify the payload, reviewer decision, source freshness, readiness build, or proof record. + +Direct price refresh has a separate defense-in-depth mutation boundary. Concrete Stooq, Yahoo, FMP, Alpha Vantage, Finnhub, and IBKR implementations expose exact source IDs; automatic-ladder labels must equal the child identity. In explicit Commercial Research mode, every reachable source must independently have approved rights and literal `prices` scope before fetch, and the exact source that returns rows is rechecked before it can enter the merge or status set. A supplied source with missing, changed, composite, unapproved, or scope-incomplete identity is refused without canonical or status mutation. Provider aliases and ladder labels cannot grant or borrow rights. This local guard does not supply row-level `source_ref` or `retrieved_at`, approve a checked-in source, prove provider access or payload quality, or authorize readiness rebuild. + +Focused-cohort saved rows use lane-specific provenance and scope. A technically populated value needs an exact source plus durable source reference, approved commercial rights, and the literal registered field for that lane before Commercial Research coverage becomes usable. Margin aliases are reviewed as their exact populated margin field; FCF aliases require `free_cash_flow`; cash and debt are independent; shares require `shares_outstanding`; filing and earnings dates require `filing_dates` and `earnings_dates`; each populated consensus metric requires `revenue_consensus` or `eps_consensus`; and every trusted relationship row requires `trusted_peers`. One supported field never grants a sibling field. Candidate relationships do not enter the trusted decision. These Advanced coverage decisions do not edit canonical evidence, readiness, source rights, forecasts, valuation, or peer roles. + +Focused-cohort adjusted price history conjuncts saved readiness with canonical row evidence. At least one row must have a valid date and positive adjusted close or close; every retained row must carry exact source, durable reference, and retrieval timestamp, and every exact source must have approved rights plus literal `prices` scope. Mixed histories fail closed without provider inference. This display review does not validate retrieval chronology, rewrite canonical history, or change research-mode readiness. + +Focused-cohort quarterly Revenue/EPS display eligibility conjuncts the existing technical trend state with a metric-specific commercial review of every populated accepted canonical row. Each row needs exact source, durable reference, retrieval timestamp, approved commercial rights, and literal `revenue` or `eps` scope. One unapproved, unknown, or scope-incomplete populated row blocks only that metric. Revenue permission never grants EPS permission. This review does not repair rejected rows, resolve revision conflicts, verify EPS split basis, establish explicit-Q4 evidence, change readiness, or activate Earnings Nowcast. + +Before that metric-specific review, dashboard consumption treats the canonical quarterly-actual CSV as one integrity unit. If any row fails the canonical contract, accepted sibling rows cannot build a Business Trend or focused-cohort quarterly packet. The blocked packet carries the original CSV row number and validation reason for Advanced display only. This boundary does not delete, rewrite, infer, or repair canonical evidence. + +The preview also preserves the method reason for newly excluded DCF transitions. Named reasons are a deterministic explanation of the existing scope decision; absent metadata remains unexplained instead of inferred. Ready, partial, excluded, added, and removed transition counts are independent and may overlap. They are not current readiness totals, evidence of company quality, a ranking, or a recommendation. + ## Research Change Event Contract A research change event requires two comparable generated snapshots from the same selected profile. Each event preserves a deterministic event ID, ticker, event family/subtype, prior/current values, source and source reference, source publication time when available, retrieval and detection time, selected profile, prior/current snapshot identities, evidence status, materiality, and a research-only review task. @@ -77,12 +153,22 @@ Publication time, retrieval time, and detection time are different facts and mus Event review outcomes are append-only evidence. The latest valid outcome controls whether an event remains in the open queue, but historical rows are retained. Review resolution never mutates canonical sources or readiness. Candidate context, provider reachability, or a detected event cannot become trusted evidence without the existing source review, validation, preview, rejected-row, apply, readiness rebuild, and proof gates. +GitHub pull-request hygiene evidence is bound to the event's explicit base and head commit SHAs. Generated-artifact classification and whitespace must inspect that same three-dot commit range after sufficient history is fetched; a clean checkout working tree is not an acceptable substitute. The gate has read-only repository permission and does not establish source rights, data correctness, human review, deployment approval, or merge safety by itself. + ## Research Thesis Journal Contract `data/research_thesis_journal.csv` is the canonical reviewed, append-only journal ledger. Every row identifies schema version, entry ID, selected profile, ticker, thesis ID, entry type, recorded/effective timestamps, reviewer, summary, evidence direction, source, durable source reference, source publication timestamp, confidence, review due date, and `supersedes_entry_id`. Source publication and effective timestamps cannot be later than the recorded timestamp. Evidence, catalyst, risk, and invalidation rows require source provenance. Thesis revisions must remain within the same profile, ticker, and thesis chain and must preserve the prior row. Generated report text, detected changes, queue outcomes, and synthetic fixtures cannot create a reviewed journal entry automatically. +## Research Decision Lab Contract + +`src/research_decision_lab.py` composes immutable selected-profile results; it is not a source, ledger, readiness system, or report generator. It writes no journal, outcome, source, readiness, proof, report, screenshot, or timing artifact. Company Workbench uses the selected report, journal, scorecard, outcome status, and change items. Monitor loads the saved journal and outcome ledgers once, derives each focused ticker independently, and converts an invalid ticker input into one unavailable state without promoting or suppressing another ticker. + +Plan, Evidence, Invalidation, Scenario, Review trigger, and Learning retain separate evidence identities and fail-closed states. A valid lane cannot promote, repair, or clear another lane. Missing journal or outcome rows create `not_started` states; invalid evidence creates `unavailable`; blocked or excluded DCF remains blocked or excluded; commercial outcome-rights failures remain `commercial_evidence_blocked`. Candidate context, generated narrative, technical price history, and synthetic fixtures cannot populate reviewer-authored lanes. + +The deterministic Decision Lab identity binds the selected profile, ticker, lane states, evidence summaries, and next process step. The Monitor table preserves the saved focused-cohort order; identity detail stays under Advanced. These identities prove deterministic local composition only. They do not prove source truth, predictive accuracy, investment performance, reviewer adoption, hosted reliability, demand, or product-market fit. + ## Scenario Lab Contract The Scenario Lab never writes canonical data, readiness, reports, or proof records. It derives an immutable input identity from the selected profile, source-backed valuation input, provenance, and bounded session parameters. Only DCF-ready operating companies with source references, revenue, FCF margin, and shares can display numerical scenario math. If the selected row is blocked or excluded, the product shows the reason and withholds baseline, adjusted, sensitivity, and terminal-value figures. diff --git a/docs/PUBLIC_DEMO_WALKTHROUGH.md b/docs/PUBLIC_DEMO_WALKTHROUGH.md index 4c76919a7..e7f67b64e 100644 --- a/docs/PUBLIC_DEMO_WALKTHROUGH.md +++ b/docs/PUBLIC_DEMO_WALKTHROUGH.md @@ -9,7 +9,7 @@ Use this when sharing the project from GitHub or LinkedIn. The walkthrough is re - Share under the controlled portfolio/demo license; do not describe the repository as open source or reusable software. - Keep the demo research-only: no investment advice, broker action, order routing, auto-trading, or direct buy/sell instructions. - No public hosted Streamlit URL is configured yet; the shareable path is the GitHub project, curated screenshots, and tracked `make demo-dashboard` instructions. Use `docs/HOSTED_DEMO_DEPLOYMENT.md` only when preparing a verified hosted app link. -- Public UX status is `review_limited`: repeated local cold/warm evidence passes, but hosted-route verification and external reviewer workflow evidence remain pending. +- Public UX status is `share_review_ready` for the local GitHub/demo workflow: public checks, browser evidence, and repeated local cold/warm evidence pass. Hosted-route verification and external reviewer workflow evidence remain pending before calling it a hosted or externally validated product. - FMP, Alpha Vantage, and Finnhub are optional provider fallbacks and are not configured by default. Provider setup does not prove coverage until a reviewed source-backed smoke, validation, preview, and readiness rebuild pass. - Coverage is intentionally readiness-gated rather than complete. Fundamentals, share count, peer mapping, earnings, and analyst estimates can remain blocked or locked while the product is still useful. - Stop before claiming a blocked lane is ready unless source proof, validation, preview, apply, rebuilt readiness, and proof history all support it. diff --git a/docs/PUBLIC_RELEASE_CHECKLIST.md b/docs/PUBLIC_RELEASE_CHECKLIST.md index 7266db258..6b430b3db 100644 --- a/docs/PUBLIC_RELEASE_CHECKLIST.md +++ b/docs/PUBLIC_RELEASE_CHECKLIST.md @@ -6,7 +6,7 @@ Use this checklist before sharing the repository on GitHub or LinkedIn. - Keep the top of `README.md` focused on what the project does, why it matters, and how to run it. - Confirm `README.md` starts with `External Reviewer Start Here` so a GitHub or LinkedIn visitor sees the GitHub/local-app boundary, current-readiness source, and stop rules before deeper operator detail. -- Keep the guided product flow near the top: README preview, `docs/PUBLIC_DEMO_WALKTHROUGH.md`, then `make dashboard` and the Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History path. +- Keep the primary product flow near the top: README Workbench preview, `make dashboard`, then Research Desk -> Discover -> Company Workbench -> Monitor. Keep Home -> Stock Selector -> Single-Stock Report -> Data Health -> Proof History as the secondary controlled Public demo. - Keep terminal proof commands secondary: use `make status-check TOP_N=5` and `make stock-report-md TICKER=NVDA` only after the visitor understands the public workflow. - Keep `make stock-report TICKER=NVDA` available for optional local report-data inspection, but prefer `make stock-report-md` for LinkedIn/GitHub visitors. - Keep `docs/OPERATOR_GUIDE.md` linked from the README as the deeper local workflow guide so LinkedIn visitors see a short landing page first and advanced users still have exact commands. @@ -34,10 +34,10 @@ Use this checklist before sharing the repository on GitHub or LinkedIn. - Keep the pilot evidence packet visible: baseline readiness, before report, focused blocker check, lane review path, validate/preview gate, apply boundary, rejected-row check, rebuild proof, and still-blocked evidence row. - Include current readiness numbers only when they are clearly labeled as local snapshots. - Keep generated examples that help visitors understand the product, such as `outputs/stock_reports/qqq.md` and `outputs/stock_reports/nvda.md`. -- Use `docs/assets/linkedin-public-dashboard.png` as the LinkedIn Featured thumbnail unless you intentionally want an operator-mode screenshot; treat image counts as illustrative and use `make status-check TOP_N=5` for current local counts. -- Use GitHub as the LinkedIn link target unless a hosted app has been separately deployed and verified. The current public-share package is GitHub plus curated screenshots plus local run instructions. +- Use `docs/assets/linkedin-public-dashboard.png` as the LinkedIn Featured thumbnail only when it is the reviewed `1200x627` Company Workbench answer-first capture. Do not publish changing coverage or readiness figures in the social image. +- Use the stable GitHub repository as the LinkedIn link target only after the reviewed feature reaches the default branch. Until then, keep the existing stable item or label any non-default review link `Draft engineering preview`. Do not substitute a hosted-app link until it is separately deployed and verified. - Before replacing the GitHub link with a hosted app link, complete `docs/HOSTED_DEMO_DEPLOYMENT.md` and rerun the public gates. -- Confirm the LinkedIn Featured description mentions Python + Streamlit, data readiness before analysis, research-only, no broker integration, no auto-trading, and no investment advice. +- Confirm the LinkedIn Featured description mentions Python + Streamlit, Research Desk -> Discover -> Company Workbench -> Monitor, usable versus withheld evidence, research-only, no broker integration, no auto-trading, and no investment advice. - Confirm LinkedIn copy does not imply complete coverage, provider-key activation, public hosted app availability, data freshness proof, or investment recommendations. - Run `make browser-qa-evidence` before sharing or replacing public screenshots. It shows the current public-share image recommendation, committed asset checks, current real-app capture status, route expectations, and capture boundaries without refreshing data or writing reports. - Run `make linkedin-share-check` for the final LinkedIn Featured-card checklist. It is read-only: it does not open LinkedIn, upload files, edit your profile, refresh data, stage files, commit, or push. diff --git a/docs/SCHEDULER_ACTIVATION_CHECKLIST.md b/docs/SCHEDULER_ACTIVATION_CHECKLIST.md index dfca1b56b..83f340656 100644 --- a/docs/SCHEDULER_ACTIVATION_CHECKLIST.md +++ b/docs/SCHEDULER_ACTIVATION_CHECKLIST.md @@ -2,7 +2,19 @@ Use this checklist before turning any refresh command into a recurring job. -The default scheduler posture is read-only monitoring. Do not schedule unattended imports, applies, commits, pushes, hosted deployment changes, broker access, or recommendation workflows. +The default scheduler posture is read-only monitoring. Automatic application is disabled for every refresh lane. Do not schedule unattended imports, applies, commits, pushes, hosted deployment changes, broker access, or recommendation workflows. + +## Refresh Operations Contract + +Every scheduled review follows the same read-only lifecycle: + +`fetch -> normalize -> validate -> quarantine -> preview -> publish snapshot -> rebuild readiness -> detect changes` + +- A job plan records provider order, batch limit, freshness policy, schema identity, attempt history, state, and failure reason before any provider call. +- A failed provider path is not retried again in the same session. Retry caps apply across prior attempts; if every provider is unavailable, already attempted, or capped, record the blocked result and pivot. +- Missing or changed schema identity, missing provenance, duplicate rows, and stale rows are quarantined. Partial batches are withheld from preview, snapshot publication, readiness rebuild, and change detection. A batch that is both partial and invalid remains explicitly `partial_invalid`; neither condition is flattened away. +- A clean batch can become `ready_for_preview`, but it never publishes or applies automatically. Snapshot publication, readiness rebuild, and any later apply remain separate reviewed actions. +- The plan and status commands are descriptive only. They do not fetch, normalize, validate, quarantine, publish, rebuild, detect changes, or write data. ## 1. Start With Status-Only Monitoring @@ -13,14 +25,16 @@ make project-status-check make provider-setup-checklist make auto-refresh-status SCHEDULE=daily make auto-refresh-runbook SCHEDULE=daily +make refresh-operations-status SCHEDULE=daily +make refresh-operations-runbook SCHEDULE=daily make diff-hygiene-summary ``` Use this mode while provider keys, hosted URL setup, trusted peer rows, or optional context rows are still missing. -## 2. Activation Preconditions +## 2. Manual Review Preconditions -Do not schedule a mutating refresh or apply path until all of these are true: +Do not schedule a mutating refresh or apply path. Automatic application remains disabled even when every gate below is satisfied. These checks define the evidence needed for a separately reviewed, manual action: | Gate | Required evidence | | --- | --- | @@ -36,12 +50,16 @@ Do not schedule a mutating refresh or apply path until all of these are true: Provider setup alone is never activation proof. The compact activation shortcut is: source available, narrow scope, validation, preview, zero rejected rows, provenance, no fabrication, and proof recorded. +The default refresh-operations status and runbook never print stage, apply, or publication commands. They stop at source preflight and read-only scope preview. A mutating command belongs in a separate, human-reviewed handoff and is never part of the scheduler plan while `auto_apply=false`. + +Provider availability and attempt history are session inputs. An unavailable provider fails closed. A failed provider is skipped for the rest of that session, and the bounded retry cap prevents older failures from being retried indefinitely. The CLI accepts `--available-providers`, `--session-id`, `--retry-cap`, and repeated `--provider-attempt provider:session:outcome` arguments for already-reviewed scheduler state. + ## 3. Allowed Recurring Jobs | Schedule | Allowed now | Boundary | | --- | --- | --- | | Daily | Status checks, source preflight, hosted/demo readiness checks, public wording checks, dry-run price plans. | Do not auto-apply imports. | -| Daily after source smoke | Capped price refresh or SEC filing/share-count review for a reviewed scope. | Apply only through `make auto-apply-gate` plus validate/preview/proof gates. | +| Daily after source smoke | Capped price refresh or SEC filing/share-count review for a reviewed scope. | Produce a manual-review handoff only; automatic apply remains disabled. | | Weekly | Peer candidate review and candidate-context reporting. | Candidate peers stay `candidate_context_only` until source-backed relationships are reviewed. | | Optional | Earnings and analyst-estimate source ladder checks. | Date-only or target-price-only rows stay `candidate_context_only`. | @@ -50,7 +68,7 @@ The compact activation shortcut is: source available, narrow scope, validation, - Stop if `make project-status-check` says source-proof queues are exhausted. - Stop if FMP, Alpha Vantage, or Finnhub keys are missing and the selected provider requires a key. - Stop if the hosted URL is not configured or cannot be opened. -- Stop if validation fails, preview widens unexpectedly, rejected rows are present, provenance is missing, or fabricated values are detected. +- Stop if validation fails, preview widens unexpectedly, rejected rows are present, provenance is missing, schema identity changes, duplicate/stale rows are present, a batch is partial, or fabricated values are detected. - Stop if generated CSV/JSON/report churn would be staged by default. When a stop rule triggers, record the outcome as `still_blocked`, `skipped`, `excluded`, or `candidate_context_only`, then pivot to the next executable product or source-review item. @@ -68,4 +86,4 @@ make diff-hygiene-summary git diff --check ``` -Scheduling is operational monitoring. It does not change the research-only boundary, unlock blocked data, create recommendations, or turn the project into a hosted public data platform. +Scheduling is operational monitoring. It does not change the research-only boundary, unlock blocked data, create recommendations, turn the project into a hosted public data platform, or enable automatic data application. diff --git a/docs/TRUSTED_PEER_PILOT_SOURCE_TEMPLATE.csv b/docs/TRUSTED_PEER_PILOT_SOURCE_TEMPLATE.csv index 5dc39b408..5682a157c 100644 --- a/docs/TRUSTED_PEER_PILOT_SOURCE_TEMPLATE.csv +++ b/docs/TRUSTED_PEER_PILOT_SOURCE_TEMPLATE.csv @@ -1,2 +1,2 @@ -ticker,peer_ticker,peer_group,sector,industry,source_type,source_title,source,source_accessed_date,as_of_date,relationship_rationale,source_evidence_note,reviewer,review_date,source_proof_status,import_row_ready,notes -TICKER,PEER,Comparable operating-company peer group,Source-backed sector,Source-backed industry,company filing / company website / investor deck / exchange profile / other trusted source,Source title or document name,https://source.example/path,YYYY-MM-DD,YYYY-MM-DD,Short reason the source supports this peer relationship,Short non-sensitive note describing the exact source evidence reviewed,reviewer-id,YYYY-MM-DD,source_backed / candidate_context_only,false,Template row only; do not import until source proof is reviewed and writeback guard passes; copy only import-schema fields into the guard +ticker,peer_ticker,peer_group,sector,industry,peer_role,comparability_basis,valuation_anchor_eligible,source_type,source_title,source,source_accessed_date,as_of_date,relationship_rationale,source_evidence_note,reviewer,review_date,source_proof_status,import_row_ready,notes +TICKER,PEER,Comparable operating-company peer group,Source-backed sector,Source-backed industry,core_peer / secondary_peer / aspirational_peer / negative_peer / excluded_close_peer / not_clean_comp,Reviewed business-model and economic comparability plus material caveats,yes / no,company filing / company website / investor deck / exchange profile / other trusted source,Source title or document name,https://source.example/path,YYYY-MM-DD,YYYY-MM-DD,Short reason the source supports this peer relationship,Short non-sensitive note describing the exact source evidence reviewed,reviewer-id,YYYY-MM-DD,source_backed / candidate_context_only,false,Template row only; do not import until source proof is reviewed and writeback guard passes; copy only import-schema fields into the guard diff --git a/docs/analysis_capability_audit.md b/docs/analysis_capability_audit.md index 4f612424e..83d222760 100644 --- a/docs/analysis_capability_audit.md +++ b/docs/analysis_capability_audit.md @@ -16,6 +16,7 @@ In plain terms: local or provider-assisted data supplies rows; this product chec - **Readiness gating:** every ticker is checked before deeper analysis appears. - **Price and momentum review:** moving averages, returns, relative strength, volume context, and volatility/ATR proxy are calculated from local price rows. +- **Quant interpretation boundary:** the shared overlay at `195ea18da9d1d6e06c36f8320509ccde46cdaa57` keeps valuation, indicator, and review/risk calculation availability separate from interpretation eligibility. Without structured external provenance and exact-source rights proof, local results remain historical/review-only or are withheld. - **Fundamentals readiness:** company fundamentals are validated for the fields needed by DCF and quality checks. - **DCF workflow:** DCF uses explicit assumptions, conservative caps, scenario outputs, and sensitivity tables when required inputs are available. - **Peer workflow:** peer comparison is withheld until source-backed peer mappings and peer metrics exist. @@ -28,8 +29,8 @@ In plain terms: local or provider-assisted data supplies rows; this product chec | Function area | Quality verdict | Best use today | Needs trusted data | What it refuses to do | Main implementation | | --- | --- | --- | --- | --- | --- | | Readiness gates | Strong today. | Decide whether a ticker can support deeper review. | Local ticker, price, fundamentals, peer, earnings, and estimate readiness rows. | It does not turn missing data into a weak conclusion. | `src/readiness_engine.py` | -| Price and momentum | Good when local price history is ready. | Review setup, trend, liquidity, and market context. | Daily OHLCV rows with enough history for returns, averages, liquidity, and volatility context. | It does not invent missing price history or fill broad-universe gaps silently. | `src/indicators.py`, `src/momentum_engine.py` | -| Fundamentals and DCF | Good for DCF-ready companies only. | Review assumptions, scenarios, and sensitivity with trusted local inputs. | Trusted fundamentals with revenue, free cash flow or FCF margin, shares outstanding, price, cash, and debt where available. | It does not label not-ready companies undervalued or overvalued. | `src/value_engine.py`, `src/valuation.py` | +| Price and momentum | Good for local historical/review context when calculation inputs exist. | Review setup, trend, liquidity, and market context subject to the shared interpretation gate. | Daily OHLCV rows with enough history for returns, averages, liquidity, and volatility context, plus structured provenance and exact-source rights proof for any current-context claim. | It does not invent missing price history, treat calculation availability as a current-market claim, or fill broad-universe gaps silently. | `src/indicators.py`, `src/momentum_engine.py`, `src/quant_interpretation_eligibility.py` | +| Fundamentals and DCF | Good for DCF-ready companies only, with interpretation gated separately. | Review assumptions, scenarios, and sensitivity with trusted local inputs; absent required proof, preserve historical/review-only context or withhold. | Trusted fundamentals with revenue, free cash flow or FCF margin, shares outstanding, price, cash, and debt where available, plus structured provenance and exact-source rights proof for current-context interpretation. | It does not label not-ready companies undervalued or overvalued, or present unproven local inputs as current-market or commercially eligible. | `src/value_engine.py`, `src/valuation.py`, `src/quant_interpretation_eligibility.py` | | Peer comparison | Ready when peer data exists. | Use as a peer missing-data list until source-backed peers and peer metrics are ready. | Source-backed peer mappings plus peer price/fundamentals rows. | It does not treat sector or industry fallback as trusted peer valuation. | `src/readiness_engine.py`, `src/valuation.py` | | ETF/index monitor context | Good for monitor context only. | Review market, theme, liquidity, and risk context without operating-company DCF. | Price, liquidity, correlation, and theme context. | It does not run operating-company DCF for ETFs, index proxies, or funds. | `src/research_decisions.py`, `src/stock_report.py` | | Single-stock report | Clearest visitor-facing review. | See one ticker's At A Glance mode, Evaluation Snapshot, Proof Checklist, Best Review Path, ready, blocked, excluded, optional, methodology, read-only proof steps, and source readiness states step by step. | Current local readiness, price, decision, DCF, peer, and optional-context outputs. | It does not run imports or refreshes, connect to external accounts, provide allocation instructions, or issue recommendations. | `src/stock_report.py`, `src/dashboard.py` | @@ -103,6 +104,6 @@ Any external review or research input still has to be translated into determinis ## Supported-Today Assessment -The current functions are strong enough for a transparent local research prototype, single-stock review, market/ETF monitoring, and DCF-ready company analysis when trusted data exists. They are not yet a full-market data platform because fundamentals, peer data, earnings, and analyst estimates are intentionally sparse until trusted rows are imported. +The current functions are strong enough for a transparent local research prototype, single-stock review, market/ETF monitoring, and DCF-ready company analysis when trusted data exists. The calculation layer is working, but this is not real quantitative investing capability and is not yet a full-market data platform: checked-in observations do not establish current-market recency, and fundamentals, peer data, earnings, analyst estimates, point-in-time universe evidence, consensus, and calibration remain intentionally source-gated. -The next quality improvement is not more indicators or another broad source loop. It is a measured performance release candidate on the fixed demo profile, followed by controlled external review. Deeper trusted peer and provider coverage remains source-gated and should resume only when reviewed evidence exists. +The performance release candidate is complete locally, observation-recency separation is implemented, and the bounded observation-recency UX repair is complete locally. Each primary Research route now shows one route-relevant human interpretation while exact selected/profile/SPY/QQQ evidence stays in responsive Advanced cards. Historical-valuation loading now rejects blank or malformed numeric evidence per row without turning it into zero or losing valid sibling rows. The shared provenance and recency eligibility overlay is implemented locally at `195ea18da9d1d6e06c36f8320509ccde46cdaa57`; it preserves calculation and readiness states, but absent structured external provenance or exact-source rights proof keeps local results historical/review-only or withheld. The semantic-main bridge and bounded automated media-preference checks are completed local engineering evidence. Remaining accessibility maturity requires direct human and assistive-technology evidence, including zoom/reflow, forced colors, reduced motion, keyboard, and screen-reader review. External maturity still requires one permitted independently reviewed point-in-time universe/benchmark package, one consensus snapshot, one reviewed peer relationship, and 10-20 independent target-persona sessions. Do not add more indicators or restart broad provider loops merely to increase feature count. diff --git a/docs/assets/linkedin-public-dashboard.png b/docs/assets/linkedin-public-dashboard.png index 6d251a816..bd74333fb 100644 Binary files a/docs/assets/linkedin-public-dashboard.png and b/docs/assets/linkedin-public-dashboard.png differ diff --git a/docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md b/docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md new file mode 100644 index 000000000..5d32c9ba9 --- /dev/null +++ b/docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md @@ -0,0 +1,552 @@ +# Commercial Research Beta Continuation Goal Prompt + +Use this prompt to continue the Stock Research Command Center in a new Codex task. It is intentionally repo-truth-first: commit hashes, counts, readiness states, and external dependencies must be reverified rather than copied forward as facts. + +```text +/goal + +Continue the Stock Research Command Center in: + +/Users/yjian070/Documents/New project/.worktrees/personal-research-mode-mvp + +Objective: + +Advance the Stock Research Command Center from its current local Commercial Research Beta release-candidate stage through every applicable local, source, hosted-preview, beta-validation, evidence-depth, calibration, and operating-maturity gate. Continue one coherent verified slice at a time while any safe executable in-scope work remains. Do not redefine completion around the work that is already easy or local. + +Start from current repository truth, not chat memory. +Verify authoritative remote commit `b69badfc80424d3a97fae5f77706aa6ed1533167` or a later descendant before relying on this implementation evidence. At that exact head the branch and draft PR were synchronized and GitHub Actions run `30726301045` passed; later descendants require their own direct verification. +When external evidence is unavailable, record its exact unblock condition once and continue to the next safe executable lane. +Never claim overall completion without direct current evidence for every applicable exit gate. +Do not run readiness rebuilds or generated-artifact commands without explicit approval. + +Persistence contract: + +1. Start every continuation from live repository, roadmap, generated-artifact, test, runtime, remote-branch, and PR truth. Chat memory and the expected-state notes below are navigation aids only. +2. Continue automatically while any safe, meaningful, in-scope local task remains. Do not ask for approval for ordinary reversible local implementation, testing, documentation, or draft-PR updates already authorized by this goal. +3. Do not mark the goal blocked because one lane, provider, dataset, hosted account, reviewer, source, or calibration cohort is unavailable when another executable workstream exists. +4. When an external dependency is unavailable, classify it once, record the exact unblock condition and last evidence, avoid identical retry loops, and move immediately to the next executable local task. +5. Recheck an external dependency only when a credential, supplied dataset, account, URL, reviewer cohort, provider entitlement, source-rights decision, or other relevant external state has verifiably changed. +6. Do not repeat exhausted provider probes, broad-coverage refreshes, speculative peer sourcing, or identical source-limit loops. +7. Work one coherent independently tested slice at a time. Finish verification, exact staging, commit, push, roadmap/docs updates, and draft-PR updates before beginning the next slice. +8. Preserve explicit approval requirements for merging, public deployment, external account changes, credential use, destructive actions, purchases, public communication, or material scope expansion. +9. Never fabricate or infer data, forecasts, probabilities, evidence, events, peers, roles, comparability, outcomes, timestamps, sources, rights, reviewer results, hosted properties, recommendations, or completion evidence. +10. Do not mark the objective complete until the requirement-by-requirement completion audit directly proves every applicable exit gate. Passing local tests is not proof of source access, hosting, external beta validation, evidence depth, calibration, or operating maturity. +11. If every remaining task genuinely requires unavailable external input or new authority, leave completion unclaimed, produce the exact dependency ledger and resume checklist, and follow the active goal system's strict blocked audit. “Non-blocking” means pivoting to executable work, not retrying an unavailable dependency forever or pretending it is complete. +12. Keep the goal active whenever any applicable gate remains incomplete or unproven. A single unavailable dependency is not a reason to stop while another safe executable local slice remains; completion requires direct evidence for all applicable gates, not persistence theater or repeated external retries. + +Expected lineage to verify, never assume: + +- Branch: `codex/personal-research-mode-mvp`. +- Draft PR: https://github.com/YuzeJ21/Stock-Analysis/pull/113. +- Performance sampling reconciliation implementation anchor: `6328c8cead7c27cb901e7878cd6d7d23fa11bb0e`. Warm shell/first-useful p90 and cold shell/first-useful maximum separately enforce the unchanged one-second and three-second limits. A controlled local Chrome run recorded 48 successful samples with zero failures; the aggregate release check, 4,474-test full suite, six-route/two-viewport accessibility browser gate, state harness, push, and exact-head GitHub Actions run `30634355602` passed. PR #113 remained open, draft, and mergeable. This is local engineering evidence only; later descendants must reverify it, and category-specific failures must not be hidden by unchanged retry loops. +- Answer-first workflow design anchor: `0dd9a56d3` or a later verified descendant. Discover truth-separation implementation anchors: `ea92d2c6e`, `df6e72b11`, `c084cc274`, and accessibility contract `38e0cef0f`, or later verified descendants. These commits separate strict screen eligibility from readiness-only alphabetical saved-company browsing without reading legacy ranking outputs. They remain local implementation evidence until current-head full, browser, release, push, draft-PR, and exact-head CI gates pass. +- Roadmap truth-reconciliation contract: `ROADMAP.md` is the concise current decision index with `Now`, `Next`, `Externally blocked`, `Later`, and `Completed with evidence`; detailed remediation history stays in its named evidence documents. Reverify the line-budget contract, focused documentation tests, release gates, commit, push, draft-PR update, and exact-head CI before calling the reconciliation slice complete. +- Priority 4 freeze synchronization anchor: remote commit `69c49968e77bfd55fa259695089e1f34ac2fddfb`; exact-head GitHub Actions run `30185232040` passed. Reverify both instead of treating the local read-only index as authoritative remote state. +- Evidence-quality lineage anchor: commit `781ba2481` or a later verified descendant. +- SEC quarterly cash-generation pilot anchor: commit `a262eda9f` or a later verified descendant. +- Explicit Company Workbench cash-generation preview anchor: commit `973bb32aa` or a later verified descendant. +- Bounded two-company cash-preview design anchor: commit `11ac530ae` or a later verified descendant. +- No-write readiness-preview lineage anchor: commit `1f72a6d90` or a later verified descendant. +- Stale readiness continuation gate lineage anchor: commit `54f3977d7` or a later verified descendant. +- Readiness promotion evidence lineage anchor: commit `5e6e01d1a` or a later verified descendant. +- Readiness change-cause lineage anchor: commit `8fd61e201` or a later verified descendant. +- DCF price-lineage review design/plan anchor: commit `cc85cb5f1` or a later verified descendant. +- Prospective price-lineage preservation design/plan anchor: commit `85495b088` or a later verified descendant. +- Staged price rights/scope review design/plan anchor: commit `b1970004d` or a later verified descendant. +- Commercial price-apply guard design/plan anchor: commit `aa12d0e5f` or a later verified descendant. +- Commercial direct-price refresh enforcement design/plan anchor: commit `5997d84bb` or a later verified descendant. +- Cohort saved-row field-scope enforcement design/plan anchor: commit `474445d17` or a later verified descendant. +- Cohort adjusted-price history scope design/plan anchor: commit `e466d083f` or a later verified descendant. +- Cohort quarterly Revenue/EPS field-scope design/plan anchor: commit `20f973f62` or a later verified descendant. +- EPS split-basis downstream enforcement design/plan anchor: commit `dc7ed7c18` or a later verified descendant. +- Evidence-integrity hardening anchor: commit `7d463bae7` or a later verified descendant. +- Prospective consensus source-rights guard design/plan anchor: commit `9b15fafac` or a later verified descendant. +- Prospective consensus batch-preflight design/plan anchor: commit `b2d5f85fe` or a later verified descendant. +- Consensus source-validation rights design/plan anchor: commit `d03cd572f` or a later verified descendant. +- Consensus source temporal-integrity design/plan anchor: commit `e48cab7e6` or a later verified descendant. +- Shared consensus commercial field-scope design/plan anchor: commit `b7013f161` or a later verified descendant. +- Consensus source-review command implementation anchor: commit `c7e81d667` or a later verified descendant. +- PR engineering gate hosted-verification anchor: commit `1fdcaaa78` or a later verified descendant; reverify the current revision's check instead of copying the earlier result forward. +- Company Workbench HTML Research Brief implementation anchors: `b10ebcfbd`, `923fa850e`, `9ce1a6ce4`, `84562a7f6`, and `8218af401` (all later descendants of `6ad7f34310652f1b172525a0b8f00becf874c44c`). On 2026-08-01 the historical pre-fix local matrix for this anchor passed: 630 focused tests; 5,146 full tests in 448.74s; dashboard and six-route render smoke; 101 HTML-browser tests; six-route/two-viewport automated accessibility; public wording/public-check; and six withheld modal-passive reproductions. This is historical automated engineering evidence only. Exact-head repair evidence: commit `b69badfc80424d3a97fae5f77706aa6ed1533167` passed the 5,828-test full suite, the required dashboard, render, HTML, accessibility, public, and hygiene gates, branch/PR synchronization, and exact-head GitHub Actions run `30726301045`. The brief remains a local read-only presentation of existing saved evidence and prepared Python scenario math, not a readiness activation, refresh, new source, new calculation engine, human-accessibility result, hosted operation, or market-validation claim. +- Public UX post-fix outcome-contract implementation anchor: commit `74eb4055e` or a later verified descendant. +- Prospective field-proof Stage A implementation anchor: commit `2c7398402` or a later verified descendant. +- Company Workbench answer-first handoff design anchor: commit `1a144c9a1` or a later verified descendant; reverify the implementation, responsive measurements, and current exact-head CI rather than treating the design commit as implementation evidence. +- Company Workbench answer-first handoff implementation anchor: commit `ca4772ffb` or a later verified descendant; reverify the current revision's responsive behavior and exact-head CI rather than copying this implementation result forward. +- Public packaging reconciliation design anchor: commit `5dead2a72` or a later verified descendant; reverify the README hierarchy, LinkedIn link boundary, curated Workbench image, answer-level performance marker, and current exact-head CI rather than treating the design as implementation evidence. +- Public packaging reconciliation plan anchor: commit `e9dc19f41` or a later verified descendant; verify the implementation and current branch state instead of assuming the plan was completed. +- Research Decision Lab approved-design anchor: commit `54e06e1c3`; composition anchor: `1cfed7490`; Company Workbench anchor: `a4786bb25`; Monitor anchor: `c7ad977b3`, or later verified descendants. Read `docs/superpowers/specs/2026-07-22-research-decision-lab-design.md`, reverify the current implementation and exact-head CI, and do not treat the design commit alone as product evidence. +- Point-in-time universe production-validator lineage anchor: commit `1361472bce6d23cc537ef222c3735bb640c9838a` or a later verified descendant. Task 8 documentation-evidence reliance requires commit `1ece7a3e4adc70450d6318c06571cc8bf54368b0` or a later descendant. Reverify the local implementation and every applicable current gate; neither anchor is permitted-real-data, provider-rights, independent-review, hosted, or commercial-validation evidence. +- Do not assume branch cleanliness, push state, or upstream alignment. The tracked PR readiness snapshot remains the June 7 snapshot and is stale under the declared-date policy; an excluded July 21 local generated working-data snapshot may exist but is not committed PR evidence. Reverify with Git and `make readiness-preview TOP_N=20`; the verified local run reported zero stable readiness changes, which does not authorize staging or a readiness rebuild. +- PR #113 must remain open and draft. Do not merge it. +- Generated CSV, JSON, readiness reports, stock reports, sample reports, screenshots, browser timing output, and other generated churn must remain excluded unless one exact artifact is intentionally reviewed and explicitly required. + +Current locally implemented capabilities to verify: + +- Personal Research Mode with Research Desk -> Discover -> Company Workbench -> Monitor. Research Desk presents one read-only **Today's Research Brief** with a deduplicated saved-attention count, one traceable reason, one freshness warning, one Monitor-or-Discover action, and an explicit not-market-complete boundary; weekly, cohort, observation, coverage, and source-change detail remains under Advanced Evidence. Discover presents the unchanged strict Daily Momentum & Valuation contract as **Screen eligibility — when supported**, then separately provides alphabetical **Browse saved companies** evidence access from focused-cohort ticker readiness. Saved rows answer why they are inspectable, what evidence is usable, and the main evidence gap; browsing is not screen eligibility, opportunity ranking, expected return, or recommendation. The research path never reads legacy decisions, rankings, or watchlist outputs and writes no artifacts. +- Company Workbench authoritative-task composition: the change answer explicitly distinguishes `none`, `snapshot_only`, and `source_backed`, so an empty queue receives a neutral no-queued-change label rather than a snapshot badge. One overall task gives priority only to a change with separate explicit source-backed eligibility. Open items keep their suggested task; still-blocked and intentionally deferred items preserve their existing wait/monitor routing and wait condition. Snapshot-only context is labeled separately and cannot outrank the ordered Research Conclusion priority. Forward View guidance is lane-specific, not a competing overall task; readiness and evidence states remain independent and unchanged. +- Answer-first hierarchy on all four Personal Research routes at desktop and phone widths. +- Company Workbench anchored answer handoff: a Workbench-only compact header retains the semantic page identity, ticker/profile scope, and research-only boundary while omitting duplicated header freshness/action metadata already carried by the profile strip and selected answer. The same fail-closed fast and final summary renders through one optional target before Review path and lane coverage, exactly once, with a ticker-preserving Data Health handoff. The selected answer has an explicit Personal Research desktop/phone style contract instead of depending on Public-mode-only CSS. Local AVGO measurement at `1280x720` verified a four-column answer; at `390x844`, the answer began near `409px`, Data Health ended near `669px`, the stop condition near `705px`, Review path near `746px`, lane coverage near `806px`, document width equaled viewport width, and one answer was present. These measurements are committed local implementation evidence on anchor `ca4772ffb`; they must still be reverified on later revisions and prove no readiness, source, hosted, accessibility, reviewer, demand, or market gate. +- Company Workbench peer-readiness projection keeps peer trend and peer valuation independent across the fast answer, settled answer, next-step routing, reader guide, quick read, methodology, and source audit. `peer_ready` or `peer_trend_comparison_ready` may support mapped-price trend context, but only `peer_valuation_comparison_ready` can unlock peer-relative valuation, premium/discount, or peer DCF comparison; an absent explicit valuation field falls back only for legacy loaded-report payload compatibility. The local AVGO workflow reproduced and closed the former fast-versus-settled contradiction without changing source rows, readiness artifacts, report calculations, or generated files. +- Mobile first-action density that keeps all five profile facts visible in two phone rows, removes only duplicated route-card freshness on phone, exposes route tasks sooner, and preserves the complete Company Workbench review path in a collapsed disclosure without changing readiness. +- SEC quarterly actual lineage with explicit Q4-table and EPS split-basis boundaries. +- Filed-Q4 EPS uses `primary_split_basis_unverified` when primary split language is absent or malformed; an explicit Q4 table never supplies split proof by itself. Revenue remains independent and arbitrary nonempty basis labels do not unlock EPS. +- Earnings Nowcast readiness and five-company cohort board. +- Prospective append-only point-in-time consensus collection contracts. +- Prospective-only per-field proof operations: `make prospective-field-proof-status`, `make prospective-field-proof-preview INPUT= AS_OF=`, and explicit `make prospective-field-proof-record INPUT= AS_OF= PREVIEW_RECEIPT= CONFIRM_REVIEWED=1`. An absent ledger is a valid empty state; legacy narrative proof is not upgraded, and no sample rows are product evidence. Preview reports `technical_write_eligible` and `commercial_evidence_eligible` independently, and its preview receipt binds ledger, input, cutoff, commercial mode, and source-rights registry. The ledger does not activate readiness, does not update canonical data, does not update proof-readiness reconciliation, and does not activate Company Workbench. Any mapping requires a separate design. +- Historical Valuation Regime, Research Outcome Review, and Catalyst Evidence Timeline. Historical-valuation numeric loading rejects blank or malformed numerator/denominator evidence per row rather than coercing it to zero or discarding valid sibling rows. +- Forward View, Scenario Lab, Source Freshness Timeline, Research Comparison, Peer Read-Through Map, and Decision-Process Scorecard. +- Local Decision Lab implementation is complete when current repository evidence reconfirms the immutable six-lane contract, Workbench placement, Monitor stable-order Research Discipline Review, per-ticker fail-closed isolation, responsive runtime behavior, release gates, exact staging, push, draft-PR update, and exact-head CI. It adds no route, ledger, readiness state, transaction field, position sizing, stop/profit rule, recommendation, or broker action. Do not reimplement it unless a current regression is directly reproduced. +- The local Decision Lab does not prove source coverage, predictive accuracy, investment performance, independent adoption, hosted reliability, commercial demand, competitive superiority, or product-market fit. +- Fail-closed provenance, source-rights, freshness, candidate-context, and synthetic-fixture controls. +- Local market-observation recency across Research Desk, Discover, Company Workbench, and Monitor: one read-only local `prices.csv` evaluation per dashboard run, using the dashboard UTC review date. The evaluator independently fails closed for unreadable, invalid, future, or missing observations and applies an exact seven-calendar-day policy. It is not an exchange-session SLA, does not make saved readiness current, and does not prove permitted market-data source rights or hosted freshness; those remain external gates. +- The observation-recency UX repair is complete locally. Research Desk, Discover, and Monitor show one profile-lane `Current`, `Stale`, or `Unavailable` interpretation; Company Workbench shows one selected-ticker interpretation. Exact selected ticker, profile lane, SPY, and QQQ scopes, dates, machine states, messages, policy, source path, and excluded-date counts remain in responsive Advanced cards. A direct browser matrix covers all four routes at `1280x720` and `390x844`, requires one primary summary, four Advanced cards, phone single-column placement, and no evidence-container horizontal overflow, and writes no screenshot, report, timing, readiness, canonical-data, or ledger artifact. This remains local UI/runtime evidence only and proves no source rights, hosted freshness, external accessibility, reviewer understanding, current-market suitability, or recommendation. +- Shared quant interpretation eligibility is implemented locally at `195ea18da9d1d6e06c36f8320509ccde46cdaa57`. One pure overlay keeps valuation, indicator, and review/risk calculation and readiness states independent from the decision to show current context, historical/review-only context, or withholding. Structured external provenance and exact-source rights proof remain absent for current local quant inputs, so results remain historical/review-only or are withheld where proof is absent. This does not prove current-market suitability, hosted behavior, commercial eligibility, source rights, nowcast activation, or calibration. +- Shared pilot and reviewed-batch freshness that treats declared source dates newer than the saved readiness build as stale even when file mtimes look current; the read-only gate never runs `make readiness` or writes generated artifacts. +- No-write readiness impact preview via `make readiness-preview TOP_N=20`; it runs production readiness logic in memory, compares stable saved-versus-proposed states, writes nothing, and does not make saved readiness current. +- Fail-closed readiness promotion evidence review inside the same no-write command: proposed fundamentals/DCF promotions retain independent exact-source rights, source/as-of/durable-reference provenance, registered field-scope, and DCF price-provenance states. Composite or unregistered source values are not split or granted inferred rights. +- Semantic readiness change-cause review inside the same no-write command: added/removed and ready/partial/excluded transitions remain separate, while newly excluded DCF rows retain named behavior-equivalent company-scope reasons. Transition counts can overlap and exclusions are method-fit states, not negative company signals. +- Independent DCF price-lineage review inside the same no-write command: each proposed DCF promotion requires one unambiguous latest valid positive-close row, then keeps row-level `source`/`source_ref`/`retrieved_at`, exact-source rights, and registered `prices` scope separate. File origin, observation date, adapter availability, refresh history, and row shape cannot supply missing provider provenance. +- Prospective price-lineage preservation through manual normalization, validation, preview, and later reviewed apply: explicit `source_ref` and `retrieved_at` survive the staged path, while missing/invalid lineage remains independently review-required and is never generated from local file or processing metadata. +- Independent staged-price source review in validation and preview: each exact retained `source` value keeps commercial-rights and registered `prices` scope states separate from technical validity and lineage; unknown, blank, unverified, scope-incomplete, and mixed batches fail closed without aliases or provider inference. +- Explicit Commercial Research mode staged-price apply guard: incomplete lineage, unapproved exact-source rights, or missing registered `prices` scope returns before backup or canonical mutation; ordinary research mode retains the existing separately reviewed local path. +- Explicit Commercial Research mode direct-price refresh guard: every concrete provider has an exact source ID, automatic ladders admit only independently approved and `prices`-scoped legs before provider execution, and the selected exact source is rechecked before merge or status mutation. Missing, changed, composite, unapproved, or scope-incomplete identity fails closed; research mode retains the existing ladder. +- Field-specific focused-cohort saved-row review: margins, free cash flow, cash, debt, shares, filing dates, earnings dates, populated Revenue/EPS consensus, and every trusted-peer row keep technical availability, provenance, exact-source rights, and literal registered scope independent. Cash/debt and Revenue/EPS consensus cannot borrow sibling permission; candidate peers remain context-only and blockers stay under Advanced. +- Focused-cohort adjusted-price review: saved price readiness is necessary but not sufficient in Commercial Research mode; every retained canonical history row needs technical date/positive-close validity, exact `source`/`source_ref`/`retrieved_at` provenance, approved exact-source rights, and registered `prices` scope. Mixed or unlined history fails closed. +- Focused-cohort quarterly Revenue/EPS review: technical trend state is conjunctive with a metric-specific review of every populated accepted canonical row. Exact source/reference/retrieval provenance, approved rights, and literal `revenue` or `eps` scope remain independent; one mixed or unapproved row blocks only its metric, SEC Companyfacts Revenue scope cannot unlock EPS, and Research mode retains packet behavior. +- Central EPS split-basis sentinel enforcement: `companyfacts_split_basis_unverified` EPS observations cannot enter Business Trend values/comparisons, commercial cohort EPS usability, target or prior-year backtest outcomes, or consensus EPS benchmark math. Mixed verified/unverified trend history remains partial with named withheld periods, EPS-only sentinel targets are excluded, and Revenue remains independent. A direct filed-Q4 integration proves contiguous Revenue can be ready while Companyfacts Q1-Q3 EPS keeps EPS withheld. +- Evidence-integrity hardening: the final broad branch review found and the branch fixed the valuation and backtest gaps. Historical valuation observations reject non-finite valuation inputs, require a canonical real `YYYY-MM-DD` denominator period end, reject blank, malformed, and non-calendar denominator period ends, and reject post-cutoff retrieval evidence. Nowcast targets and prior-year benchmarks canonicalize Revenue/EPS independently through explicit `supersedes_source_ref` lineage, retain one event per ticker/period, and withhold ambiguous leaves per metric so one metric does not suppress the other; they use cutoff-bounded prior-year benchmarks so post-cutoff revisions cannot leak. It does not make stale readiness current or complete consensus, calibration, hosted, reviewer, operating, demand, or product-market-fit gates. The exact next external lane remains permitted point-in-time consensus review/collection preview. +- GitHub Actions runtime maintenance: the pull-request engineering gate pins `actions/checkout@v6` and `actions/setup-python@v6`; its contract rejects the retired Node 20 pins while preserving pull-request-only, read-only, exact-head, full-history, test, render, wording, PR-range hygiene, and whitespace behavior. Require a fresh exact-head hosted pass after this change; action-version maintenance proves no product, data, source, hosted-product, reviewer, calibration, demand, or market gate. +- Prospective consensus source-rights guard: preview keeps append-only technical validity, exact-source rights, and required `revenue_consensus` / `eps_consensus` scope independent; explicit Commercial Research mode blocks before ledger or directory mutation when rights or populated-metric scope is incomplete, while ordinary research mode retains explicit reviewed append compatibility. +- Prospective consensus ledger integrity: status, preview, and record validate every saved row and require one unique append-ordered, timestamp-increasing root-to-current-leaf chain per ticker/period. Preview receipts bind the exact normalized cutoff, mode, proposed input, and saved ledger; record requires and recomputes the receipt before mutation. This is deterministic pre-write integrity, not concurrent locking or crash-safe transactionality. +- Price temporal/apply integrity: normalization, staged validation/preview/apply, and DCF lineage share one explicit-timezone daily-price temporal decision. Declared retrieval must fall between next-day UTC availability and the review cutoff. Apply carries one validated staged frame through a flushed same-directory atomic replacement instead of re-reading the import; this is not source publication proof, concurrent locking, or a database transaction. +- Prospective consensus whole-batch preflight: preview and record simulate rows in supplied order against saved plus earlier technically reviewable evidence, reject empty or technically invalid batches before mutation, apply the commercial conjunction only in explicit Commercial Research mode, and append a valid batch through one handle. Deterministic later rejections cannot leave partial proposed rows. +- Registry-derived consensus source validation: caller labels cannot grant rights; technically accepted rows retain independent candidate or historical-reviewable state while exact-source commercial rights and each populated Revenue/EPS scope come only from the checked-in registry. Invalid rows do not enter commercial-ready counts, and composite provider IDs remain unknown. +- Consensus source temporal integrity: every validation call supplies a UTC review cutoff, every row declares exactly `current_only` or `point_in_time`, and technical acceptance requires `snapshot_at <= retrieved_at <= review_cutoff`. Candidate context receives no cutoff exemption, and rejected rows never enter commercial review. +- Shared consensus commercial field-scope review: prospective collection and upstream source validation use one immutable exact-source rights/scope decision with ordered required and missing metric fields; blank or duplicate requirements fail locally, composite IDs remain unknown, and consumer technical/write states stay independent. +- Read-only consensus source-review command: set `SOURCE_INPUT=` and run `make earnings-consensus-source-review INPUT=$SOURCE_INPUT PROVIDER= AS_OF=`; it requires explicit provider/cutoff, rejects ambiguous CSV shape, and exposes ordered technical, temporal, rights, and Revenue/EPS scope evidence before collection preview without writing an artifact. Source review and prospective collection are distinct input contracts; set `COLLECTION_INPUT=` only after separate human review and explicit evidence-preserving mapping into the existing collection schema. +- Pull-request engineering gate: `.github/workflows/commercial-research-beta.yml` runs the full tests plus the minimal dashboard, Personal Research render, wording, range hygiene, and range whitespace contract only for pull requests to `main`, with `contents: read` and no provider, readiness, deployment, secret, schedule, or artifact-upload path. It fetches full history, checks out the exact event head, and binds both hygiene commands to the explicit event base/head SHAs. Direct GitHub evidence must pass on every current revision, and automation never substitutes for human review. +- Same-mode Advanced Evidence continuity: Data Health and Proof History stay inside Personal Research mode, preserve the selected ticker, and expose Return to Company Workbench before evidence content. The navigation does not change readiness or evidence state. +- Public mobile workflow audit: fresh desktop and `390x844` phone review passed for all five Public routes; Home and Single-Stock Report are `resolved_post_fix` on complete stop-rule visibility at owner-approved implementation anchor `18417d90766e9aeb629eef6bb1ef4fc40805773c`. The Single-Stock Report direct-open loading state still renders the compact selected-ticker answer before provider and report work, preserving Selected ticker -> `Use now` -> `Still withheld` -> `Open Data Health`. Fresh zero-scroll evidence at `390x844` recorded Home `stop_bottom=843.609375`, `action_height=44`, and `visible_stops=1`; Single-Stock Report `stop_bottom=836.421875`, `action_height=44`, and `trust_gap=2.21875`; both recorded `scroll_width=390`, all four measured scroll offsets were zero, no horizontal overflow, no traceback, and zero open Advanced details. Home phone DOM and visual order is primary -> stop -> metrics. At `1280x720`, desktop retained two Home grid tracks and four Single-Stock Report grid tracks, one breakpoint-appropriate Home stop, 44px actions, and no scroll, overflow, or runtime errors. The earlier `stop_bottom=886.3125` zero-scroll failure and unscrolled `843.4296875` result remain historical superseded evidence only. This is local product-layout evidence only—not hosted, accessibility-conformance, external-reviewer, freshness, demand, or market evidence—and changes no readiness, source, research, or generated-artifact state; quant state also remains unchanged. +- Public packaging reconciliation: README begins with one external-reviewer entry point, names Research Desk -> Discover -> Company Workbench -> Monitor as the primary product workflow, and keeps the five-page Public path secondary. LinkedIn copy uses `Stock Research Command Center | Evidence-First Company Research`, requires the stable GitHub link only after the feature reaches the default branch, and labels any draft branch/PR link `Draft engineering preview`. `docs/assets/linkedin-public-dashboard.png` is a reviewed real-app `1200x627` AVGO Workbench answer image without volatile readiness figures. Browser QA binds it to the Workbench route and answer markers, while Commercial Research performance uses `Use now` as the Workbench first-useful marker. This is packaging and local runtime evidence only, not proof of external sharing, hosted availability, reviewer understanding, demand, competitive position, or market validation. +- Implemented locally: read-only immutable-package status/preview with ten independent states: manifest, technical, temporal, identity, membership, corporate action, delisting, source rights, reproduction, and leakage. +- Reproduction contract: `membership_count_and_sha256_at_cutoff_v1`. +- The only point-in-time universe operating entries are `make point-in-time-universe-status MANIFEST=` and `make point-in-time-universe-preview MANIFEST= TOP_N=20`; they do not fetch, write, apply, rebuild readiness, or activate analysis. +- Synthetic fixtures remain test-only and local software evidence cannot complete Priority 4. +- This local evidence does not change independent readiness for actuals, consensus, Revenue, EPS, valuation, catalysts, outcomes, backtesting, or calibration. +- It does not provide investment advice; numerical probability remains unavailable without calibration; Q4 evidence and EPS split-basis compatibility remain explicit; synthetic evidence stays test-only; candidate peer evidence remains candidate-context-only. +- Proof-readiness reconciliation: run `make proof-readiness-reconciliation TOP_N=20` before reusing a supporting proof outcome. It maps only explicit fundamentals, DCF, share-count, price, peer-mapping, and peer-valuation proof lanes to their independent current fields. Only `explicit_ticker_change` is ticker-level support when the latest proof explicitly names that ticker in `changed_tickers`; keep scope-only outcomes non-supporting even when the ticker appears in a batch scope. `proof_applicability` and `current_blocker_code` are independent; route each current blocker, including `current_canonical_row_missing`, to its named safe review. Current blocker diagnosis does not establish the historical cause; never infer historical source, rights, scope, or cause from narrative proof. `historical_supported_currently_blocked` means historical support cannot be treated as current support; current saved readiness remains authoritative. Advanced Proof History shows global and selected-ticker conflicts before raw ledger detail, while the four primary research routes remain unchanged. The command is read-only and does not restore canonical data, rewrite proof history, promote readiness, prove source rights, field scope, provenance, payload truth, commercial use, or any external gate. Any reported count is a current-snapshot audit finding, not a durable coverage total. The implemented structured per-ticker/per-field record is prospective-only and does not retroactively upgrade existing narrative proof. +- Readiness continuation gate across project status, Session Source Preflight, provider setup, coverage frontier, Auto-Refresh Status, its runbook, Advanced Data Health cards, and the commercial-beta release path: while readiness is stale, incomplete, unverified, or current but untracked, `make readiness-preview TOP_N=20` is the only continuation-safe command; source availability, provider, scheduled-operation, and ranked coverage details remain planning context only, and `make readiness` is a separate intentional reviewed write. +- Peer evidence-quality contract separating relationship provenance, peer role, economic comparability, result context, trend readiness, and valuation-anchor eligibility. +- In-memory quarterly cash-generation evidence contract with independent operating-margin, free-cash-flow, and FCF-margin states, explicit filed-Q4 enforcement, Advanced-only source lineage, and no supplemental data file, writer, template, report, or generated artifact. +- Quarterly adapter acceptance harness for one in-memory company batch, with deterministic identity, commercial-rights, supported-field, cutoff, revision, component, compatibility, complete-period, and Q4 blockers; `accepted_for_review` always leaves production activation false and readiness promotions empty. +- Read-only SEC quarterly cash-generation source preview: `make sec-quarterly-cash-preview AS_OF=` fetches only exact Companyfacts, submissions, and primary-filing endpoints in memory. NVIDIA Q1 FY2027 accession `0001045810-26-000052` and AMD Q1 FY2026 accession `0000002488-26-000076` passed with exact UTC acceptance time and explicit filed-table capex outflow proof; they created no generated artifact and did not activate Company Workbench or readiness. +- Explicit Company Workbench cash-generation preview: only the opt-in `cash_preview=1` NVIDIA and AMD routes compose the accepted SEC result at the fixed review cutoff. They use complete withholding, label every value preview-only rather than ready, keep technical lineage under Advanced, leave `production_activation=false` and `readiness_promotions=()`, make no canonical persistence or generated artifact, and do not alter the normal Workbench route. + +Truth boundaries that must remain unchanged unless direct evidence proves otherwise: + +- Research-only; no investment advice, broker integration, order routing, auto-trading, direct buy/sell instructions, or post-earnings price prediction. +- Real-company Earnings Nowcast remains blocked until compatible quarterly actuals and exact-period point-in-time consensus independently pass. +- Numerical Beat/Miss probability remains withheld until at least 100 valid leakage-safe events pass Brier-score, calibration-bin, and benchmark-improvement gates. +- EPS split basis remains unverified without explicit primary proof. The Companyfacts unverified sentinel must remain blocked from display comparisons, model eligibility, backtest outcomes, and benchmark math; matching sentinel labels or commercial field scope are not proof. +- Q4 actuals require explicit SEC-filed Q4 result-table evidence; do not derive Q4. +- Synthetic fixtures are test-only. +- Empty valuation, catalyst, outcome, and consensus ledgers remain visibly empty. +- Candidate context cannot modify deterministic forecasts or become trusted evidence. +- A source-backed peer relationship is not automatically a valuation anchor. Only explicitly reviewed `core_peer` or `secondary_peer` rows with source/as-of provenance, relationship rationale, comparability basis, and `valuation_anchor_eligible=yes` may enter peer medians. +- Operating margin, free cash flow, and FCF margin remain withheld in Company Workbench until a separate activation review connects compatible explicit observations. The NVIDIA source preview is `accepted_for_review`, not production input. Their readiness cannot promote Revenue, EPS, DCF, consensus, peer, catalyst, outcome, backtest, or calibration states. +- Quarterly adapter acceptance is a local review-routing decision only. It cannot change source rights, load or write an adapter file, supply Company Workbench production observations, or promote readiness. +- Declared source dates newer than the saved readiness build must keep pilot, reviewed-batch, project-status, and profile-context freshness stale. File mtimes cannot override that evidence; only an intentional reviewed `make readiness` run can rebuild the generated snapshot. +- `make readiness-preview TOP_N=20` is inspection evidence only. It must not create or modify CSV, JSON, report, sample-report, screenshot, timing, directory, or bytecode artifacts; it cannot authorize a rebuild, promote readiness, or make stale counts current. +- Proposed technical readiness movement is independent from promotion evidence review. Missing or duplicate canonical evidence, incomplete provenance, unapproved exact-source rights, missing registered support for required fundamentals fields, and unproven DCF price-source lineage must remain explicit and cannot be converted into rebuild approval. +- DCF price availability is independent from price lineage, exact-source commercial rights, and registered price scope. Missing or duplicate latest rows fail closed; missing or composite provider IDs are never inferred. The price-lineage audit cannot change technical readiness, canonical prices, valuation, source rights, freshness, or rebuild authorization. +- Prospective lineage preservation is not evidence activation. A `lineage_complete` staged row still requires exact-source rights, registered `prices` support, payload review, explicit apply intent, rebuilt readiness, and proof. Do not treat temporary fixtures or schema support as a real provider row. +- Staged rights/scope review is not source approval or apply authorization. An approved registry result only reports current checked-in rights and field-scope metadata for the exact source ID; it cannot prove the payload, reviewer decision, freshness, canonical activation, rebuilt readiness, or market validation. +- Commercial apply-guard passage is not evidence activation or automatic mutation. It is a necessary pre-write condition after an explicit apply request; payload review, reviewer intent, freshness, readiness rebuild, proof recording, and downstream use remain separate. +- Direct-refresh guard passage is not provider approval, payload proof, row-level lineage, freshness, readiness, or operating evidence. The checked-in registry still has no approved direct price provider with registered `prices` scope, and temporary test registries cannot change that state. +- Daily-price temporal passage proves only conservative local ordering against one supplied cutoff. It does not establish exchange close, provider publication time, payload correctness, source rights, reviewer approval, canonical freshness, readiness, or safe concurrent operation. Atomic replacement narrows partial-file risk only; backup plus replace is not a database transaction or crash-recovery system. +- Cohort saved-row, adjusted-price, or quarterly metric scope passage is display eligibility for one exact lane only. It cannot validate the payload or retrieval chronology, approve a source, repair rejected rows, resolve revisions, verify EPS split basis, establish Q4 proof, change readiness, activate consensus or nowcast, promote a peer role, or grant another lane. +- Canonical quarterly dashboard consumption is whole-ledger fail-closed: any rejected CSV row blocks accepted-subset Business Trend and focused-cohort quarterly packets. Row-numbered reasons remain under Advanced; no rejected row is repaired, inferred, or promoted. +- Non-empty historical valuation, supported catalyst, and reviewed outcome packets require approved rights for every exact source plus literal `valuation_history`, `catalyst_evidence`, or `research_outcomes` scope in Commercial Research composition. One failed row blocks that scoped result; research-mode review stays independent and empty ledgers remain empty. +- Consensus commercial-guard passage is not point-in-time evidence activation. Technical append eligibility, commercial rights, Revenue scope, EPS scope, payload review, historical availability, comparability, nowcast readiness, backtesting, and calibration remain independent. The checked-in registry currently has no approved prospective-consensus source or consensus scope, and temporary fixtures cannot change that state. +- Consensus batch preflight is not a database transaction, concurrent-writer lock, crash-recovery mechanism, scheduler, or evidence source. It aligns deterministic preview and record decisions only; it cannot supply or approve a provider, repair lineage, reorder revisions, change rights, create readiness, or satisfy calibration. +- Consensus preview receipts prove only that the exact locally reviewed cutoff, mode, input bytes-as-records, and saved ledger state were re-evaluated consistently. Any change requires a fresh preview; a matching receipt does not prove payload truth, publication availability, rights, reviewer judgment, readiness, backtesting, calibration, or safe concurrent operation. +- Consensus source validation is a review-routing contract, not permission, collection, or activation. `historical_evidence_reviewable` cannot prove payload correctness, source rights, freshness, point-in-time depth, nowcast readiness, backtesting, or calibration, and no caller-provided label can upgrade those states. +- Consensus source cutoff passage proves only local timestamp ordering against the supplied review boundary. It cannot infer provider publication time, establish availability, validate a payload or source reference, grant rights, create freshness, collect evidence, or promote readiness, backtesting, or calibration. +- Shared commercial field-scope review proves only consistent checked-in registry metadata for one exact source and ordered required fields. It cannot prove a payload, timestamp, comparability, reviewer intent, collection, activation, readiness, backtesting, calibration, or any non-consensus evidence domain. +- Consensus source review is a read-only local contract evaluation, not provider entitlement, payload approval, collection, activation, readiness, backtesting, or calibration. Human or JSON output and `auto_apply=false` cannot promote a row or satisfy an external gate. +- A checked-in GitHub Actions workflow proves only the intended automation contract. Independent hosted CI requires a completed GitHub result on the current commit; a green automated check is not independent human review and proves no source, hosted-product, reviewer, calibration, adoption, or market claim. +- README, LinkedIn copy, committed screenshots, browser-QA markers, and local timing are public-packaging evidence only. They cannot prove data freshness, changing readiness totals, hosted availability, external reviewer comprehension, adoption, competitive differentiation, demand, or product-market fit. Do not publish the stable default-branch link until the reviewed feature is actually present there; label branch or PR links as draft previews. +- Subagent review is engineering review only; no GitHub human reviews exist. +- Named change causes explain existing method decisions only. They cannot alter scope, upgrade partial data, establish current readiness totals, create a company-quality judgment, or authorize a rebuild. +- The readiness continuation gate follows declared source dates and keeps tracked release-evidence eligibility independent. It must suppress broad refresh, source-proof, apply, and readiness-rebuild routing when readiness is stale, incomplete, unverified, current but untracked, or cannot be compared with tracked HEAD evidence. A date-current local generated snapshot is not PR evidence. Reviewed-batch temporal routing must use the same declared-date profile context rather than a separate mtime rule. The gate does not change provider availability or ranking calculations; those remain planning context until the separate intentional reviewed write and exact artifact review are authorized. + +Current local status boundaries, not external-dependency classifications: + +- Quarterly cash-generation source adapter: `bounded_two_company_source_preview_accepted_for_review`; NVIDIA Q1 FY2027 and AMD Q1 FY2026 establish bounded two-company portability only. The superseded NVIDIA-only label was Quarterly cash-generation source adapter: `one_company_source_preview_accepted_for_review`. Production activation, broad coverage, historical depth, and Q4 coverage remain unproven; do not repeat the NVIDIA pilot or AMD pilot unless its exact filing changes or a regression is suspected. +- Quarterly adapter composition remains explicit preview only. It is not canonical evidence activation; the completed bounded second-company proof does not authorize broader scope. Do not add a third company, persistence, or readiness promotion without a separate exact-source design and review. +- Readiness promotion evidence: `local_evidence_review_required`. Continue only with local no-write lineage validation; do not authorize generated rebuilds or edit source rights without reviewed evidence. + +Current external dependency classifications to verify once, then avoid looping: + +- Point-in-time consensus and rights: `permitted_point_in_time_consensus_and_rights_required`. Last observed: no permitted point-in-time consensus input or approved exact-source consensus rights evidence is on record. Exact unblock condition: configure a permitted point-in-time provider or supply a reviewed CSV, approve rights for that exact source and every populated consensus field, then run source review and collection preview without inferring a provider or recording readiness. +- Hosted account and controls: `hosted_account_and_controls_required`. Last observed: no verified hosted URL or enforced hosted access boundary is on record. Exact unblock condition: provide an intentional host account, a directly verified URL, enforced claimed access controls, and user/workspace isolation in that environment. +- Independent reviewers: `independent_reviewers_required`. Last observed: no independent human GitHub pull-request review or completed independent beta-review cohort is on record. Exact unblock condition: obtain independent human GitHub review of PR #113 for the engineering-review gate and complete controlled sessions with 10-20 independent task-based reviewers, recording reproducible observations for beta validation. Subagent and automated review satisfy neither condition. +- Trusted peer/source review: `trustworthy_peer_source_and_review_required`. Last observed: no trustworthy reviewed peer relationship evidence is on record for the bounded pilot. Exact unblock condition: supply a trustworthy relationship source and review capacity sufficient to preserve one bounded reviewed peer relationship, including role, rationale, comparability, source/as-of evidence, and explicit valuation-anchor eligibility. +- Calibration cohort: `calibration_cohort_required`. Last observed: calibration evidence remains insufficient and numerical probability is withheld. If one immutable operator bundle is supplied, run `make calibration-evidence-bundle-preview BUNDLE=` once; `invalid`, `blocked`, and `contract_consistent_review_required` are review states only. The preview writes no artifact, does not activate readiness, and probability remains withheld. Synthetic fixtures remain test-only. Classify an absent or unchanged bundle once and move to another executable lane. Exact unblock condition: accumulate at least 100 leakage-safe out-of-sample events from permitted point-in-time inputs, obtain source-rights and independent review, and pass the declared Brier-score, calibration-bin, benchmark-improvement, exact-identity, chronology, and backtest gates. +- Operated owner/incident/rollback capacity: `operated_owner_incident_rollback_capacity_required`. Last observed: no named operated owner or directly rehearsed incident and rollback capacity is on record. Exact unblock condition: provide a named owner and directly rehearsed incident and rollback capacity, then directly verify audit, retention, entitlements, monitoring, and health-check operation. A local runbook or hosted URL is insufficient. +- Point-in-time benchmark/universe data and rights: `point_in_time_benchmark_universe_and_rights_required`. Last observed: no permitted point-in-time benchmark and universe dataset with approved exact-source rights and independent review is on record. Exact unblock condition: Priority 4 remains open until one bounded permitted real dataset is independently reviewed, reproduces the expected membership count and digest, and passes rights, identity, corporate-action, delisting, survivorship, cutoff, partition, reproduction, and leakage gates. Do not retry an unavailable provider or infer data access, source identity, field scope, or rights. +- Accessibility manual-review environment: `accessibility_manual_review_environment_required`. Last observed: `docs/ACCESSIBILITY_EVIDENCE.md` records a partial local desktop/phone workflow audit, a local automated route-and-viewport gate for one stable role-based `main` landmark, and passing forced-colors/reduced-motion browser emulation at the current implementation anchor. Manual color and motion protocol tasks and complete independent-human keyboard, zoom, reflow, screen-reader, hosted, and assistive-technology landmark-use review remain unproven. Exact unblock condition: verify a suitable local or supervised review environment, execute those remaining manual and assistive-technology tasks, and record reproducible findings and material-defect retests. +- Paper-position laboratory design approval: `paper_position_lab_design_approval_required`. Last observed: no separately approved hypothetical paper-position laboratory design is on record. Exact unblock condition: complete the research-only design, private-data policy, language review, misuse analysis, and acceptance plan, then obtain separate explicit approval before implementation. + +Approved Next-Stage Maturity Program: + +This is the authoritative priority order after the completed Research Decision Lab. If the current branch head lacks direct local matrix, branch synchronization, draft-PR update, or exact-head CI evidence, complete those release steps first; otherwise select the first incomplete safe roadmap priority. When the next requirement is unavailable, classify its last evidence and exact unblock condition once, leave it incomplete, and move to the next safe executable priority. Re-enter a blocked priority only after relevant external state changes. + +A blocked priority does not become complete, and moving past it does not weaken its exit gate. Local contracts, fixtures, screenshots, subagent reviews, and green tests cannot substitute for direct source, hosted, accessibility, independent-user, or calibration evidence. Continue cycling through the ordered program while safe executable work exists; overall completion requires direct current evidence for every applicable priority. + +Documentation and routing reconciliation is complete locally. Reopen it only when current repository evidence reproduces contract drift. + +The reproduced default-profile readiness truth drift is repaired locally: profile context reports declared-date freshness separately from tracked evidence origin, `working_artifact_uncommitted` routes status and reviewed-batch work to the no-write preview, and excluded generated readiness files remain unstaged. This local contract does not approve a readiness rebuild or promote the July working snapshot into PR evidence. + +Modal modifiers and active exposure fail closed. The only active documented-result exception is the exact reviewed-historical-assumptions methodology template; adjacent transaction-sizing variants remain withheld. A raw calibration count cannot establish calibrated probability. `calibrated` requires paired verified calibration status and a leakage-safe benchmark-passing `BacktestReport`. The pairing requires fully recalculated immutable observations matched one-for-one to those report events by ticker, fiscal period, and cutoff, with each Boolean outcome verified under the declared strict Revenue or EPS `actual > consensus` definition. The declared metric itself must be modeled and improve its matching consensus benchmark; the opposite metric's backtest cannot authorize it, and stored relative classifications must be rederived from forecast intervals before aggregate evidence is accepted. A canonical digest binds the full supplied report package, including normalized event chronology, ordered source IDs, model/input identity, scored fields, exclusions, failures, summaries, and benchmarks; any post-assessment change requires a fresh pairing. This is an internal consistency contract, not provider, source-rights, timestamp, or external-provenance attestation. Identity-less or unrelated same-count evidence cannot promote probability. Evidence publication, snapshot, and retrieval timestamps must all be at or before the cutoff. Broad-review repairs must be evaluated only through direct current-head local and exact-head CI evidence; their presence alone establishes neither gate. + +Current local execution queue: + +Current local product slice awaiting GitHub release closure: Discover truth separation, the Company Workbench primary brief, Monitor consolidation, Research Desk simplification, and shared-shell cleanup are implemented test-first. Strict eligibility remains empty when required evidence is incomplete and never relaxes thresholds. Research Desk now renders one **Today's Research Brief** instead of a weekly-card row plus four competing answers; it routes to Monitor only when saved work needs attention and otherwise routes to Discover, while keeping all supporting evidence under Advanced. Alphabetical saved-company browsing remains available from readiness-only identities without importing ranking-adjacent fields or legacy outputs. Workbench shows one Company Brief with usable evidence, withheld evidence, change state, one authoritative next task, the stop rule, and a ticker-bound Data Health action before any detailed module; one explicit session-local action restores the unchanged detailed evidence modules without writing data or changing readiness. Monitor renders one five-panel **Follow-up Queue** instead of Evidence Monitor Brief, Research Discipline Review, and Research change monitor as three competing primary answers. Its empty state appears once, preserves the external-event boundary, and returns to Discover; full process identities and source-change evidence remain under `Advanced: Monitor evidence`. Personal Research now has one visible top page-navigation authority; the sidebar retains workspace selection without repeating Research page choices, and the Research main column no longer renders the Operator command/readiness header or broad profile strip before the route answer. Public/Operator shells, direct links, ticker parameters, and secondary evidence routes remain unchanged. At shared-shell implementation anchor `2bc374ab8`, the clean read-only browser gate passed all 12 six-route/two-viewport cases while the same 18 generated paths stayed excluded; two former global-label waits were replaced with route-owned answer markers after the simplified shell exposed their hidden coupling. Release credit attaches only after full current-head gates, protected-artifact identity, an intentional push, draft-PR reconciliation, and exact-head CI; PR #113 carries the current GitHub result. Pilot readiness remains blocked on uncommitted working readiness evidence and incomplete source proof. The next external data unblock remains one bounded permitted price-history plus historical-valuation evidence package through separately reviewed source activation. + +Completed local item: `1. Add shared quant provenance/recency eligibility without coupling readiness.` This former queue item is implemented at +`195ea18da9d1d6e06c36f8320509ccde46cdaa57` and is not an active task. + +Completed local framework reliability item: `1. Close the same-document +transport framework reliability evidence.` At +`d68ab27bee9c07c450faeb866b08cbf13638b56f`, 4,381 tests and the required +render/public/commercial-beta/pilot gates passed. The clean exact local browser +run passed all 12 six-route/two-viewport results with zero deprecated-component +warnings, bridge iframes, bridge focusable descendants, and bridge height; the +owned bounded server-output proof also passed with zero warnings. The same 18 +generated paths remained excluded. Current exact-head synchronization at +`b69badfc8` and run `30726301045` close that recorded release gate; do not +treat local or GitHub engineering evidence as hosted-product evidence. + +Completed local accessibility media-preference base slice: exact implementation +anchor `a97a096e1caf2761176dc42798a8c9686179aecc` passed the then-current +forced-colors and reduced-motion evaluators in all 12 six-route/two-viewport +browser results. Final review found that anchor did not independently verify +the route-specific state marker and next action in either mode, and its +forced-colors marker check could credit the generic workflow-link border. The +later final-review contract requires exact-count visible route markers and next +actions, plus separate two-pixel active-border and positive-outline evidence; +do not reuse the historical anchor as proof of those narrower assertions. +This does not pass C01, C02, M01 or complete direct platform, +assistive-technology, hosted, independent-human, or WCAG review. + +Historical pre-fix evidence: Task 4 local matrix completed at `c8c313b9c`. +Apply the durable release-routing rule before choosing another implementation +slice. Keep exact technical evidence under Advanced, preserve the read-only +research queue and alphabetical cohort identity, and do not convert unavailable +source, rights, reviewer, hosted, or calibration evidence into product copy. + +Evidence Monitor Brief implementation anchor: `78c5ff9dc96f2d676b31ce4819053bb3f4eb555e`. +The Monitor route reuses the fixed seven-day summary and existing discipline +precedence, filters only exact `Monitor` rows from the primary table, and keeps +the complete rows and identities under Advanced. Ordinary route use remains +in-memory and read-only: it does not refresh, persist, rebuild readiness, write +generated artifacts, create a market or trade claim, or change independent +readiness, observation, research-attention, source-change, or calibration state. +External source, hosted-operation, human-accessibility, independent-session, +and calibration gates remain incomplete. + +Priority 7 manual accessibility evidence remains externally incomplete. Do not +re-run Priorities 1-3 unless a current regression is directly reproduced. +Priority 6 provider-neutral retention/deletion or audit-event work is not the active next lane and requires a separate reviewed design; provider-specific +work remains blocked on an approved environment. + +Priority 1 — Legacy portfolio, ranking, and action-language quarantine +- Priority 1 is complete locally. The five retained compatibility pages are Operator-only, Public and Personal Research deep links fail closed, and every retained page is labeled `Legacy research utility — not part of Personal Research Mode` with details behind an explicit collapsed control. +- Direct route, wording, and source-contract tests prove retained legacy output cannot feed Company Workbench, Research Decision Lab, readiness, recommendations, sizing, or transaction behavior. Historical calculations and filenames remain compatibility-only. +- Reopen only if a current route, wording, report, or no-trading regression is directly reproduced. + +Priority 2 — Stage B field-proof audit and operator hardening +- Priority 2 is complete locally. `make prospective-field-proof-audit` provides read-only append history, active-head state, current blocker explanations, and controlled invalid-ledger routing; preview provides per-row explanations and reports `preview_receipt_persisted=false` and `receipt_revalidation_required=true`. +- Direct byte-snapshot tests prove audit and preview create no ledger, readiness, canonical-data, report, screenshot, timing, or generated artifact. Audit does not activate readiness, does not update canonical data, and does not activate Company Workbench. +- Reopen only if a current read-only, append-history, blocker-explanation, no-mapping, or no-write regression is directly reproduced. + +Priority 3 — In-app research-record authoring +- Priority 3 — completed locally after direct desktop/phone runtime review and the required automated acceptance matrix. The approved specification is at `docs/superpowers/specs/2026-07-22-in-app-research-record-authoring-design.md` and its test-first implementation plan is at `docs/superpowers/plans/2026-07-22-in-app-research-record-authoring.md`. Priority 4's local validator is frozen; its permitted real-data exit gate remains externally incomplete. Priority 6's provider-neutral authorization contract is complete locally; hosted implementation remains environment-dependent. Apply the durable release-routing rule before choosing another implementation slice. +- The approved design uses one collapsed Company Workbench composer over the existing thesis-journal, catalyst, and outcome engines. A session-only receipt binds the exact draft, selected profile/ticker, destination ledger, and current ledger fingerprint; edits or concurrent appends require a fresh preview. Tests must use temporary ledgers and must not append production research data. +- Hardening commit `07758114c` closes the confirmation race: all three append engines share one resolved-ledger cooperative lock, receipts bind resolved ledger identity, every new preview resets confirmation, and uncertain post-append teardown requires one-shot read-side reload before success. +- Final integrity commit `e3a090dba` ensures confirmation appends only the receipt-matched recomputed record and enforces one readable active thesis lineage: revisions must supersede the exact active entry and preserve its thesis ID. The Company Workbench locks and explains that relationship, with temporary-ledger create -> revise -> reload coverage. +- Confirmation-integrity commit `5a6c55921` binds every displayed preview field, preview time, and destination label to the exact receipt. If an append raises after it may have written, confirmation returns one-shot `save_pending_reload` with the exact record ID unless the locked ledger is provably unchanged; it never invites a blind duplicate retry. +- Reopen only if a current desktop/phone, persistence, rejection, identity, provenance, empty-state, or no-fabrication regression is directly reproduced. + +Thesis, evidence, catalyst, and outcome records are all available in the collapsed Company Workbench composer. +A valid record requires an exact preview and explicit confirmation before save. +Drafts are untrusted and preview receipts are session-only. +Production tests never append repository ledgers; persistence tests use temporary ledgers. +A saved record cannot change readiness, forecasts, probabilities, recommendations, or any other ledger. + +Priority 4's local validator is frozen; its permitted real-data exit gate remains externally incomplete. +Priority 6's provider-neutral authorization contract is complete locally; hosted implementation remains environment-dependent. +If the current branch head lacks direct local matrix, branch synchronization, draft-PR update, or exact-head CI evidence, complete those release steps first; otherwise select the first incomplete safe roadmap priority. +That summary is necessary but not sufficient; the exact Priority 4 exit condition below also requires independent review, expected count/digest reproduction, and the partition gate. + +Priority 4 — Point-in-time benchmark and universe foundation +- The provider-neutral, read-only first slice is implemented locally against the approved design at `docs/superpowers/specs/2026-07-23-point-in-time-universe-foundation-design.md`; reverify its ten independent states and no-write behavior from current repository truth. +- Completed review and remediation chronology is preserved in `docs/internal/POINT_IN_TIME_UNIVERSE_REVIEW_HISTORY.md`. Priority 4 remains externally incomplete; this continuation contract retains only the current operating boundary and exact external exit gate. +- Local resource budgets for one supplied package: preview sample 100 rows; manifest 1 MiB; each contract CSV 32 MiB; four contract snapshots combined 64 MiB; source-rights registry 4 MiB; declared rows 250,000 per contract; package traversal 32 entries. +- Source-bearing raw rows retain independent exact-source rights review even when exact-row technical parsing fails; walk-forward history is cutoff-relative; publication must precede or equal retrieval. Evidence publication, snapshot, and retrieval timestamps must all be at or before the cutoff. Regular-file snapshot reads retain at most the declared limit plus one byte from one descriptor; deeply nested manifest JSON or source-rights YAML fails with a stable readable input error instead of a traceback. Duplicate JSON/YAML mapping keys and malformed contract headers also fail nonzero, traceback-free, and write-free through the direct validator and CLI/Make boundaries. +- These local bounds do not prove scale, hosted reliability, or market readiness. +- No permitted independently reviewed real dataset, accepted expected count/digest, or source-rights proof is on record. +- Keep the foundation isolated from the current ticker-centric universe merge path. Stable security identity, timestamped membership, corporate-action/listing-status events, evaluation cutoffs, immutable manifests, exact-source rights, revision lineage, and deterministic membership digests remain fail-closed and independent. +- Use only the two documented read-only status/preview entries. Do not fetch, write normalized artifacts, apply rows, rebuild readiness, or activate analysis. +- Explicitly control corporate actions, delistings, survivorship, and leakage; never substitute current constituents, identifiers, prices, or fundamentals for historical state. +- Preserve leakage-safe partition or walk-forward boundaries and reproducible benchmark diagnostics. +- Keep this priority open under the single registered external dependency condition above. Schemas, fixtures, and local green gates are insufficient. +- Keep the synchronized Priority 4 local validator frozen unless a newly reproduced Critical or Important defect requires reopening it. One bounded permitted real dataset remains the exact external exit gate. While that dataset and the independent Priority 5 consensus/peer evidence are unavailable, keep Priority 6's completed local policy contract separate from unavailable hosted implementation. Apply the durable release-routing rule before choosing another implementation slice. Preserve the external manual-accessibility boundary. Do not retry unavailable providers or recommit the already synchronized package. + +Priority 5 — One permitted consensus source and one reviewed peer relationship +- Review one permitted point-in-time consensus source for one ticker and exact fiscal period, then perform separate human evidence-preserving mapping into the prospective schema and run the collection preview. +- Only if the exact input, ledger, cutoff, mode, and receipt remain unchanged may the operator run `CONFIRM_REVIEWED=1 make earnings-consensus-collection-record INPUT=$COLLECTION_INPUT AS_OF= PREVIEW_RECEIPT=`. +- Treat the result as an append-only evidence record; it does not activate readiness or numerical probability, and it cannot bypass actuals, Revenue/EPS comparability, backtesting, or calibration gates. +- Establish one genuinely reviewed peer relationship with role, rationale, comparability basis, source/as-of evidence, and an explicit independent valuation-anchor decision. +- Keep the two gates independent; do not broaden coverage or infer source, rights, peer trust, or valuation eligibility. +- Exit only when one real consensus snapshot and one real peer relationship independently pass technical, temporal, provenance, rights, scope, reviewer, and append-only evidence checks. This does not unlock probability. + +Priority 6 — Controlled hosted operating boundary +- The first local slice implements + `src.hosted_access_control.evaluate_workspace_access`, a pure, + provider-neutral, deny-by-default contract for authenticated-principal, + active-membership, workspace, role/resource/action, append-only + research-record, stable reason, and privacy-safe audit obligation decisions. + It does not prove hosted authentication, deployed isolation, persistence, + audit storage, retention, monitoring, rollback, incident response, operated + capacity, or market validation. +- Reverify final trust-boundary hardening commit + `90b441d25e97655e887803d88f4730497e71e383` or a later descendant and + exact-head CI before relying on the local contract. Do not create or change + hosted accounts, choose a provider, use credentials, deploy, or publish + without explicit approval. +- The local policy slice is complete. A provider-neutral retention/deletion + or append-only audit-event interface remains a separately reviewed future + design; provider-neutral retention/deletion or audit-event work is not the + active next lane. Provider-specific integration remains blocked until the + exact identity, storage, logging, host, and operating environment are + explicitly approved. + +- Before an environment is approved, implement only provider-neutral control contracts, threat boundaries, interfaces, and denial-test harnesses; do not silently choose an identity, storage, logging, or hosting architecture. +- Provider-specific implementation begins only after explicit approval of the exact identity, storage, host, and operating environment. Then implement authentication, private-workspace isolation, least privilege, secret handling, append-only audit logs, retention/deletion, monitoring, health checks, backup, rollback, and incident ownership against it. +- Test cross-user and cross-workspace denial, audit completeness, retention execution, alerts, recovery, and rollback. +- Do not create or change hosted accounts, purchase services, use credentials, deploy, or publish without explicit approval. +- Exit only when the actual approved environment directly proves every claimed control, including a named owner and observed rollback rehearsal. Local code or a URL alone is insufficient. + +Priority 7 — Accessibility evidence beyond screenshots +- Add automated semantic checks and direct keyboard, focus, label/error, resize, zoom/reflow, color/contrast, reduced-motion, and screen-reader task testing across Research Desk -> Discover -> Company Workbench -> Monitor on desktop and phone. +- Preserve the 2026-07-23 partial local audit in `docs/ACCESSIBILITY_EVIDENCE.md`: desktop and phone routes had no document-level horizontal overflow, visible controls retained accessible names, and the Monitor primary-link contrast defect was corrected from approximately `3.3:1` to `12.4:1`. Do not broaden that result into keyboard, screen-reader, zoom, forced-colors, or WCAG evidence. +- Preserve the second test-first slice: every Personal Research route now exposes a route-preserving `Skip to page answer` link and focusable answer target, primary sections use continuous `h1` -> `h2` hierarchy, the nested Monitor `h3` is non-empty, and desktop/phone retest found no horizontal overflow. This proves structural presence and heading order only, not complete skip-link keyboard behavior. +- Preserve the 2026-07-26 live-DOM correction: Streamlit had normalized the skip link to `target="_blank"`. It now uses a fragment-only `#public-page-answer` destination and explicit `target="_self"` so same-tab activation retains every current query parameter without a rerun and focuses the answer target. Direct ticker-bound Workbench pointer retest passed; synthetic Enter could not activate ordinary links in the available browser-control environment, so K01/K02 remain `blocked_environment`, not `passed_direct`. +- Preserve the 2026-07-27 target-size correction and its adversarial follow-up: direct Workbench audit measured the framework help target at `16x16`, dataframe toolbar buttons at `22x22`, and the phone Workbench action at `102x21`. The first correction enlarged only the help wrapper; expanded P02 review caught the actual button at `16x24`. The final selector targets the nested button. Desktop/phone retest measured actual help and dataframe buttons at `24x24`, the phone action at `102x24`, and no phone document overflow. This closes only the inspected P01/P02 defects, not full route coverage, keyboard, zoom, forced-colors, screen-reader, or WCAG evidence. +- Preserve the later 2026-07-27 direct Chrome keyboard run at exact commit `15c5270070238b91343a9f6eaa26f86527bfcaf2`: K02 same-page focus transfer and K03 route-rail navigation passed directly in the recorded desktop environment. K01 failed because Workspace help, Workspace selection, route selection, and the Streamlit main section precede the skip link. K04 opened filtered AVGO by keyboard but failed because every company action exposes the same accessible name `Open review`. K05 stopped at focused native disclosure summaries with no visible outline. This supersedes only the earlier input-environment limitation for the recorded Chrome run; it is not independent human, phone-keyboard, assisted-technology, or WCAG evidence. The proposed narrow remediation is not implemented until its design is reviewed. +- Preserve the same-day follow-up at `6aef0f8a9e89e8ec94d308095ca18ef33ea4177a`: real browser zoom remained unavailable, so Z01-Z03 are `blocked_environment`; `640px`/`320px` viewport stress found no document overflow but is not zoom evidence. At `390x844`, the inspected visible primary route and opened thesis-authoring controls had no sub-24px target or adjacent overlap. This is partial P02 evidence only; mobile-sidebar choices, all record kinds, all disclosure states, and pointer misactivation remain unproved. +- Preserve the exact-HEAD continuation at `1a8727706595f87df459eeb5c7c762dafad83577`: K08 passed directly in the truthful current Monitor state, and forward/reverse traversal found no keyboard trap. K09 nevertheless failed because every focused native `summary` still had computed `outline: none`. Direct accessibility-role inspection returned zero `main` landmarks on Research Desk, Discover, ticker-bound AVGO Company Workbench, and Monitor. The connected Chrome and in-app browser surfaces exposed viewport control but no supported forced-colors or reduced-motion activation; C01-C02 and M01 therefore remain `blocked_environment`, not passed. Screen-reader tasks remain `not_run`. +- Preserve the keyboard authoring continuation at `b1dd457b574045c06496fdf4ec6772dddfc7d751`: K06 reached every empty thesis field and `Validate and preview` with direct keyboard input. Rejected validation exposed `thesis_id is required` through one global `role="alert"` and correctly exposed no confirmation action, but Thesis Id had no `aria-invalid`, `aria-describedby`, or `aria-errormessage`, and focus remained on the validation button. K06 therefore failed pending field-level association and direct retest. K07 passed: keyboard route navigation opened Monitor, the one-line thesis ledger fingerprint remained unchanged, both absent catalyst/outcome ledgers remained absent, and no research record was written. S06-S07 remain `not_run`. +- Preserve the exact `390x844` phone continuation at `c748b192dd4fe27f1c17f2feba07eaccd24e4619`: K02 passed on ticker-bound AVGO Workbench, retaining every route parameter and transferring focus to the answer. K01 failed because Workspace help, Workspace selection, and route selection received focus at negative x coordinates inside the closed off-canvas sidebar before the main section and skip link. K03 changed routes deterministically with Arrow Down but failed visible focus because the selected route remained off-canvas; the sidebar-open control was hidden at `0x0` and absent from the recorded Tab sequence. Phone K04 reached search, filtered AVGO, and opened the correct `86x44` action, but the accessible name remained only `Open review` and the preceding summary still had no outline. P02 failed for mobile route choices: each route label measured approximately `104x50` but began at `x=-279`; the visible main document retained `390px` scroll and client widths. +- Preserve the exact-tested narrow-remediation browser gate at implementation anchor `0000c97e7db17e5d4353e30e976f2b7dec6bfd46`: `make research-accessibility-browser-check` verified the rendered Stock Research Command Center `Demo` profile and a clean product/code/test/docs tree while classifying and excluding exactly 18 unstaged generated CSV/output paths, then passed all eight primary research route-and-viewport cases at `1280x720` and `390x844`. After initial focus was cleared, one physical Tab focused the sole skip link; no DOM-order enumeration or programmatic link focus substitutes for that result. The focused link was fully inside both horizontal and vertical viewport bounds at approximately `x=8.8..141.4`, `y=8.8..47.0`, and Enter retained the full route query and focused `#public-page-answer`. The labelled workflow navigation and every applicable route link were inside the viewport, with each link at least `44px` high; all four actually rendered eligible Discover actions had unique ticker-matched names without an assumed row count; directly focused summaries exposed a solid three-pixel outline; and rejected empty-thesis validation associated, described, and focused Thesis Id while retaining one global alert. The live Workbench regression then changed the draft, removed only bridge-owned Thesis Id error state, and associated the next `effective_at is required` rejection only with Effective At on desktop and phone. Non-loopback or mismatched servers, staged paths, dirty non-generated implementation paths, absent Chrome, and absent Playwright fail closed; an unverified run receives no local commit/profile attribution. The gate writes no JSON, timing, report, screenshot, readiness, canonical-data, or ledger artifact. This supersedes only the five narrow failures and the stale-error/provenance evidence gaps at the recorded local implementation anchor; it is automated engineering evidence, not independent-human, screen-reader, hosted, or WCAG-conformance evidence. +- Preserve the exact-tested framework-safe semantic-main gate at implementation anchor `d1328eaa4d08cf08ec2b70939e4e031ee5f907b0`: focused tests returned `22 passed`, and `make research-accessibility-browser-check` verified the rendered loopback Stock Research Command Center `Demo` profile plus a clean product tree while classifying and excluding exactly the same 18 unstaged generated paths. All 12 Research Desk, Discover, ticker-bound Company Workbench, Monitor, Research Data Health, and Research Proof History cases passed at `1280x720` and `390x844` with no failures. Every initial DOM and every DOM after a same-document Streamlit `notRunning` -> `running` -> `notRunning` script cycle exposed exactly one role-based main with exact `role="main"`, `id="research-main"`, `aria-label="Stock research workspace"`, one contained `#public-page-answer`, one level-one heading, and host status `data-research-main-bridge-status="applied"`; only that rerun/probe phase preserved the exact route with zero top-level navigation. A hidden inert child-list mutation forced the live observer to restore `applied`, after which the probe was removed. Separately, every case deliberately navigated to an explicit different Research route and back, waited for marker/stability/exact H1, required the full away and return URLs including the complete query string and empty fragment with the ticker parameter where present, and repeated semantic-main, runtime, and applicable primary/secondary navigation assertions. One physical Tab still focused the sole initial skip link, its activated target was inside that unique main, and no console/page error, rendered traceback, or horizontal overflow occurred across the recorded phases. The controlled native radio event and installed Streamlit test-state transition are framework engineering evidence only, not pointer/keyboard/mobile-sidebar interaction credit or a public cross-version compatibility guarantee. The gate remained repository/data read-only and in-memory/stdout-only and wrote no screenshot, timing, JSON, report, readiness, canonical-data, ledger, or generated repository artifact. +- Preserve the behavior-first same-document correction evidence at + `e8084099b3ea1b794ce8e2a0af00998602133084`: the direct local gate again + passed all 12 route-and-viewport cases with an empty failure list and the + same 18 generated paths excluded. This proves the fixed scripts retained + landmark, authoring association/cleanup/focus, rerun, mutation, routing, + query, error, and overflow behavior after replacing iframe-era + `window.frameElement` / `window.parent` assumptions with executing-document + ownership. The gate now additionally exposes fail-closed per-result + deprecation-warning, bridge-iframe, bridge-focusable, and bridge-height + fields. Focused tests return `33 passed`; exact integer validation rejects + malformed DOM counts, and the gate-owned local server's bounded merged + stdout/stderr is included in warning detection. Warning counting inspects + each full line before storage truncation/eviction; reader exceptions and + incomplete reader shutdown fail closed. Explicit `BASE_URL` mode + classifies server output as unavailable and cannot pass the strict transport + proof. The six-route render smoke emits no `st.components.v1.html` warning, + The clean instrumentation run at + `d68ab27bee9c07c450faeb866b08cbf13638b56f` then passed all 12 results with + each new field equal to zero, `server_runtime_output_status` captured + locally, the overall server warning count equal to zero, an empty failure + list, and the same 18 generated paths excluded. The surrounding verification + returned 4,381 passing tests and passed the required local release gates. + This closed the local transport reliability item. Current exact-head + synchronization at `b69badfc8` and run `30726301045` supersede the earlier + pending release step; neither proves hosted-product behavior. +- Preserve the clean-tree research-state accessibility matrix at `6328c8cead7c27cb901e7878cd6d7d23fa11bb0e`: the direct local browser gate passed all 12 route-and-viewport results and the separate synthetic state harness at `1280x720` and `390x844`, with an empty failure list and exactly 18 generated paths classified and excluded. Loading, empty, withheld, stale, failure, and validation remained visible non-live states with busy semantics only for loading. Validation rejection and reload uncertainty used one assertive alert; preview readiness, draft change, and verified reload used one polite status; unchanged rerenders retained one visible non-live message. Both harness cases had no traceback, overflow, console/page error, or repository mutation. This closes only the previously environment-limited automated matrix; it is not screen-reader, human, hosted, or WCAG evidence. +- Preserve the historical automated media-preference matrix at exact implementation anchor `a97a096e1caf2761176dc42798a8c9686179aecc`: `make research-accessibility-browser-check TIMEOUT_SECONDS=90` passed the then-current forced-colors and reduced-motion evaluators for all 12 Research Desk, Discover, ticker-bound Company Workbench, Monitor, Research Data Health, and Research Proof History cases at `1280x720` and `390x844`. Existing landmark, physical-Tab skip focus and activation, navigation, applicable Discover/authoring, same-document rerun, away-and-return, dynamic-state, runtime-warning, browser-error, overflow, and byte-for-byte repository checks remained green; the same 18 generated paths remained unstaged and excluded. Final review narrowed this claim: that anchor did not independently observe the route-specific state marker and next action in either emulated mode, and its forced-colors active-marker measurement did not distinguish the generic one-pixel border from the active two-pixel border plus outline. Require a later exact-fix-HEAD direct result before crediting the hardened assertions. This is automated engineering evidence only. It does not pass C01, C02, M01 or direct platform/human forced-colors, reduced-motion, zoom, screen-reader, assistive-technology, hosted, independent-human, or WCAG evidence. +- Execute `docs/ACCESSIBILITY_TASK_PROTOCOL.md` for direct task evidence. It records exact revision/environment metadata, keyboard-only navigation, write-free validation, zoom/reflow, forced-colors, reduced-motion, screen-reader, target-size, finding, and completion rules; the protocol is not completion evidence. +- The current Streamlit container API has no stable semantic-role parameter. Preserve the separately reviewed fixed same-origin bridge, its exact-target and fail-closed ownership rules, and its direct runtime gate; do not broaden it into research-content access, navigation, application actions, persistence, telemetry, or readiness behavior. +- Continue with the named open findings: small framework controls outside the tested matrix, independent-human complete keyboard order, true zoom/reflow, forced colors, reduced motion, screen-reader tasks, and assistive-technology confirmation of dynamic announcements. Do not reopen the completed synthetic loading/empty/withheld/stale/failure matrix unless a current regression is reproduced. +- Record environment, assistive technology, defect, severity, remediation, and retest evidence; never infer conformance from screenshots or automation alone. +- Exit only when automated and manual task evidence covers the complete workflow and all material defects are fixed or explicitly bounded. + +Priority 8 — Independent workflow validation +- The local protocol is ready: the invitation, feedback template, anonymous CSV schema, runbook, and closeout checklist use `Research Desk -> Discover -> Company Workbench -> Monitor`, prevent coaching, exclude owner/Codex/subagent/fixture/screenshot evidence, capture the complete scorecard, and require voluntary minimal-data capture with withdrawal and deletion handling. This is protocol evidence only; zero independent sessions are on record. +- Run 10-20 independent workflow sessions with the target researcher persona under a non-coaching protocol. +- Measure task completion, time to first useful answer, readiness comprehension, evidence tracing, authoring friction, trust, misuse risk, perceived performance, repeat-use intent, and the most important missing workflow without collecting investment recommendations or unnecessary personal data. +- Exit only when all sessions have anonymized evidence, reproducible findings, severity decisions, and material-defect retests. Automated, subagent, owner, or screenshot review does not count. + +Priority 9 — Out-of-sample calibration cohort +- Accumulate at least 100 valid leakage-safe out-of-sample events from permitted point-in-time inputs with immutable cutoffs, comparable actuals, revision lineage, exclusions, and reproducible cohort membership. +- Evaluate predeclared Brier-score, calibration-bin, benchmark-improvement, missingness, and stability gates without post-hoc threshold selection. A raw calibration count cannot establish calibrated probability. `calibrated` requires fully recalculated immutable observations whose exact ticker, fiscal-period, cutoff, and declared strict Revenue/EPS outcome match every event in the same leakage-safe benchmark-passing `BacktestReport`. +- Exit only when every declared gate passes on direct current evidence. Numerical Beat/Miss probability remains withheld before that point and remains withheld after a failed gate. + +Priority 10 — Separately approved hypothetical paper-position laboratory +- Produce or refine only a separate research-only design, private-data policy, language review, misuse analysis, and acceptance plan for user-authored hypothetical paper positions. +- Do not implement until that design receives separate explicit approval. It must prohibit recommendations, model-generated sizing, allocation instructions, live holdings, account imports, broker connections, order routing, auto-trading, price-triggered stop/profit instructions, and investment-performance claims. +- Keep hypothetical state isolated from evidence truth, ranking, deterministic forecasts, readiness, and calibration. Live brokerage remains out of scope permanently. +- Exit only after separate approved design and direct acceptance evidence; this continuation prompt does not itself grant that approval. + +Execution order for each continuation: + +1. Verify current branch, worktree, latest commits, upstream alignment, PR #113 draft status, ROADMAP.md, generated-artifact hygiene, current tests, and current product gates. +2. Review unresolved PR feedback and roadmap claims against live code and runtime behavior. +3. Before reusing a supporting proof outcome, run `make proof-readiness-reconciliation TOP_N=20`; keep `historical_supported_currently_blocked` lanes blocked, keep scope-only outcomes non-supporting at ticker level, route each current blocker to its named safe review, and move to fresh evidence or another executable lane. Never infer historical source, rights, scope, or cause from narrative proof; the implemented structured per-ticker/per-field record is prospective-only and does not retroactively upgrade narrative history. +4. Audit the complete user workflow: Research Desk -> Discover -> Company Workbench -> Monitor. + Also verify that README, LinkedIn copy, the curated Workbench image, and share checks describe that primary workflow consistently and keep the Public five-page path secondary. +5. Reverify the Research Decision Lab only to the depth needed for current changed scope. If current evidence remains green, preserve the recorded local milestone and do not repeat its implementation. Do not re-run Priorities 1-3 unless a current regression is directly reproduced. Apply the durable release-routing rule before resuming the first incomplete safe numbered priority. +6. Implement one coherent slice with failing tests first where behavior changes. After it is locally verified, apply the durable release-routing rule, then rescan Priorities 1-10 from the top; resume the earliest incomplete executable item and move to the next safe executable priority whenever its next gate is externally blocked. +7. Preserve independent readiness for actuals, consensus, Revenue, EPS, operating margin, free cash flow, FCF margin, valuation, trusted relationships, peer comparability, peer valuation anchors, catalysts, outcomes, backtesting, and calibration. +8. Keep technical evidence under Advanced unless it is required to explain the primary research answer. +9. Classify unavailable external dependencies once and move to the next executable local roadmap item. +10. Update tests, methodology, provenance, runbooks, ROADMAP.md, and this prompt when the verified stage or continuation contract changes. +11. Stage exact intentional paths only, commit the verified slice, push only the named branch, and update PR #113 while keeping it draft. +12. Continue to the next safe executable item rather than ending merely because one slice is complete. + +Continuation maturity lanes: + +Stage A-G labels are continuation maturity lanes only; they do not replace the numbered Stage 0-6 exit gates below. Stage A is the implemented isolated prospective field-proof primitive, and the Research Decision Lab local milestone is complete when current evidence reconfirms it. Stage B — local field-proof audit and operator hardening is the second approved local priority after legacy surface quarantine: improve read-only audit, error explanation, and operator review ergonomics only, with no readiness mapping. Mapping field proof into proof-readiness reconciliation, Company Workbench, canonical data, or any readiness lane requires a separate design and approval. Later lettered lanes remain planning labels until separately designed; they grant no release-stage exit. + +Numbered release stage gates: + +Stage 0 — Independent engineering legitimacy +- Keep the minimal PR-only workflow read-only, least privilege, and free of providers, readiness generation, schedules, secrets, deployment, and artifact uploads. +- Require a completed GitHub Actions result for the current PR revision; do not infer hosted execution from local workflow tests. +- Require independent human GitHub review of PR #113 and classify it separately from automated checks. +- Hosted automation passed on the recorded implementation lineage. Exit remains revision-specific: require the current revision's direct CI result and an explicitly known review state. A green check does not satisfy human review or any later source, hosted-preview, beta, evidence-depth, calibration, or operating gate. + +Stage 1 — Answer-first workflow hardening +- Exit only when all four Personal Research routes show the primary answer and one next action before technical evidence at desktop and phone widths. +- Current expected state: locally completed, including the verified mobile first-action density slice, Company Workbench anchored-answer correction, and public packaging reconciliation. Treat it as current-revision evidence only after focused/full tests, release and hygiene gates, exact push, and exact-head CI pass; local browser measurements, screenshots, and copy do not satisfy hosted, human-review, adoption, demand, competitive, or market gates. + +Stage 2 — Permitted source activation +- Acquire one permitted append-only prospective point-in-time consensus snapshot for one reviewed ticker. +- Require exact fiscal period, independent Revenue/EPS comparability, durable source reference, publication and retrieval timestamps, provenance, freshness, revision handling, and suitable usage rights. +- Require the exact source-rights record to approve commercial use and list each populated metric scope independently before a Commercial Research append; research-only collection does not satisfy this stage gate. +- Run validate, preview, rejection review, readiness, and provenance checks before any apply decision. +- Treat every supplied consensus CSV as one ordered batch: review batch-level technical and commercial blockers before record, and do not interpret a preflight pass as source, payload, readiness, or calibration proof. +- Treat upstream source validation as technical review plus independent registry evidence: a caller label is never rights proof, and historical-reviewable state is never activation. +- Require an explicit validation cutoff and exact history scope before source review; reject reversed or post-cutoff timestamps for candidate and historical rows alike. +- Require collection preview and upstream validation to retain parity through the shared exact-source field-scope decision; keep their technical acceptance, blocker wording, and write controls independently testable. +- Enforce the source-review-before-preview sequence across distinct input contracts: first set `SOURCE_INPUT=` and run `make earnings-consensus-source-review INPUT=$SOURCE_INPUT PROVIDER= AS_OF=`; separately review the payload and evidence; explicitly map accepted evidence into the existing prospective schema without inference; then set `COLLECTION_INPUT=` and run `make earnings-consensus-collection-preview INPUT=$COLLECTION_INPUT AS_OF=`. No local command performs or writes this mapping. Neither gate records a snapshot or rebuilds readiness. +- After the exact collection preview is separately reviewed, an explicitly authorized operator may run `CONFIRM_REVIEWED=1 make earnings-consensus-collection-record INPUT=$COLLECTION_INPUT AS_OF= PREVIEW_RECEIPT=`. This creates an append-only evidence record only; it does not activate readiness or numerical probability, build a forecast, prove comparability, or satisfy backtesting or calibration. +- Exit only when one repeatable source path demonstrates deterministic provenance, rights, freshness, failure handling, and append-only collection. + +Stage 3 — Controlled hosted preview +- Use the deterministic demo profile; keep secrets and local research outside Git. +- Verify the actual URL, claimed access control, user isolation, health checks, rollback, and desktop/phone workflow. +- Exit only when the hosted runtime directly proves those properties without unsupported claims. + +Stage 4 — Controlled beta validation +- Run 10-20 task-based reviewer sessions through the complete Personal Research workflow. +- Measure time to first useful answer, readiness comprehension, evidence tracing, authoring friction, trust, misuse risk, perceived performance, repeat-use intent, and the most important missing workflow. +- Record reproducible workflow defects only; reviewer feedback is not financial evidence. +- Exit only when reviewers complete the workflow without mistaking readiness, context, scenarios, or evidence gaps for advice or live-market certainty. + +Stage 5 — Evidence-depth expansion +- Run a 25-50 company trusted-peer pilot only after the single reviewed relationship in Priority 5 has direct evidence and trustworthy relationship sourcing and review capacity are demonstrably repeatable. +- Preserve peer role, rationale, comparability, source/as-of evidence, and explicit valuation-anchor decisions. +- Add optional earnings and estimates only from trusted, period-defined, source-backed rows. +- Expand valuation, catalyst, and outcome ledgers only from reviewed evidence. +- Continue chronological backtesting and calibration accumulation without relaxing the probability gate. +- Exit only when evidence depth expands through repeatable reviewed contracts without inferred broad coverage or readiness coupling. + +Stage 6 — Operating maturity and product direction +- Add scheduled collection, rotation, alerts, and monitoring only after one provider path proves deterministic limits, provenance, rejection handling, and proof recording. +- Keep imports behind validate, preview, reviewed apply/skip, rebuilt readiness, and proof recording. +- Directly verify authentication, isolation, audit, retention, entitlements, monitoring, health checks, incident/rollback procedures, and owner capacity before claiming an operated platform. +- Choose explicitly among portfolio-quality prototype, maintained research tool, or operated platform only after hosted-preview, beta, source, peer, and operating evidence exists. + +Verification after every meaningful implementation slice: + +- focused tests for changed modules; +- `python3 -m pytest tests -q`; +- `make dashboard-smoke`; +- `make research-dashboard-render-smoke`; +- `make public-wording-check`; +- `make public-check`; +- `make linkedin-share-check`; +- `make browser-qa-evidence`; +- `make pilot-readiness-check TOP_N=10`; +- `make diff-hygiene-summary`; +- `git diff --check`; +- relevant commercial performance or release gate when workflow/runtime behavior changes; +- `make staged-hygiene-check` and `git diff --cached --check` after exact staging. + +Release-matrix boundary: the controller owns the full release matrix, push, PR +update, and hosted exact-head CI verification. A documentation-slice worker may +run only its focused docs/render checks, stage its exact reviewed files, and +report the local evidence; it must not claim the full release matrix or hosted +CI has passed. + +Git and artifact rules: + +- Never use `git add -A`. +- Stage exact reviewed product, code, documentation, test, and template paths only. +- Never stage broad generated CSV, JSON, readiness-report, stock-report, sample-report, screenshot, or timing churn. One exact curated screenshot may be staged only when it is the explicitly reviewed product asset required by the current slice. +- Commit only coherent verified slices. +- Push only to `codex/personal-research-mode-mvp`. +- Keep PR #113 draft and update it after each verified implementation slice. +- Keep PR #113 open and draft; do not merge, deploy, or mark it ready. +- The same 18 protected generated paths remain excluded, byte-for-byte unchanged, and unstaged unless separately reviewed and explicitly approved: + - `data/analyst_estimates_readiness.csv` + - `data/dcf_readiness.csv` + - `data/earnings_readiness.csv` + - `data/price_coverage_report.csv` + - `data/reports/analyst_estimates_readiness_report.csv` + - `data/reports/data_source_status.csv` + - `data/reports/dcf_readiness_report.csv` + - `data/reports/earnings_readiness_report.csv` + - `data/reports/feature_readiness_summary.csv` + - `data/reports/fundamentals_coverage_report.csv` + - `data/reports/peer_readiness_report.csv` + - `data/reports/peer_unlock_worklist.csv` + - `data/reports/price_coverage_report.csv` + - `data/reports/ticker_readiness_report.csv` + - `data/reports/universe_coverage_report.csv` + - `data/universe_master.csv` + - `outputs/feature_readiness_summary.csv` + - `outputs/peer_unlock_worklist.csv` +- Do not merge into main or deploy publicly without explicit approval. + +Completion audit before any completion claim: + +- Derive every requirement from this goal, ROADMAP.md, PR #113, relevant specs/runbooks, and the current repository. +- Map each requirement and exit gate to authoritative evidence: code, tests, current artifacts, runtime behavior, source rows, provider rights, hosted URL behavior, reviewer records, calibration outputs, or operating controls. +- Classify every item as proven, contradicted, incomplete, indirect, or missing. +- Treat indirect, stale, synthetic-only, fixture-only, screenshot-only, local-only, or repo-contract-only evidence as insufficient for broader claims. +- Keep the goal active whenever any applicable gate remains incomplete or unproven. +- Declare completion only when every applicable stage exit gate has direct current evidence and no required work remains. + +Required handoff after each slice: + +1. Repository and PR status. +2. Current product stage and capabilities. +3. Roadmap item worked. +4. Changes made. +5. Tests, runtime checks, and product gates run. +6. Commit and push status. +7. Generated artifacts excluded. +8. External dependencies classified without repeat loops. +9. Remaining product gaps by stage. +10. Exact next executable step. +11. Whether the branch is safe for draft review. +12. Confirmation that the overall goal remains active unless every completion gate is directly proven. +``` + +The persistence language is deliberately evidence-bound. It prevents a single external dependency from stopping local progress, but it does not authorize fabricated completion, infinite retries, merging, deployment, credential use, destructive actions, or material scope expansion. diff --git a/docs/internal/POINT_IN_TIME_UNIVERSE_REVIEW_HISTORY.md b/docs/internal/POINT_IN_TIME_UNIVERSE_REVIEW_HISTORY.md new file mode 100644 index 000000000..cfd8b50d3 --- /dev/null +++ b/docs/internal/POINT_IN_TIME_UNIVERSE_REVIEW_HISTORY.md @@ -0,0 +1,105 @@ +# Point-in-Time Universe Review History + +This internal audit record preserves completed Priority 4 remediation evidence +that no longer belongs in the active roadmap. + +It is historical engineering evidence, not current source, hosted, reviewer, +market, or calibration evidence. Current execution state and exit gates remain +in `ROADMAP.md`. + +## Review And Remediation Sequence + +The second through fourth fresh whole-branch reviews drove the raw-row rights, +cutoff-relative history, publication chronology, immutable bounded-read, +aggregate-budget, and structured-input parser closures. + +The fifth fresh whole-branch review confirmed those closures and found three +Important trust-boundary defects: C0/C1 characters in structural identifiers +could render the newline-delimited membership digest ambiguous and forge +public status lines, while manifest creation could predate its cutoff or bound +evidence. + +Commits `b2bbd9961` and `c643d066b` remediate those V5 findings locally with +one shared C0/C1 plus Unicode line/paragraph-separator boundary, safe +structural-token rendering, an explicit creation-at-or-after-cutoff manifest +gate, and exact-row chronology against every contract timestamp. + +The first independent R7 review found the Unicode separator and +`listing_state_after` bypass gaps; `c643d066b` closes them locally. + +The sixth fresh whole-branch review confirmed those closures and found one +remaining Important non-scalar input defect: lone Unicode surrogate code +points could reach public output. + +Commit `f143d48ed` rejects Unicode category `Cs` through the shared boundary +and defensively ASCII-escapes it while valid supplementary-plane scalars +remain deterministic. + +The seventh fresh whole-branch review confirmed the V6 correction and found +four further trust-boundary defects (two Critical, one Important, and one +Minor): duplicate JSON/YAML mapping keys could silently change manifest and +rights meaning; invalid or unresolved successor and listing-state evidence +could authorize stale original-member digests; malformed CSV headers could +discard contract bodies and continue; and non-RFC3339 manifest or policy +timestamps were accepted. + +The local seventh-review remediation rejects duplicate keys at every mapping +depth, requires strict RFC3339 UTC manifest and policy timestamps with at most +six fractional-second digits, stops malformed headers as package-level +input-identity failures, and enforces explicit policy/event/listing-state, +successor-identity, and membership-consistency gates without inferring or +repairing a successor or membership. + +An independent scoped re-review then confirmed the four original findings and +the two compatibility regressions were addressed. + +The eighth fresh whole-branch review then found three Critical, nine Important, +and two Minor defects across sub-microsecond ordering, event-time identity, +listing chronology and rights, walk-forward bootstrap aggregation, +identity/action reconciliation, eligible provenance, package-contained bounded +reads, manifest type handling, standalone rights loading, and literal-safe Make +arguments. + +Remediation 9A through 9G closed every finding test-first. Independent scoped +re-reviews confirmed no remaining Critical or Important finding in each +corrected scope; the two Minor contracts now reject identical issuer/security +IDs and recursively freeze manifest semantics. + +Freeze reconciliation consolidated 21 overlapping remediation test files into +six domain suites and one shared fixture module, removed one exact duplicate +plus cross-remediation private imports, and closed five additional local +correctness gaps: ambiguous parents cannot authorize forks; pre-action cutoffs +do not poison later required coverage; decision-consumed listing-state evidence +is retained in eligible provenance; manifest nesting is explicitly bounded; +and structural source IDs cannot forge status output. + +Full branch verification at freeze reconciliation is 4,084 passing tests, one +environment-limited socket test skipped, and one existing dependency +deprecation warning. + +The final fresh whole-slice review found one Important cutoff-relative event +regression; it was reproduced, fixed, and confirmed closed with no remaining +Critical or Important issue. The consolidated package was synchronized at +`69c49968e77bfd55fa259695089e1f34ac2fddfb`, and exact-head GitHub Actions run +`30185232040` passed the full test, dashboard-startup, Personal Research +render, public-wording, PR-range generated-artifact hygiene, and whitespace +matrix. + +Real-data evidence remains pending; Priority 4 remains externally incomplete. + +## Resource And Input Boundaries + +Local resource budgets for one supplied package: preview sample 100 rows; +manifest 1 MiB; each contract CSV 32 MiB; four contract snapshots combined 64 +MiB; source-rights registry 4 MiB; declared rows 250,000 per contract; package +traversal 32 entries. + +Duplicate JSON/YAML mapping keys and malformed contract headers also fail +nonzero, traceback-free, and write-free through the direct validator and +CLI/Make boundaries. + +These local bounds do not prove scale, hosted reliability, or market +readiness. + +No permitted independently reviewed real dataset, accepted expected +count/digest, or source-rights proof is on record. diff --git a/docs/internal/RESEARCH_DECISION_LAB_CONTINUATION_GOAL_PROMPT.md b/docs/internal/RESEARCH_DECISION_LAB_CONTINUATION_GOAL_PROMPT.md new file mode 100644 index 000000000..0310254ea --- /dev/null +++ b/docs/internal/RESEARCH_DECISION_LAB_CONTINUATION_GOAL_PROMPT.md @@ -0,0 +1,114 @@ +# Research Decision Lab Continuation Goal Prompt + +Use this prompt to reverify the completed local Decision Lab and continue the next Commercial Research Beta maturity gate without repeating finished implementation. + +```text +/goal + +Continue the Stock Research Command Center in: + +/Users/yjian070/Documents/New project/.worktrees/personal-research-mode-mvp + +Start from current repository truth, never chat memory. + +Read: + +- docs/superpowers/specs/2026-07-22-research-decision-lab-design.md +- docs/superpowers/plans/2026-07-22-research-decision-lab.md +- ROADMAP.md +- docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md +- docs/METHODOLOGY.md +- docs/PROVENANCE_CONTRACT.md +- docs/PERSONAL_RESEARCH_MODE.md + +Expected state to verify, never assume: + +- Branch: codex/personal-research-mode-mvp +- Decision Lab Monitor anchor: c7ad977b3 or a later verified descendant +- Draft PR: https://github.com/YuzeJ21/Stock-Analysis/pull/113 +- PR remains open and draft +- Branch is pushed and aligned with origin +- Existing generated CSV/JSON/report/sample-report/screenshot/timing churn remains local and unstaged + +Current local milestone: + +Local Decision Lab implementation is complete when current evidence reconfirms all of the following: + +1. The immutable six-lane composition preserves independent Plan, Evidence, Invalidation, Scenario, Review trigger, and Learning states. +2. Company Workbench keeps Selected ticker -> Use now -> Still withheld -> Data Health first, then renders exactly one Decision Lab after What Changed. +3. Research Conclusion and Next Research Task remain authoritative and separate from the next process step. +4. Monitor keeps Weekly Research Summary first, then Research Discipline Review, then the independent Research change monitor. +5. Monitor preserves saved focused-cohort order, isolates an invalid ticker, and never ranks by severity, market value, expected return, or attractiveness. +6. Empty process evidence says only that no process item is due from saved reviewer-authored evidence; it makes no market-event, risk, or external-research claim. +7. Identities and technical evidence remain under collapsed Advanced sections. +8. The feature writes no journal, outcome, source, readiness, proof, report, screenshot, timing, or generated data artifact. + +Do not reimplement or redesign this milestone if current tests, runtime behavior, docs, branch state, and exact-head CI still prove it. Fix only a directly reproduced regression. + +Objective: + +Reverify the Decision Lab release evidence, then advance the highest-value executable Commercial Research Beta gate. Prefer a safe local reliability, evidence-depth, or operating-contract slice already approved by ROADMAP.md. If no such local slice remains, classify the exact external dependency and its unblock condition, then move to another executable lane. Never convert an unavailable provider, dataset, hosted account, reviewer, peer source, calibration cohort, or operating owner into fabricated completion evidence. + +Persistence contract: + +1. Verify branch, HEAD, upstream alignment, working tree, latest commits, PR #113 status, ROADMAP.md, generated-artifact hygiene, tests, runtime gates, and exact-head CI before editing. +2. Continue automatically while any safe, meaningful, approved local task remains. +3. Classify each unavailable external dependency once, record its exact unblock condition, avoid identical retry loops, and immediately continue another executable lane. +4. Work test-first, one coherent independently verified slice at a time. +5. After each slice, run focused and full tests, required renders, wording, performance, release, pilot-boundary, PR-range, diff, whitespace, and staged-hygiene checks. +6. Stage exact intentional code/test/docs/template paths only. Never use git add -A. +7. Commit and push only to codex/personal-research-mode-mvp, update draft PR #113, and require exact-head CI before the next slice. +8. Keep PR #113 open and draft. Do not merge or deploy publicly without explicit approval. +9. Do not run make readiness, broad refresh/import/apply commands, or generate/stage CSV, JSON, readiness reports, stock reports, sample reports, screenshots, or timing artifacts for this continuation. +10. Do not claim the broader goal complete while any applicable source, hosted, reviewer, calibration, demand, or operating gate lacks direct evidence. + +Research-only boundaries: + +- No investment advice, recommendation, company grade, expected-return score, or research-skill score. +- No buy/sell/hold/add/trim/reduce direction, allocation, recommended position size, entry, exit, stop-loss, take-profit, live holding, account import, cost basis, broker integration, order routing, auto-trading, or real transaction record. +- Candidate context cannot populate trusted evidence, change deterministic forecasts, or promote readiness. +- Synthetic fixtures are test-only; empty ledgers remain empty. +- Q4 actuals require explicit SEC-filed Q4-table evidence; EPS split basis remains unverified without explicit proof. +- Real-company Earnings Nowcast remains blocked without permitted point-in-time consensus; numerical Beat/Miss probability remains withheld without leakage-safe calibration evidence. +- Every readiness and process lane remains independent. + +Required verification for a changed slice: + +- focused tests for changed modules and contracts +- python3 -m pytest tests -q +- make dashboard-smoke +- make research-dashboard-render-smoke +- make public-wording-check +- make public-check +- make browser-qa-evidence +- make commercial-beta-performance-gate when Workbench or Monitor changes +- make commercial-beta-release-check +- make pilot-readiness-check TOP_N=10 +- make pr-range-hygiene-check +- make diff-hygiene-summary +- git diff --check +- make staged-hygiene-check and git diff --cached --check after exact staging + +Truthful maturity boundary: + +The local Decision Lab does not prove source coverage, predictive accuracy, investment performance, independent adoption, hosted reliability, commercial demand, competitive superiority, or product-market fit. Passing local tests or exact-head CI cannot satisfy those external gates. + +At each handoff report: + +1. Repository and PR status. +2. Current product stage. +3. Decision Lab regression audit result. +4. New maturity slice completed, if any. +5. Focused, full, runtime, performance, wording, release, and hygiene evidence. +6. Commit, push, and exact-head CI status. +7. Generated artifacts excluded. +8. External dependencies classified once with exact unblock conditions. +9. Remaining applicable gates. +10. Exact next executable step. +11. Whether the branch is safe for draft review. +12. Whether the local Decision Lab goal is complete and whether the broader Commercial Research Beta goal remains active. + +Do not stop after a green Decision Lab recheck while another safe approved local task remains. Do not claim external completion. If every remaining task genuinely requires unavailable external evidence or new authority, leave the broader goal active, publish the exact dependency ledger and resume condition, and do not retry unchanged blockers. +``` + +This continuation is persistent but evidence-bound: it prevents one external dependency from stopping other work without redefining completion or weakening the research-only contract. diff --git a/docs/superpowers/plans/2026-07-16-sec-quarterly-actuals-lineage.md b/docs/superpowers/plans/2026-07-16-sec-quarterly-actuals-lineage.md new file mode 100644 index 000000000..8b4b10c76 --- /dev/null +++ b/docs/superpowers/plans/2026-07-16-sec-quarterly-actuals-lineage.md @@ -0,0 +1,638 @@ +# SEC Quarterly Actuals Lineage Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a preview-only SEC evidence pipeline that stages correctly identified Q1-Q4 Revenue and diluted GAAP EPS actuals for the five-company semiconductor Earnings Nowcast pilot without cumulative facts, period relabeling, split leakage, quarter gaps, or inferred Q4 values. + +**Architecture:** A focused `earnings_nowcast_sec_actuals` module normalizes raw Companyfacts records, establishes fiscal-quarter lineage from original current-quarter filings, preserves revisions, discovers explicit Q4 earnings-release exhibits, and writes only onboarding-compatible temporary evidence plus audit output. Existing readiness and model modules gain metric-specific continuity and split-basis gates; existing onboarding remains the final validate/preview boundary. + +**Tech Stack:** Python 3.12, standard-library dataclasses/HTMLParser/urllib, existing SEC provider cache APIs, CSV/JSON, pytest, Make. + +## Global Constraints + +- Research-only; no investment advice, broker integration, order routing, auto-trading, or direct buy/sell instructions. +- Never infer Revenue, EPS, Q4 values, fiscal periods, consensus, probabilities, or recommendations. +- Q4 must come from an explicit result table in an SEC-filed primary-source exhibit; annual-minus-nine-month arithmetic is prohibited. +- Historical evidence is append-only and cutoff-aware; later revisions never overwrite earlier evidence. +- Revenue and EPS readiness remain independent. +- Generated SEC staging CSV/JSON/rejected-row artifacts stay outside default staging. +- No automatic apply path is added. +- Historical point-in-time consensus remains an external input and must continue to block packet generation when absent. +- Numerical Beat/Miss probability remains hidden until at least 100 valid events pass calibration gates. + +## File Map + +- Create `src/earnings_nowcast_sec_actuals.py`: normalized fact types, Q1-Q3 lineage, SEC filing-index/exhibit parsing, Q4 extraction, staging CLI, and audit payload. +- Create `tests/test_earnings_nowcast_sec_actuals.py`: fixture-driven parser, lineage, revision, Q4, cutoff, and output-boundary tests. +- Modify `src/earnings_nowcast_readiness.py`: metric-specific fiscal-quarter continuity and split-basis eligibility. +- Modify `src/earnings_nowcast_model.py`: consume only the contiguous canonical metric window. +- Modify `tests/test_earnings_nowcast_readiness.py`: missing-quarter and split-basis regression tests. +- Modify `tests/test_earnings_nowcast_model.py`: prove no Q3-to-next-Q1 sequential calculation. +- Modify `tests/test_earnings_nowcast_backtest.py`: prove continuity and revisions remain cutoff-safe in walk-forward evaluation. +- Modify `Makefile`: add the narrow, read-only `earnings-nowcast-sec-actuals-stage` launcher. +- Modify `tests/test_launchers.py`: lock launcher arguments and no-apply boundary. +- Modify `docs/EARNINGS_NOWCAST_PILOT.md`, `docs/METHODOLOGY.md`, `docs/PROVENANCE_CONTRACT.md`, and `ROADMAP.md`: document actuals lineage, Q4, split, continuity, and remaining consensus dependency. + +--- + +### Task 1: Normalize SEC facts and extract Q1-Q3 fiscal-quarter lineage + +**Files:** +- Create: `src/earnings_nowcast_sec_actuals.py` +- Create: `tests/test_earnings_nowcast_sec_actuals.py` + +**Interfaces:** +- Consumes: raw SEC Companyfacts JSON and an ISO cutoff timestamp. +- Produces: `normalize_sec_duration_facts(payload: Mapping[str, object]) -> tuple[SecDurationFact, ...]`. +- Produces: `extract_q1_q3_lineage(ticker: str, payload: Mapping[str, object], *, cutoff: str, retrieved_at: str) -> ExtractionResult`. +- `ExtractionResult.rows` contains onboarding `QuarterlyActual` values; `ExtractionResult.audit_rows` records accepted and rejected source facts. + +- [ ] **Step 1: Write failing normalization and cumulative-fact tests** + +Create fixtures directly in `tests/test_earnings_nowcast_sec_actuals.py` so one Q3 accession contains: + +```python +def _fact(*, val, start, end, filed="2026-06-25", fy=2026, fp="Q3", frame=None): + return { + "val": val, + "start": start, + "end": end, + "filed": filed, + "form": "10-Q", + "accn": "0000000000-26-000001", + "fy": fy, + "fp": fp, + "frame": frame, + } + + +def test_q3_lineage_keeps_aligned_quarter_and_rejects_ytd_and_comparative_period(): + payload = companyfacts_fixture( + revenue=[ + _fact(val=30, start="2025-08-29", end="2026-05-28"), + _fact(val=12, start="2026-02-27", end="2026-05-28", frame="CY2026Q2"), + _fact(val=8, start="2025-02-28", end="2025-05-29", frame="CY2025Q2"), + ], + eps=[ + _fact(val=3.0, start="2025-08-29", end="2026-05-28"), + _fact(val=1.2, start="2026-02-27", end="2026-05-28", frame="CY2026Q2"), + _fact(val=0.8, start="2025-02-28", end="2025-05-29", frame="CY2025Q2"), + ], + ) + + result = extract_q1_q3_lineage("SYN1", payload, cutoff=CUTOFF, retrieved_at=RETRIEVED_AT) + + assert [(row.fiscal_period, row.revenue_actual, row.eps_actual) for row in result.rows] == [ + ("2026-Q3", 12.0, 1.2) + ] + assert {row.state for row in result.audit_rows} >= { + "accepted_explicit_quarter", + "cumulative_fact_rejected", + "comparative_period_relabelled", + } +``` + +- [ ] **Step 2: Run the focused test and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_earnings_nowcast_sec_actuals.py::test_q3_lineage_keeps_aligned_quarter_and_rejects_ytd_and_comparative_period -q +``` + +Expected: collection/import failure because `src.earnings_nowcast_sec_actuals` does not exist. + +- [ ] **Step 3: Implement normalized fact types and deterministic Q1-Q3 selection** + +Implement these public types and functions: + +```python +@dataclass(frozen=True) +class SecDurationFact: + taxonomy: str + concept: str + unit: str + value: float + start: str + end: str + filed: str + form: str + accession: str + fiscal_year: int + fiscal_period: str + frame: str + + @property + def duration_days(self) -> int: + return (date.fromisoformat(self.end) - date.fromisoformat(self.start)).days + + +@dataclass(frozen=True) +class ExtractionAuditRow: + ticker: str + state: str + metric: str + fiscal_period: str + source_ref: str + detail: str + + +@dataclass(frozen=True) +class ExtractionResult: + rows: tuple[QuarterlyActual, ...] + audit_rows: tuple[ExtractionAuditRow, ...] + + +def normalize_sec_duration_facts(payload: Mapping[str, object]) -> tuple[SecDurationFact, ...]: ... + + +def extract_q1_q3_lineage( + ticker: str, + payload: Mapping[str, object], + *, + cutoff: str, + retrieved_at: str, +) -> ExtractionResult: ... +``` + +Implementation rules: + +- accept only `10-Q` and `10-Q/A` numeric duration facts; +- accept Revenue concepts in the design's priority order and diluted EPS only; +- accept 60-120-day quarter durations; +- choose the latest `end` within each accession/`fy`/`fp` as the current quarter; +- pair metrics only when accession/start/end/filed/`fy`/`fp` match; +- reject conflicting frame or concept values; +- construct the SEC accession URL from CIK and accession; +- set date-only Companyfacts `reported_at` metadata to `23:59:59Z` on the filed date and reject rows after cutoff; +- preserve a metric-only partial row when just Revenue or EPS is proven. + +- [ ] **Step 4: Add missing-frame, concept-conflict, partial-row, and cutoff tests** + +Add tests proving: + +```python +assert missing_frame_result.rows[0].revenue_actual == 12.0 +assert ambiguous_result.rows[0].revenue_actual is None +assert revenue_only_result.rows[0].eps_actual is None +assert post_cutoff_result.rows == () +assert "post_cutoff_rejected" in {row.state for row in post_cutoff_result.audit_rows} +``` + +Add a two-filing lineage test where the original Q2 filing establishes +`2025-Q2` for a period end and a later Q3 filing presents that same period as +comparative data with a different SEC `fy`/`fp`. Assert that the later row keeps +the original `2025-Q2` identity and becomes a revision candidate instead of +being relabelled from the later filing's metadata. Add a fail-closed case where +no original current-quarter filing can establish the comparative period. + +- [ ] **Step 5: Run the Task 1 tests and verify GREEN** + +Run: + +```bash +python3 -m pytest tests/test_earnings_nowcast_sec_actuals.py -q +``` + +Expected: all Task 1 tests pass. + +- [ ] **Step 6: Commit Task 1** + +```bash +git add -- src/earnings_nowcast_sec_actuals.py tests/test_earnings_nowcast_sec_actuals.py +git commit -m "Extract SEC quarterly actual lineage" +``` + +--- + +### Task 2: Preserve append-only revisions and write preview-only staging artifacts + +**Files:** +- Modify: `src/earnings_nowcast_sec_actuals.py` +- Modify: `tests/test_earnings_nowcast_sec_actuals.py` + +**Interfaces:** +- Consumes: per-ticker `ExtractionResult` objects and an explicit output directory. +- Produces: `link_quarter_revisions(rows: Sequence[QuarterlyActual]) -> tuple[QuarterlyActual, ...]`. +- Produces: `write_sec_actuals_stage(output_dir: Path, results: Mapping[str, ExtractionResult]) -> StageResult`. +- Produces: `stage_sec_quarterly_actuals(tickers: Sequence[str], *, output_dir: Path, cutoff: str, user_agent: str | None, ...) -> StageResult`. + +- [ ] **Step 1: Write failing revision-lineage tests** + +```python +def test_later_changed_presentation_is_append_only_revision(): + original = actual("2025-Q2", revenue=100, eps=1.0, source_ref="sec://original", reported_at="2025-08-01T00:00:00Z") + revised = actual("2025-Q2", revenue=100, eps=0.1, source_ref="sec://split-adjusted", reported_at="2025-11-01T00:00:00Z") + + linked = link_quarter_revisions([original, revised]) + + assert len(linked) == 2 + assert linked[1].supersedes_source_ref == original.source_ref + assert linked[0].eps_actual == 1.0 +``` + +Also test unchanged later presentations are de-duplicated and an unrelated conflicting source is not silently marked as a revision. + +- [ ] **Step 2: Run the revision test and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_earnings_nowcast_sec_actuals.py::test_later_changed_presentation_is_append_only_revision -q +``` + +Expected: FAIL because `link_quarter_revisions` is missing. + +- [ ] **Step 3: Implement revision linking and output types** + +```python +@dataclass(frozen=True) +class StageResult: + requested_tickers: tuple[str, ...] + accepted_tickers: tuple[str, ...] + withheld_tickers: tuple[str, ...] + accepted_row_count: int + rejected_row_count: int + quarterly_actuals_path: str + audit_path: str + rejected_path: str + automatic_apply: bool = False +``` + +Only link a later row as a revision when ticker/fiscal period/source family match and the changed row is later by `reported_at`. Keep the original row. Never mutate canonical data. + +- [ ] **Step 4: Write failing output-boundary tests** + +```python +def test_stage_writes_only_explicit_output_directory(tmp_path): + result = write_sec_actuals_stage(tmp_path / "stage", {"SYN1": extraction_result()}) + + assert Path(result.quarterly_actuals_path).parent == tmp_path / "stage" + assert (tmp_path / "stage" / "quarterly_actuals.csv").exists() + assert (tmp_path / "stage" / "consensus_snapshots.csv").read_text().count("\n") == 1 + assert result.automatic_apply is False + assert not (tmp_path / "data").exists() +``` + +Verify audit JSON contains raw concept/start/end/frame/accession metadata and rejected CSV contains explicit reason codes. + +- [ ] **Step 5: Implement CSV/JSON output and stage orchestrator** + +Reuse `SCHEMAS` and `EVIDENCE_SCHEMA_VERSION` from `src.earnings_nowcast_onboarding`. Reuse `load_sec_ticker_map` and `fetch_companyfacts` from `src.providers.sec_companyfacts`. Dependency-inject ticker-map and Companyfacts fetchers for tests. Require `output_dir`; do not default to `data/imports`. + +- [ ] **Step 6: Run Task 2 tests and existing onboarding tests** + +```bash +python3 -m pytest tests/test_earnings_nowcast_sec_actuals.py tests/test_earnings_nowcast_onboarding.py -q +``` + +Expected: all pass and no file appears outside pytest temporary directories. + +- [ ] **Step 7: Commit Task 2** + +```bash +git add -- src/earnings_nowcast_sec_actuals.py tests/test_earnings_nowcast_sec_actuals.py +git commit -m "Stage SEC actuals as append-only evidence" +``` + +--- + +### Task 3: Enforce metric-specific quarter continuity and split safety + +**Files:** +- Modify: `src/earnings_nowcast_readiness.py` +- Modify: `src/earnings_nowcast_model.py` +- Modify: `tests/test_earnings_nowcast_readiness.py` +- Modify: `tests/test_earnings_nowcast_model.py` +- Modify: `tests/test_earnings_nowcast_backtest.py` + +**Interfaces:** +- Produces: `contiguous_metric_window(rows: Sequence[QuarterlyActual], target_period: str, metric: str, minimum_quarters: int) -> tuple[QuarterlyActual, ...]`. +- `assess_nowcast_readiness` adds `quarter_history_gap` and `incompatible_eps_definition` without changing existing public state names. +- `build_baseline_nowcast` uses the contiguous windows returned by the readiness helper. + +- [ ] **Step 1: Write failing continuity tests** + +```python +def test_missing_q4_withholds_both_metrics_instead_of_treating_q3_to_q1_as_sequential(): + rows = [row for row in _actuals() if row.fiscal_period != "2024-Q4"] + + result = assess_nowcast_readiness( + ticker="SYN1", + fiscal_period="2026-Q1", + as_of_timestamp=CUTOFF, + actuals=rows, + consensus=[_consensus()], + ) + + assert result.revenue_ready is False + assert result.eps_ready is False + assert "quarter_history_gap" in result.missing_evidence +``` + +Add a model regression test that monkeypatches or directly checks the selected input periods and proves Q3-to-Q1 is never passed to `_sequential_growth`. + +- [ ] **Step 2: Run the continuity tests and verify RED** + +```bash +python3 -m pytest tests/test_earnings_nowcast_readiness.py::test_missing_q4_withholds_both_metrics_instead_of_treating_q3_to_q1_as_sequential -q +``` + +Expected: FAIL because current readiness counts five rows without checking adjacency. + +- [ ] **Step 3: Implement fiscal-period adjacency and contiguous windows** + +```python +def _next_period(period: str) -> str: + year, quarter = period.split("-Q") + return f"{int(year) + 1}-Q1" if quarter == "4" else f"{year}-Q{int(quarter) + 1}" + + +def contiguous_metric_window(rows, target_period, metric, minimum_quarters): + eligible = [row for row in rows if getattr(row, f"{metric}_actual") is not None] + # Walk backward from the period immediately before target_period. + # Return an empty tuple when any required period is missing. +``` + +Use the contiguous window for history sufficiency, stability checks, model inputs, and source IDs. Keep Revenue and EPS windows separate. + +- [ ] **Step 4: Write failing split-basis tests** + +```python +def test_split_basis_change_withholds_eps_but_keeps_revenue_ready(): + rows = _actuals() + rows[0] = replace(rows[0], split_adjustment_basis="pre_split") + consensus = replace(_consensus(), split_adjustment_basis="post_split_2024_06_10") + + result = assess_nowcast_readiness(...) + + assert result.revenue_ready is True + assert result.eps_ready is False + assert "incompatible_eps_definition" in result.missing_evidence +``` + +Also prove a source-backed append-only split-adjusted revision restores a consistent EPS window before the cutoff, while a post-cutoff revision does not. + +- [ ] **Step 5: Implement split-basis filtering and cutoff-safe revision selection** + +Reuse `_metric_definition` and `canonicalize_actuals`; do not add ticker-specific split constants to model code. The evidence row and consensus row carry the basis. If no five-quarter contiguous compatible EPS window exists, withhold EPS. + +- [ ] **Step 6: Run readiness, model, and backtest suites** + +```bash +python3 -m pytest \ + tests/test_earnings_nowcast_readiness.py \ + tests/test_earnings_nowcast_model.py \ + tests/test_earnings_nowcast_backtest.py -q +``` + +Expected: all pass; existing leakage and probability gates remain unchanged. + +- [ ] **Step 7: Commit Task 3** + +```bash +git add -- src/earnings_nowcast_readiness.py src/earnings_nowcast_model.py \ + tests/test_earnings_nowcast_readiness.py tests/test_earnings_nowcast_model.py \ + tests/test_earnings_nowcast_backtest.py +git commit -m "Require contiguous nowcast quarter history" +``` + +--- + +### Task 4: Ingest explicit Q4 results from SEC-filed earnings-release exhibits + +**Files:** +- Modify: `src/earnings_nowcast_sec_actuals.py` +- Modify: `src/providers/sec_submissions.py` +- Modify: `tests/test_earnings_nowcast_sec_actuals.py` +- Modify: `tests/test_sec_submissions.py` + +**Interfaces:** +- Produces: `sec_filing_index_url(cik: str, accession: str) -> str` in `src.providers.sec_submissions`. +- Produces: `extract_filing_exhibits(index_html: str, *, cik: str, accession: str) -> tuple[FiledExhibit, ...]`. +- Produces: `extract_explicit_q4_actual(ticker: str, exhibit: FiledExhibit, document_text: str, *, fiscal_period: str, filed_at: str, retrieved_at: str) -> ExtractionResult`. +- `stage_sec_quarterly_actuals` combines Q1-Q3 lineage and explicit Q4 rows before revision linking. + +- [ ] **Step 1: Write failing filing-index and Q4 table tests** + +Use a fixture index containing an `EX-99.1` link and an earnings-release fixture containing: + +```html +

Fourth Quarter Fiscal 2025 Summary

+ + + + +
Q4 FY25
Revenue$39,331 million
GAAP diluted earnings per share$0.89
+

All per-share amounts are retrospectively adjusted for the ten-for-one split effective June 7, 2024.

+``` + +Assertions: + +```python +assert exhibits[0].document_type == "EX-99.1" +assert result.rows[0].fiscal_period == "2025-Q4" +assert result.rows[0].revenue_actual == 39_331_000_000 +assert result.rows[0].eps_actual == 0.89 +assert result.rows[0].split_adjustment_basis == "split_adjusted_2024_06_07" +``` + +- [ ] **Step 2: Run Q4 tests and verify RED** + +```bash +python3 -m pytest tests/test_earnings_nowcast_sec_actuals.py -k 'filing_index or explicit_q4' -q +``` + +Expected: FAIL because filing-index and Q4 interfaces are missing. + +- [ ] **Step 3: Implement filing-index URL and standard-library table extraction** + +Use `html.parser.HTMLParser`; do not add a new runtime dependency. The parser must preserve table row/cell text and nearby quarter/split labels. Normalize commas, currency symbols, parentheses, and explicit `million` / `billion` scale only after the metric label is matched. + +- [ ] **Step 4: Add fail-closed Q4 cases** + +Add tests proving these produce no actual row and explicit audit states: + +- annual total without a Q4 column; +- guidance table (`outlook`, `expected`, `approximately`); +- non-GAAP EPS without GAAP diluted EPS; +- Q4 derived only by subtraction; +- ambiguous period header; +- source filed after cutoff; +- missing split note labels EPS `as_reported`; the readiness split-basis gate, + rather than the parser, withholds EPS when the surrounding lineage requires + an incompatible split-adjusted basis. + +- [ ] **Step 5: Implement exhibit discovery and Q4 selection** + +Search `8-K` / `8-K/A` filing index documents for `EX-99`, `EX-99.1`, or `EX-99.2` result exhibits. Require exact source URL, accession, filed date, quarter label, and metric labels. Issuer aliases belong in a small immutable table of labels, never values. If multiple exhibits disagree, reject Q4 as `ambiguous_concept`. + +- [ ] **Step 6: Run Q4 and SEC provider tests** + +```bash +python3 -m pytest tests/test_earnings_nowcast_sec_actuals.py tests/test_sec_submissions.py -q +``` + +Expected: all pass without network access. + +- [ ] **Step 7: Commit Task 4** + +```bash +git add -- src/earnings_nowcast_sec_actuals.py src/providers/sec_submissions.py \ + tests/test_earnings_nowcast_sec_actuals.py tests/test_sec_submissions.py +git commit -m "Stage explicit SEC filed Q4 actuals" +``` + +--- + +### Task 5: Add launcher, documentation, and five-company live verification + +**Files:** +- Modify: `Makefile` +- Modify: `tests/test_launchers.py` +- Modify: `docs/EARNINGS_NOWCAST_PILOT.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `ROADMAP.md` +- Modify: `src/earnings_nowcast_sec_actuals.py` +- Modify: `tests/test_earnings_nowcast_sec_actuals.py` + +**Interfaces:** +- CLI: `python3 -m src.earnings_nowcast_sec_actuals --tickers ... --output-dir ... --cutoff ...`. +- Make: `make earnings-nowcast-sec-actuals-stage TICKERS=... OUTPUT_DIR=... AS_OF=...`. +- JSON summary exposes per-ticker accepted rows, rejected rows, missing Q4, continuity gaps, source refs, and `automatic_apply=false`. + +- [ ] **Step 1: Write failing launcher tests** + +Add to `tests/test_launchers.py`: + +```python +assert "earnings-nowcast-sec-actuals-stage" in makefile +assert "TICKERS is required" in makefile +assert "OUTPUT_DIR is required" in makefile +assert "--cutoff $(AS_OF)" in makefile +assert "imports-apply" not in target_body(makefile, "earnings-nowcast-sec-actuals-stage") +``` + +Add a CLI test that dependency-injects cached fixtures and confirms JSON output contains `automatic_apply: false`. + +- [ ] **Step 2: Run launcher tests and verify RED** + +```bash +python3 -m pytest tests/test_launchers.py tests/test_earnings_nowcast_sec_actuals.py -q +``` + +Expected: launcher assertion fails. + +- [ ] **Step 3: Implement CLI and Make target** + +Add to `.PHONY` and Makefile: + +```make +earnings-nowcast-sec-actuals-stage: +ifndef TICKERS + $(error TICKERS is required, for example: make earnings-nowcast-sec-actuals-stage TICKERS=NVDA OUTPUT_DIR=/tmp/sec-actuals AS_OF=2026-07-16T03:59:59Z) +endif +ifndef OUTPUT_DIR + $(error OUTPUT_DIR is required; use a generated temporary/review directory) +endif +ifndef AS_OF + $(error AS_OF is required for cutoff-safe evidence staging) +endif + @python3 -m src.earnings_nowcast_sec_actuals --tickers "$(TICKERS)" --output-dir "$(OUTPUT_DIR)" --cutoff "$(AS_OF)" +``` + +The Python CLI accepts `--no-network`, `--sec-refresh`, and `--json`. It never accepts an apply flag. + +- [ ] **Step 4: Update methodology and roadmap truth** + +Document: + +- Q1-Q3 duration and fiscal-lineage rules; +- explicit Q4 source rule and no-derivation boundary; +- revision/cutoff behavior; +- quarter-continuity and EPS split-basis withholding; +- five-company pilot scope; +- real output still `awaiting_point_in_time_consensus`; +- probability still `awaiting_calibration_evidence`. + +Do not claim real Nowcast coverage merely because actuals stage successfully. + +- [ ] **Step 5: Run focused and full deterministic verification** + +```bash +python3 -m pytest \ + tests/test_earnings_nowcast_sec_actuals.py \ + tests/test_earnings_nowcast_onboarding.py \ + tests/test_earnings_nowcast_readiness.py \ + tests/test_earnings_nowcast_model.py \ + tests/test_earnings_nowcast_backtest.py \ + tests/test_sec_submissions.py \ + tests/test_launchers.py -q +python3 -m pytest tests -q +make public-wording-check +make dashboard-smoke +make browser-qa-evidence +make public-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all tests and gates pass. Generated readiness/report files remain unstaged. + +- [ ] **Step 6: Run the live read-only five-company smoke** + +```bash +rm -rf /tmp/stock-nowcast-five-company/sec-lineage-stage +make earnings-nowcast-sec-actuals-stage \ + TICKERS=NVDA,AMD,AVGO,MU,QCOM \ + OUTPUT_DIR=/tmp/stock-nowcast-five-company/sec-lineage-stage \ + AS_OF=2026-07-16T03:59:59Z +make earnings-nowcast-validate \ + INPUT_DIR=/tmp/stock-nowcast-five-company/sec-lineage-stage \ + AS_OF=2026-07-16T03:59:59Z +make earnings-nowcast-preview \ + INPUT_DIR=/tmp/stock-nowcast-five-company/sec-lineage-stage \ + AS_OF=2026-07-16T03:59:59Z +``` + +Inspect per ticker: + +- at least five consecutive Revenue quarters or an explicit truthful gap; +- EPS ready only within one source-backed split basis; +- Q4 rows have exact SEC-filed exhibit URLs; +- zero post-cutoff rows; +- zero cumulative facts; +- preview remains `ready_for_packet=false` with `point_in_time_consensus_missing` when consensus is absent. + +Do not stage `/tmp` output. + +- [ ] **Step 7: Commit Task 5** + +```bash +git add -- Makefile ROADMAP.md \ + docs/EARNINGS_NOWCAST_PILOT.md docs/METHODOLOGY.md docs/PROVENANCE_CONTRACT.md \ + src/earnings_nowcast_sec_actuals.py tests/test_earnings_nowcast_sec_actuals.py \ + tests/test_launchers.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add SEC quarterly actuals staging workflow" +``` + +## Final Review Gate + +After all task commits: + +```bash +git status --short --branch +git log -8 --oneline +git diff origin/main...HEAD --stat +python3 -m pytest tests -q +make public-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +The feature is complete only when fixture validation, full regression gates, and live read-only evidence agree. Real Earnings Nowcast remains blocked until historical point-in-time consensus snapshots are separately source-backed and validated. diff --git a/docs/superpowers/plans/2026-07-17-commercial-research-beta-foundation.md b/docs/superpowers/plans/2026-07-17-commercial-research-beta-foundation.md new file mode 100644 index 000000000..49433e257 --- /dev/null +++ b/docs/superpowers/plans/2026-07-17-commercial-research-beta-foundation.md @@ -0,0 +1,189 @@ +# Commercial Research Beta Foundation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build the local, deterministic Commercial Research Beta foundation without activating unapproved sources or fabricating analysis evidence. + +**Architecture:** Add focused contracts around the existing Personal Research Mode, quarterly actuals, Scenario Lab, thesis journal, source providers, and refresh orchestrator. Keep UI composition thin and fail closed whenever source, rights, freshness, or calibration evidence is absent. + +**Tech Stack:** Python 3.12, dataclasses, pandas, CSV/JSON contracts, Streamlit, pytest, Make. + +**Implementation status (2026-07-17):** Tasks 1-10 are implemented and locally verified; the desktop/mobile findings and corrected Workbench defect are recorded in [Dashboard QA Notes](../../DASHBOARD_QA.md). The unchecked boxes below preserve the original execution sequence; this status line is the current plan truth. The local package is not hosted, externally validated, merged, or commercially launched. + +## Global Constraints + +- Research-only; no investment advice, broker integration, trading, order routing, auto-trading, or direct buy/sell instructions. +- Do not fabricate data, forecasts, probabilities, sources, reviewers, or results. +- Candidate peers and qualitative signals never become trusted numerical inputs automatically. +- Commercial mode refuses sources with unverified commercial rights. +- Keep generated data, reports, screenshots, timing output, packets, caches, and rejected rows excluded. +- Never use `git add -A`; do not push or merge unless explicitly asked. + +--- + +### Task 1: Commercial Source-Rights Registry + +**Files:** +- Create: `config/source_rights.yml` +- Create: `src/commercial_source_rights.py` +- Create: `tests/test_commercial_source_rights.py` +- Modify: `Makefile` +- Modify: `docs/DATA_STRATEGY.md` + +**Interfaces:** +- Produces immutable source-rights records, registry validation, commercial eligibility decisions, and a read-only CLI/status command. + +- [ ] Write failing tests for complete records, unknown sources, unverified rights, commercial refusal, and approved-source acceptance. +- [ ] Run focused tests and confirm expected failures. +- [ ] Implement the minimal registry loader and commercial-mode gate. +- [ ] Add a read-only `make commercial-source-rights` command and documentation. +- [ ] Run focused tests and commit the coherent slice. + +### Task 2: Focused Cohort Coverage Matrix + +**Files:** +- Create: `src/focused_cohort_coverage.py` +- Create: `tests/test_focused_cohort_coverage.py` +- Modify: `src/research_workspace.py` +- Modify: `src/dashboard.py` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` + +**Interfaces:** +- Consumes the deterministic cohort plus saved readiness/source metadata. +- Produces per-company, per-lane states without calculating new fundamentals or changing readiness. + +- [ ] Write failing tests for usable, partial, candidate-only, blocked, excluded, missing, and maximum-50 behavior. +- [ ] Implement immutable coverage rows and matrix summaries. +- [ ] Show the concise answer in Discover/Workbench; keep the full matrix under Advanced. +- [ ] Run focused tests and commit the coherent slice. + +### Task 3: Quarterly Actuals Review Contract + +**Files:** +- Modify: `src/earnings_nowcast_onboarding.py` +- Modify: `src/quarterly_business_trend.py` +- Modify: `Makefile` +- Modify: `tests/test_earnings_nowcast_onboarding.py` +- Modify: `tests/test_quarterly_business_trend.py` +- Modify: `docs/EARNINGS_NOWCAST_PILOT.md` + +**Interfaces:** +- Preserves existing append-only templates, validation, preview, duplicate/revision handling, rejected-row reporting, and readiness. + +- [ ] Add failing tests for ambiguous periods, incompatible units, post-cutoff rows, unresolved revisions, inferred Q4, and preview-only behavior. +- [ ] Close only verified contract gaps; do not add automatic apply. +- [ ] Add or clarify canonical read-only commands and readiness output. +- [ ] Run focused tests and commit the coherent slice. + +### Task 4: Deterministic Forward View V1 + +**Files:** +- Create: `src/forward_view.py` +- Create: `tests/test_forward_view.py` +- Modify: `src/research_workspace.py` +- Modify: `src/dashboard.py` +- Modify: `docs/METHODOLOGY.md` + +**Interfaces:** +- Consumes quarterly trend, source-backed valuation readiness/scenarios, trusted-peer state, reviewer-authored thesis evidence, and Earnings Outlook readiness. +- Produces one immutable packet with trend, scenario, context, withholding, and next-task sections. + +- [ ] Write failing tests for complete, partial, stale, candidate-peer-only, invalid-assumption, and calibration-withheld cases. +- [ ] Implement the packet without introducing a second valuation or nowcast calculation. +- [ ] Integrate it into Company Workbench in the required answer order. +- [ ] Keep raw provenance under Advanced and run focused tests. +- [ ] Commit the coherent slice. + +### Task 5: Point-In-Time Validation Diagnostics + +**Files:** +- Modify: `src/earnings_nowcast_backtest.py` +- Modify: `tests/test_earnings_nowcast_backtest.py` +- Modify: `docs/METHODOLOGY.md` + +**Interfaces:** +- Preserves event snapshots, cutoff enforcement, benchmark comparisons, interval diagnostics, and probability calibration gates. + +- [ ] Add failing tests for empty evidence, revised consensus, stale snapshots, mixed valid/invalid events, and benchmark non-improvement. +- [ ] Add only missing diagnostics and fail-closed reasons. +- [ ] Confirm synthetic fixtures never produce predictive-accuracy claims. +- [ ] Run focused tests and commit the coherent slice. + +### Task 6: Controlled Refresh Operations Contract + +**Files:** +- Create: `src/refresh_operations.py` +- Create: `tests/test_refresh_operations.py` +- Modify: `src/auto_refresh_orchestrator.py` +- Modify: `Makefile` +- Modify: `docs/SCHEDULER_ACTIVATION_CHECKLIST.md` + +**Interfaces:** +- Produces read-only job plans/states for fetch, normalize, validate, quarantine, preview, snapshot publish, readiness rebuild, and change detection. + +- [ ] Write failing tests for provider unavailability, retry caps, identical attempts, schema changes, provenance loss, duplicates, stale rows, partial batches, and quarantine. +- [ ] Implement deterministic plans and failure classifications with automatic apply disabled. +- [ ] Add read-only status/runbook commands. +- [ ] Run focused tests and commit the coherent slice. + +### Task 7: Private-Beta Readiness Contract + +**Files:** +- Create: `src/private_beta_readiness.py` +- Create: `tests/test_private_beta_readiness.py` +- Create: `docs/PRIVATE_BETA_ARCHITECTURE.md` +- Modify: `Makefile` +- Modify: `docs/HOSTED_DEMO_DEPLOYMENT.md` + +**Interfaces:** +- Produces a read-only classification for authentication, workspaces, user data separation, secrets, audit, retention, entitlements, monitoring, and health checks. + +- [ ] Write failing tests for local-ready, external-account-required, and unsafe-secret states. +- [ ] Implement the readiness checklist without claiming runtime authentication or hosting. +- [ ] Document data boundaries and exact external setup steps. +- [ ] Run focused tests and commit the coherent slice. + +### Task 8: Commercial Beta Product Contract And Pilot Package + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `README.md` +- Modify: `docs/PILOT_RUNBOOK.md` +- Modify: `docs/PILOT_REVIEW_FEEDBACK_TEMPLATE.md` +- Modify: `docs/PRODUCT_DIRECTION_DECISION.md` +- Modify: `tests/test_launchers.py` +- Modify: `tests/test_pilot_review_feedback_template.py` + +**Interfaces:** +- Makes `ROADMAP.md` the sole active plan and defines beta jobs, outcomes, safe claims, external classifications, and pilot measures. + +- [ ] Write failing documentation-contract tests. +- [ ] Reconcile current-status wording without hard-coded changing coverage claims. +- [ ] Add the 10-20 user task-based runbook and measurement definitions. +- [ ] Keep absent users as `awaiting_external_review`. +- [ ] Run focused tests and commit the coherent slice. + +### Task 9: Desktop And Mobile UX Hardening + +**Files:** +- Modify only reproducibly affected product/test files after live review. + +**Interfaces:** +- Preserves Research Desk -> Discover -> Company Workbench -> Monitor and Advanced Evidence boundaries. + +- [ ] Review 1280x720 and 390x844 for all four primary pages plus supporting evidence links. +- [ ] Add a failing regression test before each reproducible fix. +- [ ] Fix first-answer order, overflow, premature raw evidence, blocked-output ambiguity, or missing next action only when observed. +- [ ] Run focused browser/UI tests and commit one coherent slice if changes exist. + +### Task 10: Final Verification And Review + +**Files:** +- No new scope; fix only findings from this plan. + +- [ ] Run all focused tests. +- [ ] Run `python3 -m pytest tests -q`. +- [ ] Run dashboard, browser, wording, public, pilot, hygiene, and whitespace gates. +- [ ] Run a fresh broad branch review and fix Critical/Important findings. +- [ ] Confirm generated artifacts remain excluded and external dependencies are classified. +- [ ] Report safe push, host, pilot, and commercialization boundaries without pushing. diff --git a/docs/superpowers/plans/2026-07-17-focused-cohort-research-workflow.md b/docs/superpowers/plans/2026-07-17-focused-cohort-research-workflow.md new file mode 100644 index 000000000..f6151a327 --- /dev/null +++ b/docs/superpowers/plans/2026-07-17-focused-cohort-research-workflow.md @@ -0,0 +1,113 @@ +# Focused Cohort Research Workflow Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a deterministic 25-50 company personal-research cohort with source-backed quarterly trend packets, deduplicated monitoring, and a traceable weekly summary. + +**Architecture:** Add three focused read-only domain modules and compose them through the existing research workspace. Reuse ticker readiness, universe classifications, quarterly actuals, change-event identities, and thesis records; never write canonical data or weaken readiness gates. + +**Tech Stack:** Python 3.12, pandas, dataclasses, pytest, Streamlit AppTest, existing CSV-first profile and readiness contracts. + +## Global Constraints + +- Research-only; no recommendations, trading, broker, or order-routing behavior. +- No fabricated or inferred quarterly data, peers, consensus, or valuation inputs. +- Public and Operator routes remain regression-protected. +- Generated data and reports remain unstaged. +- Every production behavior begins with a failing focused test. + +--- + +### Task 1: Deterministic Focused Cohort + +**Files:** +- Create: `src/focused_research_cohort.py` +- Create: `tests/test_focused_research_cohort.py` + +**Interfaces:** +- Consumes: ticker-readiness and universe-master `pandas.DataFrame` inputs. +- Produces: `FocusedCohortMember`, `FocusedCohort`, `build_focused_cohort(...)`, and `focused_cohort_frame(...)`. + +- [x] Write tests for deterministic order, operating-company eligibility, exclusions, duplicate tickers, missing inputs, truthful sub-25 state, and recommendation-free fields. +- [x] Run `python3 -m pytest tests/test_focused_research_cohort.py -q` and confirm the missing-module failure. +- [x] Implement immutable cohort contracts and deterministic selection. +- [x] Rerun the focused tests and commit the coherent cohort slice. + +### Task 2: Quarterly Business Trend + +**Files:** +- Create: `src/quarterly_business_trend.py` +- Create: `tests/test_quarterly_business_trend.py` + +**Interfaces:** +- Consumes: iterable `QuarterlyActual` rows and ticker. +- Produces: `QuarterlyMetricTrend`, `QuarterlyTrendPacket`, `build_quarterly_trend_packet(...)`, and display-row helpers. + +- [x] Write tests for sequential and year-over-year comparisons, revisions, missing periods, incompatible metric definitions, ambiguous rows, and explicit Q4-only behavior. +- [x] Run the focused test and confirm RED. +- [x] Implement revision resolution and metric-by-metric fail-closed comparisons. +- [x] Rerun tests and commit the coherent trend slice. + +### Task 3: Monitor Detail and Deduplication + +**Files:** +- Modify: `src/research_workspace.py` +- Modify: `tests/test_research_workspace.py` + +**Interfaces:** +- Consumes: existing `ResearchReviewItem` rows. +- Produces: one deterministic monitor row per event identity with previous/current state, source/effective dates, affected section, next task, and wait condition. + +- [x] Add failing tests for duplicate event identities and detailed monitor columns. +- [x] Implement minimal deduplication and field mapping. +- [x] Run focused monitor tests and commit with the Workbench composition slice. + +### Task 4: Weekly Research Summary + +**Files:** +- Create: `src/weekly_research_summary.py` +- Create: `tests/test_weekly_research_summary.py` + +**Interfaces:** +- Consumes: `FocusedCohort`, unresolved research review items, optional reviewer-authored journal summaries, and an as-of timestamp. +- Produces: `WeeklyResearchSummary`, traceable section items, and display rows. + +- [x] Add failing tests for empty weeks, duplicate events, excluded tickers, blocked lanes, stale reviews, and prohibited recommendation wording. +- [x] Implement deterministic grouping and traceable source references. +- [x] Run focused tests and commit with workspace integration. + +### Task 5: Personal Research Workspace Integration + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `src/research_workspace.py` +- Modify: `src/dashboard_render_smoke.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_dashboard_render_smoke.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` + +**Interfaces:** +- Consumes: cohort, quarterly trend, monitor, and weekly summary contracts. +- Produces: Research Desk, Discover, Company Workbench, and Monitor first-answer UI with Advanced Evidence collapsed. + +- [x] Add failing page-contract tests for cohort scope, Workbench answer order, unavailable quarterly evidence, weekly summary, and preserved Public/Operator routes. +- [x] Implement small composition helpers before changing dashboard rendering. +- [x] Run focused dashboard tests and render smoke. +- [x] Commit the coherent workspace slice. + +### Task 6: Documentation and Release Verification + +**Files:** +- Modify: `README.md` +- Modify: `ROADMAP.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify only if contracts change: `docs/METHODOLOGY.md`, `docs/PROVENANCE_CONTRACT.md` + +**Interfaces:** +- Documents verified cohort size, supported quarterly metrics, withheld states, daily/weekly workflow, automation boundary, and owner-authored review work. + +- [x] Update documentation tests before public wording. +- [x] Run focused and full pytest. +- [x] Run dashboard smoke, browser QA, public wording, public check, pilot readiness, diff hygiene, and whitespace checks. +- [x] Review desktop and mobile Research routes plus the Public Single-Stock regression route. +- [x] Stage exact product/code/docs/test files, run staged hygiene, and create a local commit without pushing. diff --git a/docs/superpowers/plans/2026-07-18-company-workbench-answer-first.md b/docs/superpowers/plans/2026-07-18-company-workbench-answer-first.md new file mode 100644 index 000000000..e322d1a81 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-company-workbench-answer-first.md @@ -0,0 +1,126 @@ +# Company Workbench Answer-First Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Keep selected-company lane coverage available under Advanced while making the existing Company Workbench report answer the first expanded research content. + +**Architecture:** Wrap the existing selected-ticker coverage cards in a collapsed route-level Streamlit expander before the unchanged single-stock report renderer. Extend source-order and real-browser marker contracts without changing report data, readiness, routes, or helper signatures. + +**Tech Stack:** Python 3.12, Streamlit, pytest, source-level dashboard contract tests, Markdown documentation. + +## Global Constraints + +- Research-only; no recommendation, expected-return ranking, broker integration, order routing, auto-trading, or direct buy/sell instruction. +- Preserve independent actuals, consensus, Revenue, EPS, valuation, catalyst, outcome, backtesting, and calibration readiness states. +- Do not refresh, import, apply, or fabricate source data. +- Keep candidate context unable to modify deterministic scenarios or become trusted evidence. +- Keep generated CSV, JSON, report, sample-report, screenshot, and timing churn unstaged. +- Keep PR #113 draft; do not merge or deploy. + +--- + +### Task 1: Make Company Workbench answer-first + +**Files:** +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_public_performance_gate.py` +- Modify: `src/dashboard.py` +- Modify: `src/public_performance_gate.py` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/DASHBOARD_QA.md` +- Modify: `ROADMAP.md` + +**Interfaces:** +- Consumes: existing `render_company_workbench`, `focused_ticker_coverage_cards`, `render_single_stock_report`, and `RESEARCH_ROUTE_SPECS` contracts. +- Produces: Company Workbench order `header -> selected-company heading -> collapsed lane coverage -> expanded report answer`, with unchanged report and readiness semantics. + +- [ ] **Step 1: Write the failing route-order contract test** + +```python +def test_company_workbench_keeps_lane_coverage_collapsed_before_report_answer(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + workbench_start = source.index("def render_company_workbench(") + workbench_end = source.index("\ndef main()", workbench_start) + workbench = source[workbench_start:workbench_end] + + selected = workbench.index('st.markdown("### Selected Company")') + advanced = workbench.index( + 'with st.expander("Advanced: selected-company lane coverage", expanded=False):' + ) + coverage = workbench.index("focused_ticker_coverage_cards(coverage, ticker)", advanced) + report = workbench.index("render_single_stock_report(", coverage) + + assert selected < advanced < coverage < report +``` + +- [ ] **Step 2: Run the route test and verify the expected failure** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py::test_company_workbench_keeps_lane_coverage_collapsed_before_report_answer -q` + +Expected: FAIL because `Advanced: selected-company lane coverage` does not exist and the cards currently render expanded. + +- [ ] **Step 3: Write the failing performance-marker contract** + +Add `Advanced: selected-company lane coverage` to the expected Company Workbench full markers in `test_research_performance_contract_covers_the_commercial_beta_workflow`. + +Run: `python3 -m pytest tests/test_public_performance_gate.py::test_research_performance_contract_covers_the_commercial_beta_workflow -q` + +Expected: FAIL because the Company Workbench route spec does not yet include the new visible label. + +- [ ] **Step 4: Implement the minimal route hierarchy** + +```python + st.markdown("### Selected Company") + with st.expander("Advanced: selected-company lane coverage", expanded=False): + render_signal_cards( + focused_ticker_coverage_cards(coverage, ticker), + show_commands=False, + variant="queue", + ) + st.caption( + "Lane coverage is technical evidence only; blocked and candidate-only states remain separate." + ) + render_single_stock_report( +``` + +Add the same Advanced label to the Company Workbench `full_markers` tuple in `src/public_performance_gate.py`. + +- [ ] **Step 5: Run focused tests and verify green** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_research_workspace.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py -q` + +Expected: all focused tests pass with no new warnings or failures. + +- [ ] **Step 6: Update workflow, QA, and roadmap documentation** + +Document the collapsed selected-company lane context, unchanged report/readiness contracts, desktop and phone first-view acceptance, and this locally implemented Stage 1 slice. + +- [ ] **Step 7: Run the full release verification bundle** + +Run: + +```text +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make commercial-beta-performance-gate +make public-wording-check +make public-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: every command exits zero; generated churn remains excluded. The pilot checklist may retain truthful manual source, hosting, or review gates while the command succeeds. + +- [ ] **Step 8: Stage exact files, verify staged hygiene, commit, and push** + +```text +git add -- src/dashboard.py src/public_performance_gate.py tests/test_research_mode_dashboard_contract.py tests/test_public_performance_gate.py ROADMAP.md docs/PERSONAL_RESEARCH_MODE.md docs/DASHBOARD_QA.md docs/superpowers/specs/2026-07-18-company-workbench-answer-first-design.md docs/superpowers/plans/2026-07-18-company-workbench-answer-first.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Prioritize Company Workbench answers" +git push origin codex/personal-research-mode-mvp +``` + +Expected: exact reviewed files only are committed and pushed; PR #113 remains open and draft. diff --git a/docs/superpowers/plans/2026-07-18-discover-answer-first.md b/docs/superpowers/plans/2026-07-18-discover-answer-first.md new file mode 100644 index 000000000..b2c10b537 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-discover-answer-first.md @@ -0,0 +1,126 @@ +# Discover Answer-First Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Put Discover's readiness-backed company selector before cohort evidence while preserving all cohort and readiness context under Advanced. + +**Architecture:** Reorder existing Streamlit composition calls in the Discover branch of `src/dashboard.py`. Keep the existing selector, focused-cohort cards, coverage cards, routes, and data contracts unchanged; only move the cards into a collapsed route-specific expander after the selector. + +**Tech Stack:** Python 3.12, Streamlit, pytest, source-level dashboard contract tests, Markdown documentation. + +## Global Constraints + +- Research-only; no recommendation, expected-return ranking, broker integration, order routing, auto-trading, or direct buy/sell instruction. +- Preserve independent readiness states and deterministic focused-cohort membership. +- Do not refresh, import, apply, or fabricate source data. +- Keep generated CSV, JSON, report, sample-report, screenshot, and timing churn unstaged. +- Keep PR #113 draft; do not merge or deploy. + +--- + +### Task 1: Make Discover selection-first + +**Files:** +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `src/dashboard.py` +- Modify: `tests/test_public_performance_gate.py` +- Modify: `src/public_performance_gate.py` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/DASHBOARD_QA.md` +- Modify: `ROADMAP.md` + +**Interfaces:** +- Consumes: existing `render_stock_selector`, `focused_cohort_cards`, `focused_cohort_coverage_cards`, `focused_cohort`, and `focused_cohort_coverage` contracts. +- Produces: Discover route order `header -> selector -> collapsed cohort readiness context`, with unchanged Company Workbench links and readiness semantics. + +- [ ] **Step 1: Write the failing route-order contract test** + +```python +def test_research_discover_renders_selector_before_advanced_cohort_context(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + discover_start = source.index('elif research_mode and selected_page == "Discover":') + discover_end = source.index('elif research_mode and selected_page == "Company Workbench":', discover_start) + discover = source[discover_start:discover_end] + + heading = discover.index('st.markdown("### Which stock can I review?")') + selector = discover.index("render_stock_selector(", heading) + advanced = discover.index('with st.expander("Advanced: cohort readiness context", expanded=False):') + cohort = discover.index("focused_cohort_cards(focused_cohort)", advanced) + coverage = discover.index("focused_cohort_coverage_cards(focused_cohort_coverage)", advanced) + + assert heading < selector < advanced < cohort < coverage +``` + +- [ ] **Step 2: Run the test and verify the expected failure** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py::test_research_discover_renders_selector_before_advanced_cohort_context -q` + +Expected: FAIL because `Advanced: cohort readiness context` does not exist and the cohort cards currently render before the selector. + +- [ ] **Step 3: Implement the minimal route reordering** + +```python + st.markdown("### Which stock can I review?") + render_stock_selector( + output_frames, + public_mode=True, + target_mode=RESEARCH_MODE, + target_page="company-workbench", + allowed_tickers=tuple(member.ticker for member in focused_cohort.members), + ) + with st.expander("Advanced: cohort readiness context", expanded=False): + render_signal_cards(focused_cohort_cards(focused_cohort), show_commands=False, variant="queue") + render_signal_cards( + focused_cohort_coverage_cards(focused_cohort_coverage), + show_commands=False, + variant="queue", + ) + st.caption( + "Cohort membership and lane coverage remain evidence context only; " + "they do not rank expected return or create a recommendation." + ) +``` + +- [ ] **Step 4: Run focused tests and verify green** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_research_workspace.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py -q` + +Expected: all focused tests pass with no new warnings or failures. + +- [ ] **Step 4a: Align the real-browser visible-marker contract test-first** + +Require Discover's performance contract to use `Which stock can I review?` as the first useful marker and `Search this review queue`, `Advanced: cohort readiness context`, and `Research-only` as visible full-settle evidence. Run the focused performance-contract test red against the stale `Focused cohort` marker, update `RESEARCH_ROUTE_SPECS`, then rerun the focused performance and route suite green. + +- [ ] **Step 5: Update the workflow, QA, and roadmap documentation** + +Document the selector-first Discover order, collapsed cohort context, unchanged readiness boundaries, and first-viewport acceptance rule. Record the slice as implemented only after focused verification passes. + +- [ ] **Step 6: Run the full release verification bundle** + +Run: + +```text +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make commercial-beta-performance-gate +make public-wording-check +make public-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: every command exits zero; generated churn remains excluded. + +- [ ] **Step 7: Stage exact files, verify staged hygiene, commit, and push** + +```text +git add -- src/dashboard.py src/public_performance_gate.py tests/test_research_mode_dashboard_contract.py tests/test_public_performance_gate.py ROADMAP.md docs/PERSONAL_RESEARCH_MODE.md docs/DASHBOARD_QA.md docs/superpowers/specs/2026-07-18-discover-answer-first-design.md docs/superpowers/plans/2026-07-18-discover-answer-first.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Prioritize Discover company selection" +git push origin codex/personal-research-mode-mvp +``` + +Expected: exact reviewed files only are committed and pushed; PR #113 remains draft. diff --git a/docs/superpowers/plans/2026-07-18-evidence-activation-layer.md b/docs/superpowers/plans/2026-07-18-evidence-activation-layer.md new file mode 100644 index 000000000..873701077 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-evidence-activation-layer.md @@ -0,0 +1,122 @@ +# Evidence Activation Layer Implementation Plan + +> **For Codex:** Execute each task with test-driven development. Keep generated evidence under `/tmp` or `outputs`, never stage it, and preserve fail-closed research-only states. + +**Goal:** Add the next evidence-bound personal research capabilities without creating new primary pages or turning candidate context into forecasts, valuation claims, or recommendations. + +**Architecture:** Extend the existing Earnings Nowcast, Valuation, Research Thesis Journal, Change Monitor, Company Workbench, and Monitor modules. New modules expose immutable/read-only packets and append-only reviewed records. Dashboard changes consume compact helpers and keep raw evidence under Advanced. + +**Tech stack:** Python 3.12, dataclasses, CSV/JSON contracts, argparse, Streamlit, pytest, Makefile launchers. + +--- + +### Task 1: Reproduce the five-company actuals baseline + +**Files:** +- Generated only: `/tmp/earnings-nowcast-sec-actuals-review-/` +- Verify: `src/earnings_nowcast_sec_actuals.py` +- Verify: `src/earnings_nowcast_onboarding.py` + +1. Stage NVDA, AMD, AVGO, MU, and QCOM SEC actuals into a new timestamped `/tmp` directory with a current UTC cutoff. +2. Validate and preview the generated onboarding rows. +3. Run per-ticker readiness and record the actual, Q4, split-basis, consensus, and calibration blockers. +4. Confirm no repository files changed. + +### Task 2: Five-company Earnings Nowcast readiness board + +**Files:** +- Create: `src/earnings_nowcast_cohort.py` +- Create: `tests/test_earnings_nowcast_cohort.py` +- Modify: `Makefile` +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` + +1. Write failing tests for one immutable summary row per ticker and independent Revenue/EPS/Q4/split/consensus/calibration states. +2. Implement CSV-loading and cohort summarization from existing onboarding contracts. +3. Add a read-only `earnings-nowcast-cohort-readiness` command. +4. Add one compact Company Workbench/Monitor summary; keep source IDs and raw rows under Advanced. + +### Task 3: Consensus source probe and prospective collector + +**Files:** +- Create: `src/earnings_consensus_sources.py` +- Create: `src/earnings_consensus_collector.py` +- Create: `tests/test_earnings_consensus_sources.py` +- Create: `tests/test_earnings_consensus_collector.py` +- Modify: `Makefile` +- Modify: `config/provider_keys.env.example` + +1. Write failing tests for deterministic provider order, missing-key classification, rights/comparability gates, and current-only estimates staying `candidate_context_only`. +2. Implement read-only probes for Alpha Vantage, FMP, Finnhub, and a generic reviewed CSV contract. Do not fetch when a key or approved source contract is absent. +3. Write failing tests for deterministic snapshot identity, deduplication, revision preservation, cutoff checks, cooldown, and no overwrite. +4. Implement plan/status/preview collection. Recording must require an explicit reviewed confirmation and append only. +5. Add source-status, collection-plan, collection-preview, and collection-status commands. No automatic promotion or apply path. + +### Task 4: Historical valuation regime + +**Files:** +- Create: `src/historical_valuation_regime.py` +- Create: `tests/test_historical_valuation_regime.py` +- Modify: `src/dashboard.py` +- Modify: `tests/test_dashboard_helpers.py` + +1. Write failing tests for aligned point-in-time price and denominator observations, definition-change segmentation, insufficient history, stale observations, and rejection of current-denominator backfills. +2. Implement descriptive multiple observations and percentile/range context only when source timestamps and period definitions align. +3. Add a compact Valuation section answer. Keep rows and provenance under Advanced. +4. Use neutral wording; never label a stock cheap, expensive, attractive, or actionable. + +### Task 5: Research outcome review + +**Files:** +- Create: `src/research_outcome_review.py` +- Create: `tests/test_research_outcome_review.py` +- Modify: `Makefile` +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` + +1. Write failing tests for append-only reviewed records, immutable thesis references, observation-window validation, source timestamps, duplicate rejection, and no return/skill scoring. +2. Implement read, validate, preview, explicitly confirmed append, and derived status helpers. +3. Add preview/record/status commands. +4. Integrate a compact learning-loop answer into Thesis Journal/Monitor; keep raw records under Advanced. + +### Task 6: Catalyst evidence timeline + +**Files:** +- Create: `src/catalyst_evidence_timeline.py` +- Create: `tests/test_catalyst_evidence_timeline.py` +- Modify: `Makefile` +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` + +1. Write failing tests for allowed event types, provenance, publication/retrieval/effective timestamps, cutoff safety, duplicate identity, and candidate-only boundaries. +2. Implement read, validate, preview, explicitly confirmed append, and timeline derivation. +3. Add preview/record/status commands. +4. Integrate a concise upcoming/recent evidence answer into Company Workbench/Monitor. No scraping, sentiment scoring, forecast mutation, or recommendation. + +### Task 7: Product integration and documentation + +**Files:** +- Modify: `src/earnings_nowcast_ui.py` +- Modify: `src/forward_view.py` +- Modify: `src/dashboard.py` +- Modify: `README.md` +- Modify: `ROADMAP.md` +- Modify: `docs/EARNINGS_NOWCAST_PILOT.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `tests/test_earnings_nowcast_ui.py` +- Modify: `tests/test_forward_view.py` + +1. Preserve the four-page Personal Research flow: Research Desk -> Discover -> Company Workbench -> Monitor. +2. Present concise answers in this order: readiness, usable evidence, withheld evidence, next research action. +3. Keep hashes, source IDs, raw events, records, and diagnostics under Advanced. +4. Document implemented software separately from real data availability, historical validation, and external dependencies. + +### Task 8: Verification, commits, and draft PR + +1. Run focused tests after each task and the full suite after integration. +2. Run dashboard smoke, browser QA, public wording, public check, pilot readiness, diff hygiene, and whitespace checks. +3. Stage exact product/code/docs/test files only; never use `git add -A`. +4. Commit coherent slices only after verification. +5. Push `codex/personal-research-mode-mvp` and update draft PR #113 without merging or deploying. diff --git a/docs/superpowers/plans/2026-07-18-mobile-research-first-action.md b/docs/superpowers/plans/2026-07-18-mobile-research-first-action.md new file mode 100644 index 000000000..ff8c0f334 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-mobile-research-first-action.md @@ -0,0 +1,309 @@ +# Mobile Research First-Action Density Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Bring each Personal Research route's first task into the 390x844 useful viewport while preserving profile truth, freshness, next action, and research-only boundaries. + +**Architecture:** Keep the existing compact profile strip and research workspace header. Add semantic classes to the two route metadata rows, apply phone-only compaction through the existing research style block, and replace Company Workbench's long always-visible path caption with a collapsed native disclosure. + +**Tech Stack:** Python 3.12, Streamlit, HTML/CSS strings, pytest, Streamlit AppTest, Git, Make. + +## Global Constraints + +- Desktop profile and route metadata remain unchanged; Company Workbench keeps the complete path available through the same collapsed disclosure. +- Data profile, Sources through, Freshness, Price-ready, DCF-ready, next action, and the research-only boundary remain visible or equivalently available. +- No readiness state, source date, source right, research input, forecast, valuation, peer, catalyst, outcome, backtest, or calibration state changes. +- No CSV, JSON, report, sample-report, committed screenshot, or timing artifact is created or staged. +- Candidate context remains untrusted and synthetic fixtures remain test-only. +- No investment advice, ranking, recommendation, price prediction, broker action, order routing, or trade instruction is added. +- Use exact staging only; never use `git add -A`. +- Keep PR #113 open and draft; do not merge or deploy. + +--- + +### Task 1: Compact the shared research context at phone width + +**Files:** +- Modify: `tests/test_research_workspace.py:353-368` +- Modify: `tests/test_research_mode_dashboard_contract.py:202-220` +- Modify: `src/research_workspace.py:376-398` +- Modify: `src/dashboard.py:33966-34057` + +**Interfaces:** +- Consumes: `research_workspace_header_html(page_title, *, ticker, profile_label, freshness, primary_action) -> str` and the existing `.profile-trust-strip.compact` markup. +- Produces: `.research-workspace-freshness` and `.research-workspace-action` semantic hooks plus phone-only CSS that leaves desktop output unchanged. + +- [ ] **Step 1: Write failing HTML and style contract tests** + +Add these assertions to `test_research_workspace_header_keeps_scope_freshness_action_and_boundary_visible`: + +```python +assert "class='research-workspace-meta-item research-workspace-freshness'" in rendered +assert "class='research-workspace-meta-item research-workspace-action'" in rendered +``` + +Add this test to `tests/test_research_mode_dashboard_contract.py`: + +```python +def test_research_workspace_phone_styles_compact_profile_and_hide_only_duplicate_freshness(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + start = source.index("def render_research_workspace_styles()") + end = source.index("\ndef render_research_workspace_header(", start) + styles = source[start:end] + + assert ".profile-trust-strip.compact" in styles + assert "grid-template-columns: repeat(3, minmax(0, 1fr));" in styles + assert ".research-workspace-freshness { display: none; }" in styles + assert ".research-workspace-action" in styles + assert "@media (max-width: 640px)" in styles +``` + +- [ ] **Step 2: Run focused tests and verify the red state** + +Run: + +```bash +python3 -m pytest tests/test_research_workspace.py::test_research_workspace_header_keeps_scope_freshness_action_and_boundary_visible tests/test_research_mode_dashboard_contract.py::test_research_workspace_phone_styles_compact_profile_and_hide_only_duplicate_freshness -q +``` + +Expected: both tests fail because the semantic classes and three-column phone CSS do not exist. + +- [ ] **Step 3: Add semantic route metadata classes** + +Change the metadata markup in `research_workspace_header_html` to: + +```python +"
" +f"
Freshness
{html.escape(str(freshness or 'Check saved readiness'))}
" +f"
Next action
{html.escape(str(primary_action or 'Review source-backed evidence'))}
" +"
" +``` + +- [ ] **Step 4: Add phone-only compaction to the existing research styles** + +Replace the current phone block in `render_research_workspace_styles` with: + +```css +@media (max-width: 640px) { + .research-desk-grid { grid-template-columns: 1fr; } + .research-workspace-header { + padding: .72rem .78rem; + margin-bottom: .65rem; + } + .research-workspace-heading h1 { font-size: 1.4rem; } + .research-workspace-meta { + grid-template-columns: 1fr; + gap: 0; + margin: .5rem 0 .4rem; + } + .research-workspace-freshness { display: none; } + .research-workspace-action { + border-top: 1px solid #e5e9e7; + padding-top: .45rem; + } + .profile-trust-strip.compact { + grid-template-columns: repeat(3, minmax(0, 1fr)); + gap: .3rem; + margin: .12rem 0 .6rem; + padding: .45rem 0; + } + .profile-trust-strip.compact > span, + .profile-trust-strip.compact .profile-trust-primary { + padding: 0 .35rem; + } + .profile-trust-strip.compact > :nth-child(3n + 1) { border-left: 0; } +} +``` + +- [ ] **Step 5: Run focused tests and verify the green state** + +Run: + +```bash +python3 -m pytest tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py -q +``` + +Expected: all focused tests pass. + +- [ ] **Step 6: Commit the semantic and responsive shell slice** + +```bash +git add -- src/research_workspace.py src/dashboard.py tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Compact mobile research context" +git push origin codex/personal-research-mode-mvp +``` + +--- + +### Task 2: Collapse the Company Workbench review path + +**Files:** +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `src/dashboard.py:34153-34185` + +**Interfaces:** +- Consumes: `company_workbench_section_contract() -> list[dict[str, object]]` and Streamlit's existing `st.expander` API. +- Produces: a collapsed `Review path` disclosure immediately after the `Selected Company` heading and before technical coverage or detailed report content. + +- [ ] **Step 1: Write the failing Workbench ordering test** + +Add: + +```python +def test_company_workbench_keeps_selected_company_before_collapsed_review_path_and_details(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + start = source.index("def render_company_workbench(") + end = source.index("\ndef main()", start) + workbench = source[start:end] + + selected = workbench.index('st.markdown("### Selected Company")') + review = workbench.index('with st.expander("Review path", expanded=False):', selected) + path = workbench.index('st.caption(" -> ".join(section_names[:-1]))', review) + coverage = workbench.index('with st.expander("Advanced: selected-company lane coverage", expanded=False):', path) + report = workbench.index("render_single_stock_report(", coverage) + + assert selected < review < path < coverage < report + assert 'st.caption("Review path: "' not in workbench +``` + +- [ ] **Step 2: Run the test and verify the red state** + +Run: + +```bash +python3 -m pytest tests/test_research_mode_dashboard_contract.py::test_company_workbench_keeps_selected_company_before_collapsed_review_path_and_details -q +``` + +Expected: FAIL because the path is an always-visible caption before the `Selected Company` heading. + +- [ ] **Step 3: Implement the collapsed disclosure** + +Replace the Workbench path block with: + +```python +st.markdown("### Selected Company") +section_names = [section["title"] for section in company_workbench_section_contract()] +with st.expander("Review path", expanded=False): + st.caption(" -> ".join(section_names[:-1])) +``` + +Keep `Advanced: selected-company lane coverage` and `render_single_stock_report(...)` directly after this block. + +- [ ] **Step 4: Run focused tests and verify the green state** + +Run: + +```bash +python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q +``` + +Expected: all focused tests pass and all four Personal Research routes still render. + +- [ ] **Step 5: Commit the Workbench disclosure slice** + +```bash +git add -- src/dashboard.py tests/test_research_mode_dashboard_contract.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Collapse the mobile Workbench review path" +git push origin codex/personal-research-mode-mvp +``` + +--- + +### Task 3: Verify the visual result and document the maturity boundary + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Test: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes: verified implementation behavior and fresh 1280x720 plus 390x844 audit screenshots stored outside the repository. +- Produces: truthful roadmap and continuation claims that describe local usability maturity without claiming source, hosted, reviewer, calibration, or market validation. + +- [ ] **Step 1: Re-capture the four routes outside the repository** + +Capture Research Desk, Discover, Company Workbench, and Monitor at 1280x720 and 390x844 to `/tmp/stock-research-workflow-audit-after/`. + +Expected visible results: + +- desktop retains all existing profile and route metadata; +- the phone profile strip uses two rows; +- the phone route card omits only its duplicate freshness row; +- Discover search is fully visible in the first phone viewport; +- Company Workbench shows `Selected Company` and collapsed `Review path` in the first phone viewport; +- Monitor shows the weekly summary in the first phone viewport; +- no route shows a traceback, horizontal overflow, fabricated content, or missing research-only boundary. + +- [ ] **Step 2: Add a failing documentation contract assertion** + +Extend the Personal Research documentation test in `tests/test_public_v1_release_docs.py` to require these phrases: + +```python +assert "mobile first-action density" in personal_mode.lower() +assert "does not change readiness" in personal_mode.lower() +``` + +- [ ] **Step 3: Run the documentation test and verify the red state** + +Run: + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py -q +``` + +Expected: FAIL because the documentation does not yet describe this slice. + +- [ ] **Step 4: Update roadmap, Personal Research documentation, and continuation truth** + +Document: + +- the four-route phone compaction is implemented locally; +- desktop profile and route metadata plus all evidence boundaries are unchanged; the Workbench path remains available in its collapsed disclosure; +- this improves usability and reviewer comprehension only; +- it does not change readiness or prove source activation, hosted operation, reviewer demand, calibration, commercial demand, or product-market fit; +- the next stage remains one permitted source path followed by controlled hosting and external reviewer validation. + +- [ ] **Step 5: Run focused and full verification** + +Run: + +```bash +python3 -m pytest tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_public_v1_release_docs.py -q +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: every command exits zero; pilot readiness remains `pilot-ready with manual gates`; generated artifact churn remains zero. + +- [ ] **Step 6: Stage exact files and verify the package** + +```bash +git add -- ROADMAP.md docs/PERSONAL_RESEARCH_MODE.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +``` + +Expected: only the four intentional documentation/test files are staged and generated CSV/JSON/report churn is zero. + +- [ ] **Step 7: Commit, push, and update draft PR #113** + +```bash +git commit -m "Document mobile research first-action maturity" +git push origin codex/personal-research-mode-mvp +gh pr comment 113 --body "Verified mobile Personal Research first-action density update: desktop profile and route metadata plus all readiness boundaries remain unchanged; the Workbench path stays available in a collapsed disclosure; phone routes expose their first task sooner; full local release gates passed; no generated CSV, JSON, report, sample-report, screenshot, or timing churn was staged. PR remains draft." +``` + +Expected: branch aligns with its remote, PR #113 remains open and draft, and the new comment records the verified boundary. diff --git a/docs/superpowers/plans/2026-07-18-monitor-answer-first.md b/docs/superpowers/plans/2026-07-18-monitor-answer-first.md new file mode 100644 index 000000000..8d39639a3 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-monitor-answer-first.md @@ -0,0 +1,196 @@ +# Monitor Answer-First Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Put Monitor's research-change answer and truthful empty-state action before technical Earnings Nowcast readiness evidence while preserving every queue and readiness contract. + +**Architecture:** Reorder existing calls inside `render_research_monitor()` so the weekly summary is followed by the deduplicated queue answer. Add the existing Discover link only for an empty queue, use the neutral context-note treatment, and move unchanged five-company readiness cards into their existing collapsed Advanced drawer. + +**Tech Stack:** Python 3.12, Streamlit, pytest, source-level dashboard contract tests, Markdown documentation. + +## Global Constraints + +- Research-only; no recommendation, expected-return ranking, broker integration, order routing, auto-trading, or direct buy/sell instruction. +- Preserve independent actuals, consensus, Revenue, EPS, valuation, catalyst, outcome, backtesting, and calibration readiness states. +- An empty queue means no saved comparable source-backed change is queued; it never proves that nothing changed in the real world. +- Do not refresh, import, apply, or fabricate source data. +- Keep candidate context unable to modify deterministic scenarios or become trusted evidence. +- Keep generated CSV, JSON, report, sample-report, screenshot, and timing churn unstaged. +- Keep PR #113 draft; do not merge or deploy. + +--- + +### Task 1: Make Monitor answer-first + +**Files:** +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_public_performance_gate.py` +- Modify: `src/dashboard.py` +- Modify: `src/public_performance_gate.py` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/DASHBOARD_QA.md` +- Modify: `ROADMAP.md` + +**Interfaces:** +- Consumes: existing `weekly_summary_cards`, `research_monitor_frame`, `render_context_note`, `load_dashboard_nowcast_cohort`, `cohort_readiness_cards`, and `RESEARCH_ROUTE_SPECS` contracts. +- Produces: Monitor order `header -> weekly summary -> change answer -> empty Discover action when applicable -> collapsed readiness evidence`, with unchanged queue rows and readiness payloads. + +- [ ] **Step 1: Write the failing Monitor route-order contract** + +Add a complete order contract alongside the existing Monitor integration assertions: + +```python +def test_monitor_renders_change_answer_before_advanced_readiness(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + monitor_start = source.index("def render_research_monitor(") + monitor_end = source.index("def render_company_workbench(", monitor_start) + monitor = source[monitor_start:monitor_end] + + weekly = monitor.index("weekly_summary_cards(weekly_summary)") + answer = monitor.index('st.markdown("### Research change monitor")', weekly) + frame = monitor.index("research_monitor_frame(state.get", answer) + empty = monitor.index("if frame.empty:", frame) + note = monitor.index("render_context_note(", empty) + discover = monitor.index('st.link_button("Open Discover"', note) + cohort = monitor.index("nowcast_cohort = load_dashboard_nowcast_cohort()", discover) + advanced = monitor.index( + 'with st.expander("Advanced: five-company Earnings Nowcast readiness", expanded=False):', + cohort, + ) + readiness_heading = monitor.index('st.markdown("### Earnings evidence readiness")', advanced) + readiness_cards = monitor.index("cohort_readiness_cards(nowcast_cohort)", readiness_heading) + readiness_frame = monitor.index("pd.DataFrame([asdict(row) for row in nowcast_cohort])", readiness_cards) + + assert weekly < answer < frame < empty < note < discover < cohort < advanced + assert advanced < readiness_heading < readiness_cards < readiness_frame + assert 'tone="success"' not in monitor[empty:discover] +``` + +Add a theme contract for the new primary action: + +```python +def test_dashboard_theme_keeps_primary_link_button_text_white(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + + assert '[data-testid="stLinkButton"] a[kind="primary"],' in source + assert '[data-testid="stLinkButton"] a[kind="primary"] * {' in source + assert "color: #ffffff !important;" in source[source.index('[data-testid="stLinkButton"] a[kind="primary"],'):] +``` + +- [ ] **Step 2: Run the route test and verify the expected failure** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py::test_monitor_renders_change_answer_before_advanced_readiness -q` + +Expected: FAIL because `Research change monitor` currently occurs after the Nowcast readiness card and there is no empty-state Discover link. + +- [ ] **Step 3: Write the failing performance-marker contract** + +Extend `test_research_performance_contract_covers_the_commercial_beta_workflow` with: + +```python + assert RESEARCH_ROUTE_SPECS[3].first_useful_marker == "WEEKLY RESEARCH SUMMARY" + assert RESEARCH_ROUTE_SPECS[3].full_markers == ( + "WEEKLY RESEARCH SUMMARY", + "Research change monitor", + "No unresolved evidence change is queued.", + "Open Discover", + "Advanced: five-company Earnings Nowcast readiness", + "Research-only", + ) +``` + +Run: `python3 -m pytest tests/test_public_performance_gate.py::test_research_performance_contract_covers_the_commercial_beta_workflow -q` + +Expected: FAIL because Monitor currently treats the route title as first useful and does not require the primary answer or action. + +- [ ] **Step 4: Implement the minimal Monitor hierarchy** + +Change `render_research_monitor()` to this composition: + +```python + render_signal_cards(weekly_summary_cards(weekly_summary), show_commands=False, variant="queue") + st.markdown("### Research change monitor") + frame = research_monitor_frame(state.get("queue") or ()) + if frame.empty: + render_context_note( + "No unresolved evidence change is queued.", + "This is a monitoring state, not a stock ranking. Continue with Discover or wait for a comparable source-backed change.", + ) + st.link_button("Open Discover", "?mode=research&page=discover", type="primary") + else: + st.dataframe(frame, width="stretch", hide_index=True) + nowcast_cohort = load_dashboard_nowcast_cohort() + with st.expander("Advanced: five-company Earnings Nowcast readiness", expanded=False): + st.markdown("### Earnings evidence readiness") + render_signal_cards(cohort_readiness_cards(nowcast_cohort), show_commands=False, variant="queue") + st.dataframe(pd.DataFrame([asdict(row) for row in nowcast_cohort]), width="stretch", hide_index=True) + st.caption("This board creates no forecast. Missing consensus, Q4, split, backtest, and calibration evidence remain separate blockers.") +``` + +Update Monitor's `PublicRouteSpec` to: + +```python + PublicRouteSpec( + "Monitor", + "/?mode=research&page=monitor", + "WEEKLY RESEARCH SUMMARY", + ( + "WEEKLY RESEARCH SUMMARY", + "Research change monitor", + "No unresolved evidence change is queued.", + "Open Discover", + "Advanced: five-company Earnings Nowcast readiness", + "Research-only", + ), + True, + ), +``` + +Keep nested Streamlit Markdown text readable on primary link buttons: + +```css +[data-testid="stLinkButton"] a[kind="primary"], +[data-testid="stLinkButton"] a[kind="primary"] * { + color: #ffffff !important; +} +``` + +- [ ] **Step 5: Run focused tests and verify green** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_research_workspace.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py -q` + +Expected: all focused tests pass with no new warnings or failures. + +- [ ] **Step 6: Update workflow, QA, and roadmap documentation** + +Document the Monitor answer-first order, neutral wait state, conditional Discover action, collapsed five-company readiness evidence, unchanged readiness boundaries, desktop and phone acceptance, and completion of local Stage 1 workflow hardening. Move the next executable roadmap step to one permitted prospective point-in-time consensus source path without claiming that an external dependency is available. + +- [ ] **Step 7: Run the full release verification bundle** + +Run: + +```text +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make commercial-beta-performance-gate +make public-wording-check +make public-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: every command exits zero; generated churn remains excluded. The pilot checklist may retain truthful manual source, hosting, or review gates while the command succeeds. + +- [ ] **Step 8: Stage exact files, verify staged hygiene, commit, and push** + +```text +git add -- ROADMAP.md docs/DASHBOARD_QA.md docs/PERSONAL_RESEARCH_MODE.md src/dashboard.py src/public_performance_gate.py tests/test_public_performance_gate.py tests/test_research_mode_dashboard_contract.py docs/superpowers/plans/2026-07-18-monitor-answer-first.md docs/superpowers/specs/2026-07-18-monitor-answer-first-design.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Prioritize Monitor answers" +git push origin codex/personal-research-mode-mvp +``` + +Expected: exact reviewed files only are committed and pushed; PR #113 remains open and draft. diff --git a/docs/superpowers/plans/2026-07-18-peer-evidence-quality.md b/docs/superpowers/plans/2026-07-18-peer-evidence-quality.md new file mode 100644 index 000000000..34c58d4a0 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-peer-evidence-quality.md @@ -0,0 +1,108 @@ +# Peer Evidence Quality Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Preserve reviewed peer role and economic-comparability evidence end to end, and withhold peer-median valuation unless a relationship is explicitly eligible to anchor it. + +**Architecture:** A small pure module owns deterministic peer evidence-quality classification. Source review, local provider, readiness generation, and Company Workbench consume that shared result so relationship, trend, read-through, and valuation readiness cannot collapse into one state. + +**Tech Stack:** Python 3.12, pandas, Streamlit, pytest, Markdown, CSV contracts. + +## Global Constraints + +- Research-only; no recommendation, transaction instruction, broker integration, order routing, or auto-trading. +- Candidate context cannot become trusted relationship or valuation evidence. +- Existing rows without explicit role and comparability evidence fail closed for valuation anchoring. +- Do not infer, backfill, or fabricate peer roles, comparability, sources, or timestamps. +- Preserve trend, result read-through, and valuation readiness as independent states. +- Keep technical evidence under Advanced unless needed to explain the primary answer. + +--- + +### Task 1: Deterministic Peer Evidence-Quality Contract + +**Files:** +- Create: `src/peer_evidence_quality.py` +- Create: `tests/test_peer_evidence_quality.py` + +**Interfaces:** +- Consumes: a mapping-shaped peer relationship row. +- Produces: `assess_peer_evidence(row: Mapping[str, object]) -> PeerEvidenceQuality` and `is_valuation_anchor_eligible(row: Mapping[str, object]) -> bool`. + +- [ ] Write failing tests for a fully eligible `core_peer`, a legacy row, a context-only role, an invalid role, missing comparability, and missing provenance. +- [ ] Run `python3 -m pytest tests/test_peer_evidence_quality.py -q` and confirm failures are caused by the missing module. +- [ ] Implement the immutable result type, allowed roles, normalization, blocker ordering, and fail-closed anchor decision. +- [ ] Rerun the focused test and confirm it passes. + +### Task 2: Preserve Review Evidence Through Import + +**Files:** +- Modify: `src/providers/local_schemas.py` +- Modify: `src/peer_mapping_source_review.py` +- Modify: `docs/TRUSTED_PEER_PILOT_SOURCE_TEMPLATE.csv` +- Modify: `tests/test_peer_mapping_source_review.py` +- Modify: `tests/test_trusted_peer_pilot_source_template.py` + +**Interfaces:** +- Consumes: reviewed `peer_role`, `relationship_rationale`, `comparability_basis`, and `valuation_anchor_eligible` fields. +- Produces: exact import header and guarded CSV row that preserve those fields. + +- [ ] Add failing tests proving incomplete evidence remains blocked and a complete row survives into the import scaffold. +- [ ] Run the two focused test modules and confirm the new assertions fail for the missing contract. +- [ ] Extend schema, review row, required fields, scaffold, CLI arguments, rendering, and template without adding an automatic apply path. +- [ ] Rerun the focused tests and confirm they pass. + +### Task 3: Gate Provider Valuation Inputs And Readiness Independently + +**Files:** +- Modify: `src/providers/local_market_data.py` +- Modify: `src/readiness_engine.py` +- Modify: `tests/test_local_market_data_provider.py` +- Modify: `tests/test_readiness_engine.py` + +**Interfaces:** +- Consumes: `assess_peer_evidence` and canonical peer rows. +- Produces: relationship-quality fields in `trusted_relationships`, valuation inputs containing only eligible anchors, and readiness columns for anchor counts/blockers. + +- [ ] Add failing provider tests proving legacy and context-only rows remain visible but are excluded from valuation inputs. +- [ ] Add failing readiness tests proving trend readiness can be true while valuation-anchor readiness is false, and eligible peers can unlock valuation readiness only with financial evidence. +- [ ] Run the focused tests and confirm the expected failures. +- [ ] Implement provider filtering and independent readiness classification using the shared contract. +- [ ] Rerun focused tests and confirm they pass. + +### Task 4: Surface The Answer In Company Workbench + +**Files:** +- Modify: `src/peer_read_through_map.py` +- Modify: `src/dashboard.py` +- Modify: `tests/test_peer_read_through_map.py` +- Modify: `tests/test_dashboard_helpers.py` + +**Interfaces:** +- Consumes: provider relationship-quality fields or the shared classifier. +- Produces: compact peer rows with `Peer Role`, `Comparability`, and `Valuation Anchor`, plus answer-first summary copy. + +- [ ] Add failing tests for eligible, context-only, and legacy-withheld table states and summary copy. +- [ ] Run focused tests and confirm the table assertions fail. +- [ ] Extend map edges, deterministic identity, compact rows, frame columns, and summary cards; keep raw evidence in Advanced. +- [ ] Rerun focused tests and confirm they pass. + +### Task 5: Documentation And Release Verification + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PILOT_RUNBOOK.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` + +**Interfaces:** +- Consumes: verified behavior from Tasks 1-4. +- Produces: truthful methodology, migration boundary, pilot procedure, and next external dependency. + +- [ ] Document the new independent states and the explicit legacy-row migration boundary. +- [ ] Run focused peer/schema/dashboard tests. +- [ ] Run `python3 -m pytest tests -q`. +- [ ] Run `make dashboard-smoke`, `make public-wording-check`, `make public-check`, `make pilot-readiness-check TOP_N=10`, `make diff-hygiene-summary`, and `git diff --check`. +- [ ] Stage only the exact product, test, spec, plan, template, and documentation paths; run `make staged-hygiene-check`. +- [ ] Commit one coherent slice, push only `codex/personal-research-mode-mvp`, and update draft PR #113 without changing its draft state. diff --git a/docs/superpowers/plans/2026-07-18-personal-research-evidence-detour-continuity.md b/docs/superpowers/plans/2026-07-18-personal-research-evidence-detour-continuity.md new file mode 100644 index 000000000..368c1d302 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-personal-research-evidence-detour-continuity.md @@ -0,0 +1,273 @@ +# Personal Research Evidence Detour Continuity Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Keep Company Workbench evidence detours inside Personal Research mode and provide one direct return action without adding routes or changing evidence state. + +**Architecture:** Pure helpers in `src/research_workspace.py` own same-mode evidence URLs and the deterministic return destination. `src/dashboard.py` renders the return action only for research-mode Data Health and Proof History, before existing evidence content. AppTest render smoke expands from four primary routes to those four plus the two secondary evidence detours. + +**Tech Stack:** Python 3, Streamlit, pandas, pytest, AppTest, Markdown. + +## Global Constraints + +- Do not run `make readiness`. +- Do not create or stage CSV, JSON, report, sample-report, screenshot, timing, or bytecode churn. +- Do not add a route, alias, session state, persistence layer, or data mutation. +- Data Health and Proof History remain secondary Advanced evidence routes. +- Public and Operator mode behavior remains unchanged. +- Missing ticker returns to Research Desk; never infer a company. +- Navigation cannot mutate readiness, source, valuation, consensus, forecast, catalyst, outcome, backtest, or calibration state. +- Preserve research-only, candidate-context, source-rights, explicit-Q4, EPS split-basis, synthetic-fixture, and no-investment-advice boundaries. + +--- + +### Task 1: Same-mode evidence and return-link helpers + +**Files:** +- Modify: `src/research_workspace.py:359-376` +- Test: `tests/test_research_workspace.py:336-351` + +**Interfaces:** +- Consumes: existing URL quoting and ticker normalization in `src.research_workspace`. +- Produces: `advanced_evidence_links(ticker: str) -> list[dict[str, str]]` with research-mode URLs and `research_evidence_return_link(ticker: str) -> dict[str, str]`. + +- [ ] **Step 1: Write failing route-helper tests** + +Replace the cross-mode expectation and add return-destination coverage: + +```python +def test_advanced_evidence_links_preserve_personal_research_mode_and_ticker(): + links = advanced_evidence_links("NVDA") + + assert links[0]["href"] == "?mode=research&page=data-health&ticker=NVDA" + assert links[1]["href"] == "?mode=research&page=proof-history&ticker=NVDA" + + +def test_research_evidence_return_link_preserves_ticker_or_falls_back_to_desk(): + assert research_evidence_return_link("BRK/B") == { + "label": "Return to Company Workbench", + "href": "?mode=research&page=company-workbench&ticker=BRK%2FB&open=1", + "purpose": "Continue the selected-company review without changing evidence state.", + } + assert research_evidence_return_link("")["href"] == "?mode=research&page=research-desk" +``` + +Use a URL-encoding assertion for a ticker such as `BRK/B`. The implementation must call `quote(..., safe="")`, so the expected query value is `BRK%2FB`. + +- [ ] **Step 2: Run the tests and verify red** + +Run: `python3 -m pytest tests/test_research_workspace.py -q` + +Expected: FAIL because Advanced Evidence still switches modes and `research_evidence_return_link` does not exist. + +- [ ] **Step 3: Implement minimal pure routing helpers** + +Import `research_evidence_return_link` in the test and implement: + +```python +def _quoted_ticker(ticker: str) -> str: + return quote(str(ticker or "").strip().upper(), safe="") + + +def advanced_evidence_links(ticker: str) -> list[dict[str, str]]: + symbol = _quoted_ticker(ticker) + suffix = f"&ticker={symbol}" if symbol else "" + return [ + { + "label": "Open Data Health", + "href": f"?mode=research&page=data-health{suffix}", + "purpose": "Inspect blocked inputs and source-proof paths.", + }, + { + "label": "Open Proof History", + "href": f"?mode=research&page=proof-history{suffix}", + "purpose": "Review evidence that changed a readiness state.", + }, + ] + + +def research_evidence_return_link(ticker: str) -> dict[str, str]: + symbol = _quoted_ticker(ticker) + if symbol: + return { + "label": "Return to Company Workbench", + "href": f"?mode=research&page=company-workbench&ticker={symbol}&open=1", + "purpose": "Continue the selected-company review without changing evidence state.", + } + return { + "label": "Return to Research Desk", + "href": "?mode=research&page=research-desk", + "purpose": "Return to the primary research workflow without changing evidence state.", + } +``` + +- [ ] **Step 4: Verify helper behavior** + +Run: `python3 -m pytest tests/test_research_workspace.py -q` + +Expected: PASS, including command-free Advanced Evidence HTML tests. + +- [ ] **Step 5: Commit the helper slice** + +```bash +git add -- src/research_workspace.py tests/test_research_workspace.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Keep research evidence links in workspace" +``` + +### Task 2: Evidence-page return action and render coverage + +**Files:** +- Modify: `src/dashboard.py:330-345,34470-34493` +- Modify: `src/dashboard_render_smoke.py:84-127` +- Test: `tests/test_research_mode_dashboard_contract.py` +- Test: `tests/test_dashboard_render_smoke.py` + +**Interfaces:** +- Consumes: `research_evidence_return_link(ticker) -> dict[str, str]` from Task 1. +- Produces: a primary `st.link_button` on research-mode Data Health and Proof History, plus AppTest route contracts for both detours. + +- [ ] **Step 1: Write failing dashboard source-order tests** + +Add source-contract assertions that each research evidence branch obtains the current ticker, renders the research header, renders `research_evidence_return_link`, then renders the existing page: + +```python +def test_research_evidence_detours_offer_return_before_evidence_content(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + data_start = source.index('elif content_page == "Data Health":') + data_end = source.index('elif content_page == PROOF_HISTORY_PATH_TITLE:', data_start) + data = source[data_start:data_end] + proof_start = data_end + proof_end = source.index('elif content_page == "Universe Manager":', proof_start) + proof = source[proof_start:proof_end] + + for branch, renderer in ((data, "render_data_health("), (proof, "render_proof_history(")): + header = branch.index("render_research_workspace_header(") + return_link = branch.index("research_evidence_return_link(", header) + button = branch.index("st.link_button(", return_link) + content = branch.index(renderer, button) + assert header < return_link < button < content +``` + +- [ ] **Step 2: Add failing render-route contracts** + +Append two `DashboardRenderRoute` entries to `RESEARCH_RENDER_ROUTES`: + +```python +DashboardRenderRoute( + name="Research Data Health", + query_params=(("mode", "research"), ("page", "data-health"), ("ticker", "NVDA")), + required_markers=("Data Health", "Return to Company Workbench", "Research-only"), +), +DashboardRenderRoute( + name="Research Proof History", + query_params=(("mode", "research"), ("page", "proof-history"), ("ticker", "NVDA")), + required_markers=("Proof History", "Return to Company Workbench", "Research-only"), +), +``` + +Extend `tests/test_dashboard_render_smoke.py` to expect the six research route names in order. + +- [ ] **Step 3: Run focused tests and verify red** + +Run: + +```bash +python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q +``` + +Expected: FAIL because the return action is not integrated or rendered. + +- [ ] **Step 4: Implement the return action in existing branches** + +Import `research_evidence_return_link` beside the existing research workspace helpers. In each research-mode evidence branch: + +```python +ticker = str(st.query_params.get("ticker") or "").strip().upper() +render_research_workspace_header(..., ticker=ticker, ...) +return_link = research_evidence_return_link(ticker) +st.link_button(return_link["label"], return_link["href"], type="primary") +``` + +Do not render the button for Public or Operator modes. Leave `render_data_health(..., public_mode=not operator_mode)` and `render_proof_history(public_mode=not operator_mode)` unchanged. + +- [ ] **Step 5: Verify route contracts and render smoke** + +Run: + +```bash +python3 -m pytest tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q +make research-dashboard-render-smoke +``` + +Expected: all six research routes pass; no screenshots or output files are created. + +- [ ] **Step 6: Commit the dashboard integration** + +```bash +git add -- src/dashboard.py src/dashboard_render_smoke.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add return path for research evidence detours" +``` + +### Task 3: Documentation, complete verification, and draft PR + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/DASHBOARD_QA.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` +- Update: draft PR #113 comment only after verification. + +**Interfaces:** +- Consumes: verified same-mode evidence detours and return action. +- Produces: durable workflow and QA claims bounded to local route continuity. + +- [ ] **Step 1: Write failing documentation assertions** + +Require ROADMAP, Personal Research Mode, Dashboard QA, and the continuation prompt to state that Data Health and Proof History preserve Personal Research mode and expose a return path without changing readiness or evidence state. + +- [ ] **Step 2: Run the documentation contract and verify red** + +Run: `python3 -m pytest tests/test_public_v1_release_docs.py -q` + +Expected: FAIL because the continuity claim is not yet documented. + +- [ ] **Step 3: Update documentation without overstating maturity** + +Record that the slice improves local workflow continuity and evidence-review usability. State explicitly that it does not prove source rights, hosted behavior, accessibility compliance, reviewer adoption, market demand, calibration, or product-market fit. Preserve the four primary route hierarchy and secondary Advanced evidence boundary. + +- [ ] **Step 4: Run focused and full verification** + +```bash +python3 -m pytest tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_public_v1_release_docs.py -q +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: executable local gates pass; pilot remains blocked by stale saved readiness; zero generated artifact candidates. + +- [ ] **Step 5: Commit and push exact documentation paths** + +```bash +git add -- ROADMAP.md docs/PERSONAL_RESEARCH_MODE.md docs/DASHBOARD_QA.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Document research evidence detour continuity" +git push origin codex/personal-research-mode-mvp +``` + +- [ ] **Step 6: Update draft PR #113 and reassess the active goal** + +Comment with the behavior, exact test evidence, no-artifact proof, screenshot-audit limitation, and remaining external gates. Verify the PR remains open and draft, branch divergence is zero, and the worktree is clean. Continue the overall goal unless every source, hosted, reviewer, evidence-depth, calibration, and operating gate has direct evidence. diff --git a/docs/superpowers/plans/2026-07-18-quarterly-cash-generation-adapter-acceptance.md b/docs/superpowers/plans/2026-07-18-quarterly-cash-generation-adapter-acceptance.md new file mode 100644 index 000000000..a1c322cf7 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-quarterly-cash-generation-adapter-acceptance.md @@ -0,0 +1,461 @@ +# Quarterly Cash-Generation Adapter Acceptance Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a pure one-company acceptance harness that fails closed on source rights, identity, cutoff, revision, Q4, component, and compatibility defects without reading or writing adapter files or activating production readiness. + +**Architecture:** A focused adapter-governance module composes the immutable commercial source-rights registry with the existing quarterly cash-generation derivation. It returns one immutable acceptance result with deterministic blockers and an explicit non-activation boundary. Documentation and contract tests keep `accepted_for_review` distinct from real source review, production activation, and market validation. + +**Tech Stack:** Python 3.12 dataclasses and immutable mappings, existing `src.commercial_source_rights` and `src.quarterly_cash_generation` contracts, pytest, Markdown contract tests, Make release gates. + +## Global Constraints + +- Accept only in-memory `QuarterlyBusinessObservation` and `QuarterlyActual` objects; add no file loader or writer. +- Add no CLI, Make target, network request, credential read, data directory, CSV, JSON, report, sample report, screenshot, timing output, template, or canonical row. +- The only successful status is `accepted_for_review`; it never means reviewed, production-ready, or commercially activated. +- `production_activation` is always `False`; `readiness_promotions` is always empty. +- Commercial rights and all three required source fields must be explicitly approved in the supplied registry. +- Q4 remains explicit filed-quarter only; annual-minus-nine-month derivation is forbidden. +- Capital expenditures preserve the reported sign; free cash flow remains cash from operations plus reported capital expenditures. +- Synthetic observations and synthetic rights records remain test-only. +- Stage exact intentional files only; never use `git add -A`; keep all generated churn excluded. +- Keep PR #113 open and draft; do not merge or deploy. + +--- + +### Task 1: Pure One-Company Acceptance Contract + +**Files:** +- Create: `src/quarterly_cash_generation_adapter.py` +- Create: `tests/test_quarterly_cash_generation_adapter.py` + +**Interfaces:** +- Consumes: `Mapping[str, SourceRights]`, `Iterable[QuarterlyBusinessObservation]`, `Iterable[QuarterlyActual]`, and optional `as_of: str | None`. +- Produces: `QuarterlyAdapterAcceptance` and `assess_quarterly_cash_generation_adapter(ticker, source_id, observations, revenue_actuals, *, rights_registry, as_of=None)`. + +- [ ] **Step 1: Add shared synthetic constructors and the first failing success-path test** + +Create `tests/test_quarterly_cash_generation_adapter.py` with constructors for one `QuarterlyBusinessObservation`, one compatible `QuarterlyActual`, and an immutable registry built through `build_source_rights_registry`. The success test must assert the complete public result: + +```python +def test_complete_one_company_batch_is_accepted_for_review_without_activation(): + observations = [ + _observation(metric="operating_income", value=50.0), + _observation(metric="cash_from_operations", value=60.0), + _observation(metric="capital_expenditures", value=-20.0), + ] + + result = assess_quarterly_cash_generation_adapter( + "SYN1", + "synthetic_adapter", + observations, + [_actual(revenue=200.0)], + rights_registry=_rights_registry(), + ) + + assert result.status == "accepted_for_review" + assert result.blockers == () + assert result.accepted_observation_count == 3 + assert result.reviewed_metrics == ( + "capital_expenditures", + "cash_from_operations", + "operating_income", + ) + assert result.derived_point_count == 3 + assert result.rights_status == "approved" + assert result.production_activation is False + assert result.readiness_promotions == () +``` + +- [ ] **Step 2: Run the test and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_quarterly_cash_generation_adapter.py::test_complete_one_company_batch_is_accepted_for_review_without_activation -q +``` + +Expected: collection fails because `src.quarterly_cash_generation_adapter` does not exist. + +- [ ] **Step 3: Implement the immutable result and minimal success path** + +Create `src/quarterly_cash_generation_adapter.py` with: + +```python +from __future__ import annotations + +from dataclasses import dataclass +from typing import Iterable, Mapping + +from src.commercial_source_rights import SourceRights, commercial_eligibility +from src.earnings_nowcast_contract import QuarterlyActual +from src.quarterly_cash_generation import ( + QuarterlyBusinessObservation, + derive_quarterly_business_metrics, +) + +REQUIRED_SOURCE_FIELDS = frozenset( + {"operating_income", "cash_from_operations", "capital_expenditures"} +) + + +@dataclass(frozen=True) +class QuarterlyAdapterAcceptance: + ticker: str + source_id: str + status: str + blockers: tuple[str, ...] + accepted_observation_count: int + reviewed_metrics: tuple[str, ...] + derived_point_count: int + explicit_q4_periods: tuple[str, ...] + rights_status: str + production_activation: bool = False + readiness_promotions: tuple[str, ...] = () + + +def assess_quarterly_cash_generation_adapter( + ticker: str, + source_id: str, + observations: Iterable[QuarterlyBusinessObservation], + revenue_actuals: Iterable[QuarterlyActual], + *, + rights_registry: Mapping[str, SourceRights], + as_of: str | None = None, +) -> QuarterlyAdapterAcceptance: + symbol = str(ticker or "").strip().upper() + normalized_source = str(source_id or "").strip() + supplied = tuple(observations) + revenues = tuple(revenue_actuals) + decision = commercial_eligibility(rights_registry, normalized_source) + derivation = derive_quarterly_business_metrics(symbol, supplied, revenues, as_of=as_of) + reviewed_metrics = tuple(sorted({row.metric for row in supplied})) + q4_periods = tuple(sorted({row.fiscal_period for row in supplied if row.fiscal_period.endswith("-Q4")})) + return QuarterlyAdapterAcceptance( + ticker=symbol, + source_id=normalized_source, + status="accepted_for_review", + blockers=(), + accepted_observation_count=len(supplied), + reviewed_metrics=reviewed_metrics, + derived_point_count=len(derivation.points), + explicit_q4_periods=q4_periods, + rights_status=decision.status, + ) +``` + +- [ ] **Step 4: Run the success-path test and verify GREEN** + +Run the Step 2 command. Expected: 1 passed. + +- [ ] **Step 5: Add failing identity, rights, and required-field tests** + +Add separate tests asserting these exact blockers: + +```python +@pytest.mark.parametrize( + ("ticker", "source_id", "observations", "blocker"), + [ + ("", "synthetic_adapter", (), "ticker_required"), + ("SYN1", "", (), "source_id_required"), + ("SYN1", "synthetic_adapter", (), "observations_required"), + ], +) +def test_required_identity_and_observations_fail_closed(ticker, source_id, observations, blocker): + result = assess_quarterly_cash_generation_adapter( + ticker, + source_id, + observations, + [_actual()], + rights_registry=_rights_registry(), + ) + assert result.status == "blocked" + assert blocker in result.blockers + assert result.accepted_observation_count == 0 + + +def test_mixed_ticker_and_source_mismatch_are_reported_together(): + rows = [ + _observation(), + _observation(ticker="OTHER"), + _observation(metric="operating_income", source="other_source"), + ] + result = _assess(rows) + assert "mixed_ticker:OTHER" in result.blockers + assert "source_mismatch:other_source" in result.blockers + + +def test_unknown_or_unverified_rights_block_acceptance(): + unknown = _assess(_complete_rows(), source_id="unknown") + assert "source_rights:unknown_source" in unknown.blockers + unverified = _assess(_complete_rows(), rights_registry=_rights_registry(commercial_use="unverified")) + assert "source_rights:commercial_rights_unverified" in unverified.blockers + + +def test_approved_source_must_explicitly_support_every_component(): + result = _assess( + _complete_rows(), + rights_registry=_rights_registry(supported_fields=["operating_income"]), + ) + assert result.blockers == ( + "source_fields_missing:capital_expenditures,cash_from_operations", + ) +``` + +- [ ] **Step 6: Run the new tests and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_quarterly_cash_generation_adapter.py -q +``` + +Expected: failures because the minimal implementation always accepts. + +- [ ] **Step 7: Implement deterministic identity and source-rights blockers** + +Add one blocker list, normalize identity before derivation, evaluate `commercial_eligibility`, compare `SourceRights.supported_fields` with `REQUIRED_SOURCE_FIELDS`, and collect sorted mixed ticker/source values. Return `blocked` whenever blockers are non-empty; blocked results set `accepted_observation_count=0`. + +Use exact blocker formats: + +```python +if not symbol: + blockers.append("ticker_required") +if not normalized_source: + blockers.append("source_id_required") +if not supplied: + blockers.append("observations_required") +for other in sorted({row.ticker for row in supplied if row.ticker != symbol}): + blockers.append(f"mixed_ticker:{other}") +for other in sorted({row.source for row in supplied if row.source != normalized_source}): + blockers.append(f"source_mismatch:{other}") +if not decision.allowed: + blockers.append(f"source_rights:{decision.status}") +record = rights_registry.get(normalized_source) +if record is not None: + missing_fields = sorted(REQUIRED_SOURCE_FIELDS - set(record.supported_fields)) + if missing_fields: + blockers.append(f"source_fields_missing:{','.join(missing_fields)}") +``` + +- [ ] **Step 8: Run the test module and verify GREEN** + +Run the Step 6 command. Expected: all current tests pass. + +- [ ] **Step 9: Add failing cutoff, revision, component, compatibility, and Q4 tests** + +Add tests that prove: + +- a post-cutoff observation returns its existing `YYYY-QN:metric:post_cutoff` blocker; +- one explicit revision leaf is accepted and increments only the derived revision behavior; +- conflicting capex leaves return `YYYY-QN:capital_expenditures:ambiguous_revision`; +- missing a required component returns `YYYY-QN:missing_component:`; +- missing or incompatible Revenue returns the existing operating-margin and FCF-margin blockers and prevents complete-period acceptance; +- incompatible cash-flow component definitions preserve the existing `free_cash_flow:incompatible_components` blocker; +- a complete Q4 batch with `explicit_filed_quarter` records `explicit_q4_periods == ("2025-Q4",)`; +- constructing a Q4 observation with `q4_evidence_state="not_q4"` still raises `ValueError` before the harness runs. + +The incomplete-period assertion must be: + +```python +assert "2025-Q1:missing_component:capital_expenditures" in result.blockers +assert "complete_derived_period_required" in result.blockers +``` + +- [ ] **Step 10: Run the test module and verify RED** + +Run the Step 6 command. Expected: new blocker and complete-period assertions fail. + +- [ ] **Step 11: Compose existing derivation blockers and enforce one complete derived period** + +For observations matching the requested ticker and source, group supplied metric names by fiscal period and append missing-component blockers in fiscal-period and metric order. Derive only requested-ticker observations, append `derivation.blockers`, and calculate periods containing all three derived metrics: + +```python +points_by_period: dict[str, set[str]] = {} +for point in derivation.points: + points_by_period.setdefault(point.fiscal_period, set()).add(point.metric) +complete_periods = tuple( + sorted( + period + for period, metrics in points_by_period.items() + if {"operating_margin", "free_cash_flow", "fcf_margin"}.issubset(metrics) + ) +) +if not complete_periods: + blockers.append("complete_derived_period_required") +``` + +Deduplicate blockers in insertion order with `tuple(dict.fromkeys(blockers))`. Set `derived_point_count` from the derivation even when blocked, but keep accepted observation count at zero. + +- [ ] **Step 12: Run the test module and verify GREEN** + +Run the Step 6 command. Expected: all acceptance tests pass. + +- [ ] **Step 13: Add and satisfy no-persistence surface tests** + +Add: + +```python +def test_adapter_acceptance_module_has_no_file_network_or_cli_surface(): + source = Path("src/quarterly_cash_generation_adapter.py").read_text(encoding="utf-8") + makefile = Path("Makefile").read_text(encoding="utf-8") + for forbidden in ( + "argparse", "requests", "urllib", "Path(", ".open(", "read_text(", + "write_text(", "csv", "json", "output_dir", "__main__", + ): + assert forbidden not in source + assert "quarterly-cash-generation-adapter" not in makefile +``` + +Run: + +```bash +python3 -m pytest tests/test_quarterly_cash_generation_adapter.py tests/test_quarterly_cash_generation.py -q +``` + +Expected: all tests pass. + +- [ ] **Step 14: Commit the contract slice** + +```bash +git add -- src/quarterly_cash_generation_adapter.py tests/test_quarterly_cash_generation_adapter.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add quarterly adapter acceptance harness" +git push origin codex/personal-research-mode-mvp +``` + +--- + +### Task 2: Methodology, Roadmap, And Continuation Contract + +**Files:** +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes: the Task 1 `accepted_for_review`, blocker, rights, and non-activation contract. +- Produces: durable maturity and continuation claims plus documentation regression coverage. + +- [ ] **Step 1: Write the failing documentation contract test** + +Add to `tests/test_public_v1_release_docs.py`: + +```python +def test_quarterly_adapter_acceptance_docs_keep_review_and_activation_separate(): + methodology = _read("docs/METHODOLOGY.md") + provenance = _read("docs/PROVENANCE_CONTRACT.md") + personal_mode = _read("docs/PERSONAL_RESEARCH_MODE.md") + roadmap = _read("ROADMAP.md") + prompt = _read("docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md") + + assert "accepted_for_review is not production activation" in methodology + assert "production_activation=false" in provenance + assert "readiness_promotions=()" in provenance + assert "no adapter file is loaded or written" in personal_mode + assert "one-company adapter acceptance harness" in roadmap + assert "does not prove a real-company source payload" in roadmap + assert "Quarterly adapter acceptance" in prompt + assert "accepted_for_review" in prompt +``` + +- [ ] **Step 2: Run the test and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py::test_quarterly_adapter_acceptance_docs_keep_review_and_activation_separate -q +``` + +Expected: failure because the new acceptance distinctions are absent. + +- [ ] **Step 3: Update methodology and provenance** + +Document the one-company scope, explicit rights-field checks, complete compatible period requirement, deterministic blockers, and the exact statement `accepted_for_review is not production activation`. In provenance, record `production_activation=false`, `readiness_promotions=()`, no file/network/credential surface, and unchanged external review requirement. + +- [ ] **Step 4: Update Personal Research guidance and ROADMAP** + +State that no adapter file is loaded or written and Company Workbench remains withheld because production supplies no accepted observations. Add the implemented one-company adapter acceptance harness to the current foundation and state that it does not prove a real-company source payload, reviewed rights expansion, or market validation. + +- [ ] **Step 5: Update the persistent continuation goal** + +Add the local acceptance harness to implemented capabilities. Add a `Quarterly adapter acceptance` boundary that requires the checked source-rights record to explicitly support all three component fields and still classifies real activation as `external_source_and_review_required`. Preserve automatic continuation across safe local work and the strict no-generated-artifact rule. + +- [ ] **Step 6: Run focused documentation and contract tests** + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py tests/test_quarterly_cash_generation_adapter.py tests/test_quarterly_cash_generation.py -q +``` + +Expected: all pass. + +- [ ] **Step 7: Commit the documentation slice** + +```bash +git add -- docs/METHODOLOGY.md docs/PROVENANCE_CONTRACT.md docs/PERSONAL_RESEARCH_MODE.md ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Document quarterly adapter acceptance boundary" +git push origin codex/personal-research-mode-mvp +``` + +--- + +### Task 3: Full Release Verification And Draft PR Update + +**Files:** +- No new product files. +- Update remote draft PR #113 only after all local evidence passes. + +**Interfaces:** +- Consumes: committed Tasks 1-2. +- Produces: clean aligned branch, verified release evidence, updated draft PR, and next-goal prompt grounded in current truth. + +- [ ] **Step 1: Run the full verification bundle** + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: tests and static/runtime gates pass; pilot readiness may retain named manual external gates; no generated artifact candidate exists. + +- [ ] **Step 2: Verify repository and PR truth** + +```bash +git status --short --branch +git rev-list --left-right --count origin/codex/personal-research-mode-mvp...HEAD +gh pr view 113 --json state,isDraft,mergeStateStatus,headRefName,url +``` + +Expected: clean tree, `0 0` divergence, PR open and draft on the named branch. + +- [ ] **Step 3: Update draft PR #113** + +Add a verified comment covering: + +- pure one-company acceptance behavior; +- deterministic rights, identity, cutoff, revision, component, compatibility, and Q4 blockers; +- `accepted_for_review` versus production activation; +- no file, CLI, network, credential, writer, or generated artifact; +- exact test and release-gate results; +- the current SEC rights-record field limitation; +- `external_source_and_review_required` as the real activation gate. + +Keep the PR draft and do not merge or deploy. + +- [ ] **Step 4: Produce the next-goal prompt** + +Deliver a copy-ready prompt that starts from current repository truth, verifies the latest commit and draft PR, preserves no-file and independent-readiness boundaries, classifies the real adapter/source dependency once, and moves to the next safe local maturity gap without repeating broad source or generated-artifact loops. diff --git a/docs/superpowers/plans/2026-07-18-quarterly-cash-generation-contract.md b/docs/superpowers/plans/2026-07-18-quarterly-cash-generation-contract.md new file mode 100644 index 000000000..176c0ac87 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-quarterly-cash-generation-contract.md @@ -0,0 +1,397 @@ +# Quarterly Cash-Generation Evidence Contract Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add source-bound quarterly operating-margin, free-cash-flow, and FCF-margin trend readiness without adding a data file, changing Earnings Nowcast, or fabricating real-company evidence. + +**Architecture:** A focused domain module validates immutable in-memory component observations and derives metric points from compatible evidence. The existing quarterly trend composer accepts observations optionally and keeps metric readiness independent; production callers supply none until a reviewed adapter exists, so real output remains withheld. Existing answer-first cards show conclusions while component lineage stays Advanced. + +**Tech Stack:** Python 3.12 dataclasses, existing `parse_utc_timestamp` and quarterly trend contracts, pytest, Streamlit AppTest. + +## Global Constraints + +- Do not add, generate, modify, or stage any CSV, JSON, report, sample report, screenshot, browser timing, canonical data, template, writer, or Make target. +- Keep `QuarterlyActual`, Earnings Nowcast input hashes, Revenue/EPS readiness, consensus readiness, ranges, backtesting, and calibration unchanged. +- Q4 requires `explicit_filed_quarter`; annual-minus-nine-month derivation is forbidden. +- Free cash flow is exactly `cash_from_operations + capital_expenditures`, preserving the reported capex cash-flow sign. +- Do not infer capex sign, accounting basis, duration basis, source time, fiscal identity, or metric compatibility. +- Supplemental readiness cannot unlock DCF, peers, catalysts, outcomes, Nowcast, backtesting, calibration, rankings, recommendations, or actions. +- Synthetic observations are test-only and use `source="synthetic_test_fixture"`. +- Empty production input stays visibly withheld. +- Stage exact code, documentation, and test paths only; never use `git add -A`. + +## File Map + +- Create `src/quarterly_cash_generation.py`: observation validation, revision resolution, compatibility, derived metric points, and blockers. +- Create `tests/test_quarterly_cash_generation.py`: pure in-memory domain tests. +- Modify `src/quarterly_business_trend.py` and `tests/test_quarterly_business_trend.py`: optional observations and independent trends. +- Modify `src/research_workspace.py`, `tests/test_research_workspace.py`, and `tests/test_dashboard_helpers.py`: answer-first cards and Advanced evidence boundary. +- Modify `docs/METHODOLOGY.md`, `docs/PROVENANCE_CONTRACT.md`, `docs/PERSONAL_RESEARCH_MODE.md`, `ROADMAP.md`, `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, and `tests/test_public_v1_release_docs.py`: durable claims and maturity boundary. + +--- + +### Task 1: Immutable Quarterly Component Evidence + +**Files:** +- Create: `src/quarterly_cash_generation.py` +- Create: `tests/test_quarterly_cash_generation.py` + +**Interfaces:** +- Produces: `QuarterlyBusinessObservation`, `QuarterlyBusinessMetricPoint`, `QuarterlyBusinessDerivation`, and `derive_quarterly_business_metrics(ticker, observations, revenue_actuals, *, as_of=None)`. +- Consumes: `QuarterlyActual` and `parse_utc_timestamp` from `src.earnings_nowcast_contract`. + +- [ ] **Step 1: Write failing constructor tests** + +Use this fixture and add parameterized failures for unsupported metric, invalid fiscal period, non-finite value, non-positive scale, naive timestamps, and invalid Q4 evidence: + +```python +def observation(period="2025-Q1", metric="cash_from_operations", value=100.0, **overrides): + values = { + "ticker": "syn1", "fiscal_period": period, "period_end_date": "2025-03-31", + "metric": metric, "value": value, "currency": "usd", "unit_scale": 1.0, + "accounting_basis": "gaap", "duration_basis": "three_months", + "source": "synthetic_test_fixture", "source_ref": f"fixture:{period}:{metric}", + "published_at": "2025-05-15T12:00:00+00:00", + "retrieved_at": "2026-07-18T12:00:00+00:00", "q4_evidence_state": "not_q4", + } + values.update(overrides) + return QuarterlyBusinessObservation(**values) + +def test_observation_requires_explicit_q4_evidence(): + with pytest.raises(ValueError, match="Q4 requires explicit filed-quarter evidence"): + observation(period="2025-Q4", period_end_date="2025-12-31") +``` + +- [ ] **Step 2: Verify the tests fail** + +Run: `python3 -m pytest tests/test_quarterly_cash_generation.py -q` + +Expected: import failure because the module does not exist. + +- [ ] **Step 3: Implement immutable types and validation** + +Implement these exact public types: + +```python +SUPPORTED_COMPONENT_METRICS = frozenset({"operating_income", "cash_from_operations", "capital_expenditures"}) + +@dataclass(frozen=True) +class QuarterlyBusinessObservation: + ticker: str + fiscal_period: str + period_end_date: str + metric: str + value: float + currency: str + unit_scale: float + accounting_basis: str + duration_basis: str + source: str + source_ref: str + published_at: str + retrieved_at: str + q4_evidence_state: str = "not_q4" + supersedes_source_ref: str | None = None + +@dataclass(frozen=True) +class QuarterlyBusinessMetricPoint: + metric: str + fiscal_period: str + period_end_date: str + value: float + definition: tuple[object, ...] + source_refs: tuple[str, ...] + +@dataclass(frozen=True) +class QuarterlyBusinessDerivation: + points: tuple[QuarterlyBusinessMetricPoint, ...] + blockers: tuple[str, ...] + revision_count: int + supplied_observation_count: int +``` + +Validate ISO dates with `date.fromisoformat`, fiscal periods with `^\d{4}-Q[1-4]$`, numeric values with `math.isfinite`, timestamps with `parse_utc_timestamp`, and required text after normalization. Permit `not_q4` only for Q1-Q3 and `explicit_filed_quarter` only for Q4. + +- [ ] **Step 4: Verify constructor tests pass** + +Run: `python3 -m pytest tests/test_quarterly_cash_generation.py -q` + +Expected: constructor tests pass. + +- [ ] **Step 5: Write failing derivation tests** + +Test the exact formulas and source lineage: + +```python +def test_derivation_uses_explicit_components(): + result = derive_quarterly_business_metrics( + "SYN1", + [observation(metric="operating_income", value=50.0), + observation(metric="cash_from_operations", value=60.0), + observation(metric="capital_expenditures", value=-20.0)], + [actual("2025-Q1", revenue=200.0)], + ) + points = {point.metric: point for point in result.points} + assert points["operating_margin"].value == 0.25 + assert points["free_cash_flow"].value == 40.0 + assert points["fcf_margin"].value == 0.20 + assert points["free_cash_flow"].source_refs == ( + "fixture:2025-Q1:cash_from_operations", "fixture:2025-Q1:capital_expenditures" + ) +``` + +Also assert cutoff filtering, explicit revision resolution, affected-component ambiguity, and incompatibility across currency, scale, accounting basis, duration basis, fiscal period, and period end. + +- [ ] **Step 6: Implement revision resolution and derivation** + +Group by `(fiscal_period, metric)`, deduplicate source references, accept exact duplicates, and resolve only one explicit revision leaf. Record stable blockers such as `2025-Q1:capital_expenditures:ambiguous_revision`. Require compatible Revenue and a non-zero denominator for margin points. Calculate: + +```python +operating_margin = operating_income.value / revenue.revenue_actual +free_cash_flow = cash_from_operations.value + capital_expenditures.value +fcf_margin = free_cash_flow / revenue.revenue_actual +``` + +- [ ] **Step 7: Verify domain behavior and absence of persistence** + +Run: + +```bash +python3 -m pytest tests/test_quarterly_cash_generation.py -q +rg -n "open\(|write_text|write_bytes|to_csv|json\.dump|csv\.writer|argparse|output_dir" src/quarterly_cash_generation.py +``` + +Expected: tests pass and the persistence scan returns no matches. + +- [ ] **Step 8: Commit Task 1** + +```bash +git add -- src/quarterly_cash_generation.py tests/test_quarterly_cash_generation.py +git commit -m "Add quarterly cash generation evidence contract" +``` + +--- + +### Task 2: Independent Trend Composition + +**Files:** +- Modify: `src/quarterly_business_trend.py:13-345` +- Modify: `tests/test_quarterly_business_trend.py` + +**Interfaces:** +- Consumes Task 1 types and derivation. +- Produces `QuarterlyTrendPacket.operating_margin`, `.free_cash_flow`, `.fcf_margin`, and the `business_observations` builder argument. + +- [ ] **Step 1: Write failing integration tests** + +```python +def test_supplemental_metrics_stay_withheld_without_observations(): + packet = build_quarterly_trend_packet("SYN1", [_actual("2025-Q1", revenue=120, eps=1.2)]) + assert packet.operating_margin.status == "withheld" + assert packet.free_cash_flow.status == "withheld" + assert packet.fcf_margin.status == "withheld" + +def test_supplemental_metric_trends_are_independent(): + packet = build_quarterly_trend_packet( + "SYN1", actual_history(), business_observations=business_history() + ) + assert packet.operating_margin.status == "ready" + assert packet.free_cash_flow.status == "ready" + assert packet.fcf_margin.status == "ready" + assert packet.revenue.latest_value == 120 + assert packet.eps.latest_value == 1.2 +``` + +- [ ] **Step 2: Verify integration tests fail** + +Run: `python3 -m pytest tests/test_quarterly_business_trend.py -q` + +Expected: missing fields and keyword argument. + +- [ ] **Step 3: Extend the packet and builder** + +Add three `QuarterlyMetricTrend` fields after `eps`, plus: + +```python +def _withheld_metric(metric: str, reason: str) -> QuarterlyMetricTrend: + return QuarterlyMetricTrend(metric, "withheld", None, "", "", None, None, (), (), reason) + +def build_quarterly_trend_packet( + ticker: str, + actuals: Iterable[QuarterlyActual], + *, + as_of: str | None = None, + business_observations: Iterable[QuarterlyBusinessObservation] = (), +) -> QuarterlyTrendPacket: +``` + +Materialize the optional iterable once. Zero observations yield three withheld trends. Otherwise derive points from the resolved Revenue rows and reuse exact previous-quarter, prior-year-quarter, definition-compatibility, and percent-change behavior. Keep packet-level status based on Revenue/EPS exactly as before. + +- [ ] **Step 4: Render all five rows from packet state** + +Replace hard-coded supplemental placeholders with one loop over Revenue, EPS, operating margin, free cash flow, and FCF margin. Format margin display values as percentages while retaining decimal values in the domain packet. Preserve component references joined with `;` only in the Advanced row. + +- [ ] **Step 5: Run focused and Nowcast regressions** + +```bash +python3 -m pytest tests/test_quarterly_cash_generation.py tests/test_quarterly_business_trend.py -q +python3 -m pytest tests/test_earnings_nowcast_contract.py tests/test_earnings_nowcast_readiness.py tests/test_earnings_nowcast_report.py -q +``` + +Expected: all pass with no schema or input-hash change. + +- [ ] **Step 6: Commit Task 2** + +```bash +git add -- src/quarterly_business_trend.py tests/test_quarterly_business_trend.py +git commit -m "Compose independent quarterly cash generation trends" +``` + +--- + +### Task 3: Answer-First Workbench Rendering + +**Files:** +- Modify: `src/research_workspace.py:238-266` +- Modify: `tests/test_research_workspace.py:91-115` +- Modify: `tests/test_dashboard_helpers.py` + +**Interfaces:** +- Consumes Task 2 packet fields. +- Produces truthful supported or withheld cards without a loader, writer, or source promotion. + +- [ ] **Step 1: Write failing card tests** + +Assert `OPERATING MARGIN`, `FREE CASH FLOW`, and `FCF MARGIN` cards are `Withheld` without observations and say a reviewed quarterly source adapter is required. With ready in-memory observations, assert values and comparison changes appear, but raw source references and formula internals do not appear in primary card text. + +- [ ] **Step 2: Verify card tests fail** + +Run: `python3 -m pytest tests/test_research_workspace.py -q` + +Expected: supplemental cards are missing. + +- [ ] **Step 3: Extend the existing card loop** + +Use five entries with `number` display for Revenue/EPS/FCF and `percent` for operating margin/FCF margin. Card bodies contain comparison changes or a concise boundary only. Source references, component values, formula details, and commands remain outside primary cards. + +- [ ] **Step 4: Add dashboard boundary assertions** + +Assert the dashboard still loads no supplemental file and keeps the table inside the existing collapsed quarterly source-evidence expander. Assert no new writer, template, output path, or Make target exists for this feature. + +- [ ] **Step 5: Verify workspace and Research routes** + +```bash +python3 -m pytest tests/test_research_workspace.py tests/test_dashboard_helpers.py -q +make research-dashboard-render-smoke +``` + +Expected: tests and all four Research routes pass; production supplemental output remains withheld. + +- [ ] **Step 6: Commit Task 3** + +```bash +git add -- src/research_workspace.py tests/test_research_workspace.py tests/test_dashboard_helpers.py +git commit -m "Render cash generation evidence without file output" +``` + +--- + +### Task 4: Methodology, Maturity Boundary, And Release Verification + +**Files:** +- Modify: `docs/METHODOLOGY.md:41-49` +- Modify: `docs/PROVENANCE_CONTRACT.md:38-48` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md:55-62` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes verified Tasks 1-3. +- Produces durable methodology and maturity claims. + +- [ ] **Step 1: Write failing documentation tests** + +```python +def test_quarterly_cash_generation_docs_preserve_no_file_boundary(): + methodology = _read("docs/METHODOLOGY.md") + provenance = _read("docs/PROVENANCE_CONTRACT.md") + roadmap = _read("ROADMAP.md") + assert "cash from operations + reported capital expenditures" in methodology + assert "explicit_filed_quarter" in provenance + assert "no new data file, writer, template, or generated artifact" in provenance + assert "methodology maturity" in roadmap + assert "does not prove real-company coverage or market validation" in roadmap +``` + +- [ ] **Step 2: Verify documentation tests fail** + +Run: `python3 -m pytest tests/test_public_v1_release_docs.py -q` + +Expected: new claims are absent. + +- [ ] **Step 3: Update methodology, provenance, mode guide, roadmap, and continuation prompt** + +Document formulas, source timestamps, revision behavior, compatibility, Q4 evidence, independent readiness, no persistence surface, unchanged downstream readiness, and withheld production state. Record that this improves methodology completeness, cash-conversion transparency, adapter extensibility, and reviewer trust—but does not prove real-company coverage, licensed source operation, hosted reliability, external reviewer success, calibration, commercial demand, or product-market fit. + +- [ ] **Step 4: Run focused tests** + +```bash +python3 -m pytest tests/test_quarterly_cash_generation.py tests/test_quarterly_business_trend.py tests/test_research_workspace.py tests/test_public_v1_release_docs.py -q +``` + +Expected: all pass. + +- [ ] **Step 5: Run the complete verification bundle** + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: every gate passes; before commit only intentional code/docs/tests appear, with zero generated artifact candidates. + +- [ ] **Step 6: Stage exact implementation paths and verify** + +```bash +git add -- src/quarterly_cash_generation.py src/quarterly_business_trend.py src/research_workspace.py tests/test_quarterly_cash_generation.py tests/test_quarterly_business_trend.py tests/test_research_workspace.py tests/test_dashboard_helpers.py tests/test_public_v1_release_docs.py docs/METHODOLOGY.md docs/PROVENANCE_CONTRACT.md docs/PERSONAL_RESEARCH_MODE.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md ROADMAP.md +make staged-hygiene-check +git diff --cached --check +``` + +Expected: product/code/docs/test paths only; zero generated, canonical-data, report, sample-report, or manual-review paths. + +- [ ] **Step 7: Commit, push, and update PR #113** + +```bash +git commit -m "Add quarterly cash generation evidence contract" +git push origin codex/personal-research-mode-mvp +``` + +Update PR #113 with independent readiness, no-file boundary, results, maturity assessment, and the external reviewed-adapter dependency. Keep it draft. + +- [ ] **Step 8: Verify final truth** + +```bash +git status --short --branch +git rev-list --left-right --count HEAD...origin/codex/personal-research-mode-mvp +gh pr view 113 --json state,isDraft,mergeStateStatus,headRefOid,url +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +``` + +Expected: clean tree, `0 0`, PR open and draft at HEAD, pilot-ready with manual gates, clean hygiene. + +## Plan Self-Review + +- Every spec field, formula, cutoff, revision, Q4, readiness, rendering, no-file, documentation, and verification requirement maps to Tasks 1-4. +- Scope is one domain contract and existing-workflow integration; no adapter, extraction, persistence, new route, scheduler, or external activation. +- Task 1 type names and Task 2-3 consumers are consistent. +- The plan creates Python code/tests and edits docs/tests only; it introduces no generated artifact path. +- Maturity claims distinguish methodology and trust gains from market validation, hosting, licensed operation, calibration, and product-market fit. diff --git a/docs/superpowers/plans/2026-07-18-readiness-change-cause-review.md b/docs/superpowers/plans/2026-07-18-readiness-change-cause-review.md new file mode 100644 index 000000000..1ca66995f --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-readiness-change-cause-review.md @@ -0,0 +1,34 @@ +# Readiness Change Cause Review Implementation Plan + +**Goal:** Make every major saved-versus-proposed readiness transition semantically inspectable without writing artifacts or changing readiness decisions. + +**Architecture:** Add a behavior-equivalent named-reason interface to `src.company_analysis_scope`, then add a pure change-transition review to `src.readiness_preview`. The existing preview builder supplies proposed metadata plus canonical fundamentals to the review after its single no-write readiness build. + +## Task 1: Named company-scope reasons + +- Add failing tests for each existing exclusion family, overlapping reasons, nonpositive-revenue inputs, and unchanged boolean results. +- Implement ordered named patterns and `company_dcf_exclusion_reasons`. +- Keep `COMPANY_DCF_EXCLUDED_TEXT_PATTERNS` and boolean helpers backward compatible. +- Run focused company-scope and readiness-engine tests. + +## Task 2: Pure readiness transition review + +- Add failing tests for added/removed rows, ready/partial/excluded set transitions, primary DCF reason counts, and unexplained exclusions. +- Add frozen change-review structures and pure set parsing. +- Integrate the review using proposed ticker metadata and canonical fundamentals without a second readiness build. +- Render counts under `Readiness Change Cause Review` with an explicit non-mutually-exclusive transition boundary. +- Prove the real command preserves a complete before/after byte fingerprint. + +## Task 3: Documentation and verification + +- Update ROADMAP, methodology, provenance, data strategy, continuation prompt, Make help, and contract tests. +- Run focused tests, full tests, dashboard and Personal Research render smoke, public wording/check, commercial beta/release, pilot readiness, diff hygiene, and whitespace checks. +- Stage exact code/test/docs/Make files only; run staged hygiene; commit and push only the feature branch. +- Update draft PR #113 with verified counts and keep it draft. + +## Stop Rules + +- Do not run `make readiness`. +- Do not change the exclusion decision or infer a reason from absent evidence. +- Do not write or stage CSV, JSON, reports, screenshots, timings, caches, or sample reports. +- Do not describe an exclusion as a negative research conclusion. diff --git a/docs/superpowers/plans/2026-07-18-readiness-impact-preview.md b/docs/superpowers/plans/2026-07-18-readiness-impact-preview.md new file mode 100644 index 000000000..0eafa9207 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-readiness-impact-preview.md @@ -0,0 +1,367 @@ +# Readiness Impact Preview Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a deterministic stdout-only preview of saved-versus-proposed readiness changes without creating or modifying any file. + +**Architecture:** Existing universe and readiness builders gain explicit no-write flags whose defaults preserve current write behavior. A focused preview module loads the saved readiness CSV, builds the proposed frames in memory, compares only stable readiness fields, and renders capped human-readable output. The pilot gate routes stale operators to the preview while keeping the intentional readiness rebuild as a separate approval boundary. + +**Tech Stack:** Python 3, pandas, argparse, pytest, GNU Make, Markdown. + +## Global Constraints + +- Do not run `make readiness` during this slice. +- Do not generate or stage CSV, JSON, report, sample-report, screenshot, or timing churn. +- Default write-mode behavior must remain unchanged for existing callers. +- The preview must not create directories, expose an output-path option, or offer JSON/file output. +- Stable comparison fields are `overall_readiness_state`, `price_ready`, `momentum_ready`, `fundamentals_ready`, `dcf_ready`, `peer_ready`, `earnings_ready`, `analyst_estimates_ready`, `ready_features`, `partial_features`, `blocked_features`, and `excluded_features`. +- Missing saved readiness fails closed; no stable changes do not make stale readiness current. +- Preserve research-only, source-rights, candidate-context, synthetic-fixture, Q4, EPS split-basis, and independent readiness boundaries. + +--- + +### Task 1: No-write universe preparation + +**Files:** +- Modify: `src/universe_model.py:170-217,299-339` +- Test: `tests/test_universe_model.py` + +**Interfaces:** +- Consumes: existing `_read_csv`, `_write_csv`, `_legacy_universe_to_master`, and `_legacy_universe_to_active` helpers. +- Produces: `ensure_universe_files(base_dir=None, *, data_dir=None, write_outputs: bool = True) -> tuple[pd.DataFrame, pd.DataFrame]` and `build_universe_coverage_report(base_dir=None, *, data_dir=None, output_path=None, write_output: bool = True) -> pd.DataFrame`. + +- [ ] **Step 1: Write failing no-write universe tests** + +Add tests that create only `data/universe.csv`, capture a manifest of all relative file paths and bytes, call both no-write helpers, and assert the frames contain the legacy ticker while the manifest is identical: + +```python +from src.universe_model import build_universe_coverage_report + + +def _file_manifest(root: Path) -> dict[str, bytes]: + return {path.relative_to(root).as_posix(): path.read_bytes() for path in root.rglob("*") if path.is_file()} + + +def test_ensure_universe_files_no_write_builds_frames_without_creating_canonical_files(tmp_path: Path): + data_dir = tmp_path / "data" + data_dir.mkdir() + pd.DataFrame([{"Ticker": "NVDA", "CompanyName": "NVIDIA", "DefaultPurpose": "Core Compounder"}]).to_csv( + data_dir / "universe.csv", index=False + ) + before = _file_manifest(tmp_path) + + master, active = ensure_universe_files(tmp_path, write_outputs=False) + + assert set(master["ticker"]) == {"NVDA"} + assert set(active["ticker"]) == {"NVDA"} + assert _file_manifest(tmp_path) == before + + +def test_universe_coverage_no_write_creates_no_report_or_directory(tmp_path: Path): + data_dir = tmp_path / "data" + data_dir.mkdir() + pd.DataFrame([{"Ticker": "NVDA", "CompanyName": "NVIDIA", "DefaultPurpose": "Core Compounder"}]).to_csv( + data_dir / "universe.csv", index=False + ) + before = _file_manifest(tmp_path) + + report = build_universe_coverage_report(tmp_path, write_output=False) + + assert set(report["ticker"]) == {"NVDA"} + assert _file_manifest(tmp_path) == before +``` + +- [ ] **Step 2: Run the tests and confirm the new keyword arguments fail** + +Run: `python3 -m pytest tests/test_universe_model.py -q` + +Expected: FAIL because `write_outputs` and `write_output` are not accepted. + +- [ ] **Step 3: Add guarded universe writes** + +Change the signatures and guard all `_write_csv` and coverage output operations: + +```python +def ensure_universe_files( + base_dir: Path | str | None = None, + *, + data_dir: Path | str | None = None, + write_outputs: bool = True, +) -> tuple[pd.DataFrame, pd.DataFrame]: +``` + +For each existing universe write use `if write_outputs: _write_csv(...)`. Add `write_output: bool = True` to the coverage signature, call `ensure_universe_files(..., write_outputs=write_output)`, and wrap `output.parent.mkdir(...)` plus `report.to_csv(...)` in `if write_output:`. Always return the in-memory coverage frame. + +- [ ] **Step 4: Verify focused universe behavior** + +Run: `python3 -m pytest tests/test_universe_model.py -q` + +Expected: PASS, including existing write-mode tests. + +- [ ] **Step 5: Commit the universe boundary** + +Run: + +```bash +git add -- src/universe_model.py tests/test_universe_model.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add no-write universe preparation" +``` + +### Task 2: No-write readiness engine + +**Files:** +- Modify: `src/readiness_engine.py:988-1174` +- Test: `tests/test_readiness_engine.py` + +**Interfaces:** +- Consumes: Task 1 no-write universe interfaces. +- Produces: `build_ticker_readiness_report(base_dir=None, *, data_dir=None, output_dir=None, write_outputs: bool = True) -> dict[str, pd.DataFrame]`. + +- [ ] **Step 1: Write a failing filesystem-preservation test** + +Reuse the existing readiness fixture setup, call the builder with `write_outputs=False`, and assert all expected in-memory report keys exist while the complete file manifest is byte-for-byte unchanged and no output directory appears: + +```python +def test_ticker_readiness_no_write_returns_reports_without_mutating_files(tmp_path: Path, monkeypatch): + # Arrange the same minimal canonical inputs used by the main readiness-state test. + data_dir = tmp_path / "data" + _write_minimal_readiness_inputs(data_dir) + before = _file_manifest(tmp_path) + + reports = build_ticker_readiness_report( + tmp_path, + data_dir=data_dir, + output_dir=tmp_path / "outputs", + write_outputs=False, + ) + + assert "ticker_readiness_report" in reports + assert "data_source_status" in reports + assert not reports["ticker_readiness_report"].empty + assert _file_manifest(tmp_path) == before + assert not (tmp_path / "outputs").exists() +``` + +Factor the existing minimal input rows into `_write_minimal_readiness_inputs(data_dir: Path) -> None`; do not reduce existing write-mode assertions. + +- [ ] **Step 2: Run the focused test and confirm failure** + +Run: `python3 -m pytest tests/test_readiness_engine.py -q` + +Expected: FAIL because `write_outputs` is not accepted. + +- [ ] **Step 3: Guard every readiness write** + +Add `write_outputs: bool = True` to the builder. Call `ensure_universe_files(..., write_outputs=write_outputs)` and `build_universe_coverage_report(..., write_output=write_outputs)`. Use the returned universe frames instead of re-reading files that no-write preparation may intentionally not create. Wrap the report loop, compatibility copies, output-directory creation, and output copies in one `if write_outputs:` block. Return the same report dictionary in both modes. + +- [ ] **Step 4: Verify no-write and regression behavior** + +Run: `python3 -m pytest tests/test_readiness_engine.py tests/test_universe_model.py -q` + +Expected: PASS; existing write-mode files are still created and no-write mode leaves the manifest unchanged. + +- [ ] **Step 5: Commit the readiness boundary** + +Run: + +```bash +git add -- src/readiness_engine.py tests/test_readiness_engine.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add no-write readiness build mode" +``` + +### Task 3: Stable readiness impact comparison and CLI + +**Files:** +- Create: `src/readiness_preview.py` +- Create: `tests/test_readiness_preview.py` + +**Interfaces:** +- Consumes: `build_ticker_readiness_report(..., write_outputs=False)` from Task 2. +- Produces: `STABLE_READINESS_FIELDS: tuple[str, ...]`, `build_readiness_impact_preview(root: Path, *, data_dir: Path | None = None, top_n: int = 20) -> ReadinessImpactPreview`, `render_readiness_impact_preview(preview: ReadinessImpactPreview) -> str`, and CLI `main() -> int`. + +- [ ] **Step 1: Write failing pure comparison tests** + +Define two small readiness frames and require timestamp-only differences to produce `no_readiness_changes`, stable state changes to produce `changes_detected`, and details to cap at `top_n` while retaining the total changed count. Include added and removed ticker rows with changed field `row_presence`. + +```python +def test_compare_ignores_updated_at_but_reports_stable_fields(): + saved = pd.DataFrame([_row("AAA", price_ready=False, updated_at="2026-01-01")]) + timestamp_only = pd.DataFrame([_row("AAA", price_ready=False, updated_at="2026-07-18")]) + changed = pd.DataFrame([_row("AAA", price_ready=True, updated_at="2026-07-18")]) + + assert compare_readiness_frames(saved, timestamp_only, top_n=20).status == "no_readiness_changes" + preview = compare_readiness_frames(saved, changed, top_n=20) + assert preview.status == "changes_detected" + assert preview.changed_ticker_count == 1 + assert preview.changed_tickers[0].fields == ("price_ready",) +``` + +- [ ] **Step 2: Run the new test module and confirm import failure** + +Run: `python3 -m pytest tests/test_readiness_preview.py -q` + +Expected: FAIL because `src.readiness_preview` does not exist. + +- [ ] **Step 3: Implement immutable comparison structures and normalization** + +Create frozen dataclasses for `ReadinessTickerChange` and `ReadinessImpactPreview`. Normalize tickers to uppercase, booleans to strict booleans, missing feature strings to `""`, and compare the union of ticker keys in sorted order. Summarize overall-state counts and true counts for the seven named readiness flags. Do not inspect `updated_at`, source attempt times, or explanatory command text. + +- [ ] **Step 4: Write failing integration tests for missing snapshots, output wording, and no writes** + +Monkeypatch `build_ticker_readiness_report` to return an in-memory proposed frame. Assert a missing `data/reports/ticker_readiness_report.csv` returns `missing_saved_snapshot`, renders a non-unlock explanation, and does not create the path. With a saved fixture, capture a byte manifest before calling `build_readiness_impact_preview` and assert it is unchanged. Require rendered output to include: + +```text +Read-only: no files were created, modified, or deleted. +This preview does not make saved readiness current. +An intentional reviewed make readiness run remains the separate rebuild boundary. +``` + +- [ ] **Step 5: Implement the preview loader, renderer, and CLI** + +The CLI accepts only `--project-root`, `--data-dir`, and `--top-n`; validates `top_n >= 1`; calls the no-write builder; prints the human-readable result; returns `0` for `changes_detected` or `no_readiness_changes`, `2` for `missing_saved_snapshot`, and `1` for a concise in-memory build error. It must not accept `--output` or `--json`. + +- [ ] **Step 6: Verify preview unit and integration coverage** + +Run: `python3 -m pytest tests/test_readiness_preview.py tests/test_readiness_engine.py tests/test_universe_model.py -q` + +Expected: PASS. + +- [ ] **Step 7: Commit the preview module** + +Run: + +```bash +git add -- src/readiness_preview.py tests/test_readiness_preview.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add no-write readiness impact preview" +``` + +### Task 4: Command, pilot routing, and user contract + +**Files:** +- Modify: `Makefile:1,315-317,1287-1291` +- Modify: `src/pilot_readiness.py:304-315` +- Modify: `tests/test_pilot_readiness.py:450-480` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes: `python3 -m src.readiness_preview --top-n N`. +- Produces: `make readiness-preview TOP_N=20` and stale pilot routing to that command without changing the blocked verdict. + +- [ ] **Step 1: Write failing command-contract assertions** + +Add tests requiring `readiness-preview` in `.PHONY`, help text describing it as stdout-only/no-write, and the recipe: + +```make +readiness-preview: + @python3 -m src.readiness_preview --top-n $(or $(TOP_N),20) +``` + +Change the stale freshness pilot expectation to `make readiness-preview TOP_N=20`, while asserting status remains `blocked` and the stop rule still prevents quoting final counts or proof deltas. + +- [ ] **Step 2: Run focused contract tests and confirm failure** + +Run: `python3 -m pytest tests/test_pilot_readiness.py tests/test_public_v1_release_docs.py -q` + +Expected: FAIL because the target and stale route do not exist. + +- [ ] **Step 3: Add the Make target and safe pilot route** + +Add the target to `.PHONY`, help, and the readiness target block. In `_freshness_check`, use `make readiness-preview TOP_N=20` when stale or missing. Extend the stop rule to say the preview is inspection evidence only and the reviewed write rebuild remains separately gated. + +- [ ] **Step 4: Prove the real command changes no project files** + +Capture `git status --porcelain=v1`, a SHA-256/size/mtime manifest for all tracked and untracked files, run `make readiness-preview TOP_N=20`, and compare the manifests. Expected: command prints saved/proposed counts and the explicit no-write/non-unlock boundary; both manifests remain identical. + +- [ ] **Step 5: Run command and pilot focused tests** + +Run: + +```bash +python3 -m pytest tests/test_readiness_preview.py tests/test_readiness_engine.py tests/test_universe_model.py tests/test_pilot_readiness.py tests/test_public_v1_release_docs.py -q +make pilot-readiness-check TOP_N=10 +``` + +Expected: tests PASS; pilot remains blocked because saved readiness is stale and names the preview as the next inspection command. + +- [ ] **Step 6: Commit the command surface** + +Run: + +```bash +git add -- Makefile src/pilot_readiness.py tests/test_pilot_readiness.py tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Route stale readiness to no-write preview" +``` + +### Task 5: Documentation, complete verification, and draft PR + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/DATA_STRATEGY.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: draft PR #113 body/comment only after local verification. + +**Interfaces:** +- Consumes: verified no-write preview and unchanged blocked freshness gate. +- Produces: durable operator guidance and PR evidence for the coherent slice. + +- [ ] **Step 1: Update the durable operating contract** + +Document that `make readiness-preview TOP_N=20` computes proposed stable readiness in memory, writes nothing, cannot make the saved snapshot current, and does not authorize `make readiness`. ROADMAP must distinguish implemented preview inspection from the still-open intentional rebuild/review gate. The continuation prompt must preserve the current no-generated-artifact boundary and name the next executable lanes after preview. + +- [ ] **Step 2: Run focused documentation and feature checks** + +Run: + +```bash +python3 -m pytest tests/test_readiness_preview.py tests/test_readiness_engine.py tests/test_universe_model.py tests/test_pilot_readiness.py tests/test_public_v1_release_docs.py -q +git diff --check +make diff-hygiene-summary +``` + +Expected: PASS; zero generated CSV/JSON/report/sample-report/screenshot/timing candidates. + +- [ ] **Step 3: Run the full approved verification bundle** + +Run: + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all executable local gates PASS; pilot remains truthfully blocked only by stale saved readiness and other already-classified external/manual gates. Do not run `make readiness`. + +- [ ] **Step 4: Stage exact documentation and verify hygiene** + +Run: + +```bash +git add -- ROADMAP.md docs/DATA_STRATEGY.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Document readiness impact preview" +``` + +- [ ] **Step 5: Push only the approved branch and update draft PR #113** + +Run `git push origin codex/personal-research-mode-mvp`, verify zero divergence, then update PR #113 with the slice summary, exact test counts/gates, explicit no-artifact evidence, and remaining stale-readiness boundary. Confirm the PR remains open and draft; do not merge. + +- [ ] **Step 6: Continue the active goal** + +Re-read ROADMAP and the continuation prompt from current HEAD. Select the next highest-value safe local/source/hosted-preview/beta/evidence/calibration/operating slice. If no safe local task remains, classify the exact external unblock conditions once and leave the overall goal active rather than claiming completion. diff --git a/docs/superpowers/plans/2026-07-18-readiness-promotion-evidence-review.md b/docs/superpowers/plans/2026-07-18-readiness-promotion-evidence-review.md new file mode 100644 index 000000000..032c9248a --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-readiness-promotion-evidence-review.md @@ -0,0 +1,77 @@ +# Readiness Promotion Evidence Review Implementation Plan + +**Goal:** Extend the existing stdout-only readiness preview with a fail-closed explanation of proposed fundamentals and DCF promotions by exact source, provenance, commercial-rights status, and registered field scope. + +**Architecture:** Add pure promotion-review structures and comparison helpers to `src.readiness_preview`. The integrated builder reuses the already computed in-memory readiness frames, reads canonical fundamentals and the checked-in immutable source-rights registry, and attaches an independent evidence review to the technical impact preview. The existing Make command, no-write behavior, and rebuild boundary remain unchanged. + +**Constraints:** Do not run `make readiness`; do not add an output path or file format; do not write or stage CSV, JSON, reports, screenshots, timings, caches, or readiness artifacts; do not split composite source identifiers or infer rights/field support; preserve technical readiness independently from commercial/evidence review. + +## Task 1: Pure promotion-evidence contract + +**Files:** + +- Modify `tests/test_readiness_preview.py` +- Modify `src/readiness_preview.py` + +1. Add failing tests for false-to-true fundamentals/DCF selection, exact source-rights decisions, required field support, provenance completeness, duplicate ticker failure, and `TOP_N` capping. +2. Run `python3 -m pytest tests/test_readiness_preview.py -q` and confirm the new contract fails before implementation. +3. Add frozen `ReadinessPromotionEvidence` and `ReadinessPromotionReview` structures. +4. Implement a pure `review_readiness_promotions(...)` helper that: + - indexes saved/proposed readiness deterministically; + - identifies unique fundamentals/DCF promotions only; + - fails closed on duplicate/missing canonical fundamentals evidence; + - evaluates the complete source value with `commercial_eligibility`; + - checks source, as-of date, SEC accession/source reference, and registered required-field coverage; + - returns counts plus capped rows without mutating inputs. +5. Run the focused tests and keep the stable readiness comparison behavior unchanged. + +## Task 2: Integrated no-write preview and operator wording + +**Files:** + +- Modify `tests/test_readiness_preview.py` +- Modify `src/readiness_preview.py` +- Modify `tests/test_public_v1_release_docs.py` +- Modify `Makefile` help text only if the existing description no longer explains the expanded review. + +1. Add failing integration assertions that the preview builder passes `write_outputs=False`, loads the checked-in registry, attaches the review, and preserves the complete filesystem manifest. +2. Require output to distinguish technical promotions from provenance, rights, field-scope, DCF price-source, and rebuild decisions. +3. Integrate the review after the one in-memory production readiness build; do not invoke a second build. +4. Render total and capped evidence summaries under `Promotion Evidence Review`. +5. Prove `make readiness-preview TOP_N=20` leaves a before/after file fingerprint unchanged. +6. Run `python3 -m pytest tests/test_readiness_preview.py tests/test_public_v1_release_docs.py -q`. + +## Task 3: Durable methodology and continuation contract + +**Files:** + +- Modify `ROADMAP.md` +- Modify `docs/METHODOLOGY.md` +- Modify `docs/PROVENANCE_CONTRACT.md` +- Modify `docs/DATA_STRATEGY.md` +- Modify `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify documentation contract tests as required + +1. Record that proposed technical promotions do not establish provenance, field support, commercial rights, complete DCF price provenance, or rebuild approval. +2. Record the exact-source fail-closed rule and the current review result without turning in-memory counts into current product claims. +3. Keep the continuation prompt's next step executable and no-write. +4. Run focused documentation and preview tests, `git diff --check`, and `make diff-hygiene-summary`. + +## Task 4: Full verification, commit, push, and PR evidence + +1. Run: + - `python3 -m pytest tests -q` + - `make dashboard-smoke` + - the repository's Personal Research render smoke target + - `make public-wording-check` + - `make public-check` + - `make commercial-beta-check` + - `make commercial-beta-release-check` + - `make pilot-readiness-check TOP_N=10` + - `make diff-hygiene-summary` + - `git diff --check` +2. Confirm the pilot remains blocked by stale saved readiness and no generated artifact is tracked or staged. +3. Stage exact code/test/docs/Make files only and run `make staged-hygiene-check`. +4. Commit the coherent implementation and documentation slice. +5. Push only `codex/personal-research-mode-mvp`. +6. Update draft PR #113 with verified counts, checks, boundaries, external dependency classification, and the next executable step. Keep it draft; do not merge or deploy. diff --git a/docs/superpowers/plans/2026-07-18-research-desk-answer-first.md b/docs/superpowers/plans/2026-07-18-research-desk-answer-first.md new file mode 100644 index 000000000..cee9b9c89 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-research-desk-answer-first.md @@ -0,0 +1,142 @@ +# Research Desk Answer-First Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Put Research Desk's weekly summary, four direct answers, and Discover action before cohort technical context while preserving all cohort evidence under Advanced. + +**Architecture:** Move existing focused-cohort and cohort-coverage cards into the top of the existing collapsed Advanced Evidence drawer. Keep all helper outputs and full evidence frames unchanged, and make the browser performance gate measure the weekly summary as the first useful answer. + +**Tech Stack:** Python 3.12, Streamlit, pytest, source-level dashboard contract tests, Markdown documentation. + +## Global Constraints + +- Research-only; no recommendation, expected-return ranking, broker integration, order routing, auto-trading, or direct buy/sell instruction. +- Preserve independent actuals, consensus, Revenue, EPS, valuation, catalyst, outcome, backtesting, and calibration readiness states. +- Do not refresh, import, apply, or fabricate source data. +- Keep candidate context unable to modify deterministic scenarios or become trusted evidence. +- Keep generated CSV, JSON, report, sample-report, screenshot, and timing churn unstaged. +- Keep PR #113 draft; do not merge or deploy. + +--- + +### Task 1: Make Research Desk answer-first + +**Files:** +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_public_performance_gate.py` +- Modify: `src/dashboard.py` +- Modify: `src/public_performance_gate.py` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/DASHBOARD_QA.md` +- Modify: `ROADMAP.md` + +**Interfaces:** +- Consumes: existing `weekly_summary_cards`, `research_desk_cards`, `research_desk_cards_html`, `focused_cohort_cards`, `focused_cohort_coverage_cards`, and `RESEARCH_ROUTE_SPECS` contracts. +- Produces: Research Desk order `header -> weekly summary -> direct answers -> Discover action -> collapsed cohort evidence`, with unchanged data and readiness semantics. + +- [ ] **Step 1: Write the failing route-order contract test** + +```python +def test_research_desk_renders_answers_before_advanced_cohort_context(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + desk_start = source.index("def render_research_desk(") + desk_end = source.index("def render_research_monitor(", desk_start) + desk = source[desk_start:desk_end] + + weekly = desk.index('st.markdown("### Weekly research summary")') + weekly_cards = desk.index("weekly_summary_cards(weekly_summary)", weekly) + answers = desk.index("cards = research_desk_cards(", weekly_cards) + answers_html = desk.index("research_desk_cards_html(cards)", answers) + discover = desk.index('st.link_button("Open Discover"', answers_html) + advanced = desk.index('with st.expander("Advanced Evidence", expanded=False):', discover) + cohort = desk.index("focused_cohort_cards(cohort)", advanced) + coverage = desk.index("focused_cohort_coverage_cards(coverage)", cohort) + cohort_frame = desk.index("focused_cohort_frame(cohort)", coverage) + coverage_frame = desk.index("focused_cohort_coverage_frame(coverage)", cohort_frame) + + assert weekly < weekly_cards < answers < answers_html < discover < advanced + assert advanced < cohort < coverage < cohort_frame < coverage_frame +``` + +- [ ] **Step 2: Run the route test and verify the expected failure** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py::test_research_desk_renders_answers_before_advanced_cohort_context -q` + +Expected: FAIL because the concise cohort calls occur before the weekly summary and cannot be found after Advanced. + +- [ ] **Step 3: Write the failing performance-marker contract** + +Require Research Desk's first-useful marker to be `Weekly research summary` and its full markers to be `Weekly research summary`, `What should I review next?`, `Open Discover`, `Advanced Evidence`, and `Research-only`. + +Run: `python3 -m pytest tests/test_public_performance_gate.py::test_research_performance_contract_covers_the_commercial_beta_workflow -q` + +Expected: FAIL because the existing route spec still uses `Research Desk` as first useful and `Focused cohort` as a visible full marker. + +- [ ] **Step 4: Implement the minimal route hierarchy** + +Remove the two concise cohort-card calls above `Weekly research summary`. Add them at the top of the existing Advanced Evidence block: + +```python + with st.expander("Advanced Evidence", expanded=False): + render_signal_cards(focused_cohort_cards(cohort), show_commands=False, variant="queue") + render_signal_cards(focused_cohort_coverage_cards(coverage), show_commands=False, variant="queue") + cohort_frame = focused_cohort_frame(cohort) +``` + +Update the Research Desk `PublicRouteSpec` to: + +```python + PublicRouteSpec( + "Research Desk", + "/?mode=research&page=research-desk", + "Weekly research summary", + ( + "Weekly research summary", + "What should I review next?", + "Open Discover", + "Advanced Evidence", + "Research-only", + ), + True, + ), +``` + +- [ ] **Step 5: Run focused tests and verify green** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_research_workspace.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py -q` + +Expected: all focused tests pass with no new warnings or failures. + +- [ ] **Step 6: Update workflow, QA, and roadmap documentation** + +Document the weekly-answer-first order, collapsed cohort context, unchanged evidence boundaries, desktop and phone first-view acceptance, and Monitor as the remaining Stage 1 route audit. + +- [ ] **Step 7: Run the full release verification bundle** + +Run: + +```text +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make commercial-beta-performance-gate +make public-wording-check +make public-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: every command exits zero; generated churn remains excluded. The pilot checklist may retain truthful manual source, hosting, or review gates while the command succeeds. + +- [ ] **Step 8: Stage exact files, verify staged hygiene, commit, and push** + +```text +git add -- src/dashboard.py src/public_performance_gate.py tests/test_research_mode_dashboard_contract.py tests/test_public_performance_gate.py ROADMAP.md docs/PERSONAL_RESEARCH_MODE.md docs/DASHBOARD_QA.md docs/superpowers/specs/2026-07-18-research-desk-answer-first-design.md docs/superpowers/plans/2026-07-18-research-desk-answer-first.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Prioritize Research Desk answers" +git push origin codex/personal-research-mode-mvp +``` + +Expected: exact reviewed files only are committed and pushed; PR #113 remains open and draft. diff --git a/docs/superpowers/plans/2026-07-18-stale-readiness-continuation-gate.md b/docs/superpowers/plans/2026-07-18-stale-readiness-continuation-gate.md new file mode 100644 index 000000000..a1cf9f519 --- /dev/null +++ b/docs/superpowers/plans/2026-07-18-stale-readiness-continuation-gate.md @@ -0,0 +1,353 @@ +# Stale Readiness Continuation Gate Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make every read-only continuation surface agree that stale selected-profile readiness permits no-write inspection only and does not authorize broad source, coverage, apply, or readiness-rebuild work. + +**Architecture:** Add one pure `ContinuationGate` derived from `ProfileContext`, then consume it in project status, provider setup, and coverage-frontier rendering. Existing readiness calculations, provider availability, external dependency classifications, and ranked lane data remain unchanged; only the continuation-safe action layer overrides contradictory execution guidance while readiness is not current. + +**Tech Stack:** Python 3.12, dataclasses, pytest, existing CLI renderers, Make release gates. + +## Global Constraints + +- Research-only; no investment advice, broker integration, order routing, auto-trading, direct buy/sell instructions, or post-earnings price prediction. +- Do not run `make readiness` or write generated CSV, JSON, report, sample-report, screenshot, timing, directory, or bytecode churn. +- `make readiness-preview TOP_N=20` is inspection evidence only and cannot make saved readiness current. +- Preserve independent readiness for actuals, consensus, Revenue, EPS, operating margin, free cash flow, FCF margin, valuation, trusted relationships, peer comparability, peer valuation anchors, catalysts, outcomes, backtesting, and calibration. +- Candidate context cannot modify forecasts or become trusted evidence. +- Stage exact intentional files only; never use `git add -A`. + +--- + +### Task 1: Pure continuation gate + +**Files:** +- Create: `src/continuation_gate.py` +- Create: `tests/test_continuation_gate.py` + +**Interfaces:** +- Consumes: `src.profile_context.ProfileContext` +- Produces: `ContinuationGate(state, next_safe_command, reason, rebuild_command, stop_rule, suppress_execution)` and `build_continuation_gate(context)` + +- [ ] **Step 1: Write the failing tests** + +```python +from pathlib import Path + +from src.continuation_gate import build_continuation_gate +from src.profile_context import CoverageCounts, ProfileContext + + +def _context(state: str) -> ProfileContext: + return ProfileContext( + profile_key="default", + profile_label="Default", + data_dir=Path("data"), + outputs_dir=Path("outputs"), + source_as_of="2026-06-26", + readiness_built_at="2026-06-07T03:00:43+00:00", + snapshot_identity="abc", + snapshot_identity_short="abc", + freshness_state=state, + freshness_message=f"Readiness is {state}.", + refresh_command="make readiness", + coverage=CoverageCounts(), + lane_source_dates=(), + snapshot_inputs=(), + ) + + +def test_stale_readiness_routes_to_no_write_preview(): + gate = build_continuation_gate(_context("stale")) + assert gate.state == "inspection_only" + assert gate.next_safe_command == "make readiness-preview TOP_N=20" + assert gate.rebuild_command == "make readiness" + assert gate.suppress_execution is True + + +def test_current_readiness_does_not_override_source_routing(): + gate = build_continuation_gate(_context("current")) + assert gate.state == "current" + assert gate.next_safe_command == "" + assert gate.suppress_execution is False +``` + +- [ ] **Step 2: Run tests to verify the module is missing** + +Run: `python3 -m pytest tests/test_continuation_gate.py -q` + +Expected: FAIL with `ModuleNotFoundError: No module named 'src.continuation_gate'`. + +- [ ] **Step 3: Implement the pure gate** + +```python +from dataclasses import dataclass + +from src.profile_context import ProfileContext + + +@dataclass(frozen=True) +class ContinuationGate: + state: str + next_safe_command: str + reason: str + rebuild_command: str + stop_rule: str + suppress_execution: bool + + +def build_continuation_gate(context: ProfileContext) -> ContinuationGate: + if context.freshness_state == "current": + return ContinuationGate("current", "", context.freshness_message, context.refresh_command, "", False) + state = "inspection_only" if context.freshness_state == "stale" else "inspection_required" + return ContinuationGate( + state=state, + next_safe_command="make readiness-preview TOP_N=20", + reason=context.freshness_message, + rebuild_command=context.refresh_command or "make readiness", + stop_rule=( + "Do not start broad refresh, source-proof, apply, or readiness-rebuild work from stale or incomplete readiness counts." + ), + suppress_execution=True, + ) +``` + +- [ ] **Step 4: Run focused tests** + +Run: `python3 -m pytest tests/test_continuation_gate.py tests/test_profile_context.py -q` + +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add -- src/continuation_gate.py tests/test_continuation_gate.py +git commit -m "Add stale readiness continuation gate" +``` + +### Task 2: Fail-closed project-status routing + +**Files:** +- Modify: `src/project_status.py` +- Modify: `tests/test_project_status.py` + +**Interfaces:** +- Consumes: `ContinuationGate` from Task 1 +- Produces: `continuation_gate` in project-status payloads and a stale-aware human rendering contract + +- [ ] **Step 1: Write the failing project-status test** + +```python +def test_project_status_stale_gate_suppresses_broad_next_steps(tmp_path, monkeypatch, capsys): + _write_fast_status_artifacts(tmp_path) + payload = project_status._fast_status_payload_from_outputs(tmp_path, top_n=5) + gate = ContinuationGate( + state="inspection_only", + next_safe_command="make readiness-preview TOP_N=20", + reason="Selected-profile source dates are newer than saved readiness.", + rebuild_command="make readiness", + stop_rule="Do not start broad refresh, source-proof, apply, or readiness-rebuild work.", + suppress_execution=True, + ) + + project_status._print_human(payload, continuation_gate=gate) + output = capsys.readouterr().out + + assert "Continuation-safe next action: make readiness-preview TOP_N=20" in output + assert "Rebuild boundary: make readiness" in output + assert "make price-refresh-loop DRY_RUN=1" not in output + assert "make trusted-data-pilot-candidates TOP_N=10" not in output +``` + +- [ ] **Step 2: Run the test and verify the signature/behavior fails** + +Run: `python3 -m pytest tests/test_project_status.py::test_project_status_stale_gate_suppresses_broad_next_steps -q` + +Expected: FAIL because `_print_human` does not accept `continuation_gate` and broad recommendations remain visible. + +- [ ] **Step 3: Apply the gate in the CLI and renderer** + +Build the profile context once in `main`, derive the gate, and pass it to `_print_human`. When `suppress_execution` is true: + +```python +print(f"Continuation gate: {continuation_gate.state}") +print(f"- Continuation-safe next action: {continuation_gate.next_safe_command}") +print(f"- Rebuild boundary: {continuation_gate.rebuild_command} requires an intentional reviewed write.") +print(f"- Stop rule: {continuation_gate.stop_rule}") +``` + +Do not render ticker-specific locked-input commands or generic recommended command rows in that state. Render one replacement row for the no-write preview and keep saved/stale coverage counts visible. + +- [ ] **Step 4: Run project-status tests** + +Run: `python3 -m pytest tests/test_project_status.py tests/test_continuation_gate.py -q` + +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add -- src/project_status.py tests/test_project_status.py +git commit -m "Route stale project status to inspection only" +``` + +### Task 3: Align provider setup and coverage frontier + +**Files:** +- Modify: `src/source_activation_guide.py` +- Modify: `src/readiness_ops.py` +- Modify: `tests/test_source_activation_guide.py` +- Modify: `tests/test_readiness_ops.py` + +**Interfaces:** +- Consumes: `build_continuation_gate(build_profile_context(...))` +- Produces: stale-aware provider `current_gate` and coverage-frontier continuation banner + +- [ ] **Step 1: Write failing provider and frontier tests** + +```python +def test_provider_checklist_stale_profile_overrides_saved_preflight_action(tmp_path, monkeypatch): + monkeypatch.setattr( + source_activation_guide, + "build_profile_context", + lambda **_kwargs: SimpleNamespace( + freshness_state="stale", + freshness_message="Selected-profile source dates are newer than saved readiness.", + refresh_command="make readiness", + ), + ) + current_preflight = { + "source_activation_console_v2": { + "operator_summary": { + "can_run_now": ["sec_fundamentals_share_count"], + "needs_setup": ["fmp", "alpha_vantage", "finnhub"], + "avoid_repeating": [], + "next_step": "make coverage-frontier TOP_N=10", + } + } + } + checklist = build_provider_setup_checklist(current_preflight, root=tmp_path) + assert checklist["current_gate"]["can_run_now"] == "inspection_only" + assert checklist["current_gate"]["next_step"] == "make readiness-preview TOP_N=20" + assert "fmp" in checklist["current_gate"]["needs_setup"] + + +def test_coverage_frontier_stale_banner_marks_rows_planning_only(): + gate = ContinuationGate( + state="inspection_only", + next_safe_command="make readiness-preview TOP_N=20", + reason="Selected-profile source dates are newer than saved readiness.", + rebuild_command="make readiness", + stop_rule="Do not start broad refresh, source-proof, apply, or readiness-rebuild work.", + suppress_execution=True, + ) + rendered = render_coverage_frontier([], continuation_gate=gate) + assert "Continuation gate: inspection_only" in rendered + assert "make readiness-preview TOP_N=20" in rendered + assert "ranked rows below are planning context only" in rendered.lower() + assert "make readiness is a separate intentional reviewed write" in rendered +``` + +- [ ] **Step 2: Run the tests and verify they fail** + +Run: `python3 -m pytest tests/test_source_activation_guide.py tests/test_readiness_ops.py -q` + +Expected: FAIL because neither surface consumes the continuation gate. + +- [ ] **Step 3: Implement provider checklist override** + +In `build_provider_setup_checklist`, build the selected profile context for `root`, derive the gate, and override only action fields when fail-closed: + +```python +current_gate.update( + { + "can_run_now": gate.state, + "avoid_repeating": "broad_refresh, source_proof, readiness_rebuild", + "next_step": gate.next_safe_command, + "next_step_reason": gate.reason, + } +) +``` + +Preserve `needs_setup`, provider rows, key-state classifications, and source-rights boundaries. + +- [ ] **Step 4: Implement coverage-frontier banner** + +Add an optional `continuation_gate` parameter to `render_coverage_frontier`. When fail-closed, render the gate before ranked lanes and state that lane commands are planning context only. In the CLI, build the selected profile context and pass the gate; keep pure frontier ranking unchanged. + +- [ ] **Step 5: Run focused status tests** + +Run: `python3 -m pytest tests/test_continuation_gate.py tests/test_project_status.py tests/test_source_activation_guide.py tests/test_readiness_ops.py -q` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add -- src/source_activation_guide.py src/readiness_ops.py tests/test_source_activation_guide.py tests/test_readiness_ops.py +git commit -m "Align stale readiness operator guidance" +``` + +### Task 4: Documentation, full verification, and release handoff + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/DATA_STRATEGY.md` +- Modify: `docs/DASHBOARD_QA.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes: verified runtime behavior from Tasks 1-3 +- Produces: durable operating-maturity and continuation-contract evidence + +- [ ] **Step 1: Add a failing documentation contract** + +Require all four documents to contain the phrases `stale readiness continuation gate`, `make readiness-preview TOP_N=20`, `planning context only`, and `separate intentional reviewed write`. + +- [ ] **Step 2: Run the documentation test and verify failure** + +Run: `python3 -m pytest tests/test_public_v1_release_docs.py -q` + +Expected: FAIL until the docs describe the verified gate. + +- [ ] **Step 3: Update documentation** + +Document that project status, provider setup, and coverage frontier now agree on inspection-only routing while selected-profile readiness is stale. State that this improves operating reliability but does not refresh data, prove source correctness, authorize a rebuild, satisfy hosted/reviewer/source/calibration gates, or establish market validation. + +- [ ] **Step 4: Run focused and full verification** + +Run: + +```bash +python3 -m pytest tests/test_continuation_gate.py tests/test_project_status.py tests/test_source_activation_guide.py tests/test_readiness_ops.py tests/test_public_v1_release_docs.py -q +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all tests and local release gates pass; pilot readiness remains truthfully blocked by stale readiness and external evidence; generated churn remains zero. + +- [ ] **Step 5: Stage exact files and verify staged hygiene** + +```bash +git add -- ROADMAP.md docs/DATA_STRATEGY.md docs/DASHBOARD_QA.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +``` + +- [ ] **Step 6: Commit, push, and update draft PR #113** + +```bash +git commit -m "Document stale readiness continuation gate" +git push origin codex/personal-research-mode-mvp +``` + +Update PR #113 with the implementation, direct checks, zero-artifact result, current external classifications, and exact next external resume condition. Keep the PR draft and do not merge or deploy. diff --git a/docs/superpowers/plans/2026-07-19-cohort-price-history-scope.md b/docs/superpowers/plans/2026-07-19-cohort-price-history-scope.md new file mode 100644 index 000000000..a366b3380 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-cohort-price-history-scope.md @@ -0,0 +1,27 @@ +# Cohort Price-History Scope Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task by task. + +**Goal:** Prevent saved price readiness from becoming commercially usable cohort evidence without row-level provenance, exact-source rights, and registered `prices` scope. + +**Architecture:** Review grouped canonical price rows inside `derive_cohort_evidence`, pass the result into the existing coverage builder, and load `prices.csv` through the dashboard’s current read-only optional loader. Keep research mode and all mutation paths unchanged. + +**Tech Stack:** Python 3.12, pandas, pytest, immutable source-rights registry + +### Task 1: Add failing price-history evidence tests + +**Files:** `tests/test_focused_cohort_coverage.py`, `tests/test_research_mode_dashboard_contract.py` + +Test missing rows, missing lineage, approved scoped rows, missing scope, mixed histories, research compatibility, and the real dashboard loader contract. Run the two focused files and confirm failures against the saved-readiness-only behavior. + +### Task 2: Implement row-group price review + +**Files:** `src/focused_cohort_coverage.py`, `src/dashboard.py` + +Add optional `prices` and injectable registry inputs. Normalize grouped rows, retain technically valid rows, require `source`, `source_ref`, and `retrieved_at` for every commercial row, review each exact source with `review_commercial_field_scope(..., ("prices",))`, and return a deterministic state/evidence message. Conjunct the commercial evidence with saved `price_ready`; preserve research mode. + +### Task 3: Document and verify + +**Files:** `ROADMAP.md`, `docs/METHODOLOGY.md`, `docs/PROVENANCE_CONTRACT.md`, `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +Record only the verified cohort display boundary and keep Priority 5 chronology plus quarterly Revenue/EPS open. Run focused tests, full tests, dashboard/render, public, commercial, pilot, hygiene, and whitespace gates. Stage exact paths, commit, push, update draft PR #113, and require hosted CI on the exact head. diff --git a/docs/superpowers/plans/2026-07-19-cohort-quarterly-field-scope.md b/docs/superpowers/plans/2026-07-19-cohort-quarterly-field-scope.md new file mode 100644 index 000000000..aba95be11 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-cohort-quarterly-field-scope.md @@ -0,0 +1,15 @@ +# Cohort Quarterly Field-Scope Implementation Plan + +1. Add failing focused-cohort tests for independent Revenue/EPS commercial + source-rights and field-scope decisions, including mixed-source failure and + unchanged Research-mode behavior. +2. Extend cohort evidence derivation with accepted quarterly actual rows and + produce independent commercial Revenue/EPS evidence states. +3. Conjoin those commercial evidence states with the existing technical trend + packet states when composing cohort coverage. +4. Pass the dashboard's accepted canonical quarterly actual rows into evidence + derivation; do not refresh, repair, or write the ledger. +5. Update methodology, provenance, roadmap, continuation contract, and dashboard + contract tests. +6. Run focused and full tests, every required product/release/hygiene gate, exact + staging, commit, push, draft-PR update, and exact-head hosted verification. diff --git a/docs/superpowers/plans/2026-07-19-cohort-saved-row-field-scope.md b/docs/superpowers/plans/2026-07-19-cohort-saved-row-field-scope.md new file mode 100644 index 000000000..83d33e821 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-cohort-saved-row-field-scope.md @@ -0,0 +1,63 @@ +# Cohort Saved-Row Field-Scope Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task by task. + +**Goal:** Prevent one source-level commercial approval from unlocking unrelated focused-cohort fields while preserving research-mode behavior and independent lane states. + +**Architecture:** Replace the shared `_source_backed` commercial boolean in `focused_cohort_coverage.py` with pure per-row field review helpers using `review_commercial_field_scope`. Compose each saved-row lane from separate technical, provenance, rights, and exact-field decisions, while retaining the existing coverage frame and Advanced dashboard boundary. + +**Tech Stack:** Python 3.12, pandas, pytest, immutable YAML source-rights registry + +## Constraints + +- Do not edit `config/source_rights.yml`, activate a provider, or fetch data. +- Do not change price readiness or canonical quarterly Revenue/EPS packets in this slice. +- Preserve candidate-only states and research-mode compatibility. +- Do not run `make readiness` or generate repository artifacts. + +### Task 1: Prove the source-level permission leak + +**Files:** +- Modify: `tests/test_focused_cohort_coverage.py` +- Modify: `src/focused_cohort_coverage.py` + +1. Add a checked-registry regression showing an SEC Companyfacts row with margin, FCF, cash, debt, shares, and filing date keeps only shares and filing date commercially usable. +2. Add injected-registry tests showing one exact supported field cannot unlock sibling fields and cash/debt remain independent. +3. Run `python3 -m pytest tests/test_focused_cohort_coverage.py -q` and confirm the old source-level behavior fails. +4. Add a frozen row review result containing technical availability, provenance completeness, rights state, required fields, missing fields, and combined usability. +5. Use `review_commercial_field_scope` for exact populated fields and make commercial evidence text name independent blockers. +6. Rerun the focused file and confirm green. + +### Task 2: Enforce earnings, consensus, and trusted-peer scope + +**Files:** +- Modify: `tests/test_focused_cohort_coverage.py` +- Modify: `src/focused_cohort_coverage.py` + +1. Add failing tests for missing `earnings_dates` scope, Revenue/EPS consensus scope independence, blocked date-only consensus, and a trusted row missing `trusted_peers` scope. +2. Review earnings dates with exact `earnings_dates` scope. +3. Require consensus values and review every populated metric with `revenue_consensus` or `eps_consensus`; retain cutoff/fiscal-period/provenance checks. +4. Require every trusted row to pass `trusted_peers`; leave candidate rows untouched. +5. Run `python3 -m pytest tests/test_focused_cohort_coverage.py -q` and confirm green. + +### Task 3: Preserve dashboard and documentation contracts + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Test: `tests/test_dashboard_research_mode.py` or the narrow existing dashboard contract test if an assertion is needed + +1. Add a dashboard contract assertion that field-scope blockers stay in Advanced cohort evidence and do not alter the primary research answer. +2. Document the saved-row lane mapping and truthful remaining Priority 2 gaps: prices and canonical quarterly Revenue/EPS. +3. Run focused cohort/dashboard/docs tests, public wording, and commercial beta checks. + +### Task 4: Verify and publish the coherent slice + +1. Run `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest -p no:cacheprovider tests -q`. +2. Run dashboard, six-route Personal Research render, public wording/check, commercial beta/release, pilot readiness, diff hygiene, and whitespace gates. +3. Confirm zero generated CSV/JSON/report/sample-report/screenshot/timing/canonical-data changes. +4. Stage exact implementation, tests, ROADMAP, methodology, provenance, continuation prompt, design, and plan paths only. +5. Run staged hygiene and cached whitespace checks, commit, and push only the feature branch. +6. Update draft PR #113 with red-green evidence and remaining audit boundaries; verify current-head hosted CI and keep the PR draft. diff --git a/docs/superpowers/plans/2026-07-19-commercial-field-scope-review.md b/docs/superpowers/plans/2026-07-19-commercial-field-scope-review.md new file mode 100644 index 000000000..a1c4a7e2e --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-commercial-field-scope-review.md @@ -0,0 +1,260 @@ +# Commercial Field-Scope Review Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Give consensus source validation and prospective collection one immutable exact-source commercial-rights and required-field scope decision without changing either consumer's public behavior. + +**Architecture:** Add a domain-neutral frozen result and pure function to the existing source-rights module. Keep consumer-specific blocker strings, technical states, cutoff/lineage checks, preview states, and write authorization in the two consensus modules while replacing their duplicate registry and missing-field calculations. + +**Tech Stack:** Python 3.12, frozen dataclasses, immutable source-rights registry, pytest, Make product gates, Git/GitHub draft PR workflow. + +## Global Constraints + +- Research-only; no investment advice, broker integration, order routing, auto-trading, direct buy/sell instructions, or post-earnings price prediction. +- Preserve exact source IDs; do not split composite labels, infer aliases, or borrow rights. +- Keep commercial rights and required-field scope independently visible. +- Keep technical validity, candidate/history state, collection, Revenue, EPS, write authorization, nowcast readiness, backtesting, and calibration independent. +- Preserve existing collector and validator result fields, blocker strings, and state transitions. +- Do not modify price, DCF, fundamentals, cash-generation, or other domain evidence models in this slice. +- Do not run `make readiness` or create/stage CSV, JSON, report, sample-report, screenshot, browser timing, or bytecode churn. +- Stage exact intentional product/code/docs/test files only; never use `git add -A`. + +--- + +### Task 1: Pure commercial field-scope decision + +**Files:** +- Modify: `tests/test_commercial_source_rights.py` +- Modify: `src/commercial_source_rights.py` + +**Interfaces:** +- Consumes: `commercial_eligibility(registry, source_id) -> CommercialEligibility` and exact `SourceRights.supported_fields`. +- Produces: `review_commercial_field_scope(registry, source_id, required_fields) -> CommercialFieldScopeReview`. + +- [ ] **Step 1: Write failing approved and missing-scope tests** + +Use the checked-in immutable registry. Assert stable supplied order, exact missing fields, and combined readiness: + +```python +complete = module.review_commercial_field_scope( + _registry(), "sec_companyfacts", ("revenue", "shares_outstanding") +) +missing = module.review_commercial_field_scope( + _registry(), "sec_companyfacts", ("revenue", "free_cash_flow") +) +assert complete.required_supported_fields == ("revenue", "shares_outstanding") +assert complete.missing_supported_fields == () +assert complete.commercial_evidence_ready is True +assert missing.missing_supported_fields == ("free_cash_flow",) +assert missing.commercial_rights_approved is True +assert missing.commercial_evidence_ready is False +``` + +- [ ] **Step 2: Write failing independence and exact-source tests** + +Assert that `yfinance` has supported `prices` scope but unverified rights, and a composite source remains unknown with every required field missing: + +```python +unverified = module.review_commercial_field_scope(_registry(), "yfinance", ("prices",)) +unknown = module.review_commercial_field_scope( + _registry(), "sec_companyfacts + yfinance", ("revenue", "prices") +) +assert unverified.missing_supported_fields == () +assert unverified.rights_status == "commercial_rights_unverified" +assert unverified.commercial_evidence_ready is False +assert unknown.rights_status == "unknown_source" +assert unknown.missing_supported_fields == ("revenue", "prices") +``` + +- [ ] **Step 3: Write failing immutability and caller-contract tests** + +Assert an approved empty-field decision is rights-ready, the result is frozen, and blank or duplicate required fields raise: + +```python +rights_only = module.review_commercial_field_scope(_registry(), "sec_companyfacts", ()) +assert rights_only.commercial_evidence_ready is True +with pytest.raises(FrozenInstanceError): + rights_only.commercial_evidence_ready = False +with pytest.raises(ValueError, match="non-empty unique strings"): + module.review_commercial_field_scope(_registry(), "sec_companyfacts", ("revenue", "")) +with pytest.raises(ValueError, match="non-empty unique strings"): + module.review_commercial_field_scope(_registry(), "sec_companyfacts", ("revenue", "revenue")) +``` + +- [ ] **Step 4: Run the source-rights tests and verify RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_commercial_source_rights.py -q +``` + +Expected: failures because `CommercialFieldScopeReview` and `review_commercial_field_scope` do not exist. + +- [ ] **Step 5: Add the frozen result** + +Add after `CommercialEligibility`: + +```python +@dataclass(frozen=True) +class CommercialFieldScopeReview: + source_id: str + rights_status: str + commercial_rights_approved: bool + required_supported_fields: tuple[str, ...] + missing_supported_fields: tuple[str, ...] + commercial_evidence_ready: bool +``` + +- [ ] **Step 6: Implement the pure exact-source review** + +Add after `commercial_eligibility`: + +```python +def review_commercial_field_scope( + registry: Mapping[str, SourceRights], + source_id: str, + required_fields: Sequence[str], +) -> CommercialFieldScopeReview: + normalized_source_id = str(source_id or "").strip() + normalized_fields = tuple(str(field or "").strip() for field in required_fields) + if any(not field for field in normalized_fields) or len(set(normalized_fields)) != len(normalized_fields): + raise ValueError("required fields must be non-empty unique strings") + rights = commercial_eligibility(registry, normalized_source_id) + record = registry.get(normalized_source_id) + supported_fields = set(record.supported_fields) if record is not None else set() + missing_fields = tuple(field for field in normalized_fields if field not in supported_fields) + return CommercialFieldScopeReview( + source_id=normalized_source_id, + rights_status=rights.status, + commercial_rights_approved=rights.allowed, + required_supported_fields=normalized_fields, + missing_supported_fields=missing_fields, + commercial_evidence_ready=rights.allowed and not missing_fields, + ) +``` + +- [ ] **Step 7: Run the source-rights tests and verify GREEN** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_commercial_source_rights.py -q +``` + +Expected: all source-rights tests pass. + +### Task 2: Migrate consensus consumers without behavior changes + +**Files:** +- Modify: `src/earnings_consensus_collector.py` +- Modify: `src/earnings_consensus_sources.py` +- Test: `tests/test_earnings_consensus_collector.py` +- Test: `tests/test_earnings_consensus_sources.py` + +**Interfaces:** +- Consumes: `review_commercial_field_scope(...) -> CommercialFieldScopeReview` from Task 1. +- Produces: unchanged `CollectionPreview`, `SourceCommercialReview`, and `SourceValidationResult` public behavior. + +- [ ] **Step 1: Refactor collector commercial evidence** + +Replace the collector's direct `commercial_eligibility`, exact-record lookup, supported-field set, and missing-field calculation with one shared review call. Map the shared fields into existing blocker strings and `CollectionPreview`: + +```python +commercial_review = review_commercial_field_scope( + rights_registry, proposed.source, required_supported_fields +) +if not commercial_review.commercial_rights_approved: + commercial_blockers.append(f"commercial_rights:{commercial_review.rights_status}") +commercial_blockers.extend( + f"registered_consensus_scope_missing:{field}" + for field in commercial_review.missing_supported_fields +) +``` + +Do not change technical `write_allowed`, `commercial_write_allowed`, state, reason, identity, batch, or mutation behavior. + +- [ ] **Step 2: Refactor source-validator commercial evidence** + +Use an empty-field shared review for source-level rights. For each technically accepted row, pass its populated Revenue/EPS tuple to the shared review and map the result into `SourceCommercialReview`. Preserve all cutoff, scope, timestamp, schema, comparability, state, count, row-number, and aggregate behavior. + +- [ ] **Step 3: Run consumer regression tests** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_commercial_source_rights.py tests/test_earnings_consensus_sources.py tests/test_earnings_consensus_collector.py -q +``` + +Expected: all source-rights, source-validator, and collector tests pass with unchanged public evidence. + +- [ ] **Step 4: Review duplication removal** + +Use `rg` and the diff to confirm both consensus modules call `review_commercial_field_scope` and no longer independently read `supported_fields` or calculate `missing_supported_fields`. Confirm their domain-specific blocker strings remain local. Run `git diff --check`. + +### Task 3: Documentation and release verification + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/DATA_STRATEGY.md` +- Modify: `docs/EARNINGS_NOWCAST_PILOT.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: the verified shared field-scope decision and unchanged consumer evidence. +- Produces: truthful roadmap, methodology, provenance, continuation, and PR evidence for the reliability slice. + +- [ ] **Step 1: Document the implemented roadmap item** + +Add item 33 and maturity commentary: both consensus paths now share one exact-source rights/field-scope decision, reducing semantic drift without changing readiness or providing source evidence. State explicitly that price, DCF, fundamentals, and cash-generation domain reviews remain separate. + +- [ ] **Step 2: Update source strategy, pilot, method, and provenance** + +Document the shared decision fields and the retained separation: consumer technical states and blocker copy remain local; registry metadata cannot prove payloads, timestamps, comparability, review intent, collection, activation, backtesting, or calibration. + +- [ ] **Step 3: Update the continuation contract** + +Add the committed design/plan lineage anchor, shared-decision capability, truthful boundary, and Stage 2 instruction. Preserve stale-readiness, external dependency, and no-generated-write rules. + +- [ ] **Step 4: Run focused and full verification** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_commercial_source_rights.py tests/test_earnings_consensus_sources.py tests/test_earnings_consensus_collector.py -q +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +``` + +Expected: zero failures; the existing third-party `dateutil` warning may remain. + +- [ ] **Step 5: Run required non-writing product gates** + +Run: + +```bash +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: executable gates pass, pilot readiness remains stale, and generated churn remains zero. + +- [ ] **Step 6: Stage exact files and verify hygiene** + +Stage only the three source modules, the three named test modules if changed, the six named product documents, and this plan if corrected during execution. Run `make staged-hygiene-check` and `git diff --cached --check`. + +- [ ] **Step 7: Commit, push, and update draft PR #113** + +Commit with message `Unify consensus commercial evidence review`, push only `codex/personal-research-mode-mvp`, confirm 0/0 alignment, and post verified scope, red-green evidence, full gates, unchanged outputs, external dependencies, and next step to PR #113. Keep it draft. + +- [ ] **Step 8: Re-audit the handoff state** + +Verify clean status, pushed HEAD, draft PR state, artifact hygiene, stale readiness, remaining stage gates, review safety, and active overall goal. diff --git a/docs/superpowers/plans/2026-07-19-commercial-price-apply-guard.md b/docs/superpowers/plans/2026-07-19-commercial-price-apply-guard.md new file mode 100644 index 000000000..06f2ac70f --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-commercial-price-apply-guard.md @@ -0,0 +1,59 @@ +# Commercial Price Apply Guard Implementation Plan + +> **For Codex:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Fail closed before staged-price mutation in explicit commercial mode unless lineage, exact-source commercial rights, and registered `prices` scope are complete for every valid row. + +**Architecture:** Extend `apply_price_import_merge(...)` with optional explicit mode and registry injection. Reuse the existing preview summary as the guard input, return a deterministic non-writing blocked result before validation replay, backup, or canonical merge, and preserve the existing research-mode path. + +**Constraints:** Use temporary pytest paths only; do not run repository `price-apply`, normalize, refresh, or readiness commands; do not edit source rights; do not infer providers; do not make guard passage automatic apply authorization or readiness evidence. + +## Task 1: Add failing guard tests + +**Files:** + +- Modify `tests/test_data_update.py` + +- [ ] Assert explicit research mode retains existing unregistered-source apply behavior. +- [ ] Assert explicit commercial mode blocks the same fixture with rights and price-scope blockers and leaves canonical bytes unchanged with no backup. +- [ ] Add approved-price fixtures proving missing lineage and missing registered price scope block independently. +- [ ] Add a complete approved-price fixture proving the existing merge path can run after the guard passes. +- [ ] Run focused tests and confirm the new signature/guard assertions fail before implementation. + +## Task 2: Implement the pre-mutation commercial guard + +**Files:** + +- Modify `src/data_update.py` + +- [ ] Import and reuse `commercial_mode_enabled`. +- [ ] Add optional `commercial_mode` and `rights_registry` parameters to apply. +- [ ] Pass the same registry into preview and later validation. +- [ ] Build the ordered blocker list from lineage, rights, and price-scope summaries. +- [ ] Return before backup/write when explicit commercial mode has blockers. +- [ ] Preserve existing missing/invalid and research-mode behavior. +- [ ] Run focused tests and require them to pass. + +## Task 3: Durable contracts + +**Files:** + +- Modify `ROADMAP.md` +- Modify `docs/DATA_STRATEGY.md` +- Modify `docs/METHODOLOGY.md` +- Modify `docs/PROVENANCE_CONTRACT.md` +- Modify `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +- [ ] Record the explicit commercial-mode mutation guard and local-research compatibility. +- [ ] Preserve the unchanged canonical 146-row audit and external source requirement. +- [ ] Record that tests use temporary fixtures and no repository apply or readiness write occurred. + +## Task 4: Full verification and delivery + +- [ ] Run focused changed-module/docs tests and public wording. +- [ ] Run the full test suite, dashboard smoke, six research-route render smoke, public check, commercial beta/release checks, pilot readiness, hygiene, and whitespace checks. +- [ ] Verify zero generated churn and no repository apply/readiness write. +- [ ] Stage exact files only and pass staged hygiene. +- [ ] Commit and push only the feature branch. +- [ ] Update draft PR #113 and verify it remains open and draft. +- [ ] Reassess remaining executable gates without claiming source, hosted, reviewer, calibration, or market completion. diff --git a/docs/superpowers/plans/2026-07-19-commercial-price-refresh-enforcement.md b/docs/superpowers/plans/2026-07-19-commercial-price-refresh-enforcement.md new file mode 100644 index 000000000..00c2f9d18 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-commercial-price-refresh-enforcement.md @@ -0,0 +1,182 @@ +# Commercial Price Refresh Enforcement Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task by task. + +**Goal:** Make every Commercial Research direct price refresh fail closed unless the exact reachable and selected provider independently has approved commercial rights and registered `prices` scope. + +**Architecture:** Add stable identities to concrete price sources, make the ladder validate and preserve exact child identity, and reuse the commercial field-scope review at both construction and mutation boundaries. Research mode remains compatible; injected registries provide deterministic tests without changing checked-in rights or calling providers. + +**Tech Stack:** Python 3.12, pandas, pytest, YAML source-rights registry + +## Global Constraints + +- Do not edit `config/source_rights.yml` or approve a provider. +- Do not call an external provider or run broad price/readiness refreshes. +- Do not generate or stage CSV, JSON, report, sample-report, screenshot, or timing churn. +- Keep exact-source rights, registered field scope, technical validity, and readiness independent. +- Use exact staging only and keep PR #113 open and draft. + +--- + +### Task 1: Lock exact price-provider identity + +**Files:** +- Modify: `src/data_update.py` +- Test: `tests/test_data_update.py` + +**Step 1: Write failing identity tests** + +Add assertions that the six concrete implementations expose these exact IDs: `stooq`, `yahoo`, `fmp`, `alpha_vantage`, `finnhub`, and `ibkr`. Add a ladder test proving a caller label that differs from the child's exact ID raises `ValueError`. + +**Step 2: Run the narrow tests and confirm RED** + +Run: + +```bash +python3 -m pytest tests/test_data_update.py -q -k 'source_id or ladder_rejects' +``` + +Expected: failures because the identities and ladder invariant do not yet exist. + +**Step 3: Implement the identity contract** + +Add `source_id` to `PriceHistorySource` and each concrete implementation. Add a helper that reads a nonblank exact ID. In `PriceSourceLadder.__init__`, require every route label to equal its child's `source_id`; store the possible IDs and a separate `last_source_id`. Set the latter only when that child returns usable rows. + +**Step 4: Run the narrow tests and confirm GREEN** + +Run the command from Step 2 and the existing ladder tests: + +```bash +python3 -m pytest tests/test_data_update.py -q -k 'PriceSourceLadder or source_id or ladder' +``` + +### Task 2: Enforce commercial rights before provider construction + +**Files:** +- Modify: `src/data_update.py` +- Test: `tests/test_data_update.py` + +**Step 1: Write failing construction tests** + +Build injected registries with `build_source_rights_registry`. Prove that: + +- an exact commercial provider with unknown/unapproved rights raises `commercial_price_source_review_required`; +- an approved provider without `prices` scope raises `commercial_price_scope_review_required`; +- automatic commercial construction retains only independently approved and scoped legs; +- an automatic ladder with no eligible leg fails closed. + +Do not fetch data in these tests. + +**Step 2: Run the construction tests and confirm RED** + +```bash +python3 -m pytest tests/test_data_update.py -q -k 'commercial and make_price_source' +``` + +**Step 3: Implement the shared review and construction filter** + +Import `commercial_mode_enabled` and `review_commercial_field_scope`. Add a deterministic review helper that distinguishes rights failure from missing `prices` scope. Extend `make_price_source(provider, *, commercial_mode=None, rights_registry=None)`. In commercial mode, review exact providers before construction and build the auto ladder from lazy provider factories only after filtering each exact ID. Preserve the existing order and behavior in research mode. + +**Step 4: Run the construction tests and confirm GREEN** + +Run the command from Step 2, then all source-construction tests. + +### Task 3: Enforce supplied and selected sources before mutation + +**Files:** +- Modify: `src/data_update.py` +- Test: `tests/test_data_update.py` + +**Step 1: Write failing mutation-boundary tests** + +Add spy sources and temporary paths proving: + +- a supplied source without exact ID fails before `fetch_history()`; +- an unapproved or scope-incomplete supplied source fails before fetch; +- no canonical or status file exists after either failure; +- an approved and scoped supplied source retains the existing successful refresh behavior; +- a source that changes its selected exact identity during fetch raises `commercial_price_source_changed` before canonical/status mutation. + +**Step 2: Run the boundary tests and confirm RED** + +```bash +python3 -m pytest tests/test_data_update.py -q -k 'commercial and update_local_price_data' +``` + +**Step 3: Implement pre-fetch and post-fetch checks** + +Extend `update_local_price_data(..., commercial_mode=None, rights_registry=None)`. Resolve every reachable exact source before loading tickers or fetching and review it once. After a nonempty response, resolve the actual selected source, require it to be in the pre-reviewed set, and repeat the exact rights/scope decision before appending rows or status. Missing, changed, or composite identity must raise before mutation. + +**Step 4: Run focused regression tests and confirm GREEN** + +```bash +python3 -m pytest tests/test_data_update.py tests/test_commercial_source_rights.py -q +``` + +### Task 4: Record the verified boundary + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Test: `tests/test_public_v1_release_docs.py` if existing contract assertions need extension + +**Step 1: Add documentation contract assertions where appropriate** + +Require public/internal documentation to state that direct commercial price refresh needs exact-provider approved rights and registered `prices` scope, while no provider approval, row lineage, current readiness, or market operation is implied. + +**Step 2: Update the roadmap and contracts** + +Record this as a verified local enforcement slice. Keep price lineage, atomic apply, an approved provider, fresh readiness, hosted beta evidence, reviewer validation, calibration, and operating controls open. + +**Step 3: Run documentation checks** + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py -q +make public-wording-check +make commercial-beta-check +``` + +### Task 5: Verify, commit, push, and update draft PR #113 + +**Files:** All intentionally changed files above only. + +**Step 1: Run fresh focused and full verification** + +```bash +python3 -m pytest tests/test_data_update.py tests/test_commercial_source_rights.py -q +python3 -m pytest tests -q +make dashboard-smoke +make research-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: engineering checks pass; pilot/readiness verdict may remain truthfully blocked solely by declared stale or unavailable evidence. + +**Step 2: Confirm generated-artifact hygiene** + +Compare the tracked generated-artifact digest to the baseline, inspect `git status`, and ensure no generated CSV, JSON, report, sample-report, screenshot, or timing file changed. + +**Step 3: Stage exact files and verify staging** + +Stage only the implementation, tests, ROADMAP, methodology/provenance docs, continuation prompt, design, and plan files. Run: + +```bash +make staged-hygiene-check +git diff --cached --check +``` + +**Step 4: Commit and push the coherent slice** + +Create one implementation commit, push only `codex/personal-research-mode-mvp`, and verify zero ahead/behind against its remote. + +**Step 5: Update and verify PR #113** + +Add a concise audit-resolution note with scope, tests, truthful open boundaries, and generated-artifact exclusion. Keep the PR draft. Confirm the PR head matches the pushed commit and inspect current hosted checks without merging or deploying. diff --git a/docs/superpowers/plans/2026-07-19-consensus-batch-preflight.md b/docs/superpowers/plans/2026-07-19-consensus-batch-preflight.md new file mode 100644 index 000000000..9d5e6a0c0 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-consensus-batch-preflight.md @@ -0,0 +1,346 @@ +# Prospective Consensus Batch Preflight Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make prospective consensus preview and record evaluate one ordered whole-batch contract so a later deterministic rejection cannot leave earlier proposed rows appended. + +**Architecture:** Add an immutable batch preview composed from the existing row preview and an in-memory virtual ledger. Add one batch append function that runs preflight once, selects the research/commercial gate without coupling their evidence states, and writes all accepted rows through one append handle only after the whole batch passes. Keep the single-row API by delegating it to the batch path. + +**Tech Stack:** Python 3.12, frozen dataclasses, standard-library CSV/JSON/path handling, pytest, Make-based repository gates. + +## Global Constraints + +- Research-only; no investment advice, broker integration, order routing, auto-trading, direct buy/sell instructions, or post-earnings price prediction. +- No provider fetch, source-rights edit, readiness rebuild, or repository CSV/JSON/report/sample-report/screenshot/timing write. +- Technical append eligibility, commercial rights, Revenue scope, EPS scope, actuals, consensus, backtesting, and calibration remain independent. +- Input order is authoritative; do not sort rows, infer providers, repair revision chains, invent supersession, or select a conflicting row. +- Promise deterministic preflight all-or-none behavior only; do not claim crash-safe transactionality or concurrent-writer locking. +- Use temporary pytest paths for every write test. +- Stage exact intentional files only; never use `git add -A`. + +--- + +### Task 1: Prove the current partial-write and preview mismatch + +**Files:** +- Modify: `tests/test_earnings_consensus_collector.py` +- Test: `tests/test_earnings_consensus_collector.py` + +**Interfaces:** +- Consumes: existing `FIELDS`, `ProspectiveConsensusRecord`, `main(...)`, and `load_snapshots(...)`. +- Produces: failing behavioral evidence for whole-batch preview and record. + +- [ ] **Step 1: Add temporary CSV and revision helpers** + +Add `csv`, `json`, and `asdict` imports, import `FIELDS` and `main`, and add helpers equivalent to: + +```python +def _write_records(path: Path, records: tuple[ProspectiveConsensusRecord, ...]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w", newline="", encoding="utf-8") as handle: + writer = csv.DictWriter(handle, fieldnames=FIELDS) + writer.writeheader() + writer.writerows(asdict(record) for record in records) + + +def _revision(record: ProspectiveConsensusRecord, *, snapshot_id: str = "snap-002") -> ProspectiveConsensusRecord: + return replace( + record, + snapshot_id=snapshot_id, + snapshot_at="2026-07-25T05:00:00Z", + retrieved_at="2026-07-25T05:00:01Z", + source_ref=f"file://reviewed/{record.ticker}/{record.fiscal_period}/20260725", + revenue_consensus="102", + supersedes_snapshot_id=record.snapshot_id, + ) +``` + +- [ ] **Step 2: Add a failing record test for a later duplicate** + +```python +def test_record_preflights_whole_batch_before_any_append(tmp_path: Path): + input_path = tmp_path / "input.csv" + ledger = tmp_path / "ledger.csv" + first = _record() + _write_records(input_path, (first, first)) + + with pytest.raises(ValueError, match="duplicate"): + main(["record", "--input", str(input_path), "--ledger", str(ledger), "--confirm-reviewed"]) + + assert not ledger.exists() +``` + +- [ ] **Step 3: Add failing preview tests for virtual-ledger lineage** + +Capture stdout with `capsys`, parse JSON, and assert: + +```python +payload = json.loads(capsys.readouterr().out) +assert payload.get("state") == "reviewable_batch" +assert [row["state"] for row in payload["rows"]] == ["reviewable_new", "reviewable_revision"] +assert payload["technical_write_allowed"] is True +``` + +Add a second input containing the same row twice and assert the later row is `duplicate`, `state` is `rejected_batch`, and `technical_write_allowed` is false. + +- [ ] **Step 4: Add a failing empty-input record test** + +Write only the header and assert `main(record...)` raises `ValueError` containing `empty_batch` while the destination and its missing parent remain absent. + +- [ ] **Step 5: Run the focused tests and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_earnings_consensus_collector.py -q +``` + +Expected: assertion failures proving current preview misses intra-batch lineage/conflicts, record leaves a partial ledger, and empty record is silently accepted. + +--- + +### Task 2: Add the pure ordered batch preview + +**Files:** +- Modify: `src/earnings_consensus_collector.py` +- Modify: `tests/test_earnings_consensus_collector.py` + +**Interfaces:** +- Consumes: `CollectionPreview`, `preview_collection(...)`, `SourceRights`, and the immutable source-rights registry. +- Produces: `BatchCollectionPreview` and `preview_collection_batch(...)`. + +- [ ] **Step 1: Add the immutable batch result** + +```python +@dataclass(frozen=True) +class BatchCollectionPreview: + mode: str + write_performed: bool + state: str + row_count: int + reviewable_count: int + technical_write_allowed: bool + commercial_evidence_ready: bool + commercial_write_allowed: bool + technical_blockers: tuple[str, ...] + commercial_blockers: tuple[str, ...] + rows: tuple[CollectionPreview, ...] +``` + +- [ ] **Step 2: Implement ordered virtual-ledger evaluation** + +Add: + +```python +def preview_collection_batch( + existing: Sequence[ProspectiveConsensusRecord], + proposed: Sequence[ProspectiveConsensusRecord], + *, + as_of: str | None = None, + cooldown_hours: int = 0, + rights_registry: Mapping[str, SourceRights] | None = None, +) -> BatchCollectionPreview: +``` + +Resolve the registry once. For every one-based row position, call `preview_collection(...)` with `as_of` when supplied or that row's `retrieved_at`. Append technically reviewable rows to a mutable virtual ledger. Build blockers as: + +```python +f"row_{index}:{preview.state}:{preview.reason}" +f"row_{index}:{blocker}" +``` + +For no proposed rows, return `state="empty_batch"`, false gates, and `("batch:empty_input",)` technical and commercial blockers. Otherwise return `reviewable_batch` only when every row is technically writeable, and compute commercial evidence independently across every row. + +- [ ] **Step 3: Route CLI preview through the batch function** + +Replace the independent list comprehension with: + +```python +result = preview_collection_batch(existing, proposed, as_of=args.as_of) +print(json.dumps(asdict(result), indent=2, sort_keys=True)) +``` + +- [ ] **Step 4: Add and verify reversed-revision behavior** + +Preview a revision before its target and assert `rejected_batch`, first-row reason contains `supersedes_snapshot_id does not exist`, and input order is unchanged in the returned row results. + +- [ ] **Step 5: Run focused tests and verify the preview tests are GREEN** + +Run: + +```bash +python3 -m pytest tests/test_earnings_consensus_collector.py -q +``` + +Expected: preview/empty tests pass; the partial-write test still fails until Task 3. + +--- + +### Task 3: Add one preflighted batch append path + +**Files:** +- Modify: `src/earnings_consensus_collector.py` +- Modify: `tests/test_earnings_consensus_collector.py` + +**Interfaces:** +- Consumes: `preview_collection_batch(...)` and `commercial_mode_enabled()`. +- Produces: `append_reviewed_batch(...)`; `append_reviewed_snapshot(...)` delegates to it. + +- [ ] **Step 1: Add failing API tests for research and commercial batches** + +Use `getattr(collector, "append_reviewed_batch", None)` and first assert it is callable so the pre-implementation run fails as an assertion rather than a collection error. Then prove: + +- an existing ledger remains byte-identical when a later same-period row is rejected; +- a later unregistered commercial row leaves a missing destination parent absent; +- a valid new row followed by its explicit revision appends both rows in order in research mode; +- the same ordered batch appends in commercial mode with an injected approved registry; +- an empty batch is rejected without filesystem mutation. + +- [ ] **Step 2: Run the focused tests and verify RED for the missing batch API** + +Run: + +```bash +python3 -m pytest tests/test_earnings_consensus_collector.py -q +``` + +Expected: failures identify the missing `append_reviewed_batch` behavior. + +- [ ] **Step 3: Implement `append_reviewed_batch(...)`** + +Use this signature: + +```python +def append_reviewed_batch( + path: Path | str, + records: Sequence[ProspectiveConsensusRecord], + *, + confirm_reviewed: bool, + commercial_mode: bool | None = None, + rights_registry: Mapping[str, SourceRights] | None = None, +) -> Path: +``` + +Convert records to a tuple, validate confirmation, load the existing ledger and registry once, run batch preview with `as_of=None`, reject technical blockers first, resolve commercial mode once, reject commercial blockers second, then create the parent and append every row using one `csv.DictWriter` and `writerows(...)` call. + +Use stable errors: + +```python +raise ValueError(f"{preview.state}: " + "; ".join(preview.technical_blockers)) +raise ValueError("batch_commercial_evidence_review_required: " + "; ".join(preview.commercial_blockers)) +``` + +- [ ] **Step 4: Delegate single-row append and CLI record** + +Make `append_reviewed_snapshot(...)` call `append_reviewed_batch(path, (record,), ...)`. Replace the CLI record loop with one `append_reviewed_batch(...)` call over the loaded tuple. + +- [ ] **Step 5: Run focused collector and rights tests and verify GREEN** + +Run: + +```bash +python3 -m pytest tests/test_earnings_consensus_collector.py tests/test_earnings_consensus_sources.py tests/test_commercial_source_rights.py -q +``` + +Expected: all focused tests pass with only the existing dependency warning. + +--- + +### Task 4: Update durable operating contracts + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/EARNINGS_NOWCAST_PILOT.md` +- Modify: `docs/DATA_STRATEGY.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: verified batch preview/record behavior. +- Produces: roadmap item 30 and a continuation anchor for this design/plan lineage. + +- [ ] **Step 1: Document the preview/record convergence** + +State that preview simulates proposed rows in input order against a virtual append-only ledger and record uses that exact result before mutation. + +- [ ] **Step 2: Preserve independent evidence and truthful limits** + +State that technical and commercial batch gates remain independent; research mode does not satisfy Commercial Research activation; no source, rights, snapshot, readiness, calibration, hosted, reviewer, or market gate changed. + +- [ ] **Step 3: Document the transaction boundary** + +State explicitly that deterministic preflight prevents known later rejections from causing partial writes but does not provide concurrent-writer locking or crash-safe filesystem transactions. + +- [ ] **Step 4: Update the continuation prompt** + +Add the design/plan commit anchor, implemented capability, truth boundary, and exact next-stage meaning without changing the external dependency classifications. + +- [ ] **Step 5: Run wording and whitespace checks** + +Run: + +```bash +make public-wording-check +git diff --check +``` + +Expected: both pass. + +--- + +### Task 5: Full verification, exact delivery, and clean-state audit + +**Files:** +- Stage only the exact changed code/test/docs paths from Tasks 1-4. + +**Interfaces:** +- Consumes: the completed implementation and durable contracts. +- Produces: one verified implementation commit, pushed draft PR update, and a clean aligned branch. + +- [ ] **Step 1: Run full tests** + +```bash +python3 -m pytest tests -q +``` + +Expected: zero failures; record the exact pass count and warnings. + +- [ ] **Step 2: Run product and release gates** + +```bash +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: local code/render/public/commercial gates pass; pilot readiness may remain blocked only by the already documented stale-readiness or external gates. + +- [ ] **Step 3: Verify zero generated churn** + +Confirm `git status --short` lists only the exact intended code/test/docs paths and no CSV, JSON, report, sample-report, screenshot, timing, readiness, or canonical-data path. + +- [ ] **Step 4: Stage exact files and verify staged hygiene** + +Run `git add --` with the explicit paths, then: + +```bash +git diff --cached --check +make staged-hygiene-check +``` + +Expected: product/code/docs/test files only; zero generated or manual-review paths. + +- [ ] **Step 5: Commit, push, and update PR #113** + +Commit with `Guard prospective consensus batches`, push only `codex/personal-research-mode-mvp`, and post the behavior, verification, maturity boundary, artifact boundary, and external dependency status to the existing draft PR. Do not merge or deploy. + +- [ ] **Step 6: Reverify final truth** + +Confirm clean 0/0 alignment, HEAD ancestry, PR open/draft/mergeable state, zero generated churn, stale-readiness truth, and the next executable local gate. Keep the overall goal active. diff --git a/docs/superpowers/plans/2026-07-19-consensus-ledger-integrity.md b/docs/superpowers/plans/2026-07-19-consensus-ledger-integrity.md new file mode 100644 index 000000000..3499984da --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-consensus-ledger-integrity.md @@ -0,0 +1,17 @@ +# Consensus Ledger Integrity Implementation Plan + +1. Add failing tests for invalid existing ledgers, all linear-chain violations, + non-leaf revisions, row-numbered errors, and unchanged bytes on failure. +2. Add failing tests for preview cutoff/input/ledger/commercial-mode receipt + binding and required record arguments. +3. Separate validated existing-ledger loading from individually validated + proposed-batch loading. +4. Implement semantic digests, full chain validation, leaf-only revision checks, + normalized cutoff output, and deterministic preview receipts. +5. Require exact cutoff and receipt at programmatic and CLI/Make record boundaries; + revalidate and recompute before append. +6. Update Make targets, methodology, provenance, operator/pilot docs, roadmap, and + continuation contract without changing the source-review input contract. +7. Run focused/full tests and every required product, release, pilot, PR-range, + and hygiene check; stage exact files, commit, push, update draft PR #113, and + verify hosted CI on the exact head. diff --git a/docs/superpowers/plans/2026-07-19-consensus-source-review-command.md b/docs/superpowers/plans/2026-07-19-consensus-source-review-command.md new file mode 100644 index 000000000..42f9737e6 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-consensus-source-review-command.md @@ -0,0 +1,639 @@ +# Consensus Source Review Command Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a supported read-only command that loads one supplied consensus CSV and exposes the existing technical, temporal, rights, and Revenue/EPS scope review before collection preview. + +**Architecture:** Extend `src.earnings_consensus_sources` with two pure presentation-boundary helpers and an explicit CLI review mode while leaving `validate_source_rows(...)` as the only validation decision. Add one bytecode-free Make target; existing provider-status behavior remains the default. + +**Tech Stack:** Python 3.12 standard library (`argparse`, `csv`, `dataclasses`, `json`, `pathlib`), pytest, GNU Make, Markdown documentation. + +## Global Constraints + +- The command is read-only: no provider fetch, normalization, directory creation, ledger append, apply, readiness rebuild, or generated output file. +- Require explicit `INPUT`, exact `PROVIDER`, and UTC `AS_OF`; supply no defaults. +- Preserve technical acceptance, candidate/history state, commercial rights, Revenue scope, EPS scope, collection, and activation as independent evidence. +- A readable review returns zero even when rows are rejected or commercially incomplete; invocation and CSV-shape errors return nonzero. +- Keep the existing `earnings-consensus-source-status` CLI and Make behavior compatible. +- Set `PYTHONDONTWRITEBYTECODE=1` on the new Make target. +- Do not create or stage CSV, JSON, report, sample-report, screenshot, browser-timing, readiness, canonical-data, or proof-ledger churn. +- Never use `git add -A`; stage only exact reviewed product/code/docs/test paths. +- Keep PR #113 open and draft; do not merge or deploy. + +--- + +### Task 1: Read and render one source-review CSV + +**Files:** +- Modify: `src/earnings_consensus_sources.py` +- Test: `tests/test_earnings_consensus_sources.py` + +**Interfaces:** +- Consumes: `validate_source_rows(provider: str, rows: Sequence[Mapping[str, object]], *, as_of: object, rights_registry: Mapping[str, SourceRights] | None = None) -> SourceValidationResult`. +- Produces: `load_source_review_csv(path: Path | str) -> tuple[dict[str, object], ...]`. +- Produces: `render_source_validation_result(result: SourceValidationResult) -> str`. + +- [ ] **Step 1: Add failing loader and renderer tests** + +Task 1 needs the existing `Path` and `pytest` imports. Extend the source import and add these tests: + +Extend the source import to include `load_source_review_csv` and `render_source_validation_result`: + +```python +from src.earnings_consensus_sources import ( + consensus_source_statuses, + load_source_review_csv, + render_source_validation_result, + validate_source_rows, +) +``` + +```python +def test_load_source_review_csv_preserves_rows_without_writing(tmp_path: Path): + input_path = tmp_path / "reviewed.csv" + input_path.write_text( + "ticker,fiscal_period,snapshot_at,retrieved_at,source_ref,revenue_consensus,eps_consensus,history_scope\n" + "NVDA,2027-Q1,2026-07-18T05:00:00Z,2026-07-18T05:00:01Z,fixture://NVDA,1,2,current_only\n", + encoding="utf-8", + ) + before = input_path.read_bytes() + + rows = load_source_review_csv(input_path) + + assert rows == ( + { + "ticker": "NVDA", + "fiscal_period": "2027-Q1", + "snapshot_at": "2026-07-18T05:00:00Z", + "retrieved_at": "2026-07-18T05:00:01Z", + "source_ref": "fixture://NVDA", + "revenue_consensus": "1", + "eps_consensus": "2", + "history_scope": "current_only", + }, + ) + assert input_path.read_bytes() == before + assert {path.name for path in tmp_path.iterdir()} == {"reviewed.csv"} + + +@pytest.mark.parametrize( + ("contents", "message"), + [ + ("", "header row"), + ("ticker,ticker\nNVDA,NVDA\n", "non-blank unique column names"), + ("ticker,,fiscal_period\nNVDA,x,2027-Q1\n", "non-blank unique column names"), + ("ticker\nNVDA,extra\n", "more values than the header"), + ], +) +def test_load_source_review_csv_rejects_ambiguous_shapes( + tmp_path: Path, + contents: str, + message: str, +): + input_path = tmp_path / "reviewed.csv" + input_path.write_text(contents, encoding="utf-8") + + with pytest.raises(ValueError, match=message): + load_source_review_csv(input_path) + + +def test_load_source_review_csv_reports_unreadable_input(tmp_path: Path): + with pytest.raises(ValueError, match="cannot read consensus source review CSV"): + load_source_review_csv(tmp_path / "missing.csv") + with pytest.raises(ValueError, match="cannot read consensus source review CSV"): + load_source_review_csv(tmp_path) + + +def test_load_source_review_csv_accepts_header_only_as_empty_review(tmp_path: Path): + input_path = tmp_path / "reviewed.csv" + input_path.write_text("ticker,fiscal_period\n", encoding="utf-8") + + rows = load_source_review_csv(input_path) + result = validate_source_rows( + "reviewed_csv", + rows, + as_of=REVIEW_CUTOFF, + ) + + assert rows == () + assert result.state == "still_blocked" + assert result.accepted_count == 0 + assert result.auto_apply is False + + +def test_render_source_validation_result_keeps_technical_and_commercial_evidence_separate(): + result = validate_source_rows( + "licensed_consensus", + [ + _historical_row(eps_consensus=""), + _current_row(retrieved_at="2026-07-18T06:00:01Z"), + ], + as_of=REVIEW_CUTOFF, + rights_registry=_rights_registry(supported_fields=("revenue_consensus",)), + ) + + rendered = render_source_validation_result(result) + + assert "Consensus Source Review" in rendered + assert "state: historical_evidence_reviewable" in rendered + assert "accepted_count: 1" in rendered + assert "rejected_count: 1" in rendered + assert "commercial_evidence_ready: true" in rendered + assert "- row 2: retrieved_at is after review cutoff" in rendered + assert "- row 1: required=revenue_consensus; missing=none; ready=true; blockers=none" in rendered + assert "auto_apply: false" in rendered + assert "collection preview remains a separate reviewed gate" in rendered +``` + +- [ ] **Step 2: Run the new tests and require the intended red state** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_earnings_consensus_sources.py::test_load_source_review_csv_preserves_rows_without_writing \ + tests/test_earnings_consensus_sources.py::test_load_source_review_csv_rejects_ambiguous_shapes \ + tests/test_earnings_consensus_sources.py::test_load_source_review_csv_reports_unreadable_input \ + tests/test_earnings_consensus_sources.py::test_load_source_review_csv_accepts_header_only_as_empty_review \ + tests/test_earnings_consensus_sources.py::test_render_source_validation_result_keeps_technical_and_commercial_evidence_separate -q +``` + +Expected: collection fails because `load_source_review_csv` and `render_source_validation_result` are not defined. + +- [ ] **Step 3: Implement strict read-only CSV loading** + +Import `csv`, then add: + +```python +def load_source_review_csv(path: Path | str) -> tuple[dict[str, object], ...]: + """Load supplied consensus source rows without normalizing or writing evidence.""" + + review_path = Path(path) + try: + with review_path.open("r", encoding="utf-8", newline="") as handle: + reader = csv.DictReader(handle, strict=True) + if reader.fieldnames is None: + raise ValueError("consensus source review CSV must contain a header row") + fieldnames = tuple(str(field or "") for field in reader.fieldnames) + if ( + any(not field.strip() for field in fieldnames) + or len(set(fieldnames)) != len(fieldnames) + ): + raise ValueError( + "consensus source review CSV headers must be non-blank unique column names" + ) + rows: list[dict[str, object]] = [] + for row_number, row in enumerate(reader, start=1): + if None in row: + raise ValueError( + f"consensus source review CSV row {row_number} has more values than the header" + ) + rows.append({str(key): value for key, value in row.items()}) + return tuple(rows) + except ValueError: + raise + except (OSError, UnicodeError, csv.Error) as exc: + raise ValueError( + f"cannot read consensus source review CSV: {review_path}" + ) from exc +``` + +Do not strip or rename headers and do not coerce cells. Core validation must expose missing exact fields. + +- [ ] **Step 4: Implement deterministic human rendering** + +Add: + +```python +def _joined(values: Sequence[object]) -> str: + return ",".join(str(value) for value in values) if values else "none" + + +def render_source_validation_result(result: SourceValidationResult) -> str: + lines = [ + "Consensus Source Review", + "Read-only: this command does not fetch, normalize, record, apply, rebuild readiness, or write artifacts.", + f"provider: {result.provider or '-'}", + f"review_cutoff: {result.review_cutoff}", + f"state: {result.state}", + f"accepted_count: {result.accepted_count}", + f"rejected_count: {result.rejected_count}", + f"historical_snapshot_count: {result.historical_snapshot_count}", + f"candidate_context_count: {result.candidate_context_count}", + f"rights_status: {result.rights_status}", + f"commercial_rights_approved: {str(result.commercial_rights_approved).lower()}", + f"commercial_ready_count: {result.commercial_ready_count}", + f"commercial_review_required_count: {result.commercial_review_required_count}", + f"commercial_evidence_ready: {str(result.commercial_evidence_ready).lower()}", + f"commercial_blockers: {_joined(result.commercial_blockers)}", + "rejected_rows:", + ] + if result.rejected_rows: + lines.extend( + f"- row {row['row_number']}: {row['reason']}" + for row in result.rejected_rows + ) + else: + lines.append("- none") + lines.append("commercial_review_rows:") + if result.commercial_review_rows: + lines.extend( + "- row " + f"{row.row_number}: required={_joined(row.required_supported_fields)}; " + f"missing={_joined(row.missing_supported_fields)}; " + f"ready={str(row.commercial_evidence_ready).lower()}; " + f"blockers={_joined(row.commercial_blockers)}" + for row in result.commercial_review_rows + ) + else: + lines.append("- none") + lines.extend( + [ + f"auto_apply: {str(result.auto_apply).lower()}", + "next_gate: collection preview remains a separate reviewed gate after the payload and evidence are accepted.", + "Boundary: reviewability is not collection, activation, readiness, backtesting, calibration, or investment advice.", + ] + ) + return "\n".join(lines) +``` + +- [ ] **Step 5: Run the full source test module** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_earnings_consensus_sources.py -q +``` + +Expected: all tests pass; no repository file changes beyond the two intentional source/test files. + +- [ ] **Step 6: Review Task 1 diff** + +Run `git diff --check` and inspect the source/test diff. Confirm no status or validator decision changed and no generated file exists. + +--- + +### Task 2: Expose explicit CLI and Make review modes + +**Files:** +- Modify: `src/earnings_consensus_sources.py` +- Modify: `Makefile` +- Test: `tests/test_earnings_consensus_sources.py` +- Test: `tests/test_launchers.py` + +**Interfaces:** +- Consumes: `load_source_review_csv(...)`, `validate_source_rows(...)`, and `render_source_validation_result(...)` from Task 1. +- Produces: `python3 -m src.earnings_consensus_sources --review-csv PATH --provider SOURCE --as-of TIMESTAMP [--json]`. +- Produces: `make earnings-consensus-source-review INPUT=... PROVIDER=... AS_OF=... [JSON=1]`. + +- [ ] **Step 1: Add failing CLI and Make contract tests** + +Add `json`, `os`, `subprocess`, and `sys` imports to `tests/test_earnings_consensus_sources.py`, then add the exact test-only CSV constant and tests below. + +Add this exact test-only constant: + +```python +HISTORICAL_REVIEW_CSV = ( + "ticker,fiscal_period,snapshot_at,retrieved_at,source_ref,revenue_consensus,eps_consensus,history_scope," + "revenue_currency,revenue_unit_scale,revenue_basis,eps_currency,eps_basis,eps_share_basis,eps_operations_basis,split_adjustment_basis\n" + "NVDA,2027-Q1,2026-07-18T05:00:00Z,2026-07-18T05:00:01Z,fixture://consensus/NVDA/2027-Q1,1,,point_in_time," + "USD,1,reported,USD,gaap,diluted,reported,as_reported\n" +) +``` + +```python +def test_source_review_cli_renders_supplied_rows_without_artifacts(tmp_path: Path): + input_path = tmp_path / "reviewed.csv" + input_path.write_text(HISTORICAL_REVIEW_CSV, encoding="utf-8") + before = input_path.read_bytes() + + result = subprocess.run( + [ + sys.executable, + "-m", + "src.earnings_consensus_sources", + "--review-csv", + str(input_path), + "--provider", + "sec_companyfacts", + "--as-of", + REVIEW_CUTOFF, + ], + check=False, + capture_output=True, + text=True, + env={**os.environ, "PYTHONDONTWRITEBYTECODE": "1"}, + ) + + assert result.returncode == 0 + assert "state: historical_evidence_reviewable" in result.stdout + assert "rights_status: approved" in result.stdout + assert "registered_consensus_scope_missing:revenue_consensus" in result.stdout + assert input_path.read_bytes() == before + assert {path.name for path in tmp_path.iterdir()} == {"reviewed.csv"} + + +def test_source_review_cli_json_matches_result_contract(tmp_path: Path): + input_path = tmp_path / "reviewed.csv" + input_path.write_text(HISTORICAL_REVIEW_CSV, encoding="utf-8") + + result = subprocess.run( + [ + sys.executable, + "-m", + "src.earnings_consensus_sources", + "--review-csv", + str(input_path), + "--provider", + "sec_companyfacts", + "--as-of", + REVIEW_CUTOFF, + "--json", + ], + check=False, + capture_output=True, + text=True, + env={**os.environ, "PYTHONDONTWRITEBYTECODE": "1"}, + ) + + payload = json.loads(result.stdout) + assert result.returncode == 0 + assert payload["provider"] == "sec_companyfacts" + assert payload["review_cutoff"] == REVIEW_CUTOFF + assert payload["historical_snapshot_count"] == 1 + assert payload["commercial_rights_approved"] is True + assert payload["commercial_evidence_ready"] is False + assert payload["auto_apply"] is False + + +@pytest.mark.parametrize( + "args", + [ + ["--review-csv", "missing.csv", "--as-of", REVIEW_CUTOFF], + ["--review-csv", "missing.csv", "--provider", "reviewed_csv"], + ["--provider", "reviewed_csv", "--as-of", REVIEW_CUTOFF], + [ + "--review-csv", + "missing.csv", + "--reviewed-csv", + "other.csv", + "--provider", + "reviewed_csv", + "--as-of", + REVIEW_CUTOFF, + ], + ], +) +def test_source_review_cli_requires_complete_review_mode(args: list[str]): + result = subprocess.run( + [sys.executable, "-m", "src.earnings_consensus_sources", *args], + check=False, + capture_output=True, + text=True, + env={**os.environ, "PYTHONDONTWRITEBYTECODE": "1"}, + ) + + assert result.returncode != 0 + assert "error:" in result.stderr + + +def test_source_review_cli_rejects_invalid_cutoff(tmp_path: Path): + input_path = tmp_path / "reviewed.csv" + input_path.write_text("ticker,fiscal_period\n", encoding="utf-8") + + result = subprocess.run( + [ + sys.executable, + "-m", + "src.earnings_consensus_sources", + "--review-csv", + str(input_path), + "--provider", + "reviewed_csv", + "--as-of", + "not-a-cutoff", + ], + check=False, + capture_output=True, + text=True, + env={**os.environ, "PYTHONDONTWRITEBYTECODE": "1"}, + ) + + assert result.returncode != 0 + assert "review cutoff" in result.stderr +``` + +In `tests/test_launchers.py`, add: + +```python +def test_makefile_exposes_bytecode_free_consensus_source_review_target(): + makefile = Path("Makefile").read_text(encoding="utf-8") + + assert "earnings-consensus-source-review:" in makefile + assert "PYTHONDONTWRITEBYTECODE=1 python3 -m src.earnings_consensus_sources" in makefile + assert '--review-csv "$(INPUT)"' in makefile + assert '--provider "$(PROVIDER)"' in makefile + assert '--as-of "$(AS_OF)"' in makefile + assert "$(if $(JSON),--json,)" in makefile +``` + +- [ ] **Step 2: Run the new CLI/Make tests and require failure** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_earnings_consensus_sources.py \ + tests/test_launchers.py \ + -k "source_review_cli or makefile_exposes_bytecode_free_consensus_source_review_target" -q +``` + +Expected: CLI tests fail because the review arguments are unknown and the Make target assertion fails. + +- [ ] **Step 3: Extend CLI argument routing without changing status mode** + +Add arguments: + +```python +parser.add_argument("--review-csv", type=Path) +parser.add_argument("--provider") +parser.add_argument("--as-of") +``` + +Before the existing status rendering, add: + +```python +if args.review_csv is not None: + if args.reviewed_csv is not None: + parser.error("--review-csv cannot be combined with --reviewed-csv") + if not str(args.provider or "").strip(): + parser.error("--provider is required with --review-csv") + if not str(args.as_of or "").strip(): + parser.error("--as-of is required with --review-csv") + try: + result = validate_source_rows( + args.provider, + load_source_review_csv(args.review_csv), + as_of=args.as_of, + ) + except ValueError as exc: + parser.error(str(exc)) + if args.json: + print(json.dumps(asdict(result), indent=2, sort_keys=True)) + else: + print(render_source_validation_result(result)) + return 0 + +if args.provider is not None or args.as_of is not None: + parser.error("--provider and --as-of require --review-csv") +``` + +Leave the existing `consensus_source_statuses(...)` branch and its JSON/list shape unchanged. + +- [ ] **Step 4: Add the Make target** + +Add `earnings-consensus-source-review` to the existing `.PHONY` line and add after status: + +```make +earnings-consensus-source-review: + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.earnings_consensus_sources --review-csv "$(INPUT)" --provider "$(PROVIDER)" --as-of "$(AS_OF)" $(if $(JSON),--json,) +``` + +- [ ] **Step 5: Run focused CLI, launcher, and existing-status tests** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_earnings_consensus_sources.py \ + tests/test_launchers.py -q +``` + +Expected: all tests pass and existing status tests remain unchanged. + +- [ ] **Step 6: Run a temporary-directory CLI smoke** + +Use `mktemp -d`, copy a test-only CSV into it, run the Python review command with `PYTHONDONTWRITEBYTECODE=1`, inspect the output, and confirm the directory still contains only the input. Do not place the fixture under `data/`, `outputs/`, or the repository. + +--- + +### Task 3: Document the Stage 2 review gate and release the slice + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/DATA_STRATEGY.md` +- Modify: `docs/EARNINGS_NOWCAST_PILOT.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes: the verified command and Make target from Task 2. +- Produces: roadmap item 34, Stage 2 command ordering, design/plan lineage anchor, and truthful review boundaries. + +- [ ] **Step 1: Update roadmap and methodology documentation** + +Add roadmap item 34: one supplied consensus CSV can now enter a read-only, explicit-provider, explicit-cutoff source review before collection preview. State that it improves Stage 2 operating reliability and rejection visibility but supplies no source, rights, payload proof, collection, readiness, backtesting, calibration, hosting, reviewers, or market validation. + +Document the command sequence: + +```bash +SOURCE_INPUT= +make earnings-consensus-source-review INPUT=$SOURCE_INPUT PROVIDER= AS_OF= +COLLECTION_INPUT= +make earnings-consensus-collection-preview INPUT=$COLLECTION_INPUT AS_OF= +``` + +Clarify that these are distinct input contracts: the second command is appropriate only after separate payload/evidence review and explicit evidence-preserving mapping into the checked-in prospective schema. No command infers or writes that mapping, and neither gate writes without the explicit later record command. + +- [ ] **Step 2: Update data strategy, pilot, provenance, and continuation contract** + +Record: + +- strict CSV header/shape checks; +- original one-based rejected row numbers; +- explicit provider and cutoff; +- human/JSON stdout only; +- `auto_apply=false`; +- checked-in registry evidence remains metadata, not payload permission; +- no default provider, cutoff, collection, or readiness promotion; +- current consensus source/data/right classifications remain external and unchanged. + +Add design/plan lineage anchor `6aa9c0c44` or the later plan commit, the capability bullet, truthful boundary, and Stage 2 source-review-before-preview instruction to the continuation prompt. + +- [ ] **Step 3: Add a failing documentation contract test, then make it pass** + +Add to `tests/test_public_v1_release_docs.py`: + +```python +def test_consensus_source_review_docs_keep_review_collection_and_activation_separate(): + roadmap = _read("ROADMAP.md") + data_strategy = _read("docs/DATA_STRATEGY.md") + pilot = _read("docs/EARNINGS_NOWCAST_PILOT.md") + methodology = _read("docs/METHODOLOGY.md") + provenance = _read("docs/PROVENANCE_CONTRACT.md") + prompt = _read("docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md") + + for text in (roadmap, data_strategy, pilot, methodology, provenance, prompt): + assert "earnings-consensus-source-review" in text + assert "collection preview" in text.lower() + assert "read-only" in text.lower() + assert "explicit provider" in roadmap.lower() + assert "original one-based" in provenance.lower() + assert "auto_apply=false" in pilot + assert "source-review-before-preview" in prompt.lower() +``` + +Run this exact test before the docs edit and require failure, then rerun it after the six documentation changes and require a pass. + +- [ ] **Step 4: Run focused and full verification** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_earnings_consensus_sources.py \ + tests/test_launchers.py \ + tests/test_public_v1_release_docs.py -q +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +``` + +Expected: zero failures; the existing third-party dateutil warning may remain. + +- [ ] **Step 5: Run all required non-writing product gates** + +Run: + +```bash +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: executable gates pass, pilot readiness remains blocked by stale saved readiness, and generated churn remains zero. + +- [ ] **Step 6: Stage exact files and verify hygiene** + +Stage only the source module, Makefile, the two or three named test files actually changed, the six named product documents, and this plan if corrected during execution. Run: + +```bash +make staged-hygiene-check +git diff --cached --check +``` + +Expected: only intentional product/code/docs/test files; zero generated or manual-review paths. + +- [ ] **Step 7: Commit, push, and update draft PR #113** + +Commit the implementation/docs slice with message `Add consensus source review command`. Push only `codex/personal-research-mode-mvp`. Post the exact command, red-green evidence, full gate results, no-write proof, unchanged external classifications, and next external unblock to PR #113. Keep it draft. + +- [ ] **Step 8: Re-audit the handoff** + +Verify clean status, 0/0 upstream alignment, open/draft/mergeable PR state, generated-artifact hygiene, stale readiness, and active overall goal. Do not merge or deploy. diff --git a/docs/superpowers/plans/2026-07-19-consensus-source-rights-guard.md b/docs/superpowers/plans/2026-07-19-consensus-source-rights-guard.md new file mode 100644 index 000000000..64f2468b2 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-consensus-source-rights-guard.md @@ -0,0 +1,79 @@ +# Prospective Consensus Source-Rights Guard Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Expose independent Revenue/EPS commercial source-rights evidence in prospective consensus preview and block unapproved ledger writes only in explicit Commercial Research mode. + +**Architecture:** Reuse the immutable commercial source-rights registry and exact-source eligibility decision. Add a pure preview helper that derives required metric scopes from the populated record, keep technical `write_allowed` separate, and make append enforce the combined evidence gate before filesystem mutation when commercial mode is enabled. + +**Constraints:** Use temporary pytest paths only; do not collect provider data, run repository record/apply/readiness commands, edit source rights, infer providers, or generate/stage CSV, JSON, report, sample-report, screenshot, or timing churn. + +## Task 1: Add failing preview evidence tests + +**Files:** + +- Modify `tests/test_earnings_consensus_collector.py` + +- [ ] Build injected immutable rights fixtures for approved, unverified, and scope-limited sources. +- [ ] Assert approved Revenue and EPS scopes produce independent, deterministic preview evidence. +- [ ] Assert a Revenue-only record does not require EPS scope. +- [ ] Assert a mixed record reports only its missing metric scope. +- [ ] Assert unknown and composite exact sources fail closed without provider inference. +- [ ] Run the focused collector tests and confirm the new contract fails before implementation. + +## Task 2: Add failing commercial append tests + +**Files:** + +- Modify `tests/test_earnings_consensus_collector.py` + +- [ ] Assert research mode preserves reviewed local append compatibility. +- [ ] Assert commercial mode blocks an unregistered row before ledger or parent-directory creation. +- [ ] Assert an approved, fully scoped source can append to a temporary ledger in commercial mode. +- [ ] Assert technical rejection remains independently authoritative. +- [ ] Run the focused collector tests and confirm the guard assertions fail before implementation. + +## Task 3: Implement preview and pre-write guard + +**Files:** + +- Modify `src/earnings_consensus_collector.py` + +- [ ] Import the existing mode, registry, and eligibility contracts. +- [ ] Derive ordered required fields from populated Revenue/EPS values. +- [ ] Add exact-source rights, supported-scope, evidence-ready, combined-write, and blocker fields to `CollectionPreview`. +- [ ] Preserve existing technical preview states and `write_allowed` meaning. +- [ ] Add optional mode and registry injection to append. +- [ ] Use one registry decision for preview and append. +- [ ] Reject a commercial evidence failure before directory creation or ledger mutation. +- [ ] Keep CLI preview non-writing and make the record command inherit the environment gate. +- [ ] Run the focused collector and source-rights tests and require them to pass. + +## Task 4: Durable research and operating contracts + +**Files:** + +- Modify `ROADMAP.md` +- Modify `docs/EARNINGS_NOWCAST_PILOT.md` +- Modify `docs/DATA_STRATEGY.md` +- Modify `docs/METHODOLOGY.md` +- Modify `docs/PROVENANCE_CONTRACT.md` +- Modify `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +- [ ] Record the independent Revenue/EPS scope states and explicit commercial pre-write guard. +- [ ] Preserve the research-only append path and all actuals, Q4, split-basis, calibration, synthetic-fixture, and candidate-context boundaries. +- [ ] Record that the checked-in registry still has no approved prospective-consensus source or scope. +- [ ] Record that this local guard is not provider, hosted, reviewer, data-depth, calibration, or market evidence. +- [ ] Update the continuation prompt with the new commit anchor after verified implementation. + +## Task 5: Full verification and delivery + +- [ ] Run focused collector/source-rights tests. +- [ ] Run `python3 -m pytest tests -q`. +- [ ] Run dashboard smoke and all six Research-route render smokes. +- [ ] Run public wording, public, commercial beta/release, pilot readiness, diff hygiene, and whitespace checks. +- [ ] Verify zero generated churn and no repository consensus write or readiness rebuild. +- [ ] Stage exact intentional files only and pass staged hygiene. +- [ ] Commit and push only `codex/personal-research-mode-mvp`. +- [ ] Update draft PR #113 and verify it remains open and draft. +- [ ] Reassess the next executable gate without claiming source, hosted, reviewer, calibration, operating, or market completion. diff --git a/docs/superpowers/plans/2026-07-19-consensus-source-temporal-integrity.md b/docs/superpowers/plans/2026-07-19-consensus-source-temporal-integrity.md new file mode 100644 index 000000000..f2851f706 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-consensus-source-temporal-integrity.md @@ -0,0 +1,216 @@ +# Consensus Source Temporal Integrity Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Require an explicit review cutoff, valid history scope, and ordered source timestamps before consensus rows can become candidate or historical-reviewable evidence. + +**Architecture:** Parse one mandatory cutoff before row processing, keep one ordered row-validation path, and add the normalized cutoff to the immutable result. Temporal failures remain technical rejections; exact-source rights and Revenue/EPS scope are calculated only for technically accepted rows and remain independent. + +**Tech Stack:** Python 3.12, frozen dataclasses, existing UTC timestamp parser, pytest, Make product gates, Git/GitHub draft PR workflow. + +## Global Constraints + +- Research-only; no investment advice, broker integration, order routing, auto-trading, direct buy/sell instructions, or post-earnings price prediction. +- Every accepted source row must prove `snapshot_at <= retrieved_at <= review_cutoff`. +- `history_scope` must be exactly `current_only` or `point_in_time`; unknown values cannot become candidate context. +- Keep technical acceptance, source rights, Revenue scope, EPS scope, historical availability, collection, nowcast readiness, backtesting, and calibration independent. +- Candidate context cannot modify deterministic forecasts or become trusted evidence. +- Do not run `make readiness` or create/stage CSV, JSON, report, sample-report, screenshot, browser timing, or bytecode churn. +- Stage exact intentional product/code/docs/test files only; never use `git add -A`. + +--- + +### Task 1: Mandatory source-row temporal contract + +**Files:** +- Modify: `tests/test_earnings_consensus_sources.py` +- Modify: `src/earnings_consensus_sources.py` + +**Interfaces:** +- Consumes: `parse_utc_timestamp(value, *, label) -> datetime` and the existing registry-derived commercial-review contract. +- Produces: `validate_source_rows(provider, rows, *, as_of, rights_registry=None) -> SourceValidationResult` with normalized `review_cutoff`. + +- [ ] **Step 1: Update valid test calls to use one explicit cutoff** + +Add: + +```python +REVIEW_CUTOFF = "2026-07-18T06:00:00Z" +``` + +Pass `as_of=REVIEW_CUTOFF` to every valid or row-focused validator call. Keep the caller-rights-label signature test focused on `rights_status` by also supplying `as_of`. + +- [ ] **Step 2: Write failing cutoff contract tests** + +Assert that a valid result exposes `review_cutoff == REVIEW_CUTOFF`, and invalid cutoff input raises before reviewing rows: + +```python +with pytest.raises(ValueError, match="review cutoff"): + validate_source_rows("reviewed_csv", (), as_of="not-a-cutoff") +``` + +- [ ] **Step 3: Write failing explicit-scope tests** + +Parametrize missing and unknown values and assert rejection, zero accepted rows, `still_blocked`, and no commercial-review rows: + +```python +@pytest.mark.parametrize("scope", ["", "historical", "latest"]) +def test_source_rows_require_an_explicit_supported_history_scope(scope): + result = validate_source_rows( + "licensed_consensus", + [_current_row(history_scope=scope)], + as_of=REVIEW_CUTOFF, + rights_registry=_rights_registry(), + ) + assert "history_scope must be current_only or point_in_time" in result.rejected_rows[0]["reason"] + assert result.commercial_review_rows == () +``` + +- [ ] **Step 4: Write failing ordering and cutoff tests** + +Add separate tests for snapshot after retrieval, snapshot after cutoff, and retrieval after cutoff. Assert the field-specific technical reason and that approved source rights remain source-level evidence while no row becomes commercially ready. + +```python +result = validate_source_rows( + "licensed_consensus", + [_current_row(snapshot_at="2026-07-18T05:00:02Z")], + as_of=REVIEW_CUTOFF, + rights_registry=_rights_registry(), +) +assert "snapshot_at cannot be after retrieved_at" in result.rejected_rows[0]["reason"] +assert result.commercial_rights_approved is True +assert result.commercial_ready_count == 0 +``` + +- [ ] **Step 5: Write failing equality and mixed-batch tests** + +Prove equality at retrieval/cutoff is allowed. In a two-row batch with the first row after cutoff and the second valid, assert one rejection, one accepted candidate, and `commercial_review_rows[0].row_number == 2`. + +- [ ] **Step 6: Run the focused tests and verify RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_earnings_consensus_sources.py -q +``` + +Expected: failures because `as_of` is not accepted, `review_cutoff` is absent, invalid scopes are still candidates, and reversed/future timestamps are accepted. + +- [ ] **Step 7: Add the normalized cutoff to the result** + +Add `review_cutoff: str` to `SourceValidationResult`. Change the signature to require `as_of`, parse it once, and normalize UTC: + +```python +cutoff = parse_utc_timestamp(as_of, label="review cutoff") +review_cutoff = cutoff.isoformat().replace("+00:00", "Z") +``` + +Return `review_cutoff` for every result, including a result whose rows are all rejected. + +- [ ] **Step 8: Implement explicit scope and row-time validation** + +Before `ConsensusSnapshot` construction, reject an unsupported scope. Parse the two row timestamps into a dictionary only when present; after successful parsing enforce ordering and cutoff: + +```python +if scope not in {"current_only", "point_in_time"}: + reasons.append("history_scope must be current_only or point_in_time") + +parsed_timestamps = {} +for timestamp in ("snapshot_at", "retrieved_at"): + if str(row.get(timestamp) or "").strip(): + try: + parsed_timestamps[timestamp] = parse_utc_timestamp(row[timestamp], label=timestamp) + except ValueError as exc: + reasons.append(str(exc)) + +snapshot_at = parsed_timestamps.get("snapshot_at") +retrieved_at = parsed_timestamps.get("retrieved_at") +if snapshot_at is not None and retrieved_at is not None and snapshot_at > retrieved_at: + reasons.append("snapshot_at cannot be after retrieved_at") +for field, timestamp in parsed_timestamps.items(): + if timestamp > cutoff: + reasons.append(f"{field} is after review cutoff") +``` + +Do not add a commercial review for any row with temporal or other technical reasons. + +- [ ] **Step 9: Run focused tests and verify GREEN** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_earnings_consensus_sources.py tests/test_earnings_consensus_collector.py -q +``` + +Expected: all source-validator and collector tests pass. + +- [ ] **Step 10: Review the production diff** + +Confirm there is no provider call, file writer, source-rights edit, ledger mutation, readiness mutation, or automatic apply path. Run `git diff --check`. + +### Task 2: Evidence-governance documentation and release verification + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/EARNINGS_NOWCAST_PILOT.md` +- Modify: `docs/DATA_STRATEGY.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: the verified mandatory-cutoff `SourceValidationResult` contract from Task 1. +- Produces: truthful roadmap, method, provenance, pilot, continuation, and draft-PR evidence for the temporal boundary. + +- [ ] **Step 1: Add the implemented roadmap item and maturity boundary** + +Add item 32 stating that source validation requires explicit scope and `snapshot_at <= retrieved_at <= review_cutoff`. State that this improves leakage resistance and review reliability but does not supply a provider, real snapshot, source rights, history depth, readiness, calibration, reviewers, or market validation. + +- [ ] **Step 2: Update pilot, strategy, method, and provenance contracts** + +Document that both candidate and historical source rows require the same cutoff truth; invalid scopes and reversed/future timestamps are technical rejections; only accepted rows enter commercial review. Make clear that cutoff passage does not prove publication availability, payload correctness, rights, freshness, collection, activation, backtesting, or calibration. + +- [ ] **Step 3: Update the continuation contract** + +Add the committed design/plan lineage anchor, implemented capability, truthful boundary, and Stage 2 review instruction. Preserve the stale-readiness no-write instruction and all external dependency classifications. + +- [ ] **Step 4: Run focused and full verification** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_earnings_consensus_sources.py tests/test_earnings_consensus_collector.py -q +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +``` + +Expected: zero failures; the existing third-party `dateutil` deprecation warning may remain. + +- [ ] **Step 5: Run every required non-writing product gate** + +Run: + +```bash +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: executable gates pass; pilot readiness still reports stale saved readiness; no readiness build or generated artifact appears. + +- [ ] **Step 6: Stage exact files and verify hygiene** + +Stage only the source module, its test, the six named product documents, and this plan if corrected during execution. Run `make staged-hygiene-check` and `git diff --cached --check`. Expected: zero staged generated CSV/JSON/report/sample-report/screenshot/timing churn. + +- [ ] **Step 7: Commit, push, and update draft PR #113** + +Commit with message `Enforce consensus source review cutoff`, push only `codex/personal-research-mode-mvp`, confirm 0/0 remote alignment, and post verified scope, red-green evidence, full gates, truthful boundaries, generated-artifact exclusion, unchanged dependencies, and the next executable step to PR #113. Keep it open and draft. + +- [ ] **Step 8: Re-audit the handoff state** + +Verify clean status, pushed HEAD, draft PR state, artifact hygiene, stale readiness, review safety, remaining stage gates, and that the overall goal remains active. diff --git a/docs/superpowers/plans/2026-07-19-consensus-source-validation-rights.md b/docs/superpowers/plans/2026-07-19-consensus-source-validation-rights.md new file mode 100644 index 000000000..e2b7121b9 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-consensus-source-validation-rights.md @@ -0,0 +1,219 @@ +# Consensus Source Validation Rights Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace caller-declared consensus source rights with exact registry-derived commercial rights and independent Revenue/EPS scope evidence while preserving technically valid research review. + +**Architecture:** Keep schema/comparability validation in `earnings_consensus_sources.py`, inject or load the existing immutable source-rights registry, and add immutable row-level commercial reviews plus aggregate counts to the result. Commercial evidence never changes technical acceptance, candidate-context classification, or the non-writing boundary. + +**Tech Stack:** Python 3.12, frozen dataclasses, PyYAML-backed `commercial_source_rights`, pytest, Make product gates, Git/GitHub draft PR workflow. + +## Global Constraints + +- Research-only; no investment advice, broker integration, order routing, auto-trading, direct buy/sell instructions, or post-earnings price prediction. +- Do not add, infer, or approve a provider, entitlement, source-rights record, consensus value, source, timestamp, or recommendation. +- Candidate context cannot modify deterministic forecasts or become trusted evidence. +- Keep technical acceptance, historical availability, exact-source rights, Revenue scope, EPS scope, nowcast readiness, backtesting, and calibration independent. +- Historical rows are reviewable evidence only; they are not activated evidence or nowcast readiness. +- Do not run `make readiness` or create/stage CSV, JSON, report, sample-report, screenshot, browser timing, or bytecode churn. +- Stage exact intentional product/code/docs/test files only; never use `git add -A`. + +--- + +### Task 1: Registry-derived validator contract + +**Files:** +- Modify: `tests/test_earnings_consensus_sources.py` +- Modify: `src/earnings_consensus_sources.py` + +**Interfaces:** +- Consumes: `commercial_eligibility(registry: Mapping[str, SourceRights], source_id: str) -> CommercialEligibility` and `load_source_rights_registry() -> Mapping[str, SourceRights]`. +- Produces: `SourceCommercialReview` and `validate_source_rows(provider, rows, *, rights_registry=None) -> SourceValidationResult` with registry-derived aggregate evidence. + +- [ ] **Step 1: Add test-only exact-source rights fixtures** + +Add a `_rights_registry` helper using `build_source_rights_registry` with explicit `source_id`, `commercial_use`, and `supported_fields`. Add `_current_row` and `_historical_row` helpers that populate real validator fields but use only synthetic values and references. + +```python +def _rights_registry(*, source_id="licensed_consensus", commercial_use="approved", supported_fields=None): + return build_source_rights_registry( + [{ + "source_id": source_id, + "display_name": "Licensed consensus fixture", + "permitted_use": "test_only", + "commercial_use": commercial_use, + "redistribution": "test_only", + "storage_limits": "temporary in-memory tests only", + "attribution": "fixture", + "rate_limits": "not applicable", + "authentication": "not applicable", + "expected_freshness": "fixture cutoff", + "supported_fields": supported_fields or ["revenue_consensus", "eps_consensus"], + "fallback_priority": 1, + }] + ) +``` + +- [ ] **Step 2: Write failing tests for default fail-closed evidence and removed caller approval** + +Update existing calls to omit `rights_status`. Assert that a technically valid Alpha Vantage current-only row remains accepted candidate context while its exact source is unknown, both populated scopes are missing, `commercial_ready_count == 0`, and `auto_apply is False`. Add a test proving `rights_status=` is no longer accepted by the function signature. + +```python +assert result.state == "candidate_context_only" +assert result.rights_status == "unknown_source" +assert result.commercial_review_rows[0].missing_supported_fields == ("revenue_consensus", "eps_consensus") +assert result.commercial_evidence_ready is False +with pytest.raises(TypeError, match="rights_status"): + validate_source_rows("alpha_vantage", (), rights_status="approved_for_project_use") +``` + +- [ ] **Step 3: Write failing tests for historical state and metric-specific scope** + +Assert that a technically valid point-in-time row is `historical_evidence_reviewable`. Cover approved Revenue-only, EPS-only, and mixed rows; a source approved only for Revenue must report EPS as the sole missing scope for a mixed row. Assert commercial counts and ordered row numbers. + +```python +assert historical.state == "historical_evidence_reviewable" +assert revenue_only.commercial_review_rows[0].required_supported_fields == ("revenue_consensus",) +assert eps_only.commercial_review_rows[0].required_supported_fields == ("eps_consensus",) +assert mixed.commercial_review_rows[0].missing_supported_fields == ("eps_consensus",) +assert mixed.commercial_rights_approved is True +assert mixed.commercial_review_required_count == 1 +``` + +- [ ] **Step 4: Write a failing exact composite-source test** + +Inject a registry that approves `licensed_consensus` and validate a row whose provider is `licensed_consensus + reviewed_csv`. Assert `unknown_source`, missing populated scopes, and no inferred identity. + +- [ ] **Step 5: Run the focused tests and verify RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_earnings_consensus_sources.py -q +``` + +Expected: failures because the old function still requires `rights_status`, lacks registry-derived fields, and returns `historical_evidence_ready`. + +- [ ] **Step 6: Add immutable commercial-review result types** + +Import `SourceRights`, `commercial_eligibility`, and `load_source_rights_registry`. Add: + +```python +@dataclass(frozen=True) +class SourceCommercialReview: + row_number: int + required_supported_fields: tuple[str, ...] + missing_supported_fields: tuple[str, ...] + commercial_evidence_ready: bool + commercial_blockers: tuple[str, ...] +``` + +Extend `SourceValidationResult` with `commercial_rights_approved`, `commercial_ready_count`, `commercial_review_required_count`, `commercial_evidence_ready`, `commercial_blockers`, and `commercial_review_rows`. Keep `rights_status` registry-derived and `auto_apply=False`. + +- [ ] **Step 7: Implement independent exact-source commercial evidence** + +Change the function signature to optional injected rights registry, normalize the provider once, and calculate the source decision once. For every technically accepted row, derive populated metric scopes in stable order, exact-record missing scopes, and stable blockers. + +```python +source_id = str(provider).strip().lower() +registry = load_source_rights_registry() if rights_registry is None else rights_registry +rights = commercial_eligibility(registry, source_id) +rights_record = registry.get(source_id) +supported_fields = set(rights_record.supported_fields) if rights_record else set() +required_supported_fields = tuple( + field for field in ("revenue_consensus", "eps_consensus") + if str(row.get(field) or "").strip() +) +missing_supported_fields = tuple( + field for field in required_supported_fields if field not in supported_fields +) +``` + +Technical reasons must no longer include source rights. Count only technically accepted rows in the row-level commercial evidence. Aggregate readiness is true only when at least one row is accepted and every accepted row is commercially ready. Return `historical_evidence_reviewable` when any accepted point-in-time row exists. + +- [ ] **Step 8: Run the focused tests and verify GREEN** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_earnings_consensus_sources.py tests/test_earnings_consensus_collector.py -q +``` + +Expected: all source-validator and collector contract tests pass. + +- [ ] **Step 9: Review the production diff** + +Confirm there is no provider call, file writer, source-rights config change, readiness mutation, automatic apply path, or generated artifact change. Run `git diff --check`. + +### Task 2: Product contract documentation and release verification + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/EARNINGS_NOWCAST_PILOT.md` +- Modify: `docs/DATA_STRATEGY.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: the verified `SourceValidationResult` evidence contract from Task 1. +- Produces: truthful product-stage, boundary, continuation-anchor, and next-step documentation for draft PR #113. + +- [ ] **Step 1: Document the completed local slice without activation claims** + +Add ROADMAP implemented item 31 and maturity commentary. State that upstream source rows now use exact registry-derived rights and row-specific Revenue/EPS scope while technical acceptance and candidate context remain independent. Explicitly state that the checked-in registry still unlocks no prospective consensus source, no real snapshot was added, and nowcast/calibration readiness did not change. + +- [ ] **Step 2: Update method, provenance, strategy, and pilot guidance** + +Document these exact boundaries: + +- caller labels cannot grant rights; +- invalid technical rows do not enter commercial-ready counts; +- `historical_evidence_reviewable` is not activation; +- composite IDs remain unknown exact sources; +- commercial evidence readiness is not payload approval, collection, history depth, nowcast readiness, backtesting, or calibration. + +- [ ] **Step 3: Update the continuation contract** + +Add the committed design/plan lineage anchor, add the upstream validation capability and its truthful boundary, retain the stale-readiness no-write instruction, and set the exact next executable local lane from the re-audited roadmap. Do not change external dependency classifications without new evidence. + +- [ ] **Step 4: Run focused and full verification** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_earnings_consensus_sources.py tests/test_earnings_consensus_collector.py -q +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +``` + +Expected: zero test failures; the existing third-party `dateutil` deprecation warning may remain. + +- [ ] **Step 5: Run every required non-writing product gate** + +Run: + +```bash +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all executable gates pass; pilot readiness truthfully reports the known stale saved readiness rather than regenerating it; no generated artifact changes appear. + +- [ ] **Step 6: Stage exact files and run staged hygiene** + +Stage only the source module, its test, and the six named product documents. The design and plan have separate reviewed commits. Run `make staged-hygiene-check` and `git diff --cached --check`. Expected: no staged generated CSV/JSON/report/sample-report/screenshot/timing churn. + +- [ ] **Step 7: Commit, push, and update draft PR #113** + +Commit the coherent implementation/docs slice with message `Derive consensus source validation rights`, push only `codex/personal-research-mode-mvp`, confirm 0/0 remote alignment, and post the verified scope, tests, truthful boundaries, generated-artifact exclusion, external dependency status, and next executable step to PR #113. Keep the PR open and draft. + +- [ ] **Step 8: Re-audit the handoff state** + +Verify clean status, pushed HEAD, draft PR state, generated-artifact hygiene, current external dependency ledger, review safety, and that the overall `/goal` remains active because hosted, reviewer, real-source, evidence-depth, backtesting, calibration, and operating gates remain incomplete or unproven. diff --git a/docs/superpowers/plans/2026-07-19-dcf-price-lineage-review.md b/docs/superpowers/plans/2026-07-19-dcf-price-lineage-review.md new file mode 100644 index 000000000..d70e0f859 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-dcf-price-lineage-review.md @@ -0,0 +1,118 @@ +# DCF Price Lineage Review Implementation Plan + +> **For Codex:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Extend the existing stdout-only readiness preview with a fail-closed review of the exact latest canonical price row supporting each proposed DCF promotion. + +**Architecture:** Add a pure `src.dcf_price_lineage` module that selects one latest usable row per proposed DCF promotion and independently evaluates selection, lineage, exact-source rights, and registered `prices` support. Integrate its frozen review result into `src.readiness_preview` after the existing in-memory readiness build. Keep the Make target, production readiness, price provider, canonical schema, valuation math, and no-write boundary unchanged. + +**Tech Stack:** Python 3, frozen dataclasses, pandas, PyYAML-backed immutable source-rights registry, pytest, Make. + +**Constraints:** Do not run `make readiness`; do not mutate or migrate canonical price data; do not write or stage CSV, JSON, reports, sample reports, screenshots, timing files, caches, or readiness artifacts; do not infer a provider from file names, adapters, dates, values, or history; do not split composite source identifiers; preserve technical DCF readiness independently from price selection, lineage, rights, registered field scope, reviewer approval, and rebuild authorization. + +## Task 1: Pure DCF price-lineage contract + +**Files:** + +- Create `tests/test_dcf_price_lineage.py` +- Create `src/dcf_price_lineage.py` + +- [ ] Add a test where `AAA` moves `dcf_ready=False -> True`, `BBB` remains ready, and `CCC` only promotes fundamentals; assert only `AAA` enters the DCF price review. +- [ ] Define test price rows where `AAA` has two valid dates and complete latest-row evidence: `source=approved_prices`, `source_ref=https://example.test/prices/AAA/2026-01-03`, and `retrieved_at=2026-01-03T23:00:00Z`. +- [ ] Define an exact rights record with `commercial_use=approved` and `supported_fields=("prices",)`; require `status="price_lineage_review_complete"`. +- [ ] Run `python3 -m pytest tests/test_dcf_price_lineage.py -q` and confirm collection fails because `src.dcf_price_lineage` does not exist. +- [ ] Add frozen `DcfPriceLineageEvidence` and `DcfPriceLineageReview` dataclasses with explicit aggregate counts, capped evidence rows, and `top_n`. +- [ ] Implement `review_dcf_price_lineage(saved, proposed, prices, *, rights_registry, top_n=20)` as a pure function. +- [ ] Normalize ticker text without mutating inputs; identify only false-to-true `dcf_ready` changes. +- [ ] Select valid rows using parsed `date`, numeric positive `close`, maximum date, and exactly one row at that date. +- [ ] Evaluate exact `source`, `source_ref`, and `retrieved_at`; call `commercial_eligibility` with the complete source string; require explicit registered `prices` support. +- [ ] Return `no_dcf_promotions`, `price_lineage_review_complete`, or `price_lineage_review_required` without changing either readiness frame. +- [ ] Run `python3 -m pytest tests/test_dcf_price_lineage.py -q` and require the initial success case to pass. + +## Task 2: Fail-closed edge cases and independent counts + +**Files:** + +- Modify `tests/test_dcf_price_lineage.py` +- Modify `src/dcf_price_lineage.py` + +- [ ] Add failing tests for a missing price row, invalid date, nonpositive close, and duplicate latest-date rows. +- [ ] Add failing tests for independently missing `source`, `source_ref`, and `retrieved_at`. +- [ ] Add cases for an unknown exact source, a composite exact source, unverified commercial rights, and an approved source whose registry omits `prices`. +- [ ] Assert a missing/ambiguous latest row yields rights `not_evaluated_missing_evidence` or `not_evaluated_ambiguous_evidence` rather than evaluating guessed data. +- [ ] Assert `TOP_N=1` caps evidence rows while all aggregate counts retain the full promotion population. +- [ ] Assert `top_n < 1` raises `ValueError`. +- [ ] Implement explicit blockers: `missing_latest_price_row`, `ambiguous_latest_price_row`, `missing_provenance:`, `commercial_rights:`, and `registered_price_scope_incomplete`. +- [ ] Count usable latest rows, ambiguous rows, lineage completeness, approved rights, registered price scope, exact sources, and rights statuses independently. +- [ ] Run `python3 -m pytest tests/test_dcf_price_lineage.py -q` and require all pure-contract tests to pass. + +## Task 3: Integrate the audit into the one no-write preview + +**Files:** + +- Modify `tests/test_readiness_preview.py` +- Modify `src/readiness_preview.py` +- Modify `Makefile` only if its help text needs clarification + +- [ ] Extend the integrated preview fixture with a proposed DCF promotion and a canonical `data/prices.csv` row carrying complete test provenance. +- [ ] Assert `build_ticker_readiness_report(..., write_outputs=False)` remains the only readiness build. +- [ ] Assert the attached price review is independent from `promotion_review` and contains the expected complete result. +- [ ] Add a missing-price-file integration case that produces `price_lineage_review_required` without writing a file. +- [ ] Require rendered output under `DCF Price Lineage Review` to show technical promotions, latest-row selection, lineage, rights, registered price scope, exact sources, rights statuses, and capped evidence rows. +- [ ] Require wording that file origin, observation date, and adapter availability are not provider provenance; the audit changes no readiness state and authorizes no rebuild. +- [ ] Run `python3 -m pytest tests/test_readiness_preview.py tests/test_dcf_price_lineage.py -q` and confirm the integration assertions fail before implementation. +- [ ] Add `dcf_price_lineage_review: DcfPriceLineageReview | None` to `ReadinessImpactPreview`. +- [ ] Load `data/prices.csv` read-only after a saved snapshot exists, normalize columns, and call `review_dcf_price_lineage(...)` with the existing saved/proposed frames and rights registry. +- [ ] Render the independent review after `Promotion Evidence Review`; retain all existing final no-write and research-only boundaries. +- [ ] Run the focused tests and require them to pass. + +## Task 4: Prove no-write behavior on current repository data + +**Files:** + +- Modify `tests/test_readiness_preview.py` only if the integration manifest needs additional coverage + +- [ ] Capture a byte-level manifest for tracked and untracked repository files before the command. +- [ ] Run `PYTHONDONTWRITEBYTECODE=1 make readiness-preview TOP_N=20`. +- [ ] Capture the same manifest afterward and assert it is identical. +- [ ] Record the live aggregate result without calling it current readiness or rebuild approval. +- [ ] Confirm no source provider is inferred for rows whose canonical evidence is absent. +- [ ] Run `git status --short` and verify no CSV, JSON, report, sample-report, screenshot, timing, cache, or bytecode artifact changed. + +## Task 5: Durable methodology and continuation contract + +**Files:** + +- Modify `ROADMAP.md` +- Modify `docs/METHODOLOGY.md` +- Modify `docs/PROVENANCE_CONTRACT.md` +- Modify `docs/DATA_STRATEGY.md` +- Modify `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify `tests/test_public_v1_release_docs.py` if contract assertions need expansion + +- [ ] Record the exact latest-price selection rule and required row-level `source`, `source_ref`, and `retrieved_at` contract. +- [ ] Record that `as_of_date`, observation date, local file label, file modification time, adapter presence, and refresh warning text do not substitute for row-level provenance. +- [ ] Record independent technical DCF, price selection, lineage, rights, registered field scope, freshness, reviewer, and rebuild states. +- [ ] Add the implementation lineage anchor and current live no-write result to the continuation prompt without turning proposed counts into current product claims. +- [ ] Set the next executable local lane based on the new evidence; do not authorize canonical migration, source-rights edits, or `make readiness` without reviewed proof and explicit write approval. +- [ ] Run `python3 -m pytest tests/test_dcf_price_lineage.py tests/test_readiness_preview.py tests/test_public_v1_release_docs.py -q`. +- [ ] Run `git diff --check` and `make diff-hygiene-summary`. + +## Task 6: Full verification, exact staging, commit, push, and PR evidence + +- [ ] Run `python3 -m pytest tests -q`. +- [ ] Run `make dashboard-smoke`. +- [ ] Run the repository's six Personal Research render smoke targets. +- [ ] Run `make public-wording-check`. +- [ ] Run `make public-check`. +- [ ] Run `make commercial-beta-check`. +- [ ] Run `make commercial-beta-release-check`. +- [ ] Run `make pilot-readiness-check TOP_N=10` and preserve the truthful stale-readiness blocker. +- [ ] Run `make diff-hygiene-summary` and `git diff --check`. +- [ ] Review `git status --short`, `git diff --stat`, and exact changed paths. +- [ ] Stage only the intentional source, test, documentation, and Make paths; never use `git add -A`. +- [ ] Run `make staged-hygiene-check` and `git diff --cached --check`. +- [ ] Commit one coherent implementation/documentation slice. +- [ ] Push only `codex/personal-research-mode-mvp` and verify 0/0 alignment. +- [ ] Add a concise evidence comment to draft PR #113 with live counts, checks, boundaries, external-dependency classification, and exact next step. +- [ ] Verify PR #113 remains open and draft; do not merge or deploy. diff --git a/docs/superpowers/plans/2026-07-19-eps-split-basis-enforcement.md b/docs/superpowers/plans/2026-07-19-eps-split-basis-enforcement.md new file mode 100644 index 000000000..30e245366 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-eps-split-basis-enforcement.md @@ -0,0 +1,14 @@ +# EPS Split-Basis Enforcement Implementation Plan + +1. Add failing quarterly-trend, focused-cohort, and backtest tests for the + Companyfacts unverified split-basis sentinel. +2. Centralize the sentinel constant in the earnings evidence contract. +3. Filter sentinel EPS observations from Business Trend values/comparisons and + retain an explicit partial or blocked reason without affecting Revenue. +4. Withhold sentinel target/prior-year EPS outcomes from backtest metrics and + exclude events with no comparable target metric. +5. Update methodology, provenance, roadmap, continuation contract, and relevant + UI/coverage contracts. +6. Run focused/full tests and every required product, release, pilot, and hygiene + check; stage exact files, commit, push, update draft PR #113, and verify hosted + CI on the exact pushed head. diff --git a/docs/superpowers/plans/2026-07-19-pr-engineering-legitimacy.md b/docs/superpowers/plans/2026-07-19-pr-engineering-legitimacy.md new file mode 100644 index 000000000..4ebc13a17 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-pr-engineering-legitimacy.md @@ -0,0 +1,76 @@ +# PR Engineering Legitimacy Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add one minimal pull-request-only GitHub Actions gate that reproduces the approved local engineering checks for PR #113 without data writes or external product claims. + +**Architecture:** A single least-privilege workflow checks out the PR revision, installs the existing Python requirements, and runs six explicit read-only commands. A focused repository contract test prevents future expansion into schedules, secrets, readiness generation, deployment, or artifact upload. + +**Tech Stack:** GitHub Actions YAML, Python 3.12, pytest, GNU Make, Markdown. + +## Global Constraints + +- Trigger only on pull requests targeting `main`. +- Grant only `contents: read` permission. +- Do not use providers, secrets, schedules, deployment, hosted probes, readiness generation, or artifact uploads. +- Do not generate or stage CSV, JSON, report, sample-report, screenshot, timing, readiness, or canonical-data churn. +- Never use `git add -A`; stage exact intentional paths only. +- Push only to `codex/personal-research-mode-mvp`; keep PR #113 open and draft. + +--- + +### Task 1: Add the minimal PR engineering gate + +**Files:** +- Create: `.github/workflows/commercial-research-beta.yml` +- Create: `tests/test_github_actions_workflow.py` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: the existing `pyproject.toml` package contract and Make targets `dashboard-smoke`, `research-dashboard-render-smoke`, `public-wording-check`, and `diff-hygiene-summary`. +- Produces: one GitHub pull-request status check named `Commercial Research Beta / local-engineering-gate`. + +- [ ] **Step 1: Write the failing workflow contract test** + +Create `tests/test_github_actions_workflow.py` with assertions that the workflow exists, targets pull requests to `main`, uses `contents: read`, selects Python 3.12, installs the checked-in package plus `pytest`, contains every required command, and contains none of the prohibited trigger or action strings. + +- [ ] **Step 2: Run the focused test and verify the intended red state** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_github_actions_workflow.py -q +``` + +Expected: fail because `.github/workflows/commercial-research-beta.yml` does not exist. + +- [ ] **Step 3: Add the minimal workflow** + +Create `.github/workflows/commercial-research-beta.yml` with `pull_request` as its only trigger, `contents: read`, one Ubuntu job, Python 3.12, editable installation of the checked-in package plus `pytest`, and the six approved commands in order. Do not add caching, uploads, provider access, readiness, deployment, or schedules. + +- [ ] **Step 4: Run the focused test and verify green** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_github_actions_workflow.py -q +``` + +Expected: `1 passed`. + +- [ ] **Step 5: Update stage documentation** + +Record the locally implemented CI contract in `ROADMAP.md` and the continuation prompt. State that independent hosted CI evidence is pending until GitHub completes the workflow and that human review remains separately unproven. + +- [ ] **Step 6: Run the complete required verification** + +Run the focused test, full pytest suite, dashboard and research render smokes, public wording and public checks, commercial beta and release checks, pilot readiness, diff hygiene, and whitespace checks. The pilot may truthfully remain blocked on stale readiness. + +- [ ] **Step 7: Stage exact files and verify the package** + +Stage only the workflow, focused test, design, plan, roadmap, and continuation contract. Run `make staged-hygiene-check` and `git diff --cached --check`. + +- [ ] **Step 8: Commit, push, and update the draft PR** + +Commit the coherent slice, push only the current feature branch, update PR #113 with the new scope and verification, and verify it remains open and draft. Do not claim the hosted check passed until GitHub directly reports success. diff --git a/docs/superpowers/plans/2026-07-19-prospective-price-lineage-preservation.md b/docs/superpowers/plans/2026-07-19-prospective-price-lineage-preservation.md new file mode 100644 index 000000000..6ea83762f --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-prospective-price-lineage-preservation.md @@ -0,0 +1,108 @@ +# Prospective Price Lineage Preservation Implementation Plan + +> **For Codex:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Let one prospectively reviewed price row preserve an explicit durable source reference and retrieval timestamp through the existing normalize, validate, preview, and apply contracts without coupling technical OHLCV validity to commercial evidence. + +**Architecture:** Widen the staged/canonical optional price metadata contract in `src.price_import_normalizer` and `src.data_update`. Normalization accepts explicit metadata only and never invents it. Validation normalizes retrieval timestamps and attaches an independent lineage summary to the existing technical result. Preview and later reviewed apply inherit the retained columns through the existing merge path. + +**Tech Stack:** Python 3, pandas, argparse, pytest, Make. + +**Constraints:** Do not run repository `price-normalize`, `price-apply`, `make readiness`, or any source refresh; use only temporary test fixtures; do not create or stage CSV, JSON, reports, sample reports, screenshots, timings, caches, or canonical data; do not generate defaults for `source_ref` or `retrieved_at`; do not make lineage completeness equivalent to rights approval, registered `prices` scope, reviewer approval, freshness, readiness, or apply authorization. + +## Task 1: Normalizer preserves explicit lineage metadata + +**Files:** + +- Modify `tests/test_price_import_normalizer.py` +- Modify `src/price_import_normalizer.py` + +- [ ] Add a failing test that passes `source_ref="https://example.test/prices/NVDA/2026-01-02"` and `retrieved_at="2026-01-03T23:00:00Z"` into `normalize_price_imports(...)` and asserts both exact fields exist in the staged temporary CSV. +- [ ] Add a test that omits both arguments and asserts both columns are present but blank; normalization time, file time, observation date, and `as_of_date` must not appear in either field. +- [ ] Run `python3 -m pytest tests/test_price_import_normalizer.py -q` and confirm the new signature/columns fail before implementation. +- [ ] Add `source_ref` and `retrieved_at` to `STAGED_PRICE_COLUMNS` after `source`. +- [ ] Pass explicit values through `_normalize_one_file(...)` and `normalize_price_imports(...)`; default both to `None`/blank. +- [ ] Add CLI `--source-ref` and `--retrieved-at` arguments and pass them unchanged into normalization. +- [ ] Preserve existing date/ticker/duplicate/invalid-row behavior and existing output ordering. +- [ ] Run the focused normalizer tests and require them to pass. + +## Task 2: Validation reports independent lineage completeness + +**Files:** + +- Modify `tests/test_data_update.py` +- Modify `src/data_update.py` + +- [ ] Extend the temporary import fixture with `source_ref` and `retrieved_at` values for valid rows. +- [ ] Add assertions that `validate_price_imports(...)` retains both fields in `valid_frame` and returns `lineage_status="lineage_complete"`, complete row count 2, review-required row count 0, and no missing lineage fields. +- [ ] Add a separate technically valid fixture with missing source reference and invalid retrieval timestamp; require technical status to remain valid or valid-with-warnings while lineage is `lineage_review_required`. +- [ ] Run the focused validation tests and confirm the summary assertions fail before implementation. +- [ ] Add `source_ref` and `retrieved_at` to optional/output price import columns. +- [ ] Normalize nonblank retrieval timestamps to UTC ISO-8601 and blank invalid values without inventing current time. +- [ ] Compute lineage completeness over retained valid rows using exact nonblank `source`, nonblank `source_ref`, and parseable `retrieved_at`. +- [ ] Return `lineage_status`, `lineage_complete_rows`, `lineage_review_required_rows`, and sorted `lineage_missing_fields` for valid, invalid, and missing-file paths. +- [ ] Add a concise warning when otherwise valid rows require lineage review; do not change the technical row count or reject the row. +- [ ] Run focused data-update tests and require them to pass. + +## Task 3: Preview and temporary apply preserve lineage + +**Files:** + +- Modify `tests/test_data_update.py` +- Modify `src/data_update.py` only if the existing merge logic needs correction + +- [ ] Assert `preview_price_import_merge(...)` returns the same lineage summary as validation and still reports existing new/updated/unchanged counts. +- [ ] In the existing temporary apply test, assert updated and new NVDA rows preserve exact `source_ref` and normalized `retrieved_at` values. +- [ ] Assert the unrelated MSFT canonical row remains present and its missing new fields remain empty. +- [ ] Run `python3 -m pytest tests/test_data_update.py -q` and confirm any preservation gap fails before correction. +- [ ] If needed, widen `PRICE_IMPORT_OUTPUT_COLUMNS` and retain optional columns through canonical reindex/merge; do not change merge keys or backup/no-delete behavior. +- [ ] Run focused normalizer/data-update tests and require them to pass. + +## Task 4: Safe CLI and Make routing + +**Files:** + +- Modify `Makefile` +- Modify `tests/test_launchers.py` +- Modify public or operator docs only where the reviewed command contract is described + +- [ ] Add failing launcher assertions that `price-normalize` conditionally passes quoted `--source-ref "$(SOURCE_REF)"` and `--retrieved-at "$(RETRIEVED_AT)"` only when supplied. +- [ ] Update Make help with one reviewed example containing `SOURCE`, `SOURCE_REF`, and `RETRIEVED_AT`; keep the default example valid for local research. +- [ ] Keep user-supplied values quoted and never echo credentials; the fields are evidence identifiers/timestamps, not secrets. +- [ ] Run focused launcher tests and `make public-wording-check`. + +## Task 5: Durable contracts and current next step + +**Files:** + +- Modify `ROADMAP.md` +- Modify `docs/METHODOLOGY.md` +- Modify `docs/PROVENANCE_CONTRACT.md` +- Modify `docs/DATA_STRATEGY.md` +- Modify `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify documentation contract tests as needed + +- [ ] Record that the staged path can now preserve prospective lineage but current canonical history remains unchanged and unproven. +- [ ] Record that missing lineage does not invalidate technical local-research rows, while complete lineage still does not establish rights, registered field scope, reviewer approval, or apply authorization. +- [ ] Add the design/plan lineage anchor and exact next external unblock condition to the continuation prompt. +- [ ] Keep the current 146-row DCF audit result unchanged; no new canonical row exists from this code slice. +- [ ] Run focused documentation/launcher/normalizer/data-update tests, `git diff --check`, and `make diff-hygiene-summary`. + +## Task 6: Full verification, exact staging, commit, push, and PR update + +- [ ] Run `python3 -m pytest tests -q`. +- [ ] Run `make dashboard-smoke`. +- [ ] Run `make research-dashboard-render-smoke`. +- [ ] Run `make public-wording-check`. +- [ ] Run `make public-check`. +- [ ] Run `make commercial-beta-check`. +- [ ] Run `make commercial-beta-release-check`. +- [ ] Run `make pilot-readiness-check TOP_N=10` and preserve the stale-readiness blocker. +- [ ] Run `make diff-hygiene-summary` and `git diff --check`. +- [ ] Verify no repository price-normalize/apply or generated-data command ran and no generated artifact changed. +- [ ] Stage only the exact source, test, Make, and documentation paths; never use `git add -A`. +- [ ] Run `make staged-hygiene-check` and `git diff --cached --check`. +- [ ] Commit one coherent implementation/documentation slice. +- [ ] Push only `codex/personal-research-mode-mvp` and verify 0/0 alignment. +- [ ] Update draft PR #113 with verified behavior, test counts, artifact boundary, remaining rights/source requirement, and next executable step. +- [ ] Verify PR #113 remains open and draft; do not merge or deploy. diff --git a/docs/superpowers/plans/2026-07-19-staged-price-rights-scope-review.md b/docs/superpowers/plans/2026-07-19-staged-price-rights-scope-review.md new file mode 100644 index 000000000..138837b81 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-staged-price-rights-scope-review.md @@ -0,0 +1,66 @@ +# Staged Price Rights And Scope Review Implementation Plan + +> **For Codex:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Add independent exact-source commercial-rights and registered `prices` scope review to staged price validation and preview without changing technical OHLCV validity or authorizing apply. + +**Architecture:** Reuse `SourceRights` and `commercial_eligibility` from the checked-in fail-closed registry. Evaluate only technically valid retained rows after normalization, aggregate exact-source decisions, and attach the result to existing validation/preview summaries. Allow explicit registry injection for deterministic tests; production uses the checked-in registry. + +**Constraints:** Do not edit `config/source_rights.yml`; do not infer providers or split composite source strings; do not run repository normalize/apply/readiness/source commands; do not create or stage generated artifacts; do not make rights/scope review change valid-row counts, technical status, lineage state, merge counts, readiness, or apply authorization. + +## Task 1: Specify fail-closed rights and scope states with tests + +**Files:** + +- Modify `tests/test_data_update.py` + +- [ ] Add an injected approved source with `prices` and require approved/complete row counts and exact-source review detail. +- [ ] Add or extend a `yfinance` fixture and require `commercial_rights_unverified` with registered price scope complete. +- [ ] Add unknown and blank source cases and require `unknown_source`, incomplete price scope, and no alias inference. +- [ ] Add a mixed batch and require deterministic mixed states/counts. +- [ ] Assert technically invalid rows do not enter rights/scope counts. +- [ ] Run focused tests and confirm the new summary assertions fail before implementation. + +## Task 2: Implement pure staged-source evidence review + +**Files:** + +- Modify `src/data_update.py` + +- [ ] Import the existing immutable source-rights types and evaluator. +- [ ] Add a pure helper that reviews technically valid rows by exact trimmed source ID. +- [ ] Report independent aggregate states, row counts, status counts, and deterministic distinct-source details. +- [ ] Keep missing/blank source fail-closed and require exact `prices` membership in `supported_fields`. +- [ ] Add concise review-required warnings without changing technical validity. +- [ ] Make `validate_price_imports(...)` accept optional registry injection and default to the checked-in registry. +- [ ] Pass the same registry through `preview_price_import_merge(...)`; preserve all merge results. +- [ ] Run focused data-update tests and require them to pass. + +## Task 3: Durable contracts + +**Files:** + +- Modify `ROADMAP.md` +- Modify `docs/METHODOLOGY.md` +- Modify `docs/PROVENANCE_CONTRACT.md` +- Modify `docs/DATA_STRATEGY.md` +- Modify `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify documentation contract tests only if required + +- [ ] Record the independent staged rights/scope review and its exact non-activation boundary. +- [ ] Preserve the current 146-row canonical price-lineage audit and external dependency classification. +- [ ] Add the design/plan anchor and next external unblock condition to the continuation prompt. +- [ ] Run focused documentation and public wording checks. + +## Task 4: Full verification and delivery + +- [ ] Run focused changed-module tests. +- [ ] Run `python3 -m pytest tests -q`. +- [ ] Run dashboard smoke and all six research render routes. +- [ ] Run public wording/check, commercial beta/check release, pilot readiness, hygiene, and whitespace checks. +- [ ] Verify readiness remains stale and no repository data command or generated artifact ran. +- [ ] Stage exact product/code/docs/test paths only; never use `git add -A`. +- [ ] Run staged hygiene and cached whitespace checks. +- [ ] Commit one coherent implementation slice and push only `codex/personal-research-mode-mvp`. +- [ ] Update draft PR #113 and verify it remains open and draft. +- [ ] Re-audit remaining safe local tasks; keep the overall goal active while any applicable gate remains unproven. diff --git a/docs/superpowers/plans/2026-07-20-canonical-quarterly-fail-closed.md b/docs/superpowers/plans/2026-07-20-canonical-quarterly-fail-closed.md new file mode 100644 index 000000000..a92b9e2d7 --- /dev/null +++ b/docs/superpowers/plans/2026-07-20-canonical-quarterly-fail-closed.md @@ -0,0 +1,14 @@ +# Canonical Quarterly Ledger Fail-Closed Plan + +1. Add failing loader-to-packet and dashboard/cohort regression tests for a CSV + containing one accepted and one rejected row. +2. Extend the quarterly trend packet with immutable canonical rejection + evidence and a load-result-aware fail-closed builder. +3. Route both dashboard canonical consumers through that boundary and suppress + accepted-subset cohort derivation when rejection evidence exists. +4. Show row-numbered rejection reasons only under Advanced quarterly evidence. +5. Update methodology, provenance, roadmap, continuation contract, and focused + public/dashboard contract tests. +6. Run focused tests, the full suite, all product/release/pilot gates, PR-range + and generated-artifact hygiene, exact staging checks, commit, push, update + draft PR #113, and verify hosted CI at the exact head. diff --git a/docs/superpowers/plans/2026-07-20-company-workbench-cash-generation-portability.md b/docs/superpowers/plans/2026-07-20-company-workbench-cash-generation-portability.md new file mode 100644 index 000000000..624abcaa5 --- /dev/null +++ b/docs/superpowers/plans/2026-07-20-company-workbench-cash-generation-portability.md @@ -0,0 +1,572 @@ +# Company Workbench Cash-Generation Portability Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add AMD Q1 FY2026 as the second immutable, explicit, preview-only Company Workbench quarterly cash-generation filing while preserving the existing NVIDIA route and every no-activation boundary. + +**Architecture:** Replace the NVIDIA-only loader constants with a frozen filing specification and a read-only two-entry ticker registry. Keep the loader signature, SEC fetcher, extractor, source-rights review, acceptance contract, preview composition, query gate, and presentation contract unchanged; only the registry may select exact reviewed filing identity. Prove AMD through fixtures, explicit route rendering, one bounded live no-write check, documentation contracts, full local gates, and exact-head GitHub CI. + +**Tech Stack:** Python 3.12, frozen dataclasses, `types.MappingProxyType`, pytest, Streamlit AppTest render smoke, existing SEC Companyfacts/submissions/primary-filing clients, Make release gates, GitHub Actions. + +## Global Constraints + +- Research-only; no investment advice, recommendation, broker integration, order routing, auto-trading, direct buy/sell instruction, or post-earnings price prediction. +- The registry contains exactly `NVDA` and `AMD`; do not add a broad company loop or accept caller-supplied CIK, accession, filing, dates, fiscal period, or cutoff. +- Both previews retain `production_activation=false`, `readiness_promotions=()`, `persistence=false`, complete withholding, and Advanced-only technical lineage. +- Keep Revenue, EPS, operating margin, free cash flow, FCF margin, valuation, catalysts, outcomes, consensus, backtesting, and calibration readiness independent. +- Do not run `make readiness`, refresh broad coverage, persist a source row, or generate/stage CSV, JSON, report, sample-report, screenshot, timing, readiness, canonical-data, or manual-review churn. +- Q4 requires explicit SEC-filed three-month Q4 table evidence and is never derived; this slice uses Q1 only. +- Stage exact intentional paths only; never use `git add -A`. +- Push only `codex/personal-research-mode-mvp`; keep PR #113 open and draft; do not merge or deploy. + +--- + +### Task 1: Immutable two-company filing registry and bounded loader + +**Files:** +- Modify: `tests/test_company_workbench_cash_generation_preview_loader.py` +- Modify: `src/company_workbench_cash_generation_preview_loader.py` + +**Interfaces:** +- Consumes: `fetch_sec_quarterly_pilot_payloads(...)`, `extract_sec_quarterly_cash_generation(...)`, `preview_sec_quarterly_cash_generation(...)`, and `compose_company_workbench_cash_generation_preview(...)`. +- Produces: frozen `CashGenerationPreviewFiling`, read-only `CASH_GENERATION_PREVIEW_FILINGS`, and the unchanged `load_company_workbench_cash_generation_preview(ticker, *, user_agent=None, fetcher=None, retrieved_at=None)` signature. + +- [ ] **Step 1: Add failing registry and AMD loader tests** + +Replace the NVIDIA constant imports with the registry import and add exact AMD fixtures and assertions: + +```python +from src.company_workbench_cash_generation_preview_loader import ( + CASH_GENERATION_PREVIEW_FILINGS, + load_company_workbench_cash_generation_preview, +) + +AMD_START = "2025-12-28" +AMD_END = "2026-03-28" +AMD_ACCESSION = "0000002488-26-000076" + + +def _amd_fact(value: float) -> dict[str, object]: + return { + "start": AMD_START, + "end": AMD_END, + "val": value, + "accn": AMD_ACCESSION, + "fy": 2026, + "fp": "Q1", + "form": "10-Q", + "filed": "2026-05-06", + } + + +def _amd_companyfacts() -> dict[str, object]: + return { + "cik": 2488, + "entityName": "ADVANCED MICRO DEVICES INC", + "facts": {"us-gaap": { + "RevenueFromContractWithCustomerExcludingAssessedTax": { + "units": {"USD": [_amd_fact(10_253_000_000)]} + }, + "OperatingIncomeLoss": {"units": {"USD": [_amd_fact(1_476_000_000)]}}, + "NetCashProvidedByUsedInOperatingActivities": { + "units": {"USD": [_amd_fact(2_955_000_000)]} + }, + "PaymentsToAcquirePropertyPlantAndEquipment": { + "units": {"USD": [_amd_fact(389_000_000)]} + }, + }}, + } + + +def _amd_submissions() -> dict[str, object]: + return {"cik": "0000002488", "filings": {"recent": { + "accessionNumber": [AMD_ACCESSION], + "filingDate": ["2026-05-06"], + "acceptanceDateTime": ["2026-05-05T18:06:27.000-04:00"], + "form": ["10-Q"], + "primaryDocument": ["amd-20260328.htm"], + }}} + + +def _amd_filing() -> str: + return f""" + + {AMD_START} + {AMD_END} + + + + + +
Net revenue10,253
Operating income1,476
Net cash provided by operating activities2,955
Purchases of property and equipment(389)
+ """ + + +def _amd_fetcher(calls: list[tuple[str, str]]): + def fetch(url: str, user_agent: str) -> bytes: + calls.append((url, user_agent)) + if "companyfacts" in url: + return json.dumps(_amd_companyfacts()).encode("utf-8") + if "submissions" in url: + return json.dumps(_amd_submissions()).encode("utf-8") + return _amd_filing().encode("utf-8") + return fetch + + +def test_registry_contains_only_two_exact_reviewed_filings(): + assert tuple(CASH_GENERATION_PREVIEW_FILINGS) == ("NVDA", "AMD") + nvda = CASH_GENERATION_PREVIEW_FILINGS["NVDA"] + amd = CASH_GENERATION_PREVIEW_FILINGS["AMD"] + assert (nvda.cik, nvda.accession, nvda.primary_document) == ( + "0001045810", "0001045810-26-000052", "nvda-20260426.htm" + ) + assert (amd.cik, amd.fiscal_period, amd.period_start, amd.period_end) == ( + "0000002488", "2026-Q1", "2025-12-28", "2026-03-28" + ) + assert (amd.accession, amd.primary_document, amd.as_of) == ( + "0000002488-26-000076", "amd-20260328.htm", "2026-07-20T23:59:59-04:00" + ) + + +def test_amd_loader_uses_exact_reviewed_identity_and_composes_in_memory(): + calls: list[tuple[str, str]] = [] + result = load_company_workbench_cash_generation_preview( + "AMD", + user_agent="Researcher research@example.com", + fetcher=_amd_fetcher(calls), + retrieved_at="2026-07-20T23:00:00+00:00", + ) + assert [url for url, _agent in calls] == [ + "https://data.sec.gov/api/xbrl/companyfacts/CIK0000002488.json", + "https://data.sec.gov/submissions/CIK0000002488.json", + "https://www.sec.gov/Archives/edgar/data/2488/000000248826000076/amd-20260328.htm", + ] + assert result.status == "accepted_for_review" + assert result.fiscal_period == "2026-Q1" + assert result.free_cash_flow.value == 2_566_000_000 + assert result.operating_margin.status == "preview_available" + assert result.fcf_margin.status == "preview_available" + assert result.production_activation is False + assert result.readiness_promotions == () + assert result.persistence is False +``` + +Strengthen the unsupported-ticker test with: + +```python +assert result.fiscal_period == "" +assert result.accession == "" +assert result.source_url == "" +assert result.cutoff == "" +``` + +Add the unsigned AMD filing test: + +```python +def test_unsigned_amd_capex_withholds_complete_preview(): + calls: list[tuple[str, str]] = [] + base_fetcher = _amd_fetcher(calls) + + def unsigned_fetch(url: str, user_agent: str) -> bytes: + payload = base_fetcher(url, user_agent) + if url.endswith("amd-20260328.htm"): + return payload.replace(b"(", b"").replace( + b")", b"" + ) + return payload + + result = load_company_workbench_cash_generation_preview( + "AMD", + user_agent="Researcher research@example.com", + fetcher=unsigned_fetch, + retrieved_at="2026-07-20T23:00:00+00:00", + ) + assert result.status == "withheld" + assert result.operating_margin.value is None + assert result.free_cash_flow.value is None + assert result.fcf_margin.value is None + assert result.components == () +``` + +- [ ] **Step 2: Run the focused loader tests and verify the expected failure** + +Run: + +```bash +python3 -m pytest tests/test_company_workbench_cash_generation_preview_loader.py -q +``` + +Expected: FAIL during import because `CASH_GENERATION_PREVIEW_FILINGS` does not exist. + +- [ ] **Step 3: Implement the immutable registry and registry-driven loader** + +Replace the NVIDIA-only constants and field substitutions with: + +```python +from dataclasses import dataclass +from types import MappingProxyType +from typing import Mapping + + +@dataclass(frozen=True) +class CashGenerationPreviewFiling: + ticker: str + cik: str + fiscal_period: str + period_start: str + period_end: str + accession: str + primary_document: str + as_of: str + + +CASH_GENERATION_PREVIEW_FILINGS: Mapping[str, CashGenerationPreviewFiling] = ( + MappingProxyType({ + "NVDA": CashGenerationPreviewFiling( + ticker="NVDA", + cik="0001045810", + fiscal_period="2027-Q1", + period_start="2026-01-26", + period_end="2026-04-26", + accession="0001045810-26-000052", + primary_document="nvda-20260426.htm", + as_of="2026-07-20T23:59:59-04:00", + ), + "AMD": CashGenerationPreviewFiling( + ticker="AMD", + cik="0000002488", + fiscal_period="2026-Q1", + period_start="2025-12-28", + period_end="2026-03-28", + accession="0000002488-26-000076", + primary_document="amd-20260328.htm", + as_of="2026-07-20T23:59:59-04:00", + ), + }) +) +``` + +At the beginning of the loader, resolve `filing = CASH_GENERATION_PREVIEW_FILINGS.get(symbol)`. When absent, call `blocked_company_workbench_cash_generation_preview` with only `symbol` and the unsupported blocker. When present, replace every previous `PREVIEW_*` argument with the matching `filing` property. In the exception path, retain only that selected filing's `fiscal_period`, `as_of`, and `accession`. + +- [ ] **Step 4: Run focused cash-generation tests** + +Run: + +```bash +python3 -m pytest tests/test_company_workbench_cash_generation_preview_loader.py tests/test_company_workbench_cash_generation_preview.py tests/test_sec_quarterly_cash_generation_pilot.py tests/test_quarterly_cash_generation_adapter.py tests/test_quarterly_cash_generation.py -q +``` + +Expected: PASS with no live network access. + +- [ ] **Step 5: Verify the loader has no write or broad-source surface** + +Run: + +```bash +make diff-hygiene-summary +git diff --check +``` + +Expected: only the loader and its focused test are changed; no generated artifact is reported. + +- [ ] **Step 6: Commit the loader slice** + +```bash +git add src/company_workbench_cash_generation_preview_loader.py tests/test_company_workbench_cash_generation_preview_loader.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add bounded AMD cash preview filing" +``` + +### Task 2: Explicit AMD Workbench route proof + +**Files:** +- Modify: `tests/test_dashboard_render_smoke.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` + +**Interfaces:** +- Consumes: the unchanged dashboard query gate and loader import, plus `CompanyWorkbenchCashGenerationPreview` presentation contract. +- Produces: runtime-contract evidence that explicit AMD preview works and normal AMD navigation never loads preview evidence. + +- [ ] **Step 1: Add an explicit AMD render-smoke test** + +Add an accepted AMD preview fixture using: + +```python +preview = CompanyWorkbenchCashGenerationPreview( + ticker="AMD", + fiscal_period="2026-Q1", + status="accepted_for_review", + message="Accepted SEC evidence supports a cash-generation review preview.", + operating_margin=CashGenerationPreviewMetric( + "operating_margin", "preview_available", 1_476_000_000 / 10_253_000_000, + "2026-Q1", (), "" + ), + free_cash_flow=CashGenerationPreviewMetric( + "free_cash_flow", "preview_available", 2_566_000_000, "2026-Q1", (), "" + ), + fcf_margin=CashGenerationPreviewMetric( + "fcf_margin", "preview_available", 2_566_000_000 / 10_253_000_000, + "2026-Q1", (), "" + ), + blockers=(), + withheld_metrics=(), + accession="0000002488-26-000076", + source_url="https://www.sec.gov/Archives/edgar/data/2488/000000248826000076/amd-20260328.htm", + accepted_at="2026-05-05T22:06:27+00:00", + cutoff="2026-07-21T03:59:59+00:00", + capex_sign_evidence="explicit_filed_table_outflow", + components=(), +) +``` + +Use route parameters `ticker=AMD`, `open=1`, and `cash_preview=1`. Require `Cash-generation review preview`, `not production evidence`, `14.4%`, `2,566,000,000`, and `25.0%`. Patch only the external loader boundary and assert no exception, missing marker, forbidden marker, or expanded Advanced section. + +- [ ] **Step 2: Add a normal AMD route non-invocation test** + +Add the normal-route guard: + +```python +def test_normal_amd_company_workbench_route_never_loads_cash_preview(): + from src.dashboard_render_smoke import DashboardRenderRoute, render_public_routes + + route = DashboardRenderRoute( + name="Normal AMD Company Workbench", + query_params=( + ("mode", "research"), + ("page", "company-workbench"), + ("ticker", "AMD"), + ("open", "1"), + ), + required_markers=("Company Workbench", "Business Trend", "Research-only"), + ) + with patch( + "src.company_workbench_cash_generation_preview_loader." + "load_company_workbench_cash_generation_preview", + side_effect=AssertionError("normal AMD Workbench must not load cash preview"), + ): + result = render_public_routes(Path("."), routes=(route,))[0] + assert result.exceptions == () + assert result.missing_markers == () +``` + +- [ ] **Step 3: Strengthen the static route contract** + +In `test_company_workbench_loads_cash_preview_only_for_explicit_flag`, keep: + +```python +assert 'company_workbench_cash_preview_requested(st.query_params.get("cash_preview"))' in source +assert "load_company_workbench_cash_generation_preview(ticker)" in source +``` + +In the default-navigation test, add: + +```python +assert "ticker=NVDA&open=1&cash_preview=1" not in source +assert "ticker=AMD&open=1&cash_preview=1" not in source +``` + +- [ ] **Step 4: Run the focused dashboard contracts** + +Run: + +```bash +python3 -m pytest tests/test_dashboard_render_smoke.py tests/test_research_mode_dashboard_contract.py tests/test_research_workspace.py -q +make research-dashboard-render-smoke +``` + +Expected: PASS; all standard and Advanced Evidence routes remain unchanged, and the AMD explicit route renders only through the patched loader boundary. + +- [ ] **Step 5: Commit the route-proof slice** + +```bash +git add tests/test_dashboard_render_smoke.py tests/test_research_mode_dashboard_contract.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Prove explicit AMD cash preview route" +``` + +### Task 3: Documentation truth and continuation contract + +**Files:** +- Modify: `tests/test_public_v1_release_docs.py` +- Modify: `ROADMAP.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: the verified two-entry registry, AMD live evidence, focused tests, and unchanged product boundaries. +- Produces: durable product-stage and next-lane truth for reviewers and later continuations. + +- [ ] **Step 1: Add a failing documentation contract** + +Add: + +```python +def test_two_company_cash_preview_docs_preserve_bounded_portability_boundary(): + methodology = _read("docs/METHODOLOGY.md") + provenance = _read("docs/PROVENANCE_CONTRACT.md") + personal = _read("docs/PERSONAL_RESEARCH_MODE.md") + roadmap = _read("ROADMAP.md") + prompt = _read("docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md") + + for text in (methodology, provenance, personal, roadmap, prompt): + assert "AMD Q1 FY2026" in text + assert "0000002488-26-000076" in text + assert "bounded two-company portability" in text.lower() + assert "cash_preview=1" in personal + assert "production_activation=false" in provenance + assert "readiness_promotions=()" in provenance + assert "does not prove broad company coverage" in roadmap.lower() + assert "do not add a third company" in prompt.lower() +``` + +- [ ] **Step 2: Run the documentation test and verify failure** + +Run: + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py -q +``` + +Expected: FAIL because the AMD portability evidence is not yet documented. + +- [ ] **Step 3: Update the six truth documents** + +Add this product-stage statement to `ROADMAP.md`: + +```markdown +**Implemented locally and live-source verified:** AMD Q1 FY2026 accession `0000002488-26-000076` now joins NVIDIA in one immutable, shared loader and explicit `cash_preview=1` Company Workbench path. The AMD filing supplied Revenue USD 10.253B, operating income USD 1.476B, cash from operations USD 2.955B, capital expenditures USD -0.389B, and `explicit_filed_table_outflow` evidence before free cash flow USD 2.566B was displayed. This is bounded two-company portability; it does not prove broad company coverage, arbitrary-filing support, historical depth, Q4 support, production activation, current readiness, hosting, reviewer validation, calibration, demand, or product-market fit. +``` + +Add this route statement to `docs/PERSONAL_RESEARCH_MODE.md`: + +```markdown +AMD Q1 FY2026 accession `0000002488-26-000076` is the second exact filing available through the explicit `cash_preview=1` route. NVIDIA and AMD now provide bounded two-company portability through the same immutable loader, complete-withholding contract, preview-only cards, and Advanced lineage. Normal Company Workbench routes remain canonical and never load the preview. +``` + +Add this method statement to `docs/METHODOLOGY.md`: + +```markdown +The AMD Q1 FY2026 result proves bounded two-company portability for the same exact-filing method: one immutable identity, three official SEC endpoints, compatible quarterly components, timezone-aware acceptance, explicit filed-table capex outflow, exact rights scope, and complete withholding. It does not prove broad company coverage, and no third company is inferred from cohort membership. +``` + +Add this evidence statement to `docs/PROVENANCE_CONTRACT.md`: + +```markdown +AMD Q1 FY2026 accession `0000002488-26-000076` uses the same bounded two-company portability contract as NVIDIA. Both results retain `production_activation=false`, `readiness_promotions=()`, `persistence=false`, exact component references, and no canonical write. Unsupported tickers fail before fetch and expose no configured filing identity. +``` + +In `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, add a lineage navigation anchor using the already committed design SHA, which every implementation commit must descend from: + +```markdown +- Bounded two-company cash-preview design anchor: commit `11ac530ae` or a later verified descendant. +``` + +Replace the quarterly portability classification with: + +```markdown +- Quarterly cash-generation portability: `bounded_two_company_preview_implemented`; NVIDIA Q1 FY2027 and AMD Q1 FY2026 use one immutable exact-filing loader and explicit preview-only Workbench path. This proves bounded two-company portability only. Do not add a third company, run broad coverage, infer filing identity, persist observations, or promote readiness without a separately justified exact-source design and review. +``` + +Keep all normal-route, complete-withholding, Advanced-only, no-persistence, stale-readiness, Q4, EPS, consensus, calibration, hosted, reviewer, demand, and market-validation boundaries unchanged. + +- [ ] **Step 4: Run documentation, wording, and whitespace checks** + +Run: + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py -q +make public-wording-check +git diff --check +``` + +Expected: PASS. + +- [ ] **Step 5: Commit documentation truth** + +```bash +git add ROADMAP.md docs/PERSONAL_RESEARCH_MODE.md docs/METHODOLOGY.md docs/PROVENANCE_CONTRACT.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Document bounded cash preview portability" +``` + +### Task 4: Live proof, complete verification, push, and PR update + +**Files:** +- No new repository files. +- Update draft PR #113 through GitHub after all local evidence passes. + +**Interfaces:** +- Consumes: Tasks 1-3 and configured `SEC_USER_AGENT`. +- Produces: current live no-write AMD proof, complete local release evidence, aligned remote branch, updated draft PR, and exact-head hosted CI evidence. + +- [ ] **Step 1: Run one bounded live loader proof** + +Run: + +```bash +python3 -c 'from src.company_workbench_cash_generation_preview_loader import load_company_workbench_cash_generation_preview as load; result = load("AMD"); print(result.status, result.fiscal_period, result.free_cash_flow.value, result.production_activation, result.readiness_promotions, result.persistence); assert result.status == "accepted_for_review"; assert result.free_cash_flow.value == 2566000000; assert result.production_activation is False; assert result.readiness_promotions == (); assert result.persistence is False' +``` + +Expected: `accepted_for_review 2026-Q1 2566000000.0 False () False`. Immediately run `git status --short` and require no artifact change. + +- [ ] **Step 2: Run focused and full verification** + +Run: + +```bash +python3 -m pytest tests/test_company_workbench_cash_generation_preview_loader.py tests/test_company_workbench_cash_generation_preview.py tests/test_sec_quarterly_cash_generation_pilot.py tests/test_quarterly_cash_generation_adapter.py tests/test_quarterly_cash_generation.py tests/test_dashboard_render_smoke.py tests/test_research_mode_dashboard_contract.py tests/test_research_workspace.py tests/test_public_v1_release_docs.py -q +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +make pr-range-hygiene-check BASE_SHA=origin/main HEAD_SHA=HEAD +git diff --check +``` + +Expected: all code and release gates pass; pilot readiness remains truthfully blocked by stale saved readiness; PR-range hygiene reports zero generated CSV/JSON and manual-review churn. + +- [ ] **Step 3: Verify repository package before push** + +Run: + +```bash +git status --short --branch +git log -8 --oneline --decorate +git rev-list --left-right --count HEAD...origin/codex/personal-research-mode-mvp +``` + +Expected: clean working tree and a finite intentional ahead count containing only the design, plan, loader, route-proof, and documentation commits. + +- [ ] **Step 4: Push only the approved feature branch** + +```bash +git push origin codex/personal-research-mode-mvp +``` + +- [ ] **Step 5: Update draft PR #113** + +Update the PR body with the exact new HEAD, AMD identity and values, bounded two-company claim, full test count, local gate results, generated-artifact exclusion, live no-write evidence, remaining external dependencies, and exact next maturity lane. Keep it open and draft; do not merge. + +- [ ] **Step 6: Verify exact-head GitHub CI** + +Run: + +```bash +gh pr view 113 --json isDraft,state,mergeable,headRefOid,statusCheckRollup,url +``` + +Wait until the `local-engineering-gate` for the exact pushed `headRefOid` completes successfully. Do not reuse prior-head CI. + +- [ ] **Step 7: Final completion audit for this slice** + +Map each design completion criterion to current code, tests, live result, documentation, Git status, PR state, range hygiene, and exact-head CI. Report this portability slice complete only if all ten criteria are directly proven. Keep the overall commercial-maturity goal active because point-in-time consensus, hosting, external reviewers, calibration, broader evidence depth, and operated-platform controls remain incomplete. diff --git a/docs/superpowers/plans/2026-07-20-company-workbench-cash-generation-preview.md b/docs/superpowers/plans/2026-07-20-company-workbench-cash-generation-preview.md new file mode 100644 index 000000000..c3b59c4b2 --- /dev/null +++ b/docs/superpowers/plans/2026-07-20-company-workbench-cash-generation-preview.md @@ -0,0 +1,407 @@ +# Company Workbench Cash-Generation Activation Preview Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build an explicit, read-only Company Workbench route that shows one accepted NVIDIA SEC quarterly cash-generation packet as a clearly non-production review preview while leaving the ordinary Workbench and every readiness state unchanged. + +**Architecture:** Add a pure activation-preview composer that converts only an `accepted_for_review` SEC pilot result into an immutable, fail-closed Workbench view model. Add a separate bounded live loader for the exact NVIDIA filing and reviewed cutoff, then pass the result into the existing Business Trend renderer only when `cash_preview=1`; raw lineage stays under Advanced and normal Workbench rendering never fetches preview data. + +**Tech Stack:** Python 3.12, frozen dataclasses, the existing SEC Companyfacts/submissions/inline-XBRL parser, existing quarterly trend composition, Streamlit, pytest, and Make-based repository gates. + +## Global Constraints + +- Research-only; no investment advice, broker integration, order routing, auto-trading, direct buy/sell instructions, or post-earnings price prediction. +- Consume only `SecQuarterlyPilotPreview.status == "accepted_for_review"` with an accepted adapter result. +- Keep `production_activation=False`, `readiness_promotions=()`, and `persistence=False` immutable. +- Accept only the `cash_preview=1` route flag; add no arbitrary ticker, accession, filing, or cutoff input. +- Use NVIDIA Q1 FY2027 accession `0001045810-26-000052`, document `nvda-20260426.htm`, and cutoff `2026-07-20T23:59:59-04:00`. +- Withhold all three preview metrics when evidence is missing, rejected, ambiguous, mismatched, post-cutoff, incompatible, incomplete, or has invalid capex-sign proof. +- Leave the ordinary Company Workbench canonical packet and all readiness states unchanged. +- EPS split basis remains unverified without explicit proof; Q4 requires an explicit filed three-month quarter. +- Generate and stage no CSV, JSON, report, sample-report, screenshot, timing, canonical-data, readiness, cache, or manual-review artifact. +- Never run `make readiness`, readiness writers, broad refreshes, canonical writers, screenshot regeneration, consensus apply, production activation, deployment, or PR merge. +- Stage exact files only; never use `git add -A`. Push only `codex/personal-research-mode-mvp`; keep PR #113 draft. + +## File Structure + +- Create `src/company_workbench_cash_generation_preview.py` for immutable types, query parsing, blocked construction, and pure composition. +- Create `src/company_workbench_cash_generation_preview_loader.py` for exact in-memory NVIDIA loading. +- Modify `src/research_workspace.py` for primary preview cards and Advanced lineage rows. +- Modify `src/dashboard.py` for explicit route gating and optional Business Trend rendering. +- Create `tests/test_company_workbench_cash_generation_preview.py` and `tests/test_company_workbench_cash_generation_preview_loader.py`. +- Modify `tests/test_research_workspace.py`, `tests/test_research_mode_dashboard_contract.py`, and `tests/test_dashboard_render_smoke.py`. +- Modify `ROADMAP.md`, `docs/PERSONAL_RESEARCH_MODE.md`, `docs/METHODOLOGY.md`, `docs/PROVENANCE.md`, `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, and `tests/test_public_v1_release_docs.py`. + +--- + +### Task 1: Pure Activation-Preview Contract + +**Files:** +- Create: `src/company_workbench_cash_generation_preview.py` +- Create: `tests/test_company_workbench_cash_generation_preview.py` + +**Interfaces:** +- Consumes: `SecQuarterlyPilotPreview`, `build_quarterly_trend_packet(...)`, and `parse_utc_timestamp(...)`. +- Produces: `CashGenerationPreviewMetric`, `CashGenerationPreviewComponent`, `CompanyWorkbenchCashGenerationPreview`, `company_workbench_cash_preview_requested(value: object) -> bool`, `blocked_company_workbench_cash_generation_preview(...)`, and `compose_company_workbench_cash_generation_preview(...)`. + +- [ ] **Step 1: Write the failing accepted-result test** + +Construct all evidence in memory. The fixture must contain Revenue `81_615_000_000`, operating income `53_536_000_000`, CFO `50_344_000_000`, capex `-1_757_000_000`, exact SEC lineage, accepted time `2026-05-20T20:35:52+00:00`, `accepted_for_review`, false activation, and empty promotions. + +```python +AS_OF = "2026-07-20T23:59:59-04:00" + +def test_accepted_sec_packet_composes_complete_non_activation_preview(): + result = compose_company_workbench_cash_generation_preview( + _accepted_sec_preview(), selected_ticker="NVDA", as_of=AS_OF + ) + assert result.status == "accepted_for_review" + assert result.operating_margin.value == pytest.approx(53_536_000_000 / 81_615_000_000) + assert result.free_cash_flow.value == 48_587_000_000 + assert result.fcf_margin.value == pytest.approx(48_587_000_000 / 81_615_000_000) + assert result.production_activation is False + assert result.readiness_promotions == () + assert result.persistence is False + assert result.accession == "0001045810-26-000052" + assert result.capex_sign_evidence == "explicit_filed_table_outflow" + assert result.blockers == () +``` + +- [ ] **Step 2: Verify the test fails before implementation** + +Run `python3 -m pytest tests/test_company_workbench_cash_generation_preview.py::test_accepted_sec_packet_composes_complete_non_activation_preview -q`. + +Expected: collection fails because the module does not exist. + +- [ ] **Step 3: Add immutable types and strict query parsing** + +Implement these exact public shapes: + +```python +@dataclass(frozen=True) +class CashGenerationPreviewMetric: + metric: str + status: str + value: float | None + fiscal_period: str + source_refs: tuple[str, ...] + withheld_reason: str + +@dataclass(frozen=True) +class CashGenerationPreviewComponent: + metric: str + value: float + currency: str + fiscal_period: str + source_ref: str + published_at: str + retrieved_at: str + accounting_basis: str + duration_basis: str + q4_evidence_state: str + +@dataclass(frozen=True) +class CompanyWorkbenchCashGenerationPreview: + ticker: str + fiscal_period: str + status: str + message: str + operating_margin: CashGenerationPreviewMetric + free_cash_flow: CashGenerationPreviewMetric + fcf_margin: CashGenerationPreviewMetric + blockers: tuple[str, ...] + withheld_metrics: tuple[str, ...] + accession: str + source_url: str + accepted_at: str + cutoff: str + capex_sign_evidence: str + components: tuple[CashGenerationPreviewComponent, ...] + production_activation: bool = False + readiness_promotions: tuple[str, ...] = () + persistence: bool = False + +def company_workbench_cash_preview_requested(value: object) -> bool: + if isinstance(value, (list, tuple)): + value = value[0] if value else "" + return str(value or "").strip() == "1" +``` + +- [ ] **Step 4: Implement the pure composer** + +Implement: + +```python +def compose_company_workbench_cash_generation_preview( + pilot: SecQuarterlyPilotPreview, + *, + selected_ticker: str, + as_of: str, +) -> CompanyWorkbenchCashGenerationPreview: +``` + +Validate timezone-aware cutoff/acceptance timestamps, both acceptance statuses, false activation flags, empty promotions, selected/extracted ticker identity, no extraction blockers, exact metadata, `sec_companyfacts` source identity, accepted observation count, acceptance before cutoff, all observations before cutoff, explicit Q4 evidence, and `explicit_filed_table_outflow`. Compose through `build_quarterly_trend_packet(..., business_observations=...)`. Expose success only when all three metrics have values for the extraction period. Any blocker returns three withheld metrics and `components=()`. + +- [ ] **Step 5: Verify the success test passes** + +Run the Step 2 command. Expected: `1 passed`. + +- [ ] **Step 6: Add fail-closed and query tests** + +Parameterize pilot-blocked, missing/blocked acceptance, mixed ticker, extraction blockers, activation true, promotions non-empty, missing accession/source/accepted time, post-cutoff time, wrong capex proof, incomplete components, ambiguous revisions, incompatible definitions, and Q4 without explicit proof. Every result must be `withheld`, carry the expected stable blocker, have three `None` values, and expose no components. Prove only scalar/list/tuple `"1"` enables the route; `None`, `""`, `"0"`, `"true"`, `"yes"`, and `"1,other"` do not. + +- [ ] **Step 7: Run and commit the pure contract** + +```bash +python3 -m pytest tests/test_company_workbench_cash_generation_preview.py -q +git add -- src/company_workbench_cash_generation_preview.py tests/test_company_workbench_cash_generation_preview.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add Workbench cash preview contract" +``` + +Expected: all focused tests and hygiene checks pass. + +--- + +### Task 2: Exact In-Memory NVIDIA Loader + +**Files:** +- Create: `src/company_workbench_cash_generation_preview_loader.py` +- Create: `tests/test_company_workbench_cash_generation_preview_loader.py` + +**Interfaces:** +- Consumes: the existing exact SEC fetch/extract/accept helpers and Task 1 composer. +- Produces: exact identity constants and `load_company_workbench_cash_generation_preview(ticker: str, *, user_agent: str | None = None, fetcher=None, retrieved_at: str | None = None)`. + +- [ ] **Step 1: Write the failing bounded-loader test** + +Monkeypatch fetch, extraction, acceptance, and rights loading. Assert fetch receives only CIK `0001045810`, accession `0001045810-26-000052`, and `nvda-20260426.htm`; extraction receives fiscal period `2027-Q1`, dates `2026-01-26` through `2026-04-26`, and cutoff `2026-07-20T23:59:59-04:00`. Assert the returned result is accepted and non-persistent. + +- [ ] **Step 2: Verify the loader test fails** + +Run `python3 -m pytest tests/test_company_workbench_cash_generation_preview_loader.py::test_loader_uses_only_reviewed_nvidia_identity_and_composes_in_memory -q`. + +Expected: collection fails because the loader module does not exist. + +- [ ] **Step 3: Implement the bounded loader** + +Define: + +```python +PREVIEW_TICKER = "NVDA" +PREVIEW_CIK = "0001045810" +PREVIEW_FISCAL_PERIOD = "2027-Q1" +PREVIEW_PERIOD_START = "2026-01-26" +PREVIEW_PERIOD_END = "2026-04-26" +PREVIEW_ACCESSION = "0001045810-26-000052" +PREVIEW_PRIMARY_DOCUMENT = "nvda-20260426.htm" +PREVIEW_AS_OF = "2026-07-20T23:59:59-04:00" +``` + +Return `unsupported_preview_ticker:` before fetching non-NVDA. Otherwise fetch the exact three endpoints in memory, extract using the fixed identity and cutoff, assess with the source-rights registry, and compose. Catch `SECUserAgentError`, `SecQuarterlyPreviewFetchError`, `TypeError`, and `ValueError` as `preview_load_blocked:` without secrets or raw payloads. Add no paths, cache, argparse, output, apply, readiness, refresh, or fallback surface. + +- [ ] **Step 4: Add blocked and surface tests** + +Prove non-NVDA never invokes fetch; missing SEC user agent, fetch error, malformed extraction, and blocked acceptance withhold all values. Inspect the function signature and module source to reject output/apply/refresh/readiness/accession/CIK/filing/cutoff parameters and `Path`, `open(`, `to_csv`, `to_json`, alternate providers, or fixture fallback. + +- [ ] **Step 5: Run and commit the loader** + +```bash +python3 -m pytest tests/test_company_workbench_cash_generation_preview_loader.py -q +git add -- src/company_workbench_cash_generation_preview_loader.py tests/test_company_workbench_cash_generation_preview_loader.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Load bounded Workbench cash preview" +``` + +Expected: all tests pass without network or file writes. + +--- + +### Task 3: Answer-First Workbench Rendering + +**Files:** +- Modify: `src/research_workspace.py` +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_workspace.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_dashboard_render_smoke.py` + +**Interfaces:** +- Consumes: Task 1 view model/query helper and Task 2 loader. +- Produces: `cash_generation_preview_cards(preview)`, `cash_generation_preview_rows(preview)`, and optional `cash_generation_preview` on `render_single_stock_report(...)`. + +- [ ] **Step 1: Write failing presentation tests** + +Assert accepted cards appear in this order: boundary, Operating Margin, Free Cash Flow, FCF Margin. Require title `Cash-generation review preview — not production evidence`, one-decimal margin percentages, comma-formatted FCF, `preview_available` rather than `ready`, blank commands, and non-activation badges. Require accession, timestamps, capex sign, definitions, values, and source refs only in Advanced rows. Withheld input must show three `Withheld` cards and no component rows. + +- [ ] **Step 2: Verify presentation tests fail** + +Run `python3 -m pytest tests/test_research_workspace.py -k cash_generation_preview -q`. + +Expected: missing helper imports or attributes. + +- [ ] **Step 3: Implement presentation helpers** + +In `src/research_workspace.py`, build four answer-first cards without lineage identifiers. Build Advanced rows for accession, source URL, accepted time, cutoff, capex sign, non-activation fields, and each component's value, definition, timestamps, and source ref. Do not add commands, advice, scores, recommendations, or readiness claims. + +- [ ] **Step 4: Write failing dashboard route tests** + +Require `render_company_workbench` to call the strict query helper before the loader and pass `None` otherwise. Require the ordinary `load_dashboard_quarterly_trend(ticker)` call to remain independent. Within Business Trend, require canonical cards first, preview cards second, then collapsed `Advanced: cash-generation preview evidence` rows. Require no preview flag in normal navigation or Discover links. + +- [ ] **Step 5: Verify dashboard tests fail** + +Run `python3 -m pytest tests/test_research_mode_dashboard_contract.py -k "cash_preview or cash_generation" -q`. + +Expected: failures because the route is not integrated. + +- [ ] **Step 6: Integrate the explicit route** + +Add `cash_generation_preview: CompanyWorkbenchCashGenerationPreview | None = None` to `render_single_stock_report(...)`. After ordinary `quarterly_trend_cards(trend_packet)`, render preview cards only when supplied, followed by a collapsed Advanced preview-evidence dataframe and non-activation caption. In `render_company_workbench`, use exactly: + +```python +cash_generation_preview = ( + load_company_workbench_cash_generation_preview(ticker) + if company_workbench_cash_preview_requested(st.query_params.get("cash_preview")) + else None +) +``` + +Pass it separately while retaining `quarterly_trend_packet=load_dashboard_quarterly_trend(ticker)`. Do not change other routes. + +- [ ] **Step 7: Add explicit-route render smoke** + +Keep the normal Workbench smoke route with no loader call. Add an explicit preview route whose loader is patched to a complete in-memory view model and require `Cash-generation review preview`, `not production evidence`, `Operating Margin`, `Free Cash Flow`, and `FCF Margin`. No test may contact SEC or write an artifact. + +- [ ] **Step 8: Run focused rendering verification** + +```bash +python3 -m pytest tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q +make dashboard-smoke +make research-dashboard-render-smoke +git status --short +``` + +Expected: all tests and smokes pass; status lists only intentional source/test changes. + +- [ ] **Step 9: Commit the rendering slice** + +```bash +git add -- src/research_workspace.py src/dashboard.py tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Render explicit Workbench cash preview" +``` + +--- + +### Task 4: Documentation and Roadmap Truth + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes: verified Tasks 1–3 behavior. +- Produces: durable capability, boundary, maturity, and next-stage evidence. + +- [ ] **Step 1: Write failing documentation tests** + +Require the guide to contain `cash_preview=1`, `Cash-generation review preview`, and `not production evidence`; provenance to contain `production_activation=false`, `readiness_promotions=()`, and `no canonical persistence`; roadmap to say `one explicit user-flow composition` and `does not prove a second company`; methodology to state complete withholding on any required-metric failure and Advanced-only technical lineage; continuation prompt to retain the NVIDIA stop rule and identify a bounded second-company proof as the next assessment. + +- [ ] **Step 2: Verify documentation tests fail** + +Run `python3 -m pytest tests/test_public_v1_release_docs.py -k "cash_generation or quarterly" -q`. + +Expected: failure on new preview wording. + +- [ ] **Step 3: Update exact documentation** + +Document the explicit local route, fixed cutoff, accepted-only composer, unchanged normal Workbench, all-or-nothing preview metrics, Advanced lineage, false activation, empty promotions, no persistence/readiness/artifacts, and no proof of another company, historical depth, Q4, hosting, reviewer adoption, calibration, demand, or product-market fit. Retain exact unblock conditions for point-in-time consensus, calibration, hosting/identity/operations, external review, and market evidence. Set the continuation prompt's expected HEAD only after the verified documentation commit. + +- [ ] **Step 4: Run and commit documentation** + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py -q +make public-wording-check +git diff --check +git add -- ROADMAP.md docs/PERSONAL_RESEARCH_MODE.md docs/METHODOLOGY.md docs/PROVENANCE.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Document Workbench cash preview evidence" +``` + +Expected: documentation and wording checks pass with zero generated files staged. + +--- + +### Task 5: Full Verification, GitHub Sync, and Next-Stage Assessment + +**Files:** +- Modify only if verification reveals an in-scope defect: files already named above. +- External update: draft PR #113 body only; no merge or deployment. + +**Interfaces:** +- Consumes: all committed preview changes. +- Produces: complete local gate evidence, clean range hygiene, exact-head CI, updated draft PR, and next-stage classification. + +- [ ] **Step 1: Run focused and full tests** + +```bash +python3 -m pytest tests/test_company_workbench_cash_generation_preview.py tests/test_company_workbench_cash_generation_preview_loader.py tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_public_v1_release_docs.py -q +python3 -m pytest tests -q +``` + +Expected: all tests pass; no new warning is accepted without review. + +- [ ] **Step 2: Run every required product and hygiene gate** + +```bash +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +make pr-range-hygiene-check BASE_SHA=origin/main HEAD_SHA=HEAD +git diff --check +``` + +Expected: all engineering gates pass; pilot readiness may remain truthfully blocked on stale saved readiness; hygiene reports zero generated churn. + +- [ ] **Step 3: Verify scope and push only the feature branch** + +```bash +git status --short --branch +git log -8 --oneline --decorate +git diff origin/codex/personal-research-mode-mvp...HEAD --stat +make diff-hygiene-summary +git push origin codex/personal-research-mode-mvp +``` + +Expected: the clean branch advances with only intentional commits. + +- [ ] **Step 4: Update and verify draft PR #113** + +Update the PR body with route behavior, accepted-only/non-activation/no-persistence boundaries, tests, full gates, zero generated churn, exact head, and remaining stale-readiness, consensus, calibration, hosting, reviewer, portability, and market gates. Verify with: + +```bash +gh pr view 113 --json state,isDraft,mergeable,headRefName,headRefOid,statusCheckRollup,url +``` + +Expected: open, draft, correct feature branch/head, no merge. Wait for `local-engineering-gate` success at the exact pushed SHA; do not reuse earlier CI evidence. + +- [ ] **Step 5: Assess the bounded next maturity slice** + +Classify a second-company SEC portability proof as the next local lane only if one exact non-Q4 official filing can test a materially different parser structure without broad discovery or refresh. Otherwise record `external_source_selection_required` and stop broad source loops. Retain `permitted_dataset_required` for point-in-time consensus, `calibration_cohort_required` for probabilities, `hosted_account_identity_operations_required` for hosted beta, and `external_reviewer_and_demand_evidence_required` for market maturity. + +- [ ] **Step 6: Complete the requirement-by-requirement handoff audit** + +Verify source, tests, rendered behavior, Git history, PR state, and hygiene evidence against every design requirement. Report repository/PR status, product stage, roadmap item, changes, tests, commits/push, excluded artifacts, external dependencies, remaining gaps, exact next step, review safety, and whether the overall goal remains active. Do not call the overall product complete merely because this local slice passes. diff --git a/docs/superpowers/plans/2026-07-20-filed-q4-split-proof-correction.md b/docs/superpowers/plans/2026-07-20-filed-q4-split-proof-correction.md new file mode 100644 index 000000000..5899fd853 --- /dev/null +++ b/docs/superpowers/plans/2026-07-20-filed-q4-split-proof-correction.md @@ -0,0 +1,13 @@ +# Filed-Q4 Split-Proof Correction Plan + +1. Replace the prior positive no-proof assertion with failing missing/malformed + split-proof regressions. +2. Add a distinct primary-proof unverified sentinel and conservative shared + basis predicate. +3. Route consensus unverified handling through the same predicate. +4. Add an integration test proving five unverified filed-Q4-derived EPS rows + keep EPS withheld while Revenue remains ready. +5. Update methodology, provenance, roadmap, continuation contract, and tests. +6. Run focused/full tests and every product/release/pilot/range/staged hygiene + gate; commit, push, update draft PR #113, and require independent re-audit + plus exact-head hosted CI. diff --git a/docs/superpowers/plans/2026-07-20-optional-evidence-commercial-gates.md b/docs/superpowers/plans/2026-07-20-optional-evidence-commercial-gates.md new file mode 100644 index 000000000..6e09b6b78 --- /dev/null +++ b/docs/superpowers/plans/2026-07-20-optional-evidence-commercial-gates.md @@ -0,0 +1,14 @@ +# Optional Evidence Commercial Gates Plan + +1. Add failing tests for approved-rights/wrong-scope and exact-scope passage in + valuation, catalyst, and outcome builders. +2. Add explicit Commercial Research parameters and exact-source lane-scope + reviews without changing research-mode behavior. +3. Carry blocker counts/reasons in packet state and keep unsupported rows out of + commercially supported/reviewed output. +4. Route dashboard loaders through Commercial Research composition and keep + evidence details under existing Advanced surfaces. +5. Update methodology, provenance, roadmap, continuation contract, and tests. +6. Run focused/full tests and every required product, release, pilot, range, + whitespace, generated-artifact, and staged-hygiene check; commit, push, + update draft PR #113, and verify exact-head hosted CI. diff --git a/docs/superpowers/plans/2026-07-20-pr-range-engineering-gate.md b/docs/superpowers/plans/2026-07-20-pr-range-engineering-gate.md new file mode 100644 index 000000000..89f52a3ef --- /dev/null +++ b/docs/superpowers/plans/2026-07-20-pr-range-engineering-gate.md @@ -0,0 +1,13 @@ +# Pull-Request Range Engineering Gate Plan + +1. Add failing workflow and hygiene tests for full history, explicit event SHAs, + range commands, and a generated file committed between two temporary SHAs. +2. Add commit-resolution and range-status loading to the existing read-only + hygiene script, with a generated-churn fail result. +3. Add a Make target accepting explicit `BASE_SHA` and `HEAD_SHA`. +4. Update the workflow to check out exact head with full history and run both + hygiene and whitespace against `BASE...HEAD`. +5. Update roadmap, methodology/provenance where relevant, continuation contract, + and focused launcher/workflow contracts. +6. Run focused/full tests and all product/release/pilot/local/range/staged hygiene + checks; commit, push, update draft PR #113, and require exact-head hosted CI. diff --git a/docs/superpowers/plans/2026-07-20-price-lineage-apply-integrity.md b/docs/superpowers/plans/2026-07-20-price-lineage-apply-integrity.md new file mode 100644 index 000000000..cb32aeeca --- /dev/null +++ b/docs/superpowers/plans/2026-07-20-price-lineage-apply-integrity.md @@ -0,0 +1,19 @@ +# Price Lineage and Apply Integrity Implementation Plan + +1. Add failing unit tests for timezone awareness, observation availability, + review cutoff, normalized UTC output, and stable blocker vocabulary. +2. Add failing normalization, staged validation/preview, DCF-lineage, and + commercial-apply integration tests using the same temporal cases. +3. Add a shared daily-price temporal review module and route every consumer + through it without changing source-rights or field-scope decisions. +4. Make explicit normalization retrieval require the exact review cutoff and + fail before writing on invalid temporal evidence. +5. Refactor staged preview/apply so apply carries the one validated frame rather + than reading the staged CSV twice. +6. Replace direct canonical writes with a flushed, fsynced, same-directory + temporary file and atomic replace; retain the optional reviewed backup. +7. Update Make targets, methodology, provenance, operator docs, ROADMAP, and the + continuation contract without running readiness or writing repository data. +8. Run focused/full tests and all required product, commercial, pilot, PR-range, + whitespace, generated-artifact, and staged-hygiene checks; commit, push, + update draft PR #113, and verify hosted CI on the exact head. diff --git a/docs/superpowers/plans/2026-07-20-sec-quarterly-cash-generation-pilot.md b/docs/superpowers/plans/2026-07-20-sec-quarterly-cash-generation-pilot.md new file mode 100644 index 000000000..bb756e1d3 --- /dev/null +++ b/docs/superpowers/plans/2026-07-20-sec-quarterly-cash-generation-pilot.md @@ -0,0 +1,620 @@ +# SEC Quarterly Cash-Generation Pilot Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Prove one real NVIDIA quarter can pass the existing quarterly cash-generation adapter review from exact SEC Companyfacts, submissions, and filed-table evidence without persisting data or changing readiness. + +**Architecture:** A pure parser selects one exact accession and three-month context from already-retrieved SEC payloads, matches each structured fact to its inline XBRL filing fact, and requires explicit filed-table outflow presentation before making capex negative. A separate read-only client/CLI fetches only the three exact SEC endpoints, renders a human review summary, and composes the existing acceptance contract; source field scope and production activation remain independent. + +**Tech Stack:** Python 3.12 standard library (`dataclasses`, `html.parser`, `json`, `urllib`), existing immutable earnings/cash-generation/source-rights contracts, pytest, YAML source registry, Make read-only operator target. + +## Global Constraints + +- Research-only; no investment advice, broker integration, order routing, auto-trading, buy/sell instruction, or post-earnings price prediction. +- The pilot is one exact NVIDIA Q1 FY2027 evidence path, not broad coverage. +- Use `sec_companyfacts` as the exact registered source ID; do not split, alias, or compose source IDs. +- Require accession `0001045810-26-000052`, period `2027-Q1`, start `2026-01-26`, end `2026-04-26`, and exact SEC submissions acceptance time for the first live proof. +- Companyfacts supplies fact identity and magnitude; the exact filed inline XBRL table must independently prove the capex outflow presentation. +- Never infer capex sign from concept name, taxonomy balance, history, or an unsigned Companyfacts magnitude. +- Never derive Q4 from annual or nine-month values; the initial live pilot is Q1 only. +- Never substitute filing-date midnight or retrieval time for publication time. +- `accepted_for_review` never means production activation, readiness promotion, source-wide coverage, reviewer validation, or market validation. +- `production_activation` remains `False`; `readiness_promotions` remains `()`. +- Do not run `make readiness`, broad refresh, apply, or generated report commands. +- Do not create or modify CSV, JSON, report, sample-report, screenshot, timing, canonical-data, or manual-review artifacts. +- Unit tests use minimal in-memory SEC-shaped fixtures and make no network request. +- Stage exact intentional paths only; never use `git add -A`. +- Push only `codex/personal-research-mode-mvp`; keep PR #113 draft; do not merge or deploy. + +--- + +### Task 1: Pure Exact-Accession SEC Parser + +**Files:** +- Create: `src/sec_quarterly_cash_generation_pilot.py` +- Create: `tests/test_sec_quarterly_cash_generation_pilot.py` + +**Interfaces:** +- Consumes: `ticker: str`, `cik: str`, `fiscal_period: str`, `period_start_date: str`, `period_end_date: str`, `accession: str`, in-memory Companyfacts/submissions/filing HTML payloads, `retrieved_at: str`, `as_of: str`, and an explicit source-rights registry. +- Produces: frozen `SecQuarterlyPilotExtraction`, frozen `SecQuarterlyPilotPreview`, `extract_sec_quarterly_cash_generation(...)`, and `preview_sec_quarterly_cash_generation(...)`. + +- [ ] **Step 1: Write the first failing exact-quarter test** + +Create minimal Companyfacts, submissions, and inline-XBRL fixtures in `tests/test_sec_quarterly_cash_generation_pilot.py`. Use the real concept names and NVIDIA-shaped values while labeling the payload as a test fixture: + +```python +def test_exact_q1_payload_builds_source_backed_components_and_revenue(): + result = extract_sec_quarterly_cash_generation( + ticker="NVDA", + cik="0001045810", + fiscal_period="2027-Q1", + period_start_date="2026-01-26", + period_end_date="2026-04-26", + accession="0001045810-26-000052", + companyfacts_payload=_companyfacts_fixture(), + submissions_payload=_submissions_fixture(), + filing_html=_filing_fixture(capex_outflow=True), + retrieved_at="2026-07-20T15:00:00+00:00", + as_of="2026-07-20T15:00:00+00:00", + ) + + assert result.blockers == () + assert result.accepted_at == "2026-05-20T20:35:52+00:00" + assert [row.metric for row in result.observations] == [ + "operating_income", + "cash_from_operations", + "capital_expenditures", + ] + assert [row.value for row in result.observations] == [ + 53_536_000_000.0, + 50_344_000_000.0, + -1_757_000_000.0, + ] + assert result.revenue_actuals[0].revenue_actual == 81_615_000_000.0 + assert result.capex_sign_evidence == "explicit_filed_table_outflow" + assert result.source_url.endswith("/nvda-20260426.htm") +``` + +Fixture facts must use `form="10-Q"`, `fy=2027`, `fp="Q1"`, unit `USD`, the exact accession/start/end dates, and unique inline fact IDs. The submissions fixture must expose `acceptanceDateTime="2026-05-20T20:35:52.000Z"` for the same accession. + +- [ ] **Step 2: Run the exact-quarter test and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_sec_quarterly_cash_generation_pilot.py::test_exact_q1_payload_builds_source_backed_components_and_revenue -q +``` + +Expected: test collection fails because `src.sec_quarterly_cash_generation_pilot` does not exist. + +- [ ] **Step 3: Add frozen result types and exact concept maps** + +Create `src/sec_quarterly_cash_generation_pilot.py` with these public shapes: + +```python +@dataclass(frozen=True) +class SecQuarterlyPilotExtraction: + ticker: str + cik: str + fiscal_period: str + period_start_date: str + period_end_date: str + accession: str + filing_date: str + accepted_at: str + source_url: str + observations: tuple[QuarterlyBusinessObservation, ...] + revenue_actuals: tuple[QuarterlyActual, ...] + capex_sign_evidence: str + blockers: tuple[str, ...] + + +@dataclass(frozen=True) +class SecQuarterlyPilotPreview: + extraction: SecQuarterlyPilotExtraction + acceptance: QuarterlyAdapterAcceptance | None + status: str + blockers: tuple[str, ...] + production_activation: bool = False + readiness_promotions: tuple[str, ...] = () +``` + +Use these ordered concept maps: + +```python +REVENUE_CONCEPTS = ( + "RevenueFromContractWithCustomerExcludingAssessedTax", + "Revenues", + "SalesRevenueNet", + "SalesRevenueGoodsNet", +) +OPERATING_INCOME_CONCEPTS = ("OperatingIncomeLoss",) +CASH_FROM_OPERATIONS_CONCEPTS = ( + "NetCashProvidedByUsedInOperatingActivities", + "NetCashProvidedByUsedInOperatingActivitiesContinuingOperations", +) +CAPEX_CONCEPTS = ( + "PaymentsToAcquireProductiveAssets", + "PaymentsToAcquirePropertyPlantAndEquipment", +) +``` + +- [ ] **Step 4: Implement exact Companyfacts and submissions selection** + +Add helpers that: + +1. inspect only `facts.us-gaap..units.USD`; +2. retain rows matching exact accession, start, end, `10-Q`/`10-Q/A`, fiscal year, and fiscal-quarter code; +3. choose the first ordered concept containing one unique exact value; +4. return `:fact_missing` or `:fact_ambiguous` rather than guessing; +5. locate the exact accession in aligned `filings.recent` arrays; +6. require matching form, filing date, primary document, and timezone-aware `acceptanceDateTime`; +7. block `acceptance_after_cutoff` when the SEC acceptance time exceeds `as_of`. + +Use `parse_utc_timestamp` for both `accepted_at` and cutoff. Normalize the SEC `Z` time through that existing function. Do not synthesize a time from `filingDate`. + +- [ ] **Step 5: Implement inline XBRL context and row parsing** + +Use only `html.parser.HTMLParser`. Track: + +- each `xbrli:context` ID and its `xbrli:startdate`/`xbrli:enddate`; +- each table row's ordered text tokens; +- every `ix:nonfraction` fact's `name`, `contextref`, `scale`, `id`, and displayed magnitude; and +- the nearest non-empty text token before and after each fact. + +An inline fact matches only when concept, exact context start/end, absolute scaled magnitude, and primary filing all match. For capex, require the immediately surrounding displayed tokens to be `(` and `)`. Return `capital_expenditures:explicit_outflow_evidence_missing` if magnitude exists but that presentation proof does not. + +Construct source references as `f"{source_url}#{inline_fact_id}"`. Create three `QuarterlyBusinessObservation` values with source `sec_companyfacts`, currency `USD`, scale `1.0`, accounting basis `reported`, duration basis `three_months`, exact acceptance time as `published_at`, exact retrieval time, and `q4_evidence_state="not_q4"`. Create one Revenue-only `QuarterlyActual` with the same definition and source lineage. + +- [ ] **Step 6: Run the exact-quarter test and verify GREEN** + +Run the Step 2 command. Expected: 1 passed. + +- [ ] **Step 7: Add failing sign, identity, ambiguity, and temporal tests** + +Add parameterized tests asserting these exact blockers: + +```python +@pytest.mark.parametrize( + ("mutation", "blocker"), + [ + ("missing_parentheses", "capital_expenditures:explicit_outflow_evidence_missing"), + ("wrong_accession", "operating_income:fact_missing"), + ("wrong_context", "capital_expenditures:inline_fact_missing"), + ("wrong_magnitude", "capital_expenditures:inline_fact_missing"), + ("ytd_context", "operating_income:fact_missing"), + ("duplicate_value", "cash_from_operations:fact_ambiguous"), + ("missing_acceptance", "submissions:accession_missing"), + ("naive_acceptance", "submissions:acceptance_time_invalid"), + ("post_cutoff", "acceptance_after_cutoff"), + ], +) +def test_extraction_failures_are_deterministic(mutation, blocker): + result = _extract_with_mutation(mutation) + assert blocker in result.blockers + assert result.observations == () + assert result.revenue_actuals == () +``` + +Add a Q4/YTD test that requests `2027-Q4` with a nine-month fact and asserts `q4_explicit_three_month_filing_required`. Add an unsupported currency test that asserts `:usd_fact_missing`. Add a filing primary-document mismatch test that asserts `filing:primary_document_mismatch`. + +- [ ] **Step 8: Run the failure tests and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_sec_quarterly_cash_generation_pilot.py -q +``` + +Expected: new cases fail until blocker aggregation and fail-closed empty outputs are implemented. + +- [ ] **Step 9: Complete deterministic fail-closed extraction** + +Aggregate safely knowable blockers in insertion order with `tuple(dict.fromkeys(blockers))`. If any blocker exists, return empty observations and Revenue actuals, `capex_sign_evidence="blocked"`, and no partially usable values. Reject Q4 unless an exact three-month Companyfacts context and matching filed-table context exist; never difference annual or YTD values. + +- [ ] **Step 10: Add and satisfy preview-composition tests** + +Add: + +```python +def test_preview_composes_existing_acceptance_without_activation(): + preview = preview_sec_quarterly_cash_generation( + extraction=_successful_extraction(), + rights_registry=_sec_rights_fixture(include_cash_fields=True), + as_of="2026-07-20T15:00:00+00:00", + ) + + assert preview.status == "accepted_for_review" + assert preview.blockers == () + assert preview.acceptance is not None + assert preview.acceptance.accepted_observation_count == 3 + assert preview.production_activation is False + assert preview.readiness_promotions == () + + +def test_extraction_or_rights_blocker_never_returns_accepted_preview(): + extraction_blocked = preview_sec_quarterly_cash_generation( + extraction=_blocked_extraction(), + rights_registry=_sec_rights_fixture(include_cash_fields=True), + ) + rights_blocked = preview_sec_quarterly_cash_generation( + extraction=_successful_extraction(), + rights_registry=_sec_rights_fixture(include_cash_fields=False), + ) + assert extraction_blocked.acceptance is None + assert extraction_blocked.status == "blocked" + assert rights_blocked.status == "blocked" + assert "source_fields_missing:" in " ".join(rights_blocked.blockers) +``` + +Run: + +```bash +python3 -m pytest tests/test_sec_quarterly_cash_generation_pilot.py tests/test_quarterly_cash_generation_adapter.py tests/test_quarterly_cash_generation.py -q +``` + +Expected: all focused contract tests pass. + +- [ ] **Step 11: Commit the pure parser slice** + +```bash +git add -- src/sec_quarterly_cash_generation_pilot.py tests/test_sec_quarterly_cash_generation_pilot.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Parse exact SEC quarterly cash evidence" +git push origin codex/personal-research-mode-mvp +``` + +--- + +### Task 2: Explicit SEC Field-Scope Review + +**Files:** +- Modify: `config/source_rights.yml:12-16` +- Modify: `tests/test_quarterly_cash_generation_adapter.py` +- Modify: `tests/test_sec_quarterly_cash_generation_pilot.py` + +**Interfaces:** +- Consumes: existing exact-source `commercial_eligibility` and `REQUIRED_SOURCE_FIELDS` contracts. +- Produces: explicit checked-in `sec_companyfacts` support for `operating_income`, `cash_from_operations`, and `capital_expenditures`; no change to another source or to commercial-use status. + +- [ ] **Step 1: Change the existing blocker test to require exact SEC acceptance** + +Replace `test_checked_in_sec_rights_do_not_silently_claim_component_support` with: + +```python +def test_checked_in_sec_rights_explicitly_support_reviewed_cash_components(): + rows = [ + _observation( + metric=row.metric, + value=row.value, + source="sec_companyfacts", + source_ref=f"https://www.sec.gov/Archives/edgar/data/1/example.htm#{row.metric}", + ) + for row in _complete_rows() + ] + + result = _assess( + rows, + source_id="sec_companyfacts", + rights_registry=load_source_rights_registry(), + ) + + assert result.status == "accepted_for_review" + assert result.rights_status == "approved" + assert result.production_activation is False + assert result.readiness_promotions == () +``` + +- [ ] **Step 2: Run the changed test and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_quarterly_cash_generation_adapter.py::test_checked_in_sec_rights_explicitly_support_reviewed_cash_components -q +``` + +Expected: FAIL with the three existing `source_fields_missing` values. + +- [ ] **Step 3: Extend only the exact SEC supported-field list** + +Add these literal entries under `sec_companyfacts.supported_fields`: + +```yaml + - operating_income + - cash_from_operations + - capital_expenditures +``` + +Do not change `commercial_use`, redistribution, storage, attribution, authentication, rate limits, fallback priority, or any other source. + +- [ ] **Step 4: Prove scope independence and verify GREEN** + +Add a test that builds a separate approved registry missing `capital_expenditures` and asserts acceptance remains blocked with `source_fields_missing:capital_expenditures`. Run: + +```bash +python3 -m pytest tests/test_quarterly_cash_generation_adapter.py tests/test_commercial_source_rights.py tests/test_sec_quarterly_cash_generation_pilot.py -q +``` + +Expected: all focused rights and pilot tests pass. + +- [ ] **Step 5: Commit the field-scope decision** + +```bash +git add -- config/source_rights.yml tests/test_quarterly_cash_generation_adapter.py tests/test_sec_quarterly_cash_generation_pilot.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Review SEC cash generation field scope" +git push origin codex/personal-research-mode-mvp +``` + +--- + +### Task 3: Read-Only Live Preview Command + +**Files:** +- Create: `src/sec_quarterly_cash_generation_preview.py` +- Create: `tests/test_sec_quarterly_cash_generation_preview.py` +- Modify: `Makefile` + +**Interfaces:** +- Consumes: Task 1 parser/preview, checked-in rights registry, exact SEC Companyfacts URL, exact SEC submissions URL, and exact accession primary-document URL. +- Produces: `fetch_sec_quarterly_pilot_payloads(...)`, `render_sec_quarterly_pilot_preview(...)`, `main(argv=None) -> int`, and read-only Make target `sec-quarterly-cash-preview`. + +- [ ] **Step 1: Write failing client and renderer tests** + +Create `tests/test_sec_quarterly_cash_generation_preview.py` with an injected fetcher that records URLs and returns in-memory bytes. Assert: + +```python +def test_client_fetches_only_three_exact_sec_endpoints_without_cache(tmp_path, monkeypatch): + seen = [] + payloads = fetch_sec_quarterly_pilot_payloads( + cik="0001045810", + accession="0001045810-26-000052", + primary_document="nvda-20260426.htm", + user_agent="Research Test test@example.com", + fetcher=_recording_fetcher(seen), + ) + assert seen == [ + "https://data.sec.gov/api/xbrl/companyfacts/CIK0001045810.json", + "https://data.sec.gov/submissions/CIK0001045810.json", + "https://www.sec.gov/Archives/edgar/data/1045810/000104581026000052/nvda-20260426.htm", + ] + assert not list(tmp_path.iterdir()) + assert set(payloads) == {"companyfacts", "submissions", "filing_html"} + + +def test_renderer_is_human_readable_and_keeps_non_activation_visible(): + text = render_sec_quarterly_pilot_preview(_accepted_preview()) + assert "status: accepted_for_review" in text + assert "NVIDIA Q1 FY2027" in text + assert "capex sign evidence: explicit_filed_table_outflow" in text + assert "production activation: false" in text + assert "readiness promotions: none" in text + assert "generated artifacts: none" in text + assert not text.lstrip().startswith("{") +``` + +- [ ] **Step 2: Run the preview tests and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_sec_quarterly_cash_generation_preview.py -q +``` + +Expected: collection fails because the preview module does not exist. + +- [ ] **Step 3: Implement the no-cache SEC client** + +Use `urllib.request.Request`/`urlopen`, 30-second timeout, `User-Agent`, `Accept`, and no filesystem API. Require a non-blank explicit argument or `SEC_USER_AGENT`; otherwise raise the existing `SECUserAgentError`. Decode Companyfacts and submissions through `json.loads`; keep filing HTML as text. Convert HTTP, URL, decode, and JSON failures into one stable `SecQuarterlyPreviewFetchError` naming only the failed endpoint class, not credentials. + +- [ ] **Step 4: Implement deterministic rendering and CLI arguments** + +The CLI must require or default to these exact pilot values: + +```text +--ticker NVDA +--cik 0001045810 +--fiscal-period 2027-Q1 +--period-start 2026-01-26 +--period-end 2026-04-26 +--accession 0001045810-26-000052 +--primary-document nvda-20260426.htm +--as-of +``` + +The command loads `config/source_rights.yml`, fetches the exact three endpoints, parses and assesses them, prints only the human renderer, and returns `0` for `accepted_for_review` or `2` for blocked/fetch-error states. It must expose no output-file, JSON, apply, activation, refresh, readiness, or fallback argument. + +- [ ] **Step 5: Add the read-only Make target and prove its surface** + +Add: + +```make +.PHONY: sec-quarterly-cash-preview +sec-quarterly-cash-preview: + @python3 -m src.sec_quarterly_cash_generation_preview \ + --ticker "$(or $(TICKER),NVDA)" \ + --cik "$(or $(CIK),0001045810)" \ + --fiscal-period "$(or $(FISCAL_PERIOD),2027-Q1)" \ + --period-start "$(or $(PERIOD_START),2026-01-26)" \ + --period-end "$(or $(PERIOD_END),2026-04-26)" \ + --accession "$(or $(ACCESSION),0001045810-26-000052)" \ + --primary-document "$(or $(PRIMARY_DOCUMENT),nvda-20260426.htm)" \ + --as-of "$(AS_OF)" +``` + +Add a test requiring `AS_OF` and scanning the module/target for the absence of `write_text`, `open(`, `Path(`, `--output`, `make readiness`, `apply`, cache paths, and provider fallbacks. + +- [ ] **Step 6: Verify preview behavior with injected payloads** + +Run: + +```bash +python3 -m pytest tests/test_sec_quarterly_cash_generation_preview.py tests/test_sec_quarterly_cash_generation_pilot.py tests/test_sec_companyfacts.py -q +``` + +Expected: all preview, parser, and SEC provider regression tests pass without network access. + +- [ ] **Step 7: Run the one permitted live NVIDIA preview** + +Run only after confirming `SEC_USER_AGENT` is configured: + +```bash +make sec-quarterly-cash-preview AS_OF=2026-07-20T23:59:59-04:00 +``` + +Expected evidence: + +```text +status: accepted_for_review +accession: 0001045810-26-000052 +accepted at: 2026-05-20T20:35:52+00:00 +operating income: 53536000000.0 USD +cash from operations: 50344000000.0 USD +capital expenditures: -1757000000.0 USD +capex sign evidence: explicit_filed_table_outflow +production activation: false +readiness promotions: none +generated artifacts: none +``` + +Immediately run `git status --short` and `make diff-hygiene-summary`. Any newly written generated artifact is a failure; do not stage it. + +- [ ] **Step 8: Commit the live-preview capability** + +```bash +git add -- src/sec_quarterly_cash_generation_preview.py tests/test_sec_quarterly_cash_generation_preview.py Makefile +make staged-hygiene-check +git diff --cached --check +git commit -m "Add read-only SEC cash generation preview" +git push origin codex/personal-research-mode-mvp +``` + +--- + +### Task 4: Documentation, Roadmap, Continuation, And PR Evidence + +**Files:** +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/DATA_STRATEGY.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes: verified parser, field-scope, command, and live-preview evidence from Tasks 1–3. +- Produces: truthful public/internal boundaries, next-stage instructions, and draft PR #113 evidence. + +- [ ] **Step 1: Write the failing documentation contract test** + +Add a test asserting all of these literal distinctions: + +```python +def test_sec_cash_generation_pilot_docs_preserve_review_boundary(): + methodology = _read("docs/METHODOLOGY.md") + provenance = _read("docs/PROVENANCE_CONTRACT.md") + strategy = _read("docs/DATA_STRATEGY.md") + personal = _read("docs/PERSONAL_RESEARCH_MODE.md") + roadmap = _read("ROADMAP.md") + prompt = _read("docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md") + + assert "explicit_filed_table_outflow" in methodology + assert "acceptanceDateTime" in provenance + assert "sec_companyfacts" in strategy + assert "accepted_for_review is not production activation" in personal + assert "NVIDIA Q1 FY2027" in roadmap + assert "does not activate Company Workbench" in roadmap + assert "sec-quarterly-cash-preview" in prompt + assert "do not repeat the NVIDIA pilot" in prompt +``` + +- [ ] **Step 2: Run the documentation test and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py::test_sec_cash_generation_pilot_docs_preserve_review_boundary -q +``` + +Expected: failure because the pilot evidence and continuation boundary are not documented. + +- [ ] **Step 3: Update methodology, provenance, and source strategy** + +Document the exact-quarter/accession selection, exact submissions acceptance timestamp, inline-XBRL fact/context/magnitude match, filed-table parentheses requirement, negative capex only after explicit outflow proof, and existing FCF formula. State that the source-rights edit is a field-scope review under the existing SEC record, not a new legal opinion or entitlement. + +- [ ] **Step 4: Update user guidance and ROADMAP** + +Record NVIDIA Q1 FY2027 as one official-source adapter preview accepted for review. State exactly that it does not activate Company Workbench, rebuild readiness, prove other companies/quarters, supply point-in-time consensus, calibrate probabilities, establish hosted reliability, replace an independent reviewer, or validate product-market fit. + +Move the quarterly adapter roadmap state from `external_source_and_review_required` to `one_company_source_preview_accepted_for_review`; retain separate `production_activation_required` and `broader_coverage_unproven` gates. + +- [ ] **Step 5: Update the persistent continuation prompt** + +Set expected HEAD to the latest verified descendant. Add the completed pilot evidence and the stop rule `do not repeat the NVIDIA pilot unless its source filing changes or a regression is suspected`. Set the exact next executable stage to a separately designed Company Workbench activation preview or a second-company portability proof; do not authorize either automatically, generated writes, readiness rebuild, broad refresh, consensus fabrication, or hosted deployment. + +- [ ] **Step 6: Run documentation and focused product checks** + +Run: + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py tests/test_sec_quarterly_cash_generation_pilot.py tests/test_sec_quarterly_cash_generation_preview.py tests/test_quarterly_cash_generation_adapter.py tests/test_quarterly_cash_generation.py tests/test_sec_companyfacts.py -q +make public-wording-check +git diff --check +``` + +Expected: all checks pass. + +- [ ] **Step 7: Run the complete non-writing verification matrix** + +Run in this order: + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +make pr-range-hygiene-check BASE_SHA=origin/main HEAD_SHA=HEAD +git diff --check +``` + +Do not run `make readiness`. If a gate writes an unexpected generated artifact, stop, classify it, and keep it unstaged. + +- [ ] **Step 8: Stage exact documentation and commit** + +```bash +git add -- docs/METHODOLOGY.md docs/PROVENANCE_CONTRACT.md docs/DATA_STRATEGY.md docs/PERSONAL_RESEARCH_MODE.md ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Document SEC cash generation pilot evidence" +git push origin codex/personal-research-mode-mvp +``` + +- [ ] **Step 9: Update draft PR #113 without changing draft state** + +Confirm exact-head PR state first: + +```bash +gh pr view 113 --json number,state,isDraft,mergeable,headRefName,headRefOid,statusCheckRollup,url +``` + +Update the PR body with a concise section naming the three source endpoints, NVIDIA accession/period, exact accepted values, capex-sign evidence, focused/full check results, zero generated artifacts, and remaining activation/coverage/consensus/calibration/hosted/reviewer/market gates. Keep `isDraft=true`; do not merge. + +- [ ] **Step 10: Final repository truth check** + +Run: + +```bash +git status --short --branch +git rev-list --left-right --count origin/codex/personal-research-mode-mvp...HEAD +gh pr view 113 --json state,isDraft,mergeable,headRefOid,statusCheckRollup,url +``` + +Expected: clean worktree, `0 0` branch alignment, PR open/draft, exact head equal to local HEAD, and no generated artifacts staged. diff --git a/docs/superpowers/plans/2026-07-21-company-workbench-answer-first-handoff.md b/docs/superpowers/plans/2026-07-21-company-workbench-answer-first-handoff.md new file mode 100644 index 000000000..233a0e033 --- /dev/null +++ b/docs/superpowers/plans/2026-07-21-company-workbench-answer-first-handoff.md @@ -0,0 +1,464 @@ +# Company Workbench Answer-First Handoff Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Put the truthful selected-company answer and ticker-specific Data Health handoff inside the first `390x844` Company Workbench viewport without changing readiness, evidence, or report content. + +**Architecture:** Extend the existing research workspace header with an opt-in compact form used only by Company Workbench. Create a Streamlit answer placeholder directly after that header and pass it to `render_single_stock_report`, whose existing fast and final summary paths will render through one target-aware helper while all other callers keep the current direct-render behavior. + +**Tech Stack:** Python 3, Streamlit, pandas, pytest, HTML/CSS contracts, local browser QA, GitHub draft PR checks. + +## Global Constraints + +- Research-only; no investment advice or direct buy/sell instruction. +- No broker integration, order routing, auto-trading, or post-earnings price prediction. +- Candidate context cannot alter deterministic forecasts or become trusted evidence. +- Actuals, consensus, Revenue, EPS, operating margin, free cash flow, FCF margin, valuation, peer, catalyst, outcome, backtest, and calibration readiness remain independent. +- Missing evidence remains withheld. +- EPS split basis remains unverified without explicit proof. +- Q4 requires explicit SEC-filed three-month table evidence. +- Synthetic fixtures remain test-only. +- Empty valuation, catalyst, outcome, consensus, and field-proof ledgers remain empty. +- Do not run `make readiness` or introduce generated CSV, JSON, report, sample-report, screenshot, timing, or canonical-data artifacts. +- Keep PR #113 open and draft; push only `codex/personal-research-mode-mvp`; do not merge or deploy publicly. +- Stage exact intentional files only; never use `git add -A`. + +--- + +## File Map + +- `src/research_workspace.py`: builds the semantic Personal Research workspace header HTML and owns the opt-in compact-header contract. +- `src/dashboard.py`: owns header styling, Company Workbench route order, and fast/final selected-answer rendering. +- `src/public_performance_gate.py`: defines the route markers used by the non-writing performance contract. +- `tests/test_research_workspace.py`: verifies full and compact header semantics. +- `tests/test_research_mode_dashboard_contract.py`: verifies answer-slot placement, report wiring, compact route scope, and navigation order. +- `tests/test_dashboard_helpers.py`: verifies both fast and final summaries use the target-aware renderer without duplicate direct output. +- `tests/test_dashboard_render_smoke.py`: verifies one final selected answer and the ticker-preserving Data Health handoff in a rendered Workbench run. +- `tests/test_public_performance_gate.py`: verifies the Workbench gate uses the retained semantic page identity rather than the removed redundant heading. +- `src/dashboard_render_smoke.py`: updates the rendered-route marker to the retained `Company Workbench` heading if the smoke contract depends on `Selected Company`. +- `src/browser_qa_evidence.py` and `tests/test_browser_qa_evidence.py`: update only static Workbench first-view wording that refers to the removed heading; do not claim screenshot evidence from these contracts. +- `ROADMAP.md`: records the verified answer-first usability slice and preserves external blockers. +- `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`: advances exact-HEAD continuation truth and next executable lane. +- `docs/superpowers/specs/2026-07-21-company-workbench-answer-first-handoff-design.md`: changes the design status from pending review to approved and records implementation evidence after verification. + +## Execution Note + +Live desktop verification exposed one inaccurate design assumption: Personal Research mode did not load the Public-mode CSS that supplied the selected answer's multi-column layout. The unstyled answer remained semantic but rendered as a block. A test-first correction adds an explicit `research` summary class and scoped Personal Research desktop/phone styles in `src/dashboard.py`; it does not load the Public shell, change other routes, or alter answer data. The final phone layout is shorter than the unstyled baseline and keeps the Data Health link, stop condition, Review path, and lane-coverage control inside the first `390x844` viewport. + +### Task 1: Compact Workbench Header Contract + +**Files:** +- Modify: `src/research_workspace.py:632-656` +- Modify: `src/dashboard.py:34190-34335` +- Test: `tests/test_research_workspace.py:665-690` +- Test: `tests/test_research_mode_dashboard_contract.py:398-435` + +**Interfaces:** +- Consumes: `research_workspace_header_html(page_title, *, ticker, profile_label, freshness, primary_action)`. +- Produces: `research_workspace_header_html(..., compact: bool = False) -> str` and `render_research_workspace_header(..., compact: bool = False) -> None`. + +- [ ] **Step 1: Add the failing compact-header unit contract** + +```python +def test_compact_research_workspace_header_keeps_identity_scope_and_boundary_without_duplicate_meta(): + rendered = research_workspace_header_html( + "Company Workbench", + ticker="NVDA", + profile_label="Local Research", + freshness="Current through 2026-07-16", + primary_action="Review source-backed sections", + compact=True, + ) + + assert "research-workspace-header compact" in rendered + assert "

Company Workbench

" in rendered + assert "NVDA" in rendered + assert "Local Research" in rendered + assert "Research-only" in rendered + assert "investment advice" in rendered + assert "Current through 2026-07-16" not in rendered + assert "Review source-backed sections" not in rendered + assert "research-workspace-meta" not in rendered +``` + +- [ ] **Step 2: Run the focused header tests and verify the new test fails** + +Run: `python3 -m pytest tests/test_research_workspace.py -q` + +Expected: FAIL because `research_workspace_header_html` does not accept `compact`. + +- [ ] **Step 3: Implement the opt-in compact HTML contract** + +Add `compact: bool = False`. Build the opening class as `research-workspace-header compact` only when requested, and make the existing `
...
` an empty string in compact mode. Preserve the current output byte-for-byte for the default full header except for local expression assembly. + +```python +def research_workspace_header_html( + page_title: str, + *, + ticker: str = "", + profile_label: str, + freshness: str, + primary_action: str, + compact: bool = False, +) -> str: + scope = str(ticker or "Focused research scope").strip().upper() if ticker else "Focused research scope" + header_class = "research-workspace-header compact" if compact else "research-workspace-header" + meta_html = "" if compact else ( + "
" + f"
Freshness
{html.escape(str(freshness or 'Check saved readiness'))}
" + f"
Next action
{html.escape(str(primary_action or 'Review source-backed evidence'))}
" + "
" + ) + return ( + f"
" + "
" + "Personal research mode" + f"

{html.escape(str(page_title or 'Research Desk'))}

" + f"

{html.escape(scope)} · {html.escape(str(profile_label or 'Local research'))}

" + "
" + f"{meta_html}" + "

Research-only. Not investment advice; no trade instruction is produced.

" + "
" + ) +``` + +- [ ] **Step 4: Wire the dashboard wrapper and compact CSS without changing other routes** + +Add `compact: bool = False` to `render_research_workspace_header`, pass it to the pure helper, and add only these scoped rules to the existing style block: + +```css +.research-workspace-header.compact { padding: .72rem .9rem; margin-bottom: .55rem; } +.research-workspace-header.compact .research-workspace-boundary { margin-top: .35rem; } +``` + +The Company Workbench call will opt in during Task 2. Research Desk, Discover, Monitor, Data Health, and Proof History keep the default. + +- [ ] **Step 5: Run focused tests and verify they pass** + +Run: `python3 -m pytest tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py -q` + +Expected: PASS for the compact helper and all unchanged full-header contracts. + +### Task 2: Anchored Selected-Answer Renderer + +**Files:** +- Modify: `src/dashboard.py:30178-30420` +- Test: `tests/test_dashboard_helpers.py:30195-30245` +- Test: `tests/test_dashboard_helpers.py:30625-30650` + +**Interfaces:** +- Consumes: an optional Streamlit DeltaGenerator compatible with `.markdown(body, unsafe_allow_html=True)`. +- Produces: `render_single_stock_public_summary(frame: pd.DataFrame, *, research_mode: bool, selected_answer_target=None) -> None` and `render_single_stock_report(..., selected_answer_target=None) -> None`. + +- [ ] **Step 1: Add failing source contracts for fast and final target use** + +Update the existing direct-route summary tests to require: + +```python +assert "selected_answer_target=None" in report_signature +assert "render_single_stock_public_summary(" in report_chunk +assert "selected_answer_target=selected_answer_target" in report_chunk +assert report_chunk.count("single_stock_public_summary_html(") == 0 +``` + +Add an isolated helper contract using a fake target and monkeypatched `st.markdown`: + +```python +def test_single_stock_public_summary_uses_selected_target_when_supplied(monkeypatch): + calls = [] + target = SimpleNamespace(markdown=lambda body, **kwargs: calls.append(("target", body, kwargs))) + monkeypatch.setattr(dashboard.st, "markdown", lambda body, **kwargs: calls.append(("global", body, kwargs))) + + dashboard.render_single_stock_public_summary( + dashboard.single_stock_one_answer_frame({"ticker": "NVDA", "status": "partial"}), + research_mode=True, + selected_answer_target=target, + ) + + assert [kind for kind, _, _ in calls] == ["target"] + assert "mode=research&page=data-health&ticker=NVDA" in calls[0][1] +``` + +- [ ] **Step 2: Run the focused helper tests and verify the new contracts fail** + +Run: `python3 -m pytest tests/test_dashboard_helpers.py -q -k 'fast_public_single_stock or public_summary_uses_selected_target or simplified_review_sections'` + +Expected: FAIL because the target-aware helper and report parameter do not exist. + +- [ ] **Step 3: Add the target-aware renderer and optional report argument** + +Place this helper immediately after `single_stock_public_summary_html`: + +```python +def render_single_stock_public_summary( + frame: pd.DataFrame, + *, + research_mode: bool, + selected_answer_target=None, +) -> None: + rendered = single_stock_public_summary_html( + frame, + target_mode=RESEARCH_MODE if research_mode else "public", + ) + target = selected_answer_target if selected_answer_target is not None else st + target.markdown(rendered, unsafe_allow_html=True) +``` + +Add `selected_answer_target=None` to `render_single_stock_report`. Replace both the fast and final `st.markdown(single_stock_public_summary_html(...))` branches with: + +```python +render_single_stock_public_summary( + fast_answer_frame, + research_mode=research_mode, + selected_answer_target=selected_answer_target, +) +``` + +and: + +```python +render_single_stock_public_summary( + single_answer_frame, + research_mode=research_mode, + selected_answer_target=selected_answer_target, +) +``` + +- [ ] **Step 4: Run focused helper tests and verify they pass** + +Run: `python3 -m pytest tests/test_dashboard_helpers.py -q -k 'fast_public_single_stock or public_summary_uses_selected_target or simplified_review_sections'` + +Expected: PASS; public/operator callers without a target still render through `st.markdown`. + +### Task 3: Company Workbench Answer-First Route + +**Files:** +- Modify: `src/dashboard.py:34404-34448` +- Test: `tests/test_research_mode_dashboard_contract.py:138-155` +- Test: `tests/test_research_mode_dashboard_contract.py:417-435` +- Test: `tests/test_dashboard_render_smoke.py` + +**Interfaces:** +- Consumes: `render_research_workspace_header(..., compact=True)` and `render_single_stock_report(..., selected_answer_target=selected_answer_target)`. +- Produces: one `st.empty()` answer slot declared before both collapsed controls; no redundant `Selected Company` heading. + +- [ ] **Step 1: Replace the obsolete order tests with failing answer-first contracts** + +```python +def test_company_workbench_anchors_answer_before_collapsed_navigation_and_passes_target_to_report(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + start = source.index("def render_company_workbench(") + end = source.index("\ndef main()", start) + workbench = source[start:end] + + header = workbench.index("render_research_workspace_header(") + target = workbench.index("selected_answer_target = st.empty()", header) + review = workbench.index('with st.expander("Review path", expanded=False):', target) + coverage = workbench.index('with st.expander("Advanced: selected-company lane coverage", expanded=False):', review) + report = workbench.index("render_single_stock_report(", coverage) + + assert header < target < review < coverage < report + assert "compact=True" in workbench[header:target] + assert "selected_answer_target=selected_answer_target" in workbench[report:] + assert 'st.markdown("### Selected Company")' not in workbench +``` + +Update the render smoke to assert the generated HTML contains exactly one `aria-label='Selected ticker answer'` or its double-quoted equivalent and the ticker-specific `?mode=research&page=data-health&ticker=AVGO` handoff. + +- [ ] **Step 2: Run route and smoke tests and verify the new order contract fails** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q` + +Expected: FAIL because the route has no answer placeholder, still includes the redundant heading, and does not pass a target. + +- [ ] **Step 3: Implement the approved Workbench placement** + +Change only the Workbench route: + +```python +render_research_workspace_header( + "Company Workbench", + context, + ticker=ticker, + primary_action="Review usable evidence, then record what remains uncertain", + compact=True, +) +selected_answer_target = st.empty() +section_names = [section["title"] for section in company_workbench_section_contract()] +with st.expander("Review path", expanded=False): + st.caption(" -> ".join(section_names[:-1])) +with st.expander("Advanced: selected-company lane coverage", expanded=False): + ... +render_single_stock_report( + ..., + selected_answer_target=selected_answer_target, +) +``` + +Do not move or alter `What Changed`, Business Trend, Valuation, Forward View, withheld states, Advanced evidence, or report data loading. + +- [ ] **Step 4: Run focused route, workspace, helper, and smoke tests** + +Run: `python3 -m pytest tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py -q` + +Expected: PASS. + +### Task 4: Performance And Browser-QA Contract Alignment + +**Files:** +- Modify: `src/public_performance_gate.py:135-155` +- Modify: `tests/test_public_performance_gate.py:35-80` +- Modify if required: `src/dashboard_render_smoke.py:95-115` +- Modify if required: `src/browser_qa_evidence.py:290-360` +- Modify if required: `tests/test_browser_qa_evidence.py` + +**Interfaces:** +- Consumes: retained semantic marker `Company Workbench` and selected-answer ARIA label. +- Produces: non-writing gates that no longer depend on the removed `Selected Company` heading. + +- [ ] **Step 1: Add failing marker expectations** + +Update the Workbench performance contract expectation to: + +```python +assert RESEARCH_ROUTE_SPECS[2].first_useful_marker == "Company Workbench" +assert RESEARCH_ROUTE_SPECS[2].full_markers[0] == "Company Workbench" +assert "Selected Company" not in RESEARCH_ROUTE_SPECS[2].full_markers +``` + +Update render-smoke/browser-QA static marker expectations only where they require the removed heading; retain all readiness and Advanced lane-coverage markers. + +- [ ] **Step 2: Run performance and browser-QA contract tests and verify they fail before source updates** + +Run: `python3 -m pytest tests/test_public_performance_gate.py tests/test_browser_qa_evidence.py -q` + +Expected: FAIL on obsolete `Selected Company` route markers. + +- [ ] **Step 3: Replace only obsolete markers in source contracts** + +In `RESEARCH_ROUTE_SPECS`, set the Workbench first-useful marker and first full marker to `Company Workbench`. Make the same narrow change in render-smoke or browser-QA contracts only if focused failures prove they rely on the removed heading. + +- [ ] **Step 4: Run focused gate tests and verify they pass** + +Run: `python3 -m pytest tests/test_public_performance_gate.py tests/test_browser_qa_evidence.py -q` + +Expected: PASS without generating timing or screenshot artifacts. + +### Task 5: Live Responsive Acceptance And Documentation + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `docs/superpowers/specs/2026-07-21-company-workbench-answer-first-handoff-design.md` + +**Interfaces:** +- Consumes: locally running Streamlit dashboard and current AVGO route. +- Produces: measured desktop and `390x844` acceptance evidence stored only under `/tmp`, plus current roadmap/continuation truth. + +- [ ] **Step 1: Run the dashboard locally without refreshing data** + +Run the existing dashboard launch command documented by the repository, using the current saved local artifacts only. Do not run readiness, broad refresh, source import, report generation, or timing artifact commands. + +Expected: the Company Workbench route opens at `/?mode=research&page=company-workbench&ticker=AVGO&open=1`. + +- [ ] **Step 2: Measure phone acceptance in the configured in-app browser** + +At `390x844`, record DOM bounding boxes for: + +```text +[aria-label="Selected ticker answer"] +the "Open Data Health" link inside that answer +the "Review path" details summary +the "Advanced: selected-company lane coverage" details summary +``` + +Expected: + +```text +selected_answer.top < review_path.top < lane_coverage.top +data_health_link.bottom <= 844 +document.documentElement.scrollWidth <= document.documentElement.clientWidth +exactly one selected-ticker answer +zero browser console errors +``` + +Save screenshots only under `/tmp/stock-research-workflow-audit-2026-07-22/`; never stage them. + +- [ ] **Step 3: Measure desktop acceptance** + +At `1280x720`, verify the selected-answer cards retain their existing multi-column layout, the Data Health handoff is visible, both navigation expanders follow the answer, and there is no horizontal overflow or browser error. + +- [ ] **Step 4: Make at most one scoped CSS correction if measurement fails** + +If the phone Data Health link extends below `844`, adjust only `.research-workspace-header.compact` padding/margin or the compact boundary margin. Do not hide the global profile strip, research boundary, withheld state, or Data Health handoff. Re-run focused tests and both viewport measurements after the correction. + +- [ ] **Step 5: Update product truth** + +In `ROADMAP.md`, record the measured answer-first Workbench improvement as locally verified while leaving consensus, calibration, source-rights, hosted preview, external reviewer, and operating gates incomplete. In the continuation prompt, record the new verified HEAD only after commit/push, keep readiness stale, forbid generated artifact churn, and set the exact next safe executable lane. Change the design status to `Approved and implemented` only after all acceptance checks pass. + +### Task 6: Full Verification, Exact Commit, Draft PR, And Exact-Head CI + +**Files:** +- Stage only the intentional code, test, plan, specification, roadmap, and continuation-prompt files changed by Tasks 1-5. +- Exclude all pre-existing generated CSV/report changes and all `/tmp` browser evidence. + +**Interfaces:** +- Consumes: verified local implementation and documentation. +- Produces: one coherent commit on `codex/personal-research-mode-mvp`, pushed draft PR #113 update, and exact-head CI evidence. + +- [ ] **Step 1: Run focused and full local verification** + +Run: + +```bash +python3 -m pytest tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py tests/test_browser_qa_evidence.py -q +python3 -m pytest tests -q +make dashboard-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make pr-range-hygiene-check +make diff-hygiene-summary +git diff --check +``` + +Expected: all commands pass. `pilot-readiness-check` must remain truthfully blocked/stale if current evidence is stale; that expected product state must not be converted into a false pass by rebuilding generated artifacts. + +- [ ] **Step 2: Review the exact diff and generated-artifact boundary** + +Run: + +```bash +git diff -- src/research_workspace.py src/dashboard.py src/public_performance_gate.py src/dashboard_render_smoke.py src/browser_qa_evidence.py tests ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md docs/superpowers/specs/2026-07-21-company-workbench-answer-first-handoff-design.md docs/superpowers/plans/2026-07-21-company-workbench-answer-first-handoff.md +git status --short +``` + +Expected: intentional product/test/docs changes plus exactly the pre-existing 18 unstaged generated CSV/report files; no generated file is staged. + +- [ ] **Step 3: Stage exact intentional files and run staged hygiene** + +Use `git add` with the explicit paths that actually changed. Never use a wildcard or `git add -A`. + +Run: `make staged-hygiene-check` + +Expected: PASS and staged diff contains no CSV, JSON, report, sample-report, screenshot, timing, or canonical-data artifact. + +- [ ] **Step 4: Commit and push the coherent slice** + +Run: + +```bash +git commit -m "Improve Workbench answer-first handoff" +git push origin codex/personal-research-mode-mvp +``` + +Expected: local HEAD equals `origin/codex/personal-research-mode-mvp`; generated working-tree changes remain unstaged. + +- [ ] **Step 5: Update draft PR #113 and wait for exact-head CI** + +Keep the PR draft. Add a concise evidence comment covering the approved design, responsive measurements, tests, generated-artifact exclusion, and external blockers. Wait for GitHub Actions on the exact pushed SHA and report the run URL and conclusion. Do not merge or deploy. + +- [ ] **Step 6: Final audit** + +Verify branch, HEAD, upstream divergence, staged area, PR draft/mergeability, exact-head CI, and the generated-artifact exclusion. Report the branch safe for draft review only if there are no unresolved Critical or Important engineering findings and every local acceptance check passed. diff --git a/docs/superpowers/plans/2026-07-21-company-workbench-task-arbitration.md b/docs/superpowers/plans/2026-07-21-company-workbench-task-arbitration.md new file mode 100644 index 000000000..f428e2e4b --- /dev/null +++ b/docs/superpowers/plans/2026-07-21-company-workbench-task-arbitration.md @@ -0,0 +1,348 @@ +# Company Workbench Task Arbitration Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Give Company Workbench one deterministic next research task while retaining lane-specific blockers as non-competing context. + +**Architecture:** Add a pure arbitration helper to `src/research_workspace.py` that combines the ticker-scoped change answer with the existing ordered Research Conclusion cards. Keep `src/dashboard.py` as a thin composer, and change only presentation labels in the existing conclusion and Forward View card builders. + +**Tech Stack:** Python 3.12, Streamlit composition helpers, pytest. + +## Global Constraints + +- Do not change readiness, evidence, source rights, forecasts, scenarios, providers, or data files. +- Unresolved source-backed change outranks the ordered Research Conclusion priority. +- Missing inputs return a neutral wait condition and never invent evidence. +- Technical evidence remains under Advanced. +- No investment advice, buy/sell wording, post-earnings price prediction, probability, broker action, or data mutation. +- Do not create or stage generated CSV, JSON, report, sample-report, screenshot, timing, readiness, canonical-data, or manual-review churn. + +## Final-review contract clarification + +- `company_change_answer` carries an explicit boolean `source_backed_eligible`, set only when `event.evidence_status == "source_backed"`; the arbiter never infers eligibility from routing state, text, source references, or mapping shape. +- `company_change_answer` independently carries `change_context_kind` with exactly `none`, `snapshot_only`, or `source_backed`. Dashboard badges map from this discriminator: empty queue to `no queued change`, snapshot-only context to `snapshot evidence only`, and source-backed context to `source-backed change`. +- Eligible `open`, `still_blocked`, and `intentionally_deferred` items preserve `review_now`, `wait_for_evidence`, and `monitor` routing respectively. Blocked and deferred items preserve their non-empty wait condition, with the existing queue fallbacks when absent. +- Snapshot-only items remain in change context without a source-backed badge and cannot outrank Research Conclusion. +- Malformed arbitration inputs fail closed to the exact neutral wait task as a whole; no later card is selected after an earlier malformed card. +- The focused normal AVGO AppTest render must have zero exceptions, a scoped no-queued-change Evidence Change card without a snapshot-only badge, exactly one `ONE NEXT TASK` marker whose card title is `Add peer mappings`, one `FORWARD-VIEW LANE UNBLOCK`, and no retired uppercase `NEXT RESEARCH TASK` kicker. + +--- + +### Task 1: Pure authoritative task arbitration + +**Files:** +- Modify: `tests/test_research_workspace.py` +- Modify: `src/research_workspace.py` + +**Interfaces:** +- Consumes: `company_change_answer: Mapping[str, object]` and `conclusion_cards: Iterable[Mapping[str, object]]`. +- Produces: `company_next_research_task(change_answer, conclusion_cards) -> dict[str, object]` with `title`, `body`, `state`, and `badges`; state is one of the existing research-routing states. + +- [ ] **Step 1: Write failing arbitration tests** + +Add the import and three tests: + +```python +from src.research_workspace import company_next_research_task + + +def test_company_next_research_task_prioritizes_unresolved_source_change(): + task = company_next_research_task( + {"state": "review_now", "next_task": "Review the filed evidence."}, + [{"title": "Add peer mappings", "body": "Peer context is partial.", "state": "wait_for_evidence", "badges": ["peers"]}], + ) + assert task == { + "title": "Review the filed evidence.", + "body": "Complete this source-backed evidence review before starting another research task.", + "state": "review_now", + "badges": ["source-backed change", "research-only"], + } + + +def test_company_next_research_task_uses_ordered_conclusion_priority_without_change(): + task = company_next_research_task( + {"state": "monitor", "next_task": "Continue the current review or wait."}, + [{"title": "Add peer mappings", "body": "Peer context is partial.", "badges": ["peers"]}], + ) + assert task["title"] == "Add peer mappings" + assert task["body"] == "Peer context is partial." + assert task["state"] == "wait_for_evidence" + assert task["badges"] == ["peers", "research-only"] + + +def test_company_next_research_task_fails_closed_to_neutral_wait(): + task = company_next_research_task({}, []) + assert task["title"] == "Wait for reviewed evidence or choose another company" + assert task["state"] == "wait_for_evidence" + assert task["badges"] == ["monitor", "research-only"] +``` + +- [ ] **Step 2: Run the tests and verify the intended failure** + +Run: + +```bash +python3 -m pytest \ + tests/test_research_workspace.py::test_company_next_research_task_prioritizes_unresolved_source_change \ + tests/test_research_workspace.py::test_company_next_research_task_uses_ordered_conclusion_priority_without_change \ + tests/test_research_workspace.py::test_company_next_research_task_fails_closed_to_neutral_wait -q +``` + +Expected: collection fails because `company_next_research_task` does not exist. + +- [ ] **Step 3: Implement the pure helper** + +Import `Iterable` and `Mapping` from `collections.abc`, then add: + +```python +def company_next_research_task( + change_answer: Mapping[str, object] | None, + conclusion_cards: Iterable[Mapping[str, object]] | None, +) -> dict[str, object]: + change = dict(change_answer or {}) + if str(change.get("state") or "").strip() == "review_now": + title = str(change.get("next_task") or "").strip() + if title: + return { + "title": title, + "body": "Complete this source-backed evidence review before starting another research task.", + "state": "review_now", + "badges": ["source-backed change", "research-only"], + } + + for raw_card in tuple(conclusion_cards or ()): + card = dict(raw_card or {}) + title = str(card.get("title") or "").strip() + if not title: + continue + badges = [str(value).strip() for value in tuple(card.get("badges") or ()) if str(value).strip()] + return { + "title": title, + "body": str(card.get("body") or "").strip(), + "state": ( + str(card.get("state") or "").strip() + if str(card.get("state") or "").strip() in RESEARCH_ROUTING_STATES + else "wait_for_evidence" + ), + "badges": list(dict.fromkeys([*badges, "research-only"])), + } + + return { + "title": "Wait for reviewed evidence or choose another company", + "body": "No source-backed change or executable company task is available. Do not infer one from missing data.", + "state": "wait_for_evidence", + "badges": ["monitor", "research-only"], + } +``` + +- [ ] **Step 4: Run the focused helper tests** + +Run the Step 2 command again. + +Expected: 3 passed. + +--- + +### Task 2: Remove competing next-task labels + +**Files:** +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_forward_view.py` +- Modify: `src/dashboard.py` +- Modify: `src/forward_view.py` + +**Interfaces:** +- Consumes: `stock_report_next_step_cards(...)` and `forward_view_cards(packet)`. +- Produces: first conclusion card kicker `RESEARCH PRIORITY` with explicit existing routing state; final Forward View card kicker `FORWARD-VIEW LANE UNBLOCK`. + +- [ ] **Step 1: Tighten presentation tests** + +Extend `test_stock_report_next_step_cards_route_to_fundamentals_then_peers_then_review` after each representative call: + +```python +assert cards[0]["kicker"] == "RESEARCH PRIORITY" +assert cards[0]["state"] == "wait_for_evidence" +assert all(card["kicker"] != "NEXT STEP" for card in cards) +``` + +Change `test_forward_view_rows_keep_technical_details_separate_and_research_only` to: + +```python +assert cards[-1]["kicker"] == "FORWARD-VIEW LANE UNBLOCK" +assert all(card["kicker"] != "NEXT RESEARCH TASK" for card in cards) +``` + +- [ ] **Step 2: Run the tests and verify two label failures** + +Run: + +```bash +python3 -m pytest \ + tests/test_dashboard_helpers.py::test_stock_report_next_step_cards_route_to_fundamentals_then_peers_then_review \ + tests/test_forward_view.py::test_forward_view_rows_keep_technical_details_separate_and_research_only -q +``` + +Expected: both tests fail on the old kickers. + +- [ ] **Step 3: Change only the two presentation labels** + +In every primary card branch inside `stock_report_next_step_cards`, replace: + +```python +"kicker": "NEXT STEP", +``` + +with: + +```python +"kicker": "RESEARCH PRIORITY", +``` + +Set `"state": "wait_for_evidence"` on the fix-price, stage/review-fundamentals, and add/review-peer branches. Set `"state": "review_now"` on the ETF/context-review and full-report-review branches. Do not introduce a new routing state. + +In the appended routing card inside `forward_view_cards`, replace: + +```python +"kicker": "NEXT RESEARCH TASK", +``` + +with: + +```python +"kicker": "FORWARD-VIEW LANE UNBLOCK", +``` + +- [ ] **Step 4: Run the focused presentation tests** + +Run the Step 2 command again. + +Expected: 2 passed. + +--- + +### Task 3: Compose one Workbench task and update contracts + +**Files:** +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `src/dashboard.py` +- Modify: `ROADMAP.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/DASHBOARD_QA.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: `company_next_research_task(change_answer, conclusion_cards)`. +- Produces: one `ONE NEXT TASK` card in Company Workbench, with Research Conclusion and Forward View no longer claiming separate next tasks. + +- [ ] **Step 1: Add a source contract test for thin composition** + +Add to `tests/test_research_mode_dashboard_contract.py`: + +```python +def test_company_workbench_uses_one_authoritative_task_arbitration(): + source = Path("src/dashboard.py").read_text(encoding="utf-8") + assert "company_next_research_task(" in source + assert '"kicker": "ONE NEXT TASK"' in source + assert '"title": str(authoritative_task["title"])' in source + assert source.count('"kicker": "ONE NEXT TASK"') == 1 +``` + +- [ ] **Step 2: Run the test and verify failure** + +Run: + +```bash +python3 -m pytest tests/test_research_mode_dashboard_contract.py::test_company_workbench_uses_one_authoritative_task_arbitration -q +``` + +Expected: fail because the dashboard does not call the helper. + +- [ ] **Step 3: Wire the helper into Company Workbench** + +Import `company_next_research_task` from `src.research_workspace`. In the Research Conclusion block, compute the conclusion cards once, render them, then compute the authoritative task: + +```python +conclusion_cards = stock_report_next_step_cards( + report_payload, + coverage if provider is not None and ticker else None, + peer_summary if provider is not None and ticker else None, +) +render_signal_cards(conclusion_cards, show_commands=False, variant="queue") +change_answer = company_change_answer(ticker, research_review_items) +authoritative_task = company_next_research_task(change_answer, conclusion_cards) +``` + +Render the existing `ONE NEXT TASK` card from the authoritative result: + +```python +{ + "kicker": "ONE NEXT TASK", + "title": str(authoritative_task["title"]), + "body": str(authoritative_task["body"]), + "badges": list(authoritative_task["badges"]), + "state": str(authoritative_task["state"]), + "command": "", +} +``` + +- [ ] **Step 4: Update product truth documentation** + +Add concise current-state notes stating: + +- Company Workbench now arbitrates one overall task. +- unresolved source-backed change wins; +- otherwise the existing ordered Research Conclusion priority wins; +- Forward View guidance is lane-specific, not a competing overall task; +- readiness and evidence states remain independent and unchanged. + +Update ROADMAP and the continuation prompt only with this verified local capability. Do not change any external dependency classification. + +- [ ] **Step 5: Run focused module and documentation tests** + +Run: + +```bash +python3 -m pytest \ + tests/test_research_workspace.py \ + tests/test_dashboard_helpers.py \ + tests/test_forward_view.py \ + tests/test_research_mode_dashboard_contract.py \ + tests/test_public_v1_release_docs.py -q +``` + +Expected: all pass. + +- [ ] **Step 6: Run the full verification matrix** + +Run: + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make commercial-beta-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +make pr-range-hygiene-check +git diff --check +``` + +Expected: all commands exit 0; pilot readiness may remain truthfully blocked only by its documented external/stale gates; no generated artifact appears. + +- [ ] **Step 7: Stage exact files and run staged hygiene** + +Stage only the two spec/plan files and exact intentional code, test, and documentation paths. Never use `git add -A`. Then run: + +```bash +make staged-hygiene-check +git diff --cached --check +``` + +Expected: both pass with zero generated or manual-review candidates. + +- [ ] **Step 8: Commit, push, and update draft PR #113** + +Commit the coherent verified slice, push only `codex/personal-research-mode-mvp`, add a concise verification update to PR #113, and keep it draft. Do not merge or deploy. diff --git a/docs/superpowers/plans/2026-07-21-proof-readiness-conflict-diagnosis.md b/docs/superpowers/plans/2026-07-21-proof-readiness-conflict-diagnosis.md new file mode 100644 index 000000000..da367cdde --- /dev/null +++ b/docs/superpowers/plans/2026-07-21-proof-readiness-conflict-diagnosis.md @@ -0,0 +1,978 @@ +# Proof-Readiness Conflict Diagnosis Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make proof-readiness reconciliation require explicit ticker-level historical change evidence and separately diagnose authoritative current blockers without inferring an unknowable historical cause. + +**Architecture:** Extend the existing read-only reconciliation module rather than introducing another command. Historical applicability is derived only from structured batch fields, current blocker diagnosis is derived only from saved current inputs, and the two axes are carried through immutable rows, summary counts, CLI/JSON output, and Advanced Proof History cards. + +**Tech Stack:** Python 3.12, dataclasses, pandas, pytest, Streamlit AppTest, Make, GitHub Actions. + +## Global Constraints + +- Current saved readiness remains authoritative. +- Only a valid supporting proof whose normalized `changed_tickers` explicitly contains the ticker is ticker-level supporting evidence. +- Do not parse narrative proof fields to infer source identity, rights, field scope, provenance, payload truth, or historical cause. +- Missing, malformed, placeholder, ambiguous, or scope-only proof fails closed. +- Preserve independent readiness for fundamentals, DCF, share count, price, peer mapping, and peer valuation inputs. +- Keep detailed diagnosis in the existing CLI/JSON contract and Advanced Proof History; do not change Research Desk, Discover, Company Workbench, or Monitor. +- Do not rebuild readiness or write canonical data, proof history, CSV, JSON, reports, screenshots, timing evidence, or bytecode artifacts. +- Keep the existing 18 generated-file modifications unstaged and uncommitted. +- Stage exact files only; never use `git add -A`. +- Keep PR #113 open and draft; push only `codex/personal-research-mode-mvp`. +- Preserve all research-only, no-investment-advice, no-trading, explicit-Q4, EPS split-basis, synthetic-fixture, candidate-context, consensus, and calibration boundaries. + +--- + +## File Structure + +- Modify `src/proof_readiness_reconciliation.py`: structured applicability, blocker diagnosis, immutable row/summary fields, loader, JSON, and text rendering. +- Modify `tests/test_proof_readiness_reconciliation.py`: red-green coverage for applicability, blocker diagnosis, output, input isolation, and read-only behavior. +- Modify `src/dashboard.py`: Advanced Proof History summaries only. +- Modify `tests/test_dashboard_helpers.py`: reconciliation-card contract and fixture updates. +- Modify `tests/test_dashboard_render_smoke.py` only if an existing constructor or render marker requires an explicit update. +- Modify `ROADMAP.md`: implemented result and current-snapshot counts. +- Modify `docs/OPERATOR_GUIDE.md`: two-axis interpretation and safe next-review boundary. +- Modify `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`: continuation rule and next maturity stage. +- Modify `tests/test_public_v1_release_docs.py`: durable documentation contract. + +--- + +### Task 1: Ticker-Level Applicability And Current Blocker Model + +**Files:** +- Modify: `tests/test_proof_readiness_reconciliation.py` +- Modify: `src/proof_readiness_reconciliation.py` + +**Interfaces:** +- Consumes: `ReviewedBatchProof.changed_tickers`, current ticker readiness, DCF readiness, peer readiness, and canonical fundamentals frames. +- Produces: new immutable row fields `proof_applicability`, `current_blocker_code`, `current_blocker_fields`, `current_blocker_detail`, `next_safe_review`, `historical_payload_status`, and `historical_evidence_limit`. +- Produces: new summary tuples `proof_applicability_counts` and `current_blocker_counts`. +- Preserves: existing state names, ordering, current-readiness authority, and lane independence. + +- [ ] **Step 1: Extend test helpers without changing production code** + +Update `_proof` so tests can independently set batch scope and explicit changed tickers: + +```python +def _proof( + *, + tickers: str = "ARCT", + changed_tickers: str | None = None, + lane: str = "fundamentals", + outcome: str = "auto_supported", + review_date: str = "2026-06-26", + batch_id: str = "RB-1", +) -> ReviewedBatchProof: + return ReviewedBatchProof( + batch_id=batch_id, + review_date=review_date, + reviewer="reviewer", + lane=lane, + scope="one reviewed scope", + tickers=tickers, + command_run="read-only fixture command", + validation_result="passed", + preview_result="reviewed", + apply_result="applied", + pre_run_readiness_snapshot="before", + post_run_readiness_snapshot="after", + changed_readiness_counts="one lane changed", + changed_tickers=tickers if changed_tickers is None else changed_tickers, + source_files="reviewed source", + generated_artifacts_reviewed="excluded", + final_outcome=outcome, + notes="fixture proof", + ) +``` + +Add a canonical fundamentals helper and pass it through `_summary`: + +```python +def _fundamentals(**rows: dict[str, str]) -> pd.DataFrame: + return pd.DataFrame([{"ticker": ticker, **values} for ticker, values in rows.items()]) + + +def _summary(*, proofs, ticker, dcf=None, peer=None, fundamentals=None): + return build_proof_readiness_reconciliation( + proofs=proofs, + ticker_readiness=ticker, + dcf_readiness=dcf if dcf is not None else pd.DataFrame(), + peer_readiness=peer if peer is not None else pd.DataFrame(), + fundamentals=fundamentals if fundamentals is not None else pd.DataFrame(), + ) +``` + +- [ ] **Step 2: Write failing applicability tests** + +Add these tests: + +```python +def test_scope_only_support_is_not_ticker_level_support(): + summary = _summary( + proofs=[_proof(tickers="ARCT,ARDX", changed_tickers="ARDX")], + ticker=_ticker_readiness( + ARCT={"fundamentals_ready": "False"}, + ARDX={"fundamentals_ready": "False"}, + ), + ) + + arct = _row(summary, "ARCT", "fundamentals") + ardx = _row(summary, "ARDX", "fundamentals") + + assert arct.proof_applicability == "scope_only_not_supported" + assert arct.state == "currently_blocked_with_non_supporting_history" + assert ardx.proof_applicability == "explicit_ticker_change" + assert ardx.state == "historical_supported_currently_blocked" + assert dict(summary.conflict_counts_by_lane) == {"fundamentals": 1} + + +@pytest.mark.parametrize( + "changed_tickers", + ["", "-", "none", "n/a", "not available", "unknown", "3289 changed tickers"], +) +def test_placeholder_changed_tickers_cannot_support(changed_tickers): + summary = _summary( + proofs=[_proof(changed_tickers=changed_tickers)], + ticker=_ticker_readiness(ARCT={"fundamentals_ready": "False"}), + ) + + row = _row(summary, "ARCT", "fundamentals") + assert row.proof_applicability == "missing_ticker_change_detail" + assert row.state == "currently_blocked_with_non_supporting_history" + + +def test_latest_non_supporting_proof_does_not_fall_back_to_older_explicit_support(): + summary = _summary( + proofs=[ + _proof(batch_id="RB-OLD", review_date="2026-06-25", changed_tickers="ARCT"), + _proof(batch_id="RB-NEW", review_date="2026-06-26", changed_tickers="-"), + ], + ticker=_ticker_readiness(ARCT={"fundamentals_ready": "False"}), + ) + + row = _row(summary, "ARCT", "fundamentals") + assert row.latest_batch_id == "RB-NEW" + assert row.proof_applicability == "missing_ticker_change_detail" + assert row.state == "currently_blocked_with_non_supporting_history" +``` + +Import `pytest` at the top of the test file. + +- [ ] **Step 3: Run the applicability tests and confirm RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_proof_readiness_reconciliation.py::test_scope_only_support_is_not_ticker_level_support \ + tests/test_proof_readiness_reconciliation.py::test_placeholder_changed_tickers_cannot_support \ + tests/test_proof_readiness_reconciliation.py::test_latest_non_supporting_proof_does_not_fall_back_to_older_explicit_support \ + -q +``` + +Expected: failures because the row has no `proof_applicability` field and `_summary` passes an unsupported `fundamentals` argument. + +- [ ] **Step 4: Write failing current-blocker tests** + +Add: + +```python +def test_missing_current_canonical_fundamentals_row_is_diagnosed_without_historical_cause_inference(): + summary = _summary( + proofs=[_proof()], + ticker=_ticker_readiness(ARCT={"fundamentals_ready": "False"}), + dcf=_dcf_readiness( + ARCT={ + "missing_dcf_fields": "free_cash_flow, shares_outstanding, revenue, fcf_margin, price" + } + ), + fundamentals=_fundamentals(ARDX={"source": "sec_companyfacts"}), + ) + + row = _row(summary, "ARCT", "fundamentals") + assert row.current_blocker_code == "current_canonical_row_missing" + assert row.current_blocker_fields == ( + "free_cash_flow", + "shares_outstanding", + "revenue", + "fcf_margin", + ) + assert row.historical_payload_status == "structured_payload_not_recorded" + assert "cannot distinguish" in row.historical_evidence_limit.lower() + assert "yfinance" not in row.current_blocker_detail.lower() + + +def test_incomplete_current_canonical_fundamentals_row_reports_exact_current_fields(): + summary = _summary( + proofs=[_proof()], + ticker=_ticker_readiness(ARCT={"fundamentals_ready": "False"}), + dcf=_dcf_readiness( + ARCT={"missing_dcf_fields": "free_cash_flow, revenue, fcf_margin, price"} + ), + fundamentals=_fundamentals( + ARCT={"shares_outstanding": "100", "source": "sec_companyfacts"} + ), + ) + + row = _row(summary, "ARCT", "fundamentals") + assert row.current_blocker_code == "current_required_fields_missing" + assert row.current_blocker_fields == ("free_cash_flow", "revenue", "fcf_margin") + assert "price" not in row.current_blocker_fields + + +def test_share_count_diagnosis_reports_only_shares_outstanding(): + summary = _summary( + proofs=[_proof(lane="share_count")], + ticker=_ticker_readiness(ARCT={}), + dcf=_dcf_readiness( + ARCT={ + "has_shares_outstanding": "False", + "missing_dcf_fields": "free_cash_flow, shares_outstanding, revenue, fcf_margin, price", + } + ), + ) + + row = _row(summary, "ARCT", "share_count") + assert row.current_blocker_code == "current_required_fields_missing" + assert row.current_blocker_fields == ("shares_outstanding",) + + +def test_price_and_peer_blockers_remain_independent(): + summary = _summary( + proofs=[ + _proof(lane="price_history", batch_id="RB-PRICE"), + _proof(lane="peer_mapping", batch_id="RB-PEER"), + _proof(lane="peer_valuation_inputs", batch_id="RB-PEER-VAL"), + ], + ticker=_ticker_readiness( + ARCT={"price_ready": "False", "peer_ready": "False"} + ), + peer=_peer_readiness(ARCT={"peer_valuation_ready": "False"}), + ) + + assert _row(summary, "ARCT", "price").current_blocker_code == "current_price_missing" + assert _row(summary, "ARCT", "peer_mapping").current_blocker_code == "current_peer_mapping_missing" + assert ( + _row(summary, "ARCT", "peer_valuation_inputs").current_blocker_code + == "current_peer_valuation_inputs_missing" + ) +``` + +Add an unavailable-input isolation test: + +```python +def test_missing_canonical_fundamentals_input_affects_only_dependent_diagnosis(): + summary = _summary( + proofs=[ + _proof(lane="fundamentals", batch_id="RB-FUND"), + _proof(lane="price", batch_id="RB-PRICE"), + _proof(lane="peer_mapping", batch_id="RB-PEER"), + ], + ticker=_ticker_readiness( + ARCT={ + "fundamentals_ready": "False", + "price_ready": "False", + "peer_ready": "False", + } + ), + dcf=_dcf_readiness( + ARCT={"missing_dcf_fields": "free_cash_flow, shares_outstanding, revenue, fcf_margin"} + ), + peer=_peer_readiness(ARCT={"peer_valuation_ready": "False"}), + fundamentals=pd.DataFrame(), + ) + + assert ( + _row(summary, "ARCT", "fundamentals").current_blocker_code + == "current_readiness_input_unavailable" + ) + assert _row(summary, "ARCT", "price").current_blocker_code == "current_price_missing" + assert _row(summary, "ARCT", "peer_mapping").current_blocker_code == "current_peer_mapping_missing" +``` + +- [ ] **Step 5: Run the blocker tests and confirm RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_proof_readiness_reconciliation.py \ + -k 'canonical or share_count_diagnosis or price_and_peer_blockers' -q +``` + +Expected: failures because the new blocker and evidence-limitation fields do not exist. + +- [ ] **Step 6: Add immutable contracts and constants** + +In `src/proof_readiness_reconciliation.py`, add: + +```python +PLACEHOLDER_TICKER_VALUES = frozenset({"-", "none", "n/a", "na", "not available", "unknown"}) +CANONICAL_DCF_FIELDS = ( + "free_cash_flow", + "shares_outstanding", + "revenue", + "fcf_margin", + "price", +) +FUNDAMENTALS_FIELDS = CANONICAL_DCF_FIELDS[:-1] +HISTORICAL_EVIDENCE_LIMIT = ( + "Historical batch proof cannot distinguish payload removal, readiness-contract change, " + "source-rights change, field-scope change, or another historical cause." +) +``` + +Extend `ProofReadinessReconciliationRow`: + +```python + proof_applicability: str + current_blocker_code: str + current_blocker_fields: tuple[str, ...] + current_blocker_detail: str + next_safe_review: str + historical_payload_status: str + historical_evidence_limit: str +``` + +Extend `ProofReadinessReconciliationSummary`: + +```python + proof_applicability_counts: tuple[tuple[str, int], ...] + current_blocker_counts: tuple[tuple[str, int], ...] +``` + +- [ ] **Step 7: Implement normalized explicit applicability** + +Replace `_proof_tickers` with a shared normalized-token helper that excludes placeholders and only retains current-universe tickers: + +```python +def _ticker_tokens(value: object, valid_tickers: set[str]) -> tuple[str, ...]: + tokens = (token.strip().upper() for token in re.split(r"[,;]", str(value or ""))) + return tuple( + dict.fromkeys( + token + for token in tokens + if token + and token.lower() not in PLACEHOLDER_TICKER_VALUES + and token in valid_tickers + ) + ) + + +def _proof_applicability( + latest_proof: _LatestProof | None, + *, + ticker: str, + valid_tickers: set[str], +) -> str: + if latest_proof is None: + return "no_applicable_proof" + if not latest_proof.review_date_valid: + return "malformed_review_date" + outcome = str(latest_proof.proof.final_outcome or "").strip().lower() + if outcome not in SUPPORTING_OUTCOMES: + return "non_supporting_outcome" + changed = _ticker_tokens(latest_proof.proof.changed_tickers, valid_tickers) + if ticker in changed: + return "explicit_ticker_change" + if not changed: + return "missing_ticker_change_detail" + return "scope_only_not_supported" +``` + +Use `_ticker_tokens` for scope matching. Set `supporting = proof_applicability == "explicit_ticker_change"` before calling `_reconciliation_state`. + +- [ ] **Step 8: Implement current blocker diagnosis** + +Add a frozen internal result: + +```python +@dataclass(frozen=True) +class _CurrentBlocker: + code: str + fields: tuple[str, ...] + detail: str + next_safe_review: str +``` + +Add helpers that: + +- parse `missing_dcf_fields` only into `CANONICAL_DCF_FIELDS` order; +- determine input availability from a nonempty normalized frame containing the required `ticker` and lane columns; +- return `current_readiness_input_unavailable` for fundamentals when the canonical fundamentals input is unavailable, rather than treating an unavailable file as an empty canonical universe; +- use the canonical fundamentals ticker set only to distinguish missing from incomplete payload; +- return `none` for current-ready lanes; +- return `current_readiness_input_unavailable` when the authoritative current field is `None`; +- return lane-specific codes exactly as specified in the design; +- never inspect proof notes, source files, or command text. + +Use this exact next-review copy: + +```python +NEXT_SAFE_REVIEW = { + "current_canonical_row_missing": ( + "Obtain and review a permitted source payload for the exact ticker before any import or readiness rebuild." + ), + "current_required_fields_missing": ( + "Review the named current fields through the existing source-review and preview-first workflow." + ), + "current_price_missing": ( + "Inspect the exact ticker's current price evidence without inferring a provider." + ), + "current_peer_mapping_missing": ( + "Review a source-backed relationship through the existing peer evidence contract." + ), + "current_peer_valuation_inputs_missing": ( + "Review current peer valuation inputs independently from mapping readiness." + ), + "current_readiness_input_unavailable": ( + "Restore or inspect the current saved input before drawing a conclusion." + ), + "none": "No current blocker is reported for this lane.", +} +``` + +For `scope_only_not_supported` or `missing_ticker_change_detail`, prefix the row's safe review with: + +```text +Review the proof row; do not reuse it as ticker-level support. +``` + +- [ ] **Step 9: Wire the canonical frame and summary counts** + +Change the builder signature: + +```python +def build_proof_readiness_reconciliation( + *, + proofs: Sequence[ReviewedBatchProof], + ticker_readiness: pd.DataFrame, + dcf_readiness: pd.DataFrame, + peer_readiness: pd.DataFrame, + fundamentals: pd.DataFrame, +) -> ProofReadinessReconciliationSummary: +``` + +In the loader, add: + +```python +fundamentals=_read_csv(data / "fundamentals.csv"), +``` + +Populate every new row field. For rows with an applicable proof, use: + +```python +historical_payload_status="structured_payload_not_recorded" +historical_evidence_limit=HISTORICAL_EVIDENCE_LIMIT +``` + +For rows with no proof, use empty strings for those two fields. + +Build deterministic summary counters: + +```python +proof_applicability_counts = Counter(row.proof_applicability for row in rows) +current_blocker_counts = Counter(row.current_blocker_code for row in rows) +``` + +Ensure the unavailable early return includes empty tuples for the new summary fields. + +- [ ] **Step 10: Run core tests and make them GREEN** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_proof_readiness_reconciliation.py -q +``` + +Expected: all core reconciliation tests pass. + +- [ ] **Step 11: Commit the core model** + +Run: + +```bash +git add -- src/proof_readiness_reconciliation.py tests/test_proof_readiness_reconciliation.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Diagnose proof readiness conflicts" +``` + +Expected: only the two named product/test files are committed; no generated artifacts are staged. + +--- + +### Task 2: CLI And JSON Two-Axis Evidence + +**Files:** +- Modify: `tests/test_proof_readiness_reconciliation.py` +- Modify: `src/proof_readiness_reconciliation.py` + +**Interfaces:** +- Consumes: Task 1 row and summary fields. +- Produces: stable JSON keys `proof_applicability_counts` and `current_blocker_counts`, plus new row fields through `asdict`. +- Produces: text sections `Proof applicability counts` and `Current blocker counts`. +- Preserves: current command name, ticker filtering, top-N behavior, existing payload keys, and read-only guarantee. + +- [ ] **Step 1: Write failing JSON and text tests** + +Add: + +```python +def test_payload_exposes_applicability_and_current_blocker_axes(): + summary = _summary( + proofs=[_proof(tickers="ARCT,ARDX", changed_tickers="ARDX")], + ticker=_ticker_readiness( + ARCT={"fundamentals_ready": "False"}, + ARDX={"fundamentals_ready": "False"}, + ), + dcf=_dcf_readiness( + ARCT={"missing_dcf_fields": "free_cash_flow, shares_outstanding, revenue, fcf_margin"}, + ARDX={"missing_dcf_fields": "free_cash_flow, shares_outstanding, revenue, fcf_margin"}, + ), + fundamentals=_fundamentals(), + ) + + payload = proof_readiness_reconciliation_payload(summary, top_n=20) + + assert payload["proof_applicability_counts"]["scope_only_not_supported"] == 1 + assert payload["proof_applicability_counts"]["explicit_ticker_change"] == 1 + assert payload["current_blocker_counts"]["current_canonical_row_missing"] == 2 + assert payload["rows"][0]["historical_payload_status"] == "structured_payload_not_recorded" + assert "historical cause" in payload["boundary"].lower() + + +def test_render_exposes_two_axes_without_claiming_historical_cause(): + summary = _summary( + proofs=[_proof()], + ticker=_ticker_readiness(ARCT={"fundamentals_ready": "False"}), + dcf=_dcf_readiness( + ARCT={"missing_dcf_fields": "free_cash_flow, shares_outstanding, revenue, fcf_margin"} + ), + fundamentals=_fundamentals(), + ) + + rendered = render_proof_readiness_reconciliation(summary, top_n=10) + + assert "Proof applicability counts:" in rendered + assert "Current blocker counts:" in rendered + assert "explicit_ticker_change" in rendered + assert "current_canonical_row_missing" in rendered + assert "Proof applicability | Current blocker | Next safe review" in rendered + assert "does not establish the historical cause" in rendered +``` + +- [ ] **Step 2: Run output tests and confirm RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_proof_readiness_reconciliation.py \ + -k 'payload_exposes_applicability or render_exposes_two_axes' -q +``` + +Expected: failures because the new summary keys and rendered sections are absent. + +- [ ] **Step 3: Extend JSON payload** + +Add to `proof_readiness_reconciliation_payload`: + +```python +"proof_applicability_counts": dict(summary.proof_applicability_counts), +"current_blocker_counts": dict(summary.current_blocker_counts), +``` + +Replace the boundary with: + +```python +"Current saved readiness remains authoritative; reconciliation does not restore data, promote readiness, " +"or rewrite proof history. Current blocker diagnosis describes observable saved inputs and does not establish " +"the historical cause." +``` + +- [ ] **Step 4: Extend text rendering** + +After state counts, render both deterministic summary sections. Change the table header to: + +```text +Ticker | Lane | Current ready | Latest proof | Review date | Reconciliation state | Proof applicability | Current blocker | Next safe review +``` + +Render `current_blocker_fields` in parentheses after the blocker code when nonempty. Keep the existing current-readiness and research-only boundaries, then add: + +```text +Boundary: current blocker diagnosis describes observable saved inputs; it does not establish the historical cause, source rights, field scope, provenance, payload truth, or commercial use. +``` + +- [ ] **Step 5: Verify CLI, JSON, filtering, and filesystem immutability** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_proof_readiness_reconciliation.py -q +make proof-readiness-reconciliation TOP_N=20 +make proof-readiness-reconciliation TICKERS=ARCT TOP_N=20 JSON=1 +git status --short +``` + +Expected: + +- focused tests pass; +- text and JSON expose both axes; +- current saved readiness remains authoritative; +- no file is created or modified by either command; +- only the pre-existing 18 generated modifications remain outside intentional product edits. + +- [ ] **Step 6: Commit CLI and JSON evidence** + +Run: + +```bash +git add -- src/proof_readiness_reconciliation.py tests/test_proof_readiness_reconciliation.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Expose proof conflict diagnosis" +``` + +--- + +### Task 3: Advanced Proof History Diagnosis + +**Files:** +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `src/dashboard.py` +- Inspect: `tests/test_dashboard_render_smoke.py`; modify it only when the focused RED run proves that an existing constructor or route marker must change + +**Interfaces:** +- Consumes: Task 1 summary rows and counts. +- Produces: answer-first global and selected-ticker evidence cards under Proof History. +- Preserves: empty card commands, raw-ledger collapse, primary route order, and current readiness authority. + +- [ ] **Step 1: Update the dashboard summary fixture** + +Extend `_proof_reconciliation_summary` row construction with: + +```python +proof_applicability="explicit_ticker_change", +current_blocker_code="current_canonical_row_missing" if conflict else "none", +current_blocker_fields=("free_cash_flow", "shares_outstanding", "revenue", "fcf_margin") if conflict else (), +current_blocker_detail="No current canonical fundamentals row is present." if conflict else "No current blocker.", +next_safe_review=( + "Obtain and review a permitted source payload for the exact ticker before any import or readiness rebuild." + if conflict + else "No current blocker is reported for this lane." +), +historical_payload_status="structured_payload_not_recorded", +historical_evidence_limit="Historical proof cannot distinguish the historical cause.", +``` + +Extend summary construction with: + +```python +proof_applicability_counts=(("explicit_ticker_change", 1),), +current_blocker_counts=(("current_canonical_row_missing" if conflict else "none", 1),), +``` + +- [ ] **Step 2: Write failing card tests** + +Extend the conflict-card test: + +```python +assert "current canonical row missing" in rendered +assert "observable current blocker" in rendered +assert "does not establish the historical cause" in rendered +assert "obtain and review a permitted source payload" in rendered +``` + +Add: + +```python +def test_proof_reconciliation_selected_card_does_not_infer_source_or_historical_cause(): + cards = dashboard.proof_readiness_reconciliation_cards( + _proof_reconciliation_summary(), + ticker="ARCT", + ) + selected = cards[1] + rendered = " ".join(str(value) for value in selected.values()).lower() + + assert "explicit ticker change" in rendered + assert "free cash flow" in rendered + assert "current canonical row missing" in rendered + assert "yfinance" not in rendered + assert "source rights changed" not in rendered + assert selected["command"] == "" +``` + +- [ ] **Step 3: Run dashboard tests and confirm RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py \ + -k 'proof_reconciliation' -q +``` + +Expected: failures because the cards do not yet show blocker/applicability fields. + +- [ ] **Step 4: Implement concise Advanced cards** + +In `proof_readiness_reconciliation_cards`: + +- find the largest non-`none` current blocker count from `summary.current_blocker_counts`; +- render labels by replacing underscores with spaces; +- keep the global conflict count unchanged; +- state: `Observable current blockers describe saved inputs; they do not establish the historical cause.`; +- for the selected ticker, combine only that ticker's conflict rows; +- show canonical lanes, applicability labels, blocker labels/fields, and the first deterministic safe next review; +- keep every command empty. + +Use this global body shape: + +```python +body = ( + "Historical support is not current readiness. Current saved readiness remains authoritative. " + f"Largest observable current blocker: {largest_blocker_label} ({largest_blocker_count:,}). " + "Observable current blockers describe saved inputs; they do not establish the historical cause, " + "restore data, promote readiness, or rewrite proof history." +) +``` + +Use this selected-ticker body shape: + +```python +body = ( + f"Current blocked lane(s): {lanes}. Proof applicability: {applicability}. " + f"Current blocker(s): {blockers}. Next safe review: {next_safe_review} " + "Reconciliation itself does not unlock the lane." +) +``` + +Do not add technical cards to any primary research route. + +- [ ] **Step 5: Run focused dashboard and route verification** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_dashboard_helpers.py \ + tests/test_dashboard_render_smoke.py \ + tests/test_research_mode_dashboard_contract.py -q +make research-dashboard-render-smoke +``` + +Expected: focused dashboard tests pass and all six Research Mode routes render. + +- [ ] **Step 6: Commit the Advanced UI slice** + +Run: + +```bash +git add -- src/dashboard.py tests/test_dashboard_helpers.py +git add -- tests/test_dashboard_render_smoke.py # only if intentionally changed +make staged-hygiene-check +git diff --cached --check +git commit -m "Show current blockers in Proof History" +``` + +Expected: no screenshot or generated artifact is staged. + +--- + +### Task 4: Methodology, Roadmap, Continuation Contract, And Release Verification + +**Files:** +- Modify: `tests/test_public_v1_release_docs.py` +- Modify: `ROADMAP.md` +- Modify: `docs/OPERATOR_GUIDE.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: verified Task 1-3 behavior and current-snapshot output. +- Produces: durable operator and continuation rules without turning snapshot counts into permanent claims. +- Preserves: draft PR, generated-artifact exclusion, external dependency boundaries, and release-gate wording. + +- [ ] **Step 1: Write the failing documentation contract** + +Extend `test_proof_readiness_reconciliation_docs_keep_historical_proof_separate_from_current_state`: + +```python +for text in (roadmap, operator, prompt): + assert "explicit_ticker_change" in text + assert "current_canonical_row_missing" in text + assert "does not establish the historical cause" in text.lower() + +assert "changed_tickers" in operator +assert "structured per-ticker" in prompt.lower() +assert "future" in roadmap.lower() +``` + +- [ ] **Step 2: Run the documentation test and confirm RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_public_v1_release_docs.py::test_proof_readiness_reconciliation_docs_keep_historical_proof_separate_from_current_state -q +``` + +Expected: failure because the new two-axis terms are not documented. + +- [ ] **Step 3: Capture the verified current snapshot without writing artifacts** + +Run: + +```bash +make proof-readiness-reconciliation TOP_N=20 +make proof-readiness-reconciliation TOP_N=0 JSON=1 +``` + +Record only values directly emitted by the current command. Expected methodological change from the audited starting snapshot is that 44 scope-only rows no longer count as ticker-level supporting conflicts; do not state the exact resulting count until the command proves it. + +- [ ] **Step 4: Update ROADMAP.md** + +Add the next numbered implemented item stating: + +- `changed_tickers` is the only ticker-level support attribution; +- `proof_applicability` and `current_blocker_code` remain independent; +- current observable blockers do not establish historical cause; +- no canonical data, readiness, or history was rewritten; +- current-snapshot counts come from the verified command and are not durable coverage totals; +- a prospective structured per-ticker/per-field proof contract is the next evidence-integrity maturity stage. + +- [ ] **Step 5: Update operator and continuation guidance** + +In `docs/OPERATOR_GUIDE.md`, explain: + +```text +explicit_ticker_change means the latest supporting proof explicitly names the ticker in changed_tickers. Scope membership alone is not ticker-level support. +current_canonical_row_missing and other current blocker codes describe current saved inputs. They do not establish the historical cause. +``` + +In `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, require future continuations to: + +- keep scope-only outcomes non-supporting at ticker level; +- route current blockers to the named safe review; +- never infer historical source, rights, scope, or cause from narrative proof; +- treat a structured per-ticker/per-field proof record as prospective future work, not a retroactive upgrade. + +- [ ] **Step 6: Run focused documentation and wording checks** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_public_v1_release_docs.py \ + tests/test_proof_readiness_reconciliation.py -q +make public-wording-check +git diff --check +``` + +Expected: all focused tests and wording/whitespace checks pass. + +- [ ] **Step 7: Run full local verification** + +Run each command and require exit zero: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make commercial-beta-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: + +- full tests pass; +- all six Research Mode routes render; +- public, commercial-beta, release, and pilot packaging gates report their truthful states; +- generated churn remains excluded; +- no completion claim exceeds local evidence. + +- [ ] **Step 8: Stage exact documentation/test files and verify** + +Run: + +```bash +git add -- \ + ROADMAP.md \ + docs/OPERATOR_GUIDE.md \ + docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md \ + tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +git diff --cached --name-only +``` + +Expected: exactly the four named files are staged and all generated files remain unstaged. + +- [ ] **Step 9: Commit documentation** + +Run: + +```bash +git commit -m "Document proof conflict diagnosis" +``` + +- [ ] **Step 10: Final local branch audit** + +Run: + +```bash +git status --short --branch +git log -6 --oneline --decorate +git rev-list --left-right --count origin/codex/personal-research-mode-mvp...HEAD +make diff-hygiene-summary +``` + +Expected: only the 18 pre-existing generated modifications remain; the branch is ahead only by intentional reviewed commits. + +- [ ] **Step 11: Push and update draft PR #113** + +Run: + +```bash +git push origin codex/personal-research-mode-mvp +gh pr view 113 --json number,state,isDraft,mergeable,headRefName,headRefOid,url,statusCheckRollup +``` + +Add a PR comment summarizing: + +- the false scope-level attribution defect; +- the two-axis contract; +- verified current-snapshot counts; +- focused/full verification; +- zero staged generated artifacts; +- unchanged external source, hosted, reviewer, consensus, calibration, and operating gates. + +Keep the PR draft. Do not merge or deploy. + +- [ ] **Step 12: Verify exact-head GitHub CI** + +Wait for the `Commercial Research Beta` workflow whose `headSha` equals the pushed HEAD. Require `local-engineering-gate` conclusion `SUCCESS` before reporting review safety. + +- [ ] **Step 13: Select the next safe maturity lane** + +Run the final read-only reconciliation and roadmap audit. The preferred next design subject is the prospective versioned per-ticker/per-field proof contract. Do not implement that new behavior without its own design approval. If a higher-value executable local gap is proven by current evidence, document the choice and its boundary instead. + +Report: + +1. repository and PR status; +2. product stage; +3. lane audited; +4. root cause and false-attribution count; +5. changes made; +6. tests and gates; +7. commits and push; +8. generated artifacts excluded; +9. external dependencies; +10. remaining maturity gaps; +11. exact next executable step; +12. review safety; +13. whether the overall goal remains active. diff --git a/docs/superpowers/plans/2026-07-21-proof-readiness-reconciliation.md b/docs/superpowers/plans/2026-07-21-proof-readiness-reconciliation.md new file mode 100644 index 000000000..e21bd523c --- /dev/null +++ b/docs/superpowers/plans/2026-07-21-proof-readiness-reconciliation.md @@ -0,0 +1,448 @@ +# Proof-Readiness Reconciliation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make historical proof outcomes impossible to mistake for current saved readiness by adding a read-only, lane-independent reconciliation engine, CLI, and Advanced Proof History answer. + +**Architecture:** A new pure module consumes the append-only batch proof ledger and the three current readiness reports, maps only explicit proof lanes to explicit current fields, and emits immutable reconciliation rows plus a bounded summary. The CLI and dashboard consume that same result; neither writes data, alters readiness, or rewrites proof history. + +**Tech Stack:** Python 3.12, pandas, dataclasses, argparse, pytest, Streamlit, Make. + +## Global Constraints + +- Research-only; no investment advice, rankings, recommendations, broker integration, order routing, or auto-trading. +- Current saved readiness remains authoritative for current lane availability. +- Historical proof remains append-only and cannot promote current readiness. +- Actuals, consensus, Revenue, EPS, valuation, catalysts, outcomes, backtesting, calibration, source rights, field scope, price lineage, hosted operations, and external validation stay independent. +- Do not run `make readiness`, source refreshes, imports, applies, report generation, screenshot generation, or timing generation. +- Preserve the existing 18 generated readiness files as local, unstaged churn. +- Never use `git add -A`; stage only exact intentional files. +- Keep PR #113 open and draft. + +--- + +### Task 1: Pure Reconciliation Engine + +**Files:** +- Create: `src/proof_readiness_reconciliation.py` +- Create: `tests/test_proof_readiness_reconciliation.py` + +**Interfaces:** +- Consumes: `ReviewedBatchProof` objects from `src.reviewed_batch_proof`, plus ticker, DCF, and peer readiness `pandas.DataFrame` objects. +- Produces: `ProofReadinessReconciliationRow`, `ProofReadinessReconciliationSummary`, `build_proof_readiness_reconciliation`, `load_proof_readiness_reconciliation`, and `filter_reconciliation_rows`. + +- [ ] **Step 1: Write failing state and independence tests** + +Create `tests/test_proof_readiness_reconciliation.py` with fixtures that use real `ReviewedBatchProof` objects and in-memory readiness frames. The first tests must assert these exact states: + +```python +def test_historical_supported_fundamentals_stays_blocked_when_current_readiness_is_false(): + summary = build_proof_readiness_reconciliation( + proofs=[proof(tickers="ARCT", lane="fundamentals", outcome="auto_supported")], + ticker_readiness=ticker_frame(ARCT={"fundamentals_ready": "False"}), + dcf_readiness=dcf_frame(ARCT={"has_shares_outstanding": "False"}), + peer_readiness=peer_frame(), + ) + row = row_for(summary, "ARCT", "fundamentals") + assert row.state == "historical_supported_currently_blocked" + assert row.current_ready is False + assert row.latest_batch_id == "RB-1" + + +def test_current_ready_without_supporting_latest_proof_is_not_proof_backed(): + summary = build_proof_readiness_reconciliation( + proofs=[proof(tickers="ARCT", lane="fundamentals", outcome="still_blocked")], + ticker_readiness=ticker_frame(ARCT={"fundamentals_ready": "True"}), + dcf_readiness=dcf_frame(ARCT={"has_shares_outstanding": "False"}), + peer_readiness=peer_frame(), + ) + assert row_for(summary, "ARCT", "fundamentals").state == "current_ready_proof_not_supporting" +``` + +Add separate tests for current support with matching proof, later `still_blocked` superseding earlier support, `fundamentals_dcf`, share count, price aliases, peer mapping, peer valuation inputs, no proof, unknown lanes/outcomes, malformed dates/booleans, duplicate tickers, and descriptive ticker text. + +- [ ] **Step 2: Run the new test file and verify RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_proof_readiness_reconciliation.py -q +``` + +Expected: collection fails because `src.proof_readiness_reconciliation` does not exist. + +- [ ] **Step 3: Implement immutable rows, explicit mappings, and deterministic latest-proof selection** + +Create `src/proof_readiness_reconciliation.py` with these public values and signatures: + +```python +SUPPORTING_OUTCOMES = frozenset({"supported", "auto_supported", "human_reviewed_supported"}) + +LANE_MAPPINGS = { + "fundamentals": ("fundamentals", "ticker", "fundamentals_ready"), + "fundamentals_dcf": ("dcf", "ticker", "dcf_ready"), + "share_count": ("share_count", "dcf", "has_shares_outstanding"), + "price": ("price", "ticker", "price_ready"), + "prices": ("price", "ticker", "price_ready"), + "price_coverage": ("price", "ticker", "price_ready"), + "price_history": ("price", "ticker", "price_ready"), + "peers": ("peer_mapping", "ticker", "peer_ready"), + "peer_mapping": ("peer_mapping", "ticker", "peer_ready"), + "peer_valuation_inputs": ("peer_valuation_inputs", "peer", "peer_valuation_ready"), +} + +@dataclass(frozen=True) +class ProofReadinessReconciliationRow: + ticker: str + lane: str + current_field: str + current_ready: bool | None + latest_batch_id: str + latest_review_date: str + latest_outcome: str + review_date_valid: bool + state: str + reason: str + + +@dataclass(frozen=True) +class ProofReadinessReconciliationSummary: + rows: tuple[ProofReadinessReconciliationRow, ...] + status_counts: tuple[tuple[str, int], ...] + conflict_counts_by_lane: tuple[tuple[str, int], ...] + input_status: str + input_message: str + + +def _reconciliation_state(*, current_ready: bool | None, proof_exists: bool, supporting: bool) -> str: + if current_ready is None: + return "not_applicable" + if current_ready and supporting: + return "current_supported_with_matching_proof" + if not current_ready and supporting: + return "historical_supported_currently_blocked" + if current_ready: + return "current_ready_proof_not_supporting" + if proof_exists: + return "currently_blocked_with_non_supporting_history" + return "no_proof_record" +``` + +Expose `build_proof_readiness_reconciliation` with keyword-only `proofs`, `ticker_readiness`, `dcf_readiness`, and `peer_readiness` arguments and a `ProofReadinessReconciliationSummary` return. Expose `load_proof_readiness_reconciliation` with keyword-only `root: Path`. Expose `filter_reconciliation_rows` with `summary`, keyword-only `tickers: Sequence[str] = ()`, and `top_n: int = 20`, returning an immutable tuple of rows. + +Implementation requirements: + +- create six canonical rows per valid ticker: fundamentals, DCF, share count, price, peer mapping, and peer valuation inputs; +- use current ticker-readiness membership as the valid ticker set; +- split proof tickers on commas or semicolons, uppercase, deduplicate, and ignore tokens absent from the valid set; +- choose the latest proof per canonical ticker/lane by valid ISO review date then append index; +- never let a malformed date outrank a valid date; a malformed supporting outcome is non-supporting; +- parse only explicit `true` and `false`; every other value yields `current_ready=None` and `not_applicable`; +- sort conflicts first, then current-ready proof gaps, then other states, each by lane and ticker; +- calculate global counts before presentation filtering. + +- [ ] **Step 4: Run focused tests and verify GREEN** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_proof_readiness_reconciliation.py -q +``` + +Expected: all reconciliation engine tests pass. + +- [ ] **Step 5: Commit the pure engine** + +Run: + +```bash +git add src/proof_readiness_reconciliation.py tests/test_proof_readiness_reconciliation.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Reconcile proof history with current readiness" +``` + +Expected: exactly the engine and its tests are committed; generated files remain unstaged. + +--- + +### Task 2: Read-Only CLI And Make Entry Point + +**Files:** +- Modify: `src/proof_readiness_reconciliation.py` +- Modify: `tests/test_proof_readiness_reconciliation.py` +- Modify: `Makefile` + +**Interfaces:** +- Consumes: `load_proof_readiness_reconciliation`, `filter_reconciliation_rows`. +- Produces: `render_proof_readiness_reconciliation`, `proof_readiness_reconciliation_payload`, module `main()`, and `make proof-readiness-reconciliation TOP_N=20`. + +- [ ] **Step 1: Write failing CLI rendering tests** + +Add tests asserting: + +```python +def test_render_names_conflicts_and_non_promotion_boundary(): + rendered = render_proof_readiness_reconciliation(summary_with_conflict(), top_n=10) + assert "Proof-Readiness Reconciliation" in rendered + assert "historical_supported_currently_blocked" in rendered + assert "Current saved readiness remains authoritative" in rendered + assert "does not restore data, promote readiness, or rewrite proof history" in rendered + + +def test_main_is_read_only(tmp_path, capsys): + write_fixture_inputs(tmp_path) + before = snapshot_files(tmp_path) + assert main(["--root", str(tmp_path), "--top-n", "10"]) == 0 + assert snapshot_files(tmp_path) == before + assert "Research-only" in capsys.readouterr().out +``` + +Add a JSON test asserting the payload retains global counts when `--tickers ARCT` narrows displayed rows. + +- [ ] **Step 2: Run CLI tests and verify RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_proof_readiness_reconciliation.py -q +``` + +Expected: tests fail because rendering and `main` are absent. + +- [ ] **Step 3: Implement rendering, JSON, argparse, and Make target** + +Add: + +```python +def _parse_ticker_filter(value: str) -> tuple[str, ...]: + return tuple(dict.fromkeys(token.strip().upper() for token in value.split(",") if token.strip())) +``` + +Expose `render_proof_readiness_reconciliation` and `proof_readiness_reconciliation_payload` with the same `summary`, `tickers`, and `top_n` contract as the filter. Expose `main(argv: Sequence[str] | None = None) -> int`; it parses `--root`, `--top-n`, `--tickers`, and `--json`, prints exactly one representation to stdout, and returns zero without opening any output file for writing. + +The human renderer must print input status, global state counts, conflict counts by canonical lane, bounded detail rows, the read-only boundary, and the exact next command. JSON must serialize dataclasses without changing state. + +Add the target to `.PHONY`, Make help, and the command body: + +```make +proof-readiness-reconciliation: + @python3 -m src.proof_readiness_reconciliation --root . --top-n $(or $(TOP_N),20) $(if $(TICKERS),--tickers "$(TICKERS)",) $(if $(filter 1,$(JSON)),--json,) +``` + +- [ ] **Step 4: Run focused tests and the real command** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_proof_readiness_reconciliation.py -q +make proof-readiness-reconciliation TOP_N=10 +make diff-hygiene-summary +``` + +Expected: tests pass; the command reports current conflicts; diff hygiene still reports only intentional product files plus the protected 18 generated files. + +- [ ] **Step 5: Commit the CLI slice** + +Run: + +```bash +git add Makefile src/proof_readiness_reconciliation.py tests/test_proof_readiness_reconciliation.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Expose proof readiness reconciliation" +``` + +--- + +### Task 3: Advanced Proof History Integration + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_dashboard_render_smoke.py` only if the established route-marker contract needs the new marker. + +**Interfaces:** +- Consumes: `load_proof_readiness_reconciliation`, filtered reconciliation rows, current `DATA_DIR`, and the optional `ticker` query parameter. +- Produces: `proof_readiness_reconciliation_cards` and an answer-first warning before raw ledger detail in both public and operator Proof History render paths. + +- [ ] **Step 1: Write failing dashboard helper and ordering tests** + +Add tests proving the card contract: + +```python +def test_proof_reconciliation_cards_warn_when_historical_support_is_currently_blocked(): + cards = dashboard.proof_readiness_reconciliation_cards(summary_with_conflict(), ticker="ARCT") + rendered = " ".join(str(value) for card in cards for value in card.values()).lower() + assert "historical support is not current readiness" in rendered + assert "arct" in rendered + assert "current saved readiness remains authoritative" in rendered + assert "make " not in rendered +``` + +Add source-order assertions that reconciliation cards are rendered after the latest evidence timeline/answer but before `Advanced: proof ledger details`. Add a selected-ticker test and a no-conflict test. Keep existing tests proving primary pages and public Proof History do not expose operator commands. + +- [ ] **Step 2: Run dashboard tests and verify RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py -q -k 'proof_history or proof_reconciliation' +``` + +Expected: new tests fail because the dashboard helper and rendering call are absent. + +- [ ] **Step 3: Implement the minimal answer-first integration** + +Import the reconciliation loader and row filter. Add: + +```python +def proof_readiness_reconciliation_cards( + summary: ProofReadinessReconciliationSummary, + *, + ticker: str = "", +) -> list[dict[str, object]]: + conflicts = [row for row in summary.rows if row.state == "historical_supported_currently_blocked"] + selected = ticker.strip().upper() + selected_conflicts = [row for row in conflicts if selected and row.ticker == selected] + cards = [{ + "kicker": "PROOF / CURRENT STATE", + "title": f"{len(conflicts):,} historical-support conflict(s)", + "body": ( + "Historical support is not current readiness. Current saved readiness remains authoritative; " + "reconciliation does not restore data, promote readiness, or rewrite proof history." + ), + "badges": [summary.input_status, "read-only"], + "command": "", + }] + if selected_conflicts: + lanes = ", ".join(row.lane.replace("_", " ") for row in selected_conflicts) + cards.append({ + "kicker": "SELECTED TICKER", + "title": f"{selected}: historical proof conflicts with current readiness", + "body": f"Current blocked lane(s): {lanes}. Re-review source evidence before relying on an older supported outcome.", + "badges": [selected, "currently blocked"], + "command": "", + }) + return cards +``` + +The first card must report the global conflict count. When the selected ticker has conflicts, a second card must name only that ticker's canonical lanes. When no conflict exists, the card must say that reconciliation found no historical-support/current-readiness conflict while still refusing to prove source rights or payload truth. Cards must have empty commands. + +In `render_proof_history`, load the summary from the current repository root, read `ticker` from `st.query_params`, and render reconciliation cards before the ledger expander. Do not add the cards to Research Desk, Discover, Company Workbench, or Monitor. + +- [ ] **Step 4: Run focused dashboard and render tests** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py tests/test_research_mode_dashboard_contract.py -q +make research-dashboard-render-smoke +``` + +Expected: all focused dashboard tests and six research route renders pass. + +- [ ] **Step 5: Commit the dashboard slice** + +Run: + +```bash +git add src/dashboard.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Show proof readiness conflicts in Proof History" +``` + +Stage `tests/test_dashboard_render_smoke.py` only if it changed. + +--- + +### Task 4: Methodology, Roadmap, Continuation Contract, And Release Verification + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `docs/OPERATOR_GUIDE.md` +- Modify: documentation contract tests selected by existing repository patterns. + +**Interfaces:** +- Consumes: verified CLI output, focused tests, and dashboard behavior. +- Produces: durable stage truth and the exact continuation command. + +- [ ] **Step 1: Write failing documentation contract assertions** + +Add assertions to the existing documentation tests that require: + +```text +make proof-readiness-reconciliation TOP_N=20 +historical_supported_currently_blocked +current saved readiness remains authoritative +``` + +The continuation contract must require reconciliation before reusing a supporting proof outcome and must preserve all external classifications. + +- [ ] **Step 2: Run documentation tests and verify RED** + +Run the selected documentation test module identified by `rg` and confirm the new assertions fail before documentation changes. + +- [ ] **Step 3: Update documentation truthfully** + +Document the implemented read-only behavior, the current-snapshot audit finding, and these limits: + +- reconciliation does not repair or restore canonical data; +- historical proof remains append-only; +- current readiness does not prove source rights, field scope, provenance, payload truth, or commercial use; +- the audit count is current-snapshot evidence, not a durable coverage total; +- future runs inspect reconciliation once before restarting a proof lane. + +- [ ] **Step 4: Run focused documentation checks** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q -k 'public_v1_release_docs or proof_readiness_reconciliation' +make public-wording-check +git diff --check +``` + +- [ ] **Step 5: Run the complete release matrix** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: every gate passes; the pilot remains truthful about manual source-proof and generated-artifact gates; no new generated churn appears. + +- [ ] **Step 6: Stage exact files, verify, commit, push, and update PR #113** + +Run exact `git add` commands for only changed product, tests, Makefile, and documentation. Then run: + +```bash +make staged-hygiene-check +git diff --cached --check +git diff --cached --name-only +git commit -m "Document proof readiness reconciliation" +git push origin codex/personal-research-mode-mvp +``` + +Update draft PR #113 with the defect, state contract, test evidence, commit range, generated-artifact exclusion, and remaining external gates. Confirm exact-head CI passes before claiming the branch safe for review. + +--- + +## Plan Self-Review + +- Spec coverage: engine, CLI, Advanced Proof History, selected ticker, fail-closed states, documentation, full gates, artifact hygiene, PR update, and independent readiness are covered. +- Placeholder scan: no deferred runtime behavior or incomplete acceptance step remains in the plan. +- Type consistency: all tasks use the same `ProofReadinessReconciliationRow` and `ProofReadinessReconciliationSummary` interfaces. +- Scope: one read-only proof-interpretation slice; no canonical-data repair, source activation, provider work, readiness rebuild, or external claim. diff --git a/docs/superpowers/plans/2026-07-21-prospective-field-proof.md b/docs/superpowers/plans/2026-07-21-prospective-field-proof.md new file mode 100644 index 000000000..2fc049e85 --- /dev/null +++ b/docs/superpowers/plans/2026-07-21-prospective-field-proof.md @@ -0,0 +1,282 @@ +# Prospective Per-Ticker Field Proof Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a prospective, append-only, per-ticker/per-field proof ledger with fail-closed validation, linear revisions, source-rights review, preview receipts, and explicit record commands without activating readiness or upgrading legacy proof. + +**Architecture:** Introduce one isolated module and ledger contract modeled on the repository's prospective consensus collector. The module carries independent technical and commercial evidence decisions, validates proposed rows against a virtual ledger, and permits all-or-nothing append only after exact preview-receipt revalidation. Existing readiness, canonical data, dashboard, and proof-reconciliation paths remain untouched. + +**Tech Stack:** Python 3.12, immutable dataclasses, CSV, SHA-256, argparse, pytest, Make, GitHub Actions. + +## Global Constraints + +- The new ledger is prospective only; never infer, migrate, or upgrade legacy narrative proof. +- A proof row, source reference, rights reference, or payload digest does not prove truth, licensing, commercial usability, readiness, or market validation. +- Keep technical write eligibility and commercial evidence eligibility independent. +- Do not integrate this contract with current readiness, canonical data, Company Workbench, proof reconciliation, or dashboards in this slice. +- Status and preview are filesystem read-only and network-free. +- Do not create `data/prospective_field_proofs.csv`, sample proof rows, CSV/JSON reports, screenshots, timing evidence, or bytecode artifacts during implementation verification. +- Preserve the existing 18 generated-file modifications as unstaged and uncommitted. +- Stage exact files only; never use `git add -A`. +- Push only `codex/personal-research-mode-mvp`; keep PR #113 open and draft; do not merge or deploy. +- Preserve all research-only, no-investment-advice, no-trading, explicit-Q4, EPS split-basis, synthetic-fixture, candidate-context, consensus, and calibration boundaries. + +--- + +## File Structure + +- Create `src/prospective_field_proof.py`: schema, records, validation, identity, revision integrity, preview/receipt, append, render, and CLI. +- Create `tests/test_prospective_field_proof.py`: unit and CLI contract coverage. +- Modify `Makefile`: status, preview, and record targets. +- Modify `tests/test_makefile_test_targets.py`: durable Make target contract. +- Modify `ROADMAP.md`: completed Stage A evidence primitive and explicit non-activation boundary. +- Modify `README.md`: correct artifact-writing and reviewer-cohort operating claims. +- Modify `docs/OPERATOR_GUIDE.md`: preview-before-record operating contract. +- Modify `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`: next executable stage and external classifications. +- Modify `tests/test_public_v1_release_docs.py`: durable documentation contract. + +--- + +### Task 1: Strict Record Schema And Ledger Integrity + +**Files:** +- Create: `tests/test_prospective_field_proof.py` +- Create: `src/prospective_field_proof.py` + +**Interfaces:** +- `SCHEMA_VERSION = "prospective-field-proof-v1"` +- `FIELDS` in the exact order specified by the design +- immutable `ProspectiveFieldProofRecord` +- `field_proof_identity`, `load_field_proofs`, `load_proposed_field_proofs`, `validate_field_proof_ledger` + +- [ ] **Step 1: Write schema and normalization tests** + +Create fixture helpers for a valid accepted record and exact CSV writer. Test exact header enforcement, missing/extra/reordered fields, ticker/field normalization, required non-placeholder values, controlled enums, lowercase SHA-256 payload digest, semantic proof ID, UTC timestamps, cutoff-independent timestamp ordering, and row-numbered errors. + +- [ ] **Step 2: Run schema tests and confirm RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_prospective_field_proof.py -q +``` + +Expected: import failure because `src.prospective_field_proof` does not exist. + +- [ ] **Step 3: Implement the minimum immutable schema and loaders** + +Use `csv.DictReader`, strict header equality, `parse_utc_timestamp`, canonical normalization, semantic JSON serialization, and SHA-256. A missing ledger loads as an empty tuple; a present empty or malformed ledger fails closed. + +- [ ] **Step 4: Write revision-integrity tests** + +Cover one root per normalized scope, one child per parent, global unique IDs and identities, parent-before-child append order, same-scope parent, current-leaf revision, strictly later `reviewed_at`, missing parent, cross-scope parent, fork, cycle/disconnected cycle, stale-leaf revision, duplicate root, and valid independent scopes. + +- [ ] **Step 5: Run revision tests and confirm RED** + +Expected: failures for the unimplemented chain validation. + +- [ ] **Step 6: Implement linear revision integrity and rerun focused tests** + +Keep validation deterministic and report the ledger/input row number and precise invariant. + +- [ ] **Step 7: Verify, stage exact files, and commit** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_prospective_field_proof.py -q +git diff --check +git add -- src/prospective_field_proof.py tests/test_prospective_field_proof.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add prospective field proof ledger integrity" +``` + +--- + +### Task 2: Preview Receipt, Rights Review, And Atomic Append + +**Files:** +- Modify: `tests/test_prospective_field_proof.py` +- Modify: `src/prospective_field_proof.py` + +**Interfaces:** +- immutable `FieldProofPreview` and `BatchFieldProofPreview` +- `preview_field_proof_batch` +- `append_reviewed_field_proof_batch` + +- [ ] **Step 1: Write independent eligibility tests** + +Test accepted/rejected/follow-up records; source identified/unavailable/disputed; payload reviewed/unavailable/rejected; unknown source; unapproved commercial rights; unsupported field; missing rights-decision reference; approved exact source/field; and research-mode technical eligibility independent from commercial eligibility. + +- [ ] **Step 2: Run the new tests and confirm RED** + +Expected: missing preview interfaces or eligibility fields. + +- [ ] **Step 3: Implement row and virtual-batch previews** + +Reuse `review_commercial_field_scope`. Validate every proposed row against a virtual ledger so a same-batch revision can follow its parent. Return explicit technical and commercial blockers; never collapse one into the other. + +- [ ] **Step 4: Write receipt and append tests** + +Test deterministic receipt over schema, normalized cutoff, commercial mode, existing-ledger digest, input digest, and source-rights-registry digest. Test invalidation after ledger/input/cutoff/mode/registry changes, missing confirmation, missing receipt, empty batch, mixed-validity all-or-nothing behavior, one header, ordered append, and commercial-mode fail-closed behavior. + +- [ ] **Step 5: Run receipt tests and confirm RED** + +Expected: append and receipt behavior is missing. + +- [ ] **Step 6: Implement receipt revalidation and append** + +Re-read and revalidate the destination ledger immediately before append. Require exact receipt and explicit confirmation. Write nothing unless every dependent gate passes. Append rows once; do not rewrite prior bytes. + +- [ ] **Step 7: Verify, stage exact files, and commit** + +Run the focused file, `git diff --check`, exact add of the two task files, staged hygiene, cached diff check, and commit: + +```bash +git commit -m "Add reviewed field proof preview and append" +``` + +--- + +### Task 3: Read-Only Status/Preview And Explicit Record Commands + +**Files:** +- Modify: `tests/test_prospective_field_proof.py` +- Modify: `src/prospective_field_proof.py` +- Modify: `Makefile` +- Modify: `tests/test_makefile_test_targets.py` + +- [ ] **Step 1: Write CLI and Make contract tests** + +Prove: + +- `status` reports absent/empty-valid/valid/invalid without creating or changing files; +- `preview` emits stable JSON including receipt, both eligibility states, blockers, and `write_performed=false`; +- `record` refuses absent confirmation or receipt and records only a fully revalidated batch; +- CLI errors are specific and nonzero; +- Make targets forward `INPUT`, `LEDGER`, `AS_OF`, and `PREVIEW_RECEIPT` exactly; +- the default ledger is `data/prospective_field_proofs.csv` and no default file is created by status/preview. + +- [ ] **Step 2: Run CLI/Make tests and confirm RED** + +Run the CLI-focused tests plus `tests/test_makefile_test_targets.py`. + +- [ ] **Step 3: Implement renderers, argparse commands, and Make targets** + +Add `.PHONY` entries and these targets: + +```make +prospective-field-proof-status +prospective-field-proof-preview +prospective-field-proof-record +``` + +The record target must reject blank `INPUT`, `AS_OF`, or `PREVIEW_RECEIPT` before Python execution. Wording must distinguish read-only preview from explicit append. + +- [ ] **Step 4: Prove filesystem non-mutation** + +Use test temporary directories and byte snapshots before/after status and preview. Assert no ledger, output, readiness, canonical, or legacy-proof file appears or changes. + +- [ ] **Step 5: Verify, stage exact files, and commit** + +Run focused tests, `git diff --check`, exact add of the four task files, staged hygiene, cached diff check, and commit: + +```bash +git commit -m "Expose prospective field proof workflow" +``` + +--- + +### Task 4: Operating Documentation And Release Evidence + +**Files:** +- Modify: `README.md` +- Modify: `ROADMAP.md` +- Modify: `docs/OPERATOR_GUIDE.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +- [ ] **Step 1: Write failing documentation-contract tests** + +Require the three commands, prospective-only boundary, legacy non-upgrade boundary, independent technical/commercial status, preview receipt, no readiness activation, absent-ledger empty state, and exact next external unblock conditions. + +- [ ] **Step 2: Run documentation tests and confirm RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_public_v1_release_docs.py -q +``` + +- [ ] **Step 3: Update operating documentation truthfully** + +Document the implemented contract and examples without creating example ledger rows. Correct these current documentation contradictions while in the same truth-maintenance slice: + +- the committed PR readiness snapshot is older than the excluded local generated snapshot; +- `make pilot-readiness-packet` writes a Markdown output and is not read-only; +- reviewer target is consistently 10–20 sessions; +- Stage A–G labels are maturity lanes, not replacements for numbered release stages. + +Classify external dependencies once: permitted point-in-time consensus and rights review, hosted account and operated controls, independent reviewers, peer evidence, and calibration cohort. + +- [ ] **Step 4: Run focused documentation and proof tests** + +Run the new module tests, Make target tests, release-doc tests, and `git diff --check`. + +- [ ] **Step 5: Stage exact docs/tests and commit** + +Run staged hygiene and cached diff check, then commit: + +```bash +git commit -m "Document prospective field proof operations" +``` + +--- + +### Task 5: Whole-Slice Verification, Independent Review, And Draft PR Update + +**Files:** No intended product edits unless a reviewer finds a concrete defect. + +- [ ] **Step 1: Run focused changed-module tests** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_prospective_field_proof.py \ + tests/test_makefile_test_targets.py \ + tests/test_public_v1_release_docs.py -q +``` + +- [ ] **Step 2: Run the complete local release matrix** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make commercial-beta-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make pr-range-hygiene-check +make diff-hygiene-summary +git diff --check +``` + +Interpret expected fail-closed pilot results as product truth, not test failure. Do not rebuild readiness or stage the existing generated files. + +- [ ] **Step 3: Request independent whole-slice review** + +Review from the design commit through current HEAD for spec compliance, overclaims, append integrity, receipt completeness, source-rights independence, filesystem writes, regression risk, and generated-artifact hygiene. Fix concrete findings test-first and rerun affected gates. + +- [ ] **Step 4: Verify exact staged/unstaged scope** + +Ensure no product changes remain unstaged, no generated files are staged, and the only remaining worktree changes are the pre-existing 18 generated files. Run staged hygiene if any correction is staged. + +- [ ] **Step 5: Push and update PR #113** + +Push only `codex/personal-research-mode-mvp`. Update the draft PR with implemented contract, explicit non-activation boundary, tests/checks, generated-artifact exclusion, current committed-vs-local readiness distinction, external blockers, and next executable slice. Keep the PR draft and wait for exact-head CI. + +- [ ] **Step 6: Continue to the next safe maturity lane** + +After exact-head CI passes, select the next safe local slice from current evidence. Prefer correcting the verified responsive first-viewport handoff regression or strengthening Company Workbench proof activation preview. Do not connect structured proof to readiness without a separately approved design and tests. diff --git a/docs/superpowers/plans/2026-07-21-public-mobile-evidence-handoff.md b/docs/superpowers/plans/2026-07-21-public-mobile-evidence-handoff.md new file mode 100644 index 000000000..cb1df674c --- /dev/null +++ b/docs/superpowers/plans/2026-07-21-public-mobile-evidence-handoff.md @@ -0,0 +1,163 @@ +# Public Mobile Evidence Handoff Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Keep the existing fail-closed Single-Stock Report reading order while making its Data Health evidence handoff fully visible in the first 390x844 phone viewport. + +**Architecture:** Reuse the existing public shell and ticker-summary markup. Add only phone-scoped layout rules inside the existing `max-width: 640px` media query, protect them with a source contract, and use a fresh live browser measurement as the layout proof. + +**Tech Stack:** Python 3.12, Streamlit, existing HTML/CSS renderer, pytest, in-app browser responsive audit. + +## Global Constraints + +- Selected ticker, Use now, and Still withheld remain before the evidence handoff. +- Desktop layout remains unchanged. +- No copy, readiness, evidence, source, forecast, probability, or data state changes. +- Advanced details remain collapsed. +- No generated research or manual-review artifact enters the repository. + +--- + +### Task 1: Phone-Scoped Handoff Contract and CSS + +**Files:** +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `src/dashboard.py` + +**Interfaces:** +- Consumes: existing `render_public_shell_mode_styles()`, `.public-ticker-summary`, `.public-ticker-action`, and `.public-primary-action`. +- Produces: phone-only compact summary spacing and action-link ordering; no Python interface change. + +- [ ] **Step 1: Write the failing source contract** + +Add a test that extracts the `max-width: 640px` block inside `render_public_shell_mode_styles()` and requires: + +```python +def test_public_single_stock_phone_keeps_evidence_handoff_in_first_view(): + source = Path("src/dashboard.py").read_text(encoding="utf-8") + shell_start = source.index("def render_public_shell_mode_styles") + mobile_start = source.index("@media (max-width: 640px)", shell_start) + mobile_end = source.index("", mobile_start) + mobile_css = source[mobile_start:mobile_end] + + summary_start = mobile_css.index(".public-ticker-summary {") + summary_end = mobile_css.index("}", summary_start) + summary_rule = mobile_css[summary_start:summary_end] + action_start = mobile_css.index(".public-ticker-action .public-primary-action {") + action_end = mobile_css.index("}", action_start) + action_rule = mobile_css[action_start:action_end] + + assert "grid-template-columns: 1fr;" in summary_rule + assert "gap: 0.25rem;" in summary_rule + assert "padding: 0.125rem 0 0.5rem;" in summary_rule + assert "order: -1;" in action_rule +``` + +- [ ] **Step 2: Run the test and confirm the current layout contract fails** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py::test_public_single_stock_phone_keeps_evidence_handoff_in_first_view -q +``` + +Expected during the final-review correction: failure because the exact summary +rule still uses `padding: 0.5rem 0`; declarations elsewhere in the mobile block +must not satisfy the selector-local contract. + +- [ ] **Step 3: Add the minimal phone-only CSS** + +In the existing `@media (max-width: 640px)` block, change only the ticker-summary phone rules: + +```css +.public-ticker-summary { + grid-template-columns: 1fr; + gap: 0.25rem; + padding: 0.125rem 0 0.5rem; +} +.public-ticker-action .public-primary-action { + order: -1; +} +``` + +The DOM order and desktop CSS stay unchanged. + +- [ ] **Step 4: Run focused contracts** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py::test_public_single_stock_phone_keeps_evidence_handoff_in_first_view tests/test_dashboard_helpers.py::test_public_workflow_controls_reserve_accessible_touch_targets tests/test_dashboard_helpers.py::test_public_app_shell_has_compact_mobile_rules -q +``` + +Expected: three passing tests. + +- [ ] **Step 5: Verify the live 390x844 route** + +Reload `http://localhost:8501/?mode=public&page=single-stock-report&ticker=NVDA&open=1` at 390x844 and measure the existing `Open Data Health` link. Expected: + +- the link top is at least 0; +- the link bottom is at most 844; +- `document.documentElement.scrollWidth <= window.innerWidth + 2`; +- no traceback is present; +- Selected ticker, Use now, and Still withheld precede the link in document order. + +Save the accepted screenshot only under `/tmp/stock-command-center-public-ux-review`. +The final accepted measurement is `top=792.53125px`, `bottom=836.53125px`, +leaving `7.46875px` of clearance in the `844px` viewport. The computed summary +spacing is `gap=4px`, `padding-top=2px`, and `padding-bottom=8px`. + +- [ ] **Step 6: Commit the tested product correction** + +Stage exactly `src/dashboard.py` and `tests/test_dashboard_helpers.py`, run staged hygiene, and commit with: + +```bash +git commit -m "Keep mobile evidence handoff in view" +``` + +--- + +### Task 2: QA and Continuation Evidence + +**Files:** +- Modify: `docs/DASHBOARD_QA.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: the verified phone CSS and fresh 390x844 browser result from Task 1. +- Produces: truthful current audit evidence and the persistent continuation boundary. + +- [ ] **Step 1: Update documentation contracts** + +Record: + +- all five public pages passed fresh desktop and phone review without overflow or tracebacks; +- Single-Stock Report required and received a phone-only evidence-handoff density correction; +- Data Health and Proof History remain answer/evidence destinations and do not gain invented calls to action; +- no readiness, source, research, or generated artifact state changed. + +- [ ] **Step 2: Run focused documentation and render tests** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_public_v1_release_docs.py tests/test_browser_qa_evidence.py tests/test_public_ux_review_checklist.py tests/test_dashboard_render_smoke.py -q +make research-dashboard-render-smoke +``` + +Expected: all tests and route renders pass. + +- [ ] **Step 3: Run full release verification** + +Run the full pytest suite, dashboard and public render smokes, public wording, commercial beta, public, commercial release, pilot, diff, PR-range, whitespace, and staged hygiene gates required by the continuation contract. + +- [ ] **Step 4: Commit, push, and verify the draft PR** + +Stage exactly the three documentation files, run staged hygiene, commit with: + +```bash +git commit -m "Document public mobile workflow audit" +``` + +Push only `codex/personal-research-mode-mvp`, update draft PR #113, and require a successful exact-head GitHub Actions result. Keep the PR draft; do not merge or deploy. diff --git a/docs/superpowers/plans/2026-07-21-public-packaging-reconciliation.md b/docs/superpowers/plans/2026-07-21-public-packaging-reconciliation.md new file mode 100644 index 000000000..5c9dae489 --- /dev/null +++ b/docs/superpowers/plans/2026-07-21-public-packaging-reconciliation.md @@ -0,0 +1,81 @@ +# Public Packaging Reconciliation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make the public README, LinkedIn Featured package, curated visual, and Workbench performance evidence tell one accurate answer-first Personal Research story. + +**Architecture:** Reuse the existing four-step Personal Research workflow and read-only public packaging checks. Tighten documentation and test contracts, bind Workbench first-useful timing to the rendered `Use now` answer, and replace the stale count-heavy social image with one reviewed real-app Workbench capture. No research data, forecasts, readiness state, or routing behavior changes. + +**Tech Stack:** Markdown, Python 3.12, pytest, Make, Streamlit, in-app browser QA, GitHub draft PR. + +## Global Constraints + +- Research-only; no investment advice, trading, orders, or price prediction. +- Do not fabricate research, readiness, source, rights, validation, hosting, or market evidence. +- Generated research CSV, JSON, report, sample-report, screenshot, timing, canonical-data, and readiness churn remains excluded. +- The one intentional `1200x627` LinkedIn thumbnail is reviewed before it replaces the committed curated asset. +- Stage exact files only; never use `git add -A`. +- Keep PR #113 draft; do not merge or deploy. + +--- + +### Task 1: Red Tests for the Public Story and Evidence Contracts + +**Files:** +- Modify: `tests/test_public_v1_release_docs.py` +- Modify: `tests/test_public_performance_gate.py` +- Modify: `tests/test_browser_qa_evidence.py` + +- [ ] Add contracts requiring one README reviewer entry point, the four-step Personal Research workflow as primary, and the five-page Public demo as secondary. +- [ ] Add LinkedIn contracts for the evidence-first title, default-branch/draft-preview link boundary, Workbench visual, and no stale readiness-count guidance. +- [ ] Require `Use now` as the Company Workbench first-useful performance marker. +- [ ] Require the browser-QA asset contract to describe a `1200x627` Company Workbench answer-first visual and its visible evidence markers. +- [ ] Run the focused tests and confirm they fail only because the old packaging contracts remain. + +--- + +### Task 2: Reconcile README, LinkedIn, Share Check, and Performance Marker + +**Files:** +- Modify: `README.md` +- Modify: `docs/LINKEDIN_PROJECT_BRIEF.md` +- Modify: `docs/PUBLIC_RELEASE_CHECKLIST.md` +- Modify: `Makefile` or the existing LinkedIn share-check source located during implementation +- Modify: `src/public_performance_gate.py` +- Modify: `src/browser_qa_evidence.py` + +- [ ] Consolidate the README opening into one `External Reviewer Start Here` section. +- [ ] Make Research Desk -> Discover -> Company Workbench -> Monitor the primary product path and keep the controlled five-page Public demo secondary. +- [ ] Replace LinkedIn title, description, visual guidance, and link guidance with maturity-accurate copy. +- [ ] Update the read-only share check to print the same title, link boundary, visual rule, and research-only stop rules. +- [ ] Bind Workbench first-useful timing to `Use now`, retaining route identity as a full-settle marker. +- [ ] Update the browser-QA asset record to the Workbench answer-first visual. +- [ ] Run focused tests and confirm they pass. + +--- + +### Task 3: Capture and Review the Curated Workbench Thumbnail + +**Files:** +- Modify: `docs/assets/linkedin-public-dashboard.png` + +- [ ] Open the current Company Workbench route for one selected ticker and wait for the final answer state. +- [ ] Capture an intentional `1200x627` frame showing Company Workbench, `Use now`, `Still withheld`, Data Health handoff, stop condition, and the research-only boundary. +- [ ] Exclude numerical readiness/source-date claims, loading states, errors, operator controls, raw tables, and generated artwork. +- [ ] Inspect the candidate beside the current app reference, verify readable composition, then replace the curated asset. +- [ ] Verify exact pixel dimensions and run the focused asset/browser-QA contracts. + +--- + +### Task 4: Record Verified Truth and Complete Release Gates + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: other narrowly relevant QA documentation only if an existing contract requires it + +- [ ] Record the packaging slice without claiming new research readiness, hosting, reviewer, source-rights, consensus, calibration, or market validation evidence. +- [ ] Run focused tests, full pytest, dashboard and research render smokes, public wording/checks, LinkedIn share check, browser-QA evidence, public and Commercial Research performance gates, Commercial Research release check, pilot readiness, PR-range/diff/staged hygiene, and whitespace checks. +- [ ] Confirm generated research churn remains unstaged and the intentional curated thumbnail is the only staged PNG. +- [ ] Commit the coherent verified implementation, push only `codex/personal-research-mode-mvp`, and update PR #113 with the slice and verification evidence. +- [ ] Keep PR #113 open and draft, then require exact-head CI success before handoff. diff --git a/docs/superpowers/plans/2026-07-21-public-ux-post-fix-outcome-contract.md b/docs/superpowers/plans/2026-07-21-public-ux-post-fix-outcome-contract.md new file mode 100644 index 000000000..0f037175a --- /dev/null +++ b/docs/superpowers/plans/2026-07-21-public-ux-post-fix-outcome-contract.md @@ -0,0 +1,278 @@ +# Public UX Post-Fix Outcome Contract Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make a verified `resolved_post_fix` public UX audit row satisfy the local share-review gate while preserving separate audit counts and fail-closed handling for every incomplete or unknown outcome. + +**Architecture:** `src/public_ux_review_checklist.py` remains the owner of the public UX outcome vocabulary and status decision. It will publish an explicit successful-outcome set used by both the checklist parser and `src/project_status.py`, avoiding duplicated strings and unsafe prefix matching. Tests first prove the current false-limited state and the project-status undercount, then documentation records the corrected local-only maturity claim. + +**Tech Stack:** Python 3, pytest, Make, Markdown, GitHub Actions. + +## Global Constraints + +- Research-only; this slice cannot change research conclusions, readiness, data, forecasts, probabilities, or recommendations. +- `resolved` and `resolved_post_fix` are the only successful public UX outcomes. +- Raw classification counts remain separate. +- `pending`, `intentionally_deferred`, `environment_limited`, `skipped`, `blocked_with_evidence`, and unknown outcomes remain fail closed. +- Do not edit or stage `/tmp` review notes or screenshots. +- Do not run `make readiness` or generate/stage CSV, JSON, report, sample-report, screenshot, timing, readiness, canonical-data, or manual-review churn. +- Stage exact files only; never use `git add -A`. +- Keep PR #113 open and draft; do not merge or deploy. + +--- + +### Task 1: Prove the post-fix classification defect + +**Files:** +- Modify: `tests/test_public_ux_review_checklist.py` +- Modify: `tests/test_project_status.py` + +**Interfaces:** +- Consumes: `public_ux_review_notes_status(notes_path)` and `project_status._public_ux_stage_from_status(status)`. +- Produces: regression coverage requiring an explicit successful post-fix outcome and a combined resolved-row count. + +- [ ] **Step 1: Add the failing checklist regression test** + +Add a test beside `test_public_ux_review_notes_status_marks_share_review_ready_when_all_rows_resolved` that writes all ten review rows, uses `resolved_post_fix` only for Single-Stock Report phone, and asserts: + +```python +assert status["status"] == "review_complete" +assert status["share_review_gate"] == "share_review_ready" +assert status["pending_rows"] == 0 +assert status["problem_rows"] == [] +assert status["classification_counts"]["resolved"] == 9 +assert status["classification_counts"]["resolved_post_fix"] == 1 +``` + +- [ ] **Step 2: Add the failing project-status regression test** + +Extend `test_project_status_stage_map_reports_completed_public_ux_review` with this status input: + +```python +"classification_counts": {"resolved": 9, "resolved_post_fix": 1}, +``` + +Keep the existing assertion that the evidence contains: + +```python +"10/10 public desktop/mobile review rows resolved" +``` + +- [ ] **Step 3: Run the focused tests and verify the red state** + +Run: + +```bash +python3 -m pytest \ + tests/test_public_ux_review_checklist.py::test_public_ux_review_notes_status_marks_share_review_ready_after_verified_fix \ + tests/test_project_status.py::test_project_status_stage_map_reports_completed_public_ux_review -q +``` + +Expected: both tests fail for the intended reasons—the parser reports `review_limited`, and project status reports `9/10`. + +- [ ] **Step 4: Confirm no production file changed during red** + +Run: + +```bash +git status --short +``` + +Expected: only the two test files and this plan are changed or committed; no data or generated artifact appears. + +--- + +### Task 2: Implement the explicit successful-outcome contract + +**Files:** +- Modify: `src/public_ux_review_checklist.py` +- Modify: `src/project_status.py` +- Test: `tests/test_public_ux_review_checklist.py` +- Test: `tests/test_project_status.py` + +**Interfaces:** +- Produces: `SUCCESSFUL_REVIEW_CLASSIFICATIONS`, an immutable explicit set containing `resolved` and `resolved_post_fix`. +- Consumes: raw `classification_counts` without collapsing keys. + +- [ ] **Step 1: Add the explicit vocabulary** + +Near the existing public UX review constants, add: + +```python +SUCCESSFUL_REVIEW_CLASSIFICATIONS = frozenset({"resolved", "resolved_post_fix"}) +``` + +Add `resolved_post_fix` to the rendered review-log classification vocabulary, describing it as a fresh verified recapture after a fix. + +- [ ] **Step 2: Use the vocabulary in problem-row classification** + +Change the status loop condition from the literal two-value comparison to: + +```python +if classification != "pending" and classification not in SUCCESSFUL_REVIEW_CLASSIFICATIONS: +``` + +Do not use prefix matching and do not modify raw classification counts. + +- [ ] **Step 3: Sum successful outcomes in project status** + +Import `SUCCESSFUL_REVIEW_CLASSIFICATIONS` into `src/project_status.py` and replace the literal `resolved` count with: + +```python +resolved = sum(int(counts.get(classification) or 0) for classification in SUCCESSFUL_REVIEW_CLASSIFICATIONS) +``` + +- [ ] **Step 4: Run the red tests and verify green** + +Run the exact focused command from Task 1 Step 3. + +Expected: 2 passed. + +- [ ] **Step 5: Run both complete focused modules** + +Run: + +```bash +python3 -m pytest tests/test_public_ux_review_checklist.py tests/test_project_status.py -q +``` + +Expected: all tests pass. + +- [ ] **Step 6: Prove unknown outcomes still fail closed** + +Add or extend a checklist test to record `resolved_typo` for one completed row and assert: + +```python +assert status["status"] == "review_has_deferred_or_limited_items" +assert status["share_review_gate"] == "review_limited" +assert status["problem_rows"][0]["classification"] == "resolved_typo" +``` + +Run the complete focused modules again and expect all tests to pass. + +--- + +### Task 3: Record the corrected maturity state + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/DASHBOARD_QA.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: focused green evidence and the existing ten-row `/tmp` audit status. +- Produces: truthful local-only roadmap and continuation language. + +- [ ] **Step 1: Verify the live local notes status without editing it** + +Run: + +```bash +make public-ux-review-notes-check +make project-status-check +``` + +Expected: + +- `status: review_complete` +- `share_review_gate: share_review_ready` +- raw counts `resolved: 9, resolved_post_fix: 1` +- project status evidence `10/10 public desktop/mobile review rows resolved` + +- [ ] **Step 2: Update ROADMAP.md** + +Record that the ten desktop/phone rows are locally `share_review_ready`, with nine direct resolutions and one verified post-fix recapture. Preserve the boundary that this is screenshot-based local QA, not hosted, accessibility-conformance, external-reviewer, freshness, demand, or market evidence. + +- [ ] **Step 3: Update docs/DASHBOARD_QA.md** + +Document the explicit meaning of `resolved_post_fix`, the final 390x844 recapture, and the fail-closed treatment of unknown or incomplete labels. + +- [ ] **Step 4: Update the continuation contract** + +Add the implementation anchor and corrected public UX status to `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`. Keep the exact next external maturity lane as one permitted point-in-time consensus snapshot and source review. + +- [ ] **Step 5: Run documentation and focused contract tests** + +Run: + +```bash +python3 -m pytest \ + tests/test_public_ux_review_checklist.py \ + tests/test_project_status.py \ + tests/test_launchers.py \ + tests/test_pilot_review_feedback_template.py \ + tests/test_public_v1_release_docs.py -q +``` + +Expected: all tests pass. + +--- + +### Task 4: Verify, package, and synchronize the slice + +**Files:** +- No planned repository modifications; stop and return to the relevant earlier task if verification exposes a slice-related defect. + +**Interfaces:** +- Consumes: completed implementation and documentation. +- Produces: one clean coherent commit, synchronized draft PR, and exact-head CI evidence. + +- [ ] **Step 1: Run full local verification** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make commercial-beta-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +make pr-range-hygiene-check +git diff --check +``` + +Expected: commands exit zero. The pilot product verdict may remain truthfully blocked only because readiness is stale. + +- [ ] **Step 2: Review and stage exact files** + +Run `git diff --` for the two production files, two test files, and three documentation files. Stage only those exact intentional paths. Do not stage `/tmp`, data, outputs, screenshots, or generated artifacts. + +- [ ] **Step 3: Verify the staged package** + +Run: + +```bash +make staged-hygiene-check +git diff --cached --check +``` + +Expected: only product/code/docs/test files; zero generated or manual-review paths. + +- [ ] **Step 4: Commit and push** + +Commit with: + +```bash +git commit -m "Recognize verified public UX post-fix outcomes" +git push origin codex/personal-research-mode-mvp +``` + +- [ ] **Step 5: Update draft PR #113** + +Post a concise PR update covering the root cause, explicit successful-outcome contract, raw-count preservation, fail-closed unknown labels, tests, generated-artifact exclusion, and unchanged external gates. Keep the PR draft. + +- [ ] **Step 6: Verify exact-head GitHub CI and final repository state** + +Wait for the Commercial Research Beta workflow on the exact pushed SHA. Require success, then verify: + +```bash +git status --short --branch +git rev-list --left-right --count @{upstream}...HEAD +gh pr view 113 --json state,isDraft,mergeable,headRefOid,statusCheckRollup,url +``` + +Expected: clean, `0 0`, PR open/draft/mergeable, exact head, successful CI. diff --git a/docs/superpowers/plans/2026-07-22-field-proof-stage-b-audit.md b/docs/superpowers/plans/2026-07-22-field-proof-stage-b-audit.md new file mode 100644 index 000000000..8ae46bf9d --- /dev/null +++ b/docs/superpowers/plans/2026-07-22-field-proof-stage-b-audit.md @@ -0,0 +1,43 @@ +# Stage B Prospective Field-Proof Audit Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add deterministic read-only field-proof history audit and clearer per-row preview explanations without adding any write, readiness, or Workbench mapping. + +**Architecture:** Extend the pure `prospective_field_proof` module with immutable audit rows and an aggregate audit contract derived from the already validated append-only ledger. Expose stdout-only text/JSON through one Make target. Reuse the current source-rights registry only to explain active-head commercial blockers. + +**Tech Stack:** Python 3.12, frozen dataclasses, argparse, Make, pytest. + +--- + +### Task 1: Define the read-only audit contract test-first + +**Files:** `src/prospective_field_proof.py`, `tests/test_prospective_field_proof.py` + +- [ ] Add failing tests for absent, two-revision, rejected/follow-up, invalid, deterministic ordering, and no-write audits. +- [ ] Implement frozen audit row/summary contracts, active-head resolution, revision numbers, blocker aggregation, and text rendering. +- [ ] Add the `audit` CLI with controlled invalid-ledger errors and stable JSON. +- [ ] Run `python3 -m pytest tests/test_prospective_field_proof.py -q`. + +### Task 2: Improve preview explanation and expose the Make target + +**Files:** `src/prospective_field_proof.py`, `Makefile`, `tests/test_makefile_test_targets.py` + +- [ ] Add failing tests for per-row preview answers, receipt-binding copy, and a read-only Make audit target. +- [ ] Render row state/reason/blockers and receipt persistence boundary. +- [ ] Add `prospective-field-proof-audit` without any output-file parameter or write command. +- [ ] Run focused module and Make-target tests. + +### Task 3: Reconcile documentation and maturity status + +**Files:** `README.md`, `ROADMAP.md`, `docs/OPERATOR_GUIDE.md`, `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, `tests/test_public_v1_release_docs.py` + +- [ ] Document the audit answer, receipt boundary, and no-mapping rule. +- [ ] Mark Stage B complete locally only after all direct tests pass and set Priority 3 authoring design as next. + +### Task 4: Verify and release the slice + +- [ ] Run focused tests, `python3 -m pytest tests -q`, dashboard/render/public/commercial-beta/pilot/hygiene gates, and `git diff --check`. +- [ ] Confirm audit and preview did not change any tracked or untracked scoped artifact. +- [ ] Stage exact code/test/docs/Makefile paths only; run staged hygiene. +- [ ] Commit, push only the feature branch, update draft PR #113, and require exact-head CI before Priority 3. diff --git a/docs/superpowers/plans/2026-07-22-in-app-research-record-authoring.md b/docs/superpowers/plans/2026-07-22-in-app-research-record-authoring.md new file mode 100644 index 000000000..056ee5ef1 --- /dev/null +++ b/docs/superpowers/plans/2026-07-22-in-app-research-record-authoring.md @@ -0,0 +1,931 @@ +# In-App Research-Record Authoring Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add safe command-line-free thesis, evidence, catalyst, and outcome authoring to Company Workbench through an exact `Validate -> Preview -> Confirm and save` flow over the existing append-only ledgers. + +**Architecture:** A new pure composition module owns record construction, existing-validator dispatch, cross-ledger references, preview receipts, ledger fingerprints, and one-ledger confirmation. A small Streamlit component owns widgets and session state and is called from Company Workbench after the journal answer; production dashboard code never opens a ledger directly. Tests use temporary ledgers and a test-only Streamlit fixture, while the product route renders the composer closed by default. + +**Tech Stack:** Python 3.12, frozen dataclasses, `hashlib`/`json`/`uuid`, existing CSV ledger engines, Streamlit, pytest, Streamlit AppTest, existing render/release gates. + +## Global Constraints + +- Research-only; no investment advice, recommendation, company ranking, expected-return score, direct buy/sell instruction, allocation, position sizing, live holding, account import, broker connection, order routing, auto-trading, or post-earnings price prediction. +- Reuse `research_thesis_journal.py`, `catalyst_evidence_timeline.py`, and `research_outcome_review.py`; do not create a generic ledger or duplicate their validators. +- Drafts, previews, and the session-only receipt are untrusted session state. Candidate context cannot become trusted evidence, and no receipt is persisted to a ledger or generated artifact. +- Profile and ticker are locked to the active Company Workbench context. +- Preview writes nothing. Confirmation requires an exact current draft, unchanged current ledger fingerprint, unchanged context, and explicit reviewed-source confirmation. +- A successful confirmation appends exactly one row to exactly one established ledger and consumes the preview receipt. +- Tests use temporary ledgers only. Do not append production research data during implementation or verification. +- Do not change readiness, canonical data, source rights, forecasts, DCF assumptions, scenarios, consensus, valuation, peers, backtests, calibration, or numerical probability. +- Empty ledgers remain empty until an explicit successful confirmation targets that exact ledger. +- EPS split basis remains unverified without explicit primary proof; Q4 actuals still require an explicit SEC-filed Q4 table; synthetic fixtures remain test-only. +- Do not run `make readiness`, broad refresh/import/apply commands, or generated CSV/JSON/report/sample-report/screenshot/timing commands. +- Stage exact intentional files only; never use `git add -A`; keep the existing 18 generated CSV/report changes unstaged. + +## File Structure + +- Create `src/research_record_authoring.py`: immutable authoring contracts, record mapping, preview validation, receipt and ledger fingerprints, cross-ledger reference checks, and confirmed dispatch. +- Create `src/research_record_authoring_ui.py`: record-specific field definitions, Streamlit widget rendering, session preview invalidation, confirmation, and saved-result feedback. +- Create `tests/test_research_record_authoring.py`: pure composition, read-only preview, stale receipt, cross-scope, one-ledger-only, and no-fabrication tests. +- Create `tests/fixtures/research_record_authoring_app.py`: test-only Streamlit host wired exclusively to temporary paths supplied by the test process. +- Create `tests/test_research_record_authoring_ui.py`: field-model and AppTest interaction coverage without production writes. +- Modify `src/dashboard.py`: import and render the composer after the journal/outcome answer only in Personal Research Company Workbench. +- Modify `tests/test_dashboard_helpers.py`: source-boundary and placement contracts. +- Modify `tests/test_dashboard_render_smoke.py`: closed-by-default Workbench render contract. +- Modify `src/browser_qa_evidence.py` and `tests/test_browser_qa_evidence.py`: add the collapsed authoring marker and mobile stop rule without claiming save evidence from screenshots. +- Modify `README.md`, `docs/PRODUCT_SPEC.md`, `ROADMAP.md`, `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, and `tests/test_public_v1_release_docs.py`: document the implemented workflow and truthful boundaries after runtime evidence exists. + +--- + +### Task 1: Pure Four-Kind Preview Composition + +**Files:** +- Create: `src/research_record_authoring.py` +- Create: `tests/test_research_record_authoring.py` + +**Interfaces:** +- Produces: `AuthoringPaths`, `AuthoringDraft`, `AuthoringPreview`, `build_authoring_draft()`, `authoring_draft_digest()`, and `preview_authoring_record()`. +- Consumes: existing `JournalEntry`, `CatalystEvent`, `ResearchOutcome`, their load functions, and their preview/validation functions. +- Invariant: preview reads only the targeted ledger plus the thesis journal needed for evidence/outcome references and never creates a path. + +- [ ] **Step 1: Write failing preview tests** + +Add these contracts to `tests/test_research_record_authoring.py`: + +```python +from dataclasses import replace +from pathlib import Path + +import pytest + +from src.research_record_authoring import ( + AuthoringPaths, + build_authoring_draft, + preview_authoring_record, +) +from src.research_thesis_journal import JournalEntry, append_journal_entry + + +def _paths(tmp_path: Path) -> AuthoringPaths: + return AuthoringPaths( + journal=tmp_path / "research_thesis_journal.csv", + catalysts=tmp_path / "catalyst_evidence.csv", + outcomes=tmp_path / "research_outcome_reviews.csv", + ) + + +def _thesis_entry() -> JournalEntry: + return JournalEntry( + schema_version="research-thesis-journal-v1", + entry_id="entry-existing", + profile_key="demo", + ticker="SYN1", + thesis_id="thesis-syn1", + entry_type="thesis", + recorded_at="2026-07-20T12:00:00Z", + effective_at="2026-07-20T11:00:00Z", + reviewer="fixture-reviewer", + summary="Existing synthetic thesis.", + evidence_direction="", + source="", + source_ref="", + source_published_at="", + confidence="0.50", + review_due_date="2026-08-20", + supersedes_entry_id="", + ) + + +@pytest.mark.parametrize( + ("kind", "fields", "destination"), + ( + ("thesis", {"thesis_id": "thesis-new", "summary": "Reviewed hypothesis.", "effective_at": "2026-07-22T10:00:00Z", "reviewer": "owner", "confidence": "0.60", "review_due_date": "2026-08-22", "supersedes_entry_id": ""}, "research_thesis_journal.csv"), + ("evidence", {"thesis_id": "thesis-syn1", "summary": "Source-backed evidence.", "effective_at": "2026-07-22T10:00:00Z", "reviewer": "owner", "evidence_direction": "supporting", "source": "company_ir", "source_ref": "https://example.invalid/source", "source_published_at": "2026-07-22T09:00:00Z"}, "research_thesis_journal.csv"), + ("catalyst", {"event_type": "earnings", "title": "Scheduled results", "summary": "Reviewed event context.", "effective_at": "2026-08-20T21:00:00Z", "published_at": "2026-07-22T09:00:00Z", "retrieved_at": "2026-07-22T10:00:00Z", "source": "company_ir", "source_ref": "https://example.invalid/event", "evidence_state": "candidate_context_only", "reviewer": "owner"}, "catalyst_evidence.csv"), + ("outcome", {"thesis_id": "thesis-syn1", "original_thesis_entry_id": "entry-existing", "reviewed_at": "2026-07-22T12:00:00Z", "observation_start": "2026-07-20T12:00:00Z", "observation_end": "2026-07-22T11:00:00Z", "reviewer": "owner", "outcome_state": "mixed", "summary": "Reviewed outcome.", "source": "reviewed_research_record", "source_ref": "journal://entry-existing", "source_published_at": "2026-07-22T11:00:00Z", "learning": "Separate the evidence lanes."}, "research_outcome_reviews.csv"), + ), +) +def test_preview_maps_all_four_kinds_without_writing(tmp_path, kind, fields, destination): + paths = _paths(tmp_path) + append_journal_entry(paths.journal, _thesis_entry()) + before = {path: path.read_bytes() if path.exists() else None for path in paths.all()} + draft = build_authoring_draft(kind, profile_key="demo", ticker="syn1", fields=fields) + + preview = preview_authoring_record( + draft, + paths=paths, + previewed_at="2026-07-22T12:30:00Z", + generated_id=f"{kind}-generated", + ) + + assert preview.state == "reviewable" + assert preview.profile_key == "demo" + assert preview.ticker == "SYN1" + assert preview.destination_label == destination + assert preview.write_performed is False + assert preview.receipt + assert {path: path.read_bytes() if path.exists() else None for path in paths.all()} == before + + +def test_preview_rejects_cross_scope_evidence_and_outcome_references(tmp_path): + paths = _paths(tmp_path) + append_journal_entry(paths.journal, _thesis_entry()) + fields = {"thesis_id": "thesis-syn1", "summary": "Evidence.", "effective_at": "2026-07-22T10:00:00Z", "reviewer": "owner", "evidence_direction": "supporting", "source": "company_ir", "source_ref": "ref", "source_published_at": "2026-07-22T09:00:00Z"} + + preview = preview_authoring_record( + build_authoring_draft("evidence", profile_key="other", ticker="SYN1", fields=fields), + paths=paths, + previewed_at="2026-07-22T12:30:00Z", + generated_id="evidence-generated", + ) + + assert preview.state == "rejected" + assert preview.reason == "thesis_id must reference an existing thesis in this profile and ticker" + assert preview.receipt == "" +``` + +- [ ] **Step 2: Run the tests and verify the missing-module failure** + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_record_authoring.py -q` + +Expected: collection fails with `ModuleNotFoundError: No module named 'src.research_record_authoring'`. + +- [ ] **Step 3: Implement immutable preview composition** + +Create `src/research_record_authoring.py` with these public contracts and complete internal dispatch: + +```python +from __future__ import annotations + +import hashlib +import json +from dataclasses import asdict, dataclass +from pathlib import Path +from types import MappingProxyType +from typing import Mapping + +from src.catalyst_evidence_timeline import CatalystEvent, load_catalyst_events, preview_event +from src.research_outcome_review import ResearchOutcome, load_outcomes, preview_outcome +from src.research_thesis_journal import ( + JOURNAL_SCHEMA_VERSION, + JournalEntry, + load_journal_entries, + validate_journal_entry, +) + +RECORD_KINDS = ("thesis", "evidence", "catalyst", "outcome") + + +@dataclass(frozen=True) +class AuthoringPaths: + journal: Path + catalysts: Path + outcomes: Path + + def all(self) -> tuple[Path, Path, Path]: + return (self.journal, self.catalysts, self.outcomes) + + +@dataclass(frozen=True) +class AuthoringDraft: + record_kind: str + profile_key: str + ticker: str + fields: tuple[tuple[str, str], ...] + + def field_map(self) -> Mapping[str, str]: + return MappingProxyType(dict(self.fields)) + + +@dataclass(frozen=True) +class AuthoringPreview: + state: str + reason: str + record_kind: str + profile_key: str + ticker: str + destination_label: str + previewed_at: str + persisted_fields: tuple[tuple[str, str], ...] + receipt: str + draft_digest: str + ledger_fingerprint: str + record: JournalEntry | CatalystEvent | ResearchOutcome | None + write_performed: bool = False + + +def build_authoring_draft(record_kind: str, *, profile_key: str, ticker: str, fields: Mapping[str, object]) -> AuthoringDraft: + kind = str(record_kind or "").strip().lower() + if kind not in RECORD_KINDS: + raise ValueError(f"Unsupported record kind: {record_kind!r}") + profile = str(profile_key or "").strip() + symbol = str(ticker or "").strip().upper() + if not profile: + raise ValueError("profile_key is required") + if not symbol: + raise ValueError("ticker is required") + normalized = tuple(sorted((str(key), str(value or "").strip()) for key, value in fields.items())) + return AuthoringDraft(kind, profile, symbol, normalized) + + +def _stable_digest(payload: object) -> str: + encoded = json.dumps(payload, sort_keys=True, separators=(",", ":"), ensure_ascii=True).encode() + return hashlib.sha256(encoded).hexdigest() + + +def authoring_draft_digest(draft: AuthoringDraft) -> str: + return _stable_digest(asdict(draft)) + + +def _ledger_fingerprint(path: Path) -> str: + payload = b"missing\0" if not path.exists() else b"present\0" + path.read_bytes() + return hashlib.sha256(payload).hexdigest() + + +def _destination(draft: AuthoringDraft, paths: AuthoringPaths) -> Path: + return paths.journal if draft.record_kind in {"thesis", "evidence"} else paths.catalysts if draft.record_kind == "catalyst" else paths.outcomes + + +def _scoped_theses(paths: AuthoringPaths, draft: AuthoringDraft) -> tuple[JournalEntry, ...]: + return tuple(row for row in load_journal_entries(paths.journal) if row.entry_type == "thesis" and row.profile_key == draft.profile_key and row.ticker.upper() == draft.ticker) + + +def _build_record(draft: AuthoringDraft, *, previewed_at: str, generated_id: str, paths: AuthoringPaths): + values = dict(draft.fields) + if draft.record_kind in {"thesis", "evidence"}: + if draft.record_kind == "evidence" and not any(row.thesis_id == values.get("thesis_id") for row in _scoped_theses(paths, draft)): + raise ValueError("thesis_id must reference an existing thesis in this profile and ticker") + return JournalEntry( + schema_version=JOURNAL_SCHEMA_VERSION, + entry_id=generated_id, + profile_key=draft.profile_key, + ticker=draft.ticker, + thesis_id=values.get("thesis_id", ""), + entry_type=draft.record_kind, + recorded_at=previewed_at, + effective_at=values.get("effective_at", ""), + reviewer=values.get("reviewer", ""), + summary=values.get("summary", ""), + evidence_direction=values.get("evidence_direction", ""), + source=values.get("source", ""), + source_ref=values.get("source_ref", ""), + source_published_at=values.get("source_published_at", ""), + confidence=values.get("confidence", ""), + review_due_date=values.get("review_due_date", ""), + supersedes_entry_id=values.get("supersedes_entry_id", ""), + ) + if draft.record_kind == "catalyst": + return CatalystEvent(schema_version="catalyst-evidence-v1", event_id=generated_id, profile_key=draft.profile_key, ticker=draft.ticker, **values) + theses = _scoped_theses(paths, draft) + if not any(row.thesis_id == values.get("thesis_id") and row.entry_id == values.get("original_thesis_entry_id") for row in theses): + raise ValueError("outcome must reference an existing thesis entry in this profile and ticker") + return ResearchOutcome(schema_version="research-outcome-review-v1", outcome_id=generated_id, profile_key=draft.profile_key, ticker=draft.ticker, **values) + + +def preview_authoring_record(draft: AuthoringDraft, *, paths: AuthoringPaths, previewed_at: str, generated_id: str) -> AuthoringPreview: + destination = _destination(draft, paths) + draft_digest = authoring_draft_digest(draft) + ledger_fingerprint = _ledger_fingerprint(destination) + try: + record = _build_record(draft, previewed_at=previewed_at, generated_id=generated_id, paths=paths) + if isinstance(record, JournalEntry): + validate_journal_entry(record, existing_entries=load_journal_entries(paths.journal)) + elif isinstance(record, CatalystEvent): + event_preview = preview_event(record, existing=load_catalyst_events(paths.catalysts)) + if event_preview.state == "rejected": + raise ValueError(event_preview.reason) + else: + outcome_preview = preview_outcome(record, existing=load_outcomes(paths.outcomes)) + if outcome_preview.state == "rejected": + raise ValueError(outcome_preview.reason) + except (OSError, ValueError) as exc: + return AuthoringPreview("rejected", str(exc), draft.record_kind, draft.profile_key, draft.ticker, destination.name, previewed_at, (), "", draft_digest, ledger_fingerprint, None) + persisted = tuple((key, str(value)) for key, value in asdict(record).items()) + receipt = _stable_digest({"draft": draft_digest, "ledger": ledger_fingerprint, "record": persisted, "destination": destination.name}) + return AuthoringPreview("reviewable", "", draft.record_kind, draft.profile_key, draft.ticker, destination.name, previewed_at, persisted, receipt, draft_digest, ledger_fingerprint, record) +``` + +The implementation must catch read/validation errors as rejected previews, but must not catch programmer errors such as an unexpected dataclass shape. + +- [ ] **Step 4: Run focused preview tests** + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_record_authoring.py -q` + +Expected: all Task 1 tests pass; no file other than the pre-seeded temporary journal changes. + +- [ ] **Step 5: Commit Task 1** + +Run: + +```bash +git add -- src/research_record_authoring.py tests/test_research_record_authoring.py +git diff --cached --check +make staged-hygiene-check +git commit -m "Add research record authoring previews" +``` + +### Task 2: Exact Confirmation, Stale Receipts, And One-Ledger Dispatch + +**Files:** +- Modify: `src/research_record_authoring.py` +- Modify: `tests/test_research_record_authoring.py` + +**Interfaces:** +- Produces: `AuthoringSaveResult` and `confirm_authoring_preview()`. +- Consumes: the Task 1 immutable preview and the three existing append functions. +- Invariant: any context, draft, receipt, or ledger mismatch returns `preview_stale` and writes nothing. + +- [ ] **Step 1: Add failing confirmation tests** + +Add tests that seed a valid preview, then prove confirmation is denied without the checkbox, denied after draft edits, denied after a concurrent append, and writes only its selected ledger: + +```python +from src.research_record_authoring import confirm_authoring_preview +from src.research_thesis_journal import load_journal_entries + + +def test_confirmation_requires_review_and_appends_exactly_one_ledger(tmp_path): + paths = _paths(tmp_path) + draft = build_authoring_draft("thesis", profile_key="demo", ticker="SYN1", fields={"thesis_id": "thesis-new", "summary": "Reviewed hypothesis.", "effective_at": "2026-07-22T10:00:00Z", "reviewer": "owner", "confidence": "0.60", "review_due_date": "2026-08-22", "supersedes_entry_id": ""}) + preview = preview_authoring_record(draft, paths=paths, previewed_at="2026-07-22T12:30:00Z", generated_id="thesis-generated") + + denied = confirm_authoring_preview(preview, current_draft=draft, paths=paths, active_profile_key="demo", active_ticker="SYN1", active_kind="thesis", confirm_reviewed=False) + assert denied.state == "confirmation_required" + assert not any(path.exists() for path in paths.all()) + + saved = confirm_authoring_preview(preview, current_draft=draft, paths=paths, active_profile_key="demo", active_ticker="SYN1", active_kind="thesis", confirm_reviewed=True) + assert saved.state == "saved" + assert saved.record_id == "thesis-generated" + assert saved.write_performed is True + assert [row.entry_id for row in load_journal_entries(paths.journal)] == ["thesis-generated"] + assert not paths.catalysts.exists() + assert not paths.outcomes.exists() + + +def test_changed_draft_or_ledger_invalidates_preview_without_writing(tmp_path): + paths = _paths(tmp_path) + append_journal_entry(paths.journal, _thesis_entry()) + fields = {"thesis_id": "thesis-syn1", "summary": "Evidence.", "effective_at": "2026-07-22T10:00:00Z", "reviewer": "owner", "evidence_direction": "supporting", "source": "company_ir", "source_ref": "ref", "source_published_at": "2026-07-22T09:00:00Z"} + draft = build_authoring_draft("evidence", profile_key="demo", ticker="SYN1", fields=fields) + preview = preview_authoring_record(draft, paths=paths, previewed_at="2026-07-22T12:30:00Z", generated_id="evidence-generated") + baseline = paths.journal.read_bytes() + + edited = build_authoring_draft("evidence", profile_key="demo", ticker="SYN1", fields={**fields, "summary": "Edited after preview."}) + stale_draft = confirm_authoring_preview(preview, current_draft=edited, paths=paths, active_profile_key="demo", active_ticker="SYN1", active_kind="evidence", confirm_reviewed=True) + assert stale_draft.state == "preview_stale" + assert paths.journal.read_bytes() == baseline + + append_journal_entry(paths.journal, replace(_thesis_entry(), entry_id="entry-concurrent", thesis_id="thesis-other")) + concurrent = paths.journal.read_bytes() + stale_ledger = confirm_authoring_preview(preview, current_draft=draft, paths=paths, active_profile_key="demo", active_ticker="SYN1", active_kind="evidence", confirm_reviewed=True) + assert stale_ledger.state == "preview_stale" + assert paths.journal.read_bytes() == concurrent +``` + +- [ ] **Step 2: Run and verify the missing-interface failure** + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_record_authoring.py -q` + +Expected: import fails because `confirm_authoring_preview` does not exist. + +- [ ] **Step 3: Implement exact confirmation** + +Add this public contract and explicit dispatch to `src/research_record_authoring.py`: + +```python +import hmac + +from src.catalyst_evidence_timeline import append_reviewed_event +from src.research_outcome_review import append_reviewed_outcome +from src.research_thesis_journal import append_journal_entry + + +@dataclass(frozen=True) +class AuthoringSaveResult: + state: str + reason: str + record_kind: str + record_id: str + destination_label: str + write_performed: bool + + +def _record_id(record: JournalEntry | CatalystEvent | ResearchOutcome) -> str: + if isinstance(record, JournalEntry): + return record.entry_id + if isinstance(record, CatalystEvent): + return record.event_id + return record.outcome_id + + +def confirm_authoring_preview( + preview: AuthoringPreview, + *, + current_draft: AuthoringDraft, + paths: AuthoringPaths, + active_profile_key: str, + active_ticker: str, + active_kind: str, + confirm_reviewed: bool, +) -> AuthoringSaveResult: + rejected = lambda state, reason: AuthoringSaveResult(state, reason, preview.record_kind, "", preview.destination_label, False) + if not confirm_reviewed: + return rejected("confirmation_required", "Review and confirm the exact preview before saving.") + if preview.state != "reviewable" or preview.record is None or not preview.receipt: + return rejected("rejected", "Only a valid preview can be confirmed.") + active = (str(active_profile_key or "").strip(), str(active_ticker or "").strip().upper(), str(active_kind or "").strip().lower()) + if active != (preview.profile_key, preview.ticker, preview.record_kind): + return rejected("preview_stale", "Selected profile, ticker, or record type changed; preview again.") + if not hmac.compare_digest(_stable_digest(asdict(current_draft)), preview.draft_digest): + return rejected("preview_stale", "Draft changed after preview; preview again.") + destination = _destination(current_draft, paths) + if not hmac.compare_digest(_ledger_fingerprint(destination), preview.ledger_fingerprint): + return rejected("preview_stale", "Ledger changed after preview; reload and preview again.") + refreshed = preview_authoring_record(current_draft, paths=paths, previewed_at=preview.previewed_at, generated_id=_record_id(preview.record)) + if refreshed.state != "reviewable" or not hmac.compare_digest(refreshed.receipt, preview.receipt): + return rejected("preview_stale", "Record no longer matches the validated preview; preview again.") + try: + if isinstance(preview.record, JournalEntry): + append_journal_entry(paths.journal, preview.record) + elif isinstance(preview.record, CatalystEvent): + append_reviewed_event(paths.catalysts, preview.record, confirm_reviewed=True) + else: + append_reviewed_outcome(paths.outcomes, preview.record, confirm_reviewed=True) + except (OSError, ValueError) as exc: + return rejected("save_failed", f"Record was not saved: {exc}") + return AuthoringSaveResult("saved", "Saved append-only reviewed record.", preview.record_kind, _record_id(preview.record), destination.name, True) +``` + +- [ ] **Step 4: Run focused and adjacent persistence tests** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_record_authoring.py tests/test_research_thesis_journal.py tests/test_catalyst_evidence_timeline.py tests/test_research_outcome_review.py -q +``` + +Expected: all tests pass; byte assertions prove rejected confirmations are write-free. + +- [ ] **Step 5: Commit Task 2** + +Run: + +```bash +git add -- src/research_record_authoring.py tests/test_research_record_authoring.py +git diff --cached --check +make staged-hygiene-check +git commit -m "Add exact research record confirmation" +``` + +### Task 3: Streamlit Composer And Session-State Contract + +**Files:** +- Create: `src/research_record_authoring_ui.py` +- Create: `tests/test_research_record_authoring_ui.py` +- Create: `tests/fixtures/research_record_authoring_app.py` + +**Interfaces:** +- Produces: `authoring_field_contract()`, `authoring_session_key()`, and `render_research_record_authoring()`. +- Consumes: Task 1/2 composition APIs and an injected `AuthoringPaths`. +- Invariant: widgets never receive editable profile/ticker values, and the fixture refuses paths outside the temporary directory passed by the test. +- Usability contract: controlled vocabularies use select controls, and thesis/supersession references are selected only from thesis rows in the active profile/ticker scope. + +- [ ] **Step 1: Write failing field and AppTest contracts** + +Add tests requiring the four record types, locked scope, source fields only where appropriate, preview-before-confirm order, and no production paths: + +```python +from pathlib import Path + +from streamlit.testing.v1 import AppTest + +from src.research_record_authoring import AuthoringPaths +from src.research_record_authoring_ui import authoring_field_contract, authoring_session_key +from src.research_thesis_journal import JournalEntry, append_journal_entry + + +def _paths(tmp_path: Path) -> AuthoringPaths: + return AuthoringPaths(tmp_path / "journal.csv", tmp_path / "catalysts.csv", tmp_path / "outcomes.csv") + + +def _thesis_entry() -> JournalEntry: + return JournalEntry( + "research-thesis-journal-v1", "entry-existing", "demo", "SYN1", "thesis-syn1", "thesis", + "2026-07-20T12:00:00Z", "2026-07-20T11:00:00Z", "fixture-reviewer", + "Existing synthetic thesis.", "", "", "", "", "0.50", "2026-08-20", "", + ) + + +def test_field_contract_is_kind_specific_and_never_exposes_scope_for_editing(): + thesis = authoring_field_contract("thesis") + evidence = authoring_field_contract("evidence") + catalyst = authoring_field_contract("catalyst") + outcome = authoring_field_contract("outcome") + + assert "profile_key" not in thesis and "ticker" not in thesis + assert tuple(thesis) == ("thesis_id", "summary", "effective_at", "reviewer", "confidence", "review_due_date", "supersedes_entry_id") + assert {"evidence_direction", "source", "source_ref", "source_published_at"} <= set(evidence) + assert {"event_type", "evidence_state", "retrieved_at"} <= set(catalyst) + assert {"original_thesis_entry_id", "observation_start", "observation_end", "learning"} <= set(outcome) + assert all("return" not in field and "skill" not in field and "rank" not in field for field in outcome) + + +def test_session_key_is_profile_ticker_and_kind_scoped(): + assert authoring_session_key("demo", "nvda", "preview") == "research-authoring:demo:NVDA:preview" + + +def test_fixture_renders_locked_scope_and_no_confirmation_before_preview(tmp_path, monkeypatch): + monkeypatch.setenv("RESEARCH_AUTHORING_FIXTURE_DIR", str(tmp_path)) + app = AppTest.from_file("tests/fixtures/research_record_authoring_app.py") + app.run(timeout=20) + + assert not app.exception + assert "Profile: demo | Ticker: SYN1" in "\n".join(item.value for item in app.markdown) + assert any(item.label == "Validate and preview" for item in app.button) + assert not any(item.label == "Confirm and save" for item in app.button) + assert not any(tmp_path.iterdir()) + + +def test_fixture_uses_controlled_choices_and_scoped_thesis_references(tmp_path, monkeypatch): + paths = _paths(tmp_path) + append_journal_entry(paths.journal, _thesis_entry()) + monkeypatch.setenv("RESEARCH_AUTHORING_FIXTURE_DIR", str(tmp_path)) + app = AppTest.from_file("tests/fixtures/research_record_authoring_app.py").run(timeout=20) + app.selectbox(key="research-authoring:demo:SYN1:kind").set_value("evidence").run() + + assert app.selectbox(key="research-authoring:demo:SYN1:field:evidence:thesis_id").options == ["thesis-syn1"] + assert app.selectbox(key="research-authoring:demo:SYN1:field:evidence:evidence_direction").options == ["supporting", "conflicting", "context"] +``` + +- [ ] **Step 2: Verify the missing-UI-module failure** + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_record_authoring_ui.py -q` + +Expected: collection fails because `src.research_record_authoring_ui` does not exist. + +- [ ] **Step 3: Implement field contracts and renderer** + +Create `src/research_record_authoring_ui.py` with immutable field definitions and a renderer using widgets outside a Streamlit form so every edit causes a rerun and invalidates a mismatched preview: + +```python +from __future__ import annotations + +from datetime import UTC, datetime +from typing import Any, Mapping +from uuid import uuid4 + +from src.research_record_authoring import ( + AuthoringPaths, + AuthoringPreview, + authoring_draft_digest, + build_authoring_draft, + confirm_authoring_preview, + preview_authoring_record, +) + +FIELD_CONTRACTS = { + "thesis": ("thesis_id", "summary", "effective_at", "reviewer", "confidence", "review_due_date", "supersedes_entry_id"), + "evidence": ("thesis_id", "summary", "effective_at", "reviewer", "evidence_direction", "source", "source_ref", "source_published_at"), + "catalyst": ("event_type", "title", "summary", "effective_at", "published_at", "retrieved_at", "source", "source_ref", "evidence_state", "reviewer"), + "outcome": ("thesis_id", "original_thesis_entry_id", "reviewed_at", "observation_start", "observation_end", "reviewer", "outcome_state", "summary", "source", "source_ref", "source_published_at", "learning"), +} + +SELECT_OPTIONS = { + "evidence_direction": ("supporting", "conflicting", "context"), + "event_type": ("earnings", "product", "regulatory", "customer", "industry", "capital_allocation", "management", "macro"), + "evidence_state": ("candidate_context_only", "supported", "still_blocked", "skipped", "excluded"), + "outcome_state": ("supported", "mixed", "not_supported", "inconclusive"), +} + + +def authoring_field_contract(record_kind: str) -> tuple[str, ...]: + try: + return FIELD_CONTRACTS[record_kind] + except KeyError as exc: + raise ValueError(f"Unsupported record kind: {record_kind!r}") from exc + + +def authoring_session_key(profile_key: str, ticker: str, suffix: str) -> str: + return f"research-authoring:{profile_key}:{ticker.upper()}:{suffix}" + + +def _generated_id(kind: str) -> str: + return f"{kind}-{uuid4().hex}" + + +def _utc_now() -> str: + return datetime.now(UTC).isoformat().replace("+00:00", "Z") + + +def _scoped_thesis_options(paths: AuthoringPaths, profile_key: str, ticker: str) -> tuple[tuple[str, str], ...]: + from src.research_thesis_journal import load_journal_entries + + return tuple( + (row.thesis_id, row.entry_id) + for row in load_journal_entries(paths.journal) + if row.entry_type == "thesis" and row.profile_key == profile_key and row.ticker.upper() == ticker.upper() + ) + + +def render_research_record_authoring(*, st_api: Any, profile_key: str, ticker: str, paths: AuthoringPaths) -> None: + symbol = ticker.upper() + with st_api.expander("Add a reviewed research record", expanded=False): + st_api.caption(f"Profile: {profile_key} | Ticker: {symbol} — locked to this Company Workbench.") + kind = st_api.selectbox("Record type", tuple(FIELD_CONTRACTS), format_func=str.title, key=authoring_session_key(profile_key, symbol, "kind")) + thesis_options = _scoped_thesis_options(paths, profile_key, symbol) + fields: dict[str, str] = {} + for name in authoring_field_contract(kind): + label = name.replace("_", " ").title() + key = authoring_session_key(profile_key, symbol, f"field:{kind}:{name}") + if name == "thesis_id" and kind in {"evidence", "outcome"}: + fields[name] = st_api.selectbox(label, tuple(dict.fromkeys(item[0] for item in thesis_options)), key=key) + elif name == "original_thesis_entry_id": + fields[name] = st_api.selectbox(label, tuple(item[1] for item in thesis_options if item[0] == fields.get("thesis_id")), key=key) + elif name == "supersedes_entry_id": + fields[name] = st_api.selectbox(label, ("", *(item[1] for item in thesis_options)), key=key) + elif name in SELECT_OPTIONS: + fields[name] = st_api.selectbox(label, SELECT_OPTIONS[name], key=key) + elif name in {"summary", "learning"}: + fields[name] = st_api.text_area(label, key=key) + else: + fields[name] = st_api.text_input(label, key=key) + draft = build_authoring_draft(kind, profile_key=profile_key, ticker=symbol, fields=fields) + preview_key = authoring_session_key(profile_key, symbol, "preview") + if st_api.button("Validate and preview", key=authoring_session_key(profile_key, symbol, "validate"), use_container_width=True): + st_api.session_state[preview_key] = preview_authoring_record(draft, paths=paths, previewed_at=_utc_now(), generated_id=_generated_id(kind)) + preview: AuthoringPreview | None = st_api.session_state.get(preview_key) + if preview is None: + st_api.caption("No record is saved until this draft passes preview and you confirm the exact reviewed source evidence.") + return + if preview.draft_digest != authoring_draft_digest(draft): + st_api.warning("Draft changed after preview. Validate and preview again before saving.") + return + if preview.state == "rejected": + st_api.error(preview.reason) + return + st_api.markdown("#### Exact append-only preview") + st_api.dataframe(dict(preview.persisted_fields), width="stretch") + confirmed = st_api.checkbox("I reviewed this exact record and its source evidence", key=authoring_session_key(profile_key, symbol, "confirmed")) + if st_api.button("Confirm and save", key=authoring_session_key(profile_key, symbol, "save"), use_container_width=True): + result = confirm_authoring_preview(preview, current_draft=draft, paths=paths, active_profile_key=profile_key, active_ticker=symbol, active_kind=kind, confirm_reviewed=confirmed) + if result.state == "saved": + del st_api.session_state[preview_key] + st_api.success(f"Saved {result.record_id}. Corrections require a new append-only record; history is never edited or deleted.") + else: + st_api.error(result.reason) +``` + +Create `tests/fixtures/research_record_authoring_app.py`: + +```python +import os +from pathlib import Path + +import streamlit as st + +from src.research_record_authoring import AuthoringPaths +from src.research_record_authoring_ui import render_research_record_authoring + +root = Path(os.environ["RESEARCH_AUTHORING_FIXTURE_DIR"]).resolve() +if not root.is_dir() or "pytest-" not in str(root): + raise RuntimeError("The authoring fixture requires a pytest temporary directory.") +render_research_record_authoring( + st_api=st, + profile_key="demo", + ticker="SYN1", + paths=AuthoringPaths(root / "journal.csv", root / "catalysts.csv", root / "outcomes.csv"), +) +``` + +- [ ] **Step 4: Expand AppTest through rejected preview and successful temporary save** + +Use widget keys rather than positional indexes. Enter a synthetic thesis, click `Validate and preview`, assert the exact preview and confirmation control appear while files remain absent, then check confirmation, click save, reload the temporary journal, and assert one row. Add a second test that edits the summary after preview and proves the save button is unavailable until re-preview. + +- [ ] **Step 5: Run focused UI and composition tests** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_record_authoring.py tests/test_research_record_authoring_ui.py -q +``` + +Expected: all tests pass; only pytest temporary directories contain new ledgers. + +- [ ] **Step 6: Commit Task 3** + +Run: + +```bash +git add -- src/research_record_authoring.py src/research_record_authoring_ui.py tests/test_research_record_authoring.py tests/test_research_record_authoring_ui.py tests/fixtures/research_record_authoring_app.py +git diff --cached --check +make staged-hygiene-check +git commit -m "Add research record authoring composer" +``` + +### Task 4: Company Workbench Integration And Read-Side Reload + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_dashboard_render_smoke.py` +- Modify: `tests/test_research_record_authoring_ui.py` + +**Interfaces:** +- Consumes: `render_research_record_authoring()` with production paths explicitly provided by Company Workbench. +- Produces: one collapsed composer after the journal/outcome answer and before Advanced thesis history. +- Invariant: Public and Operator routes do not render an authoring composer; only Personal Research Company Workbench can invoke it. + +- [ ] **Step 1: Add failing source and render contracts** + +Add source-boundary assertions to `tests/test_dashboard_helpers.py`: + +```python +def test_company_workbench_authoring_is_research_only_and_below_the_journal_answer(): + source = Path("src/dashboard.py").read_text(encoding="utf-8") + render_start = source.index("def render_single_stock_report(") + render_end = source.index("def ", render_start + 4) + render_source = source[render_start:render_end] + + assert "render_research_record_authoring" in render_source + assert render_source.index("research_thesis_journal_html") < render_source.index("render_research_record_authoring") + assert render_source.index("render_research_record_authoring") < render_source.index("Advanced: thesis and evidence history") + assert "if research_mode:" in render_source + assert "AuthoringPaths(" in render_source +``` + +Add a render test to `tests/test_dashboard_render_smoke.py` requiring exactly one closed `Add a reviewed research record` expander on Research Company Workbench, no expanded Advanced section, and no composer marker on the equivalent Public Single-Stock Report. + +- [ ] **Step 2: Run and verify placement failures** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py -k research_record_authoring -q +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_render_smoke.py -k authoring -q +``` + +Expected: failures show the renderer import/call and route marker are absent. + +- [ ] **Step 3: Wire the composer into Company Workbench** + +Add imports: + +```python +from src.research_record_authoring import AuthoringPaths +from src.research_record_authoring_ui import render_research_record_authoring +``` + +Immediately after `outcome_status_cards()` and before `Advanced: thesis and evidence history`, add: + +```python +if research_mode: + render_research_record_authoring( + st_api=st, + profile_key=selected_context.profile_key, + ticker=ticker, + paths=AuthoringPaths( + journal=DATA_DIR / "research_thesis_journal.csv", + catalysts=DATA_DIR / "catalyst_evidence.csv", + outcomes=DATA_DIR / "research_outcome_reviews.csv", + ), + ) +``` + +Do not add the renderer to Public or Operator branches. After a successful save, call `st.rerun()` only after the success receipt has been placed in session state; on the next run, reload all three existing read-side states and show the receipt once. The UI module must never claim persistence solely from the prior button callback. + +- [ ] **Step 4: Add read-side reload test** + +In the test-only fixture, confirm a record, rerun AppTest, and assert the saved receipt is displayed only when `load_journal_entries()` contains the record identifier. Delete the temporary row before rerun in a separate test and assert the UI reports `Saved record could not be reloaded; review the ledger` instead of a success claim. + +- [ ] **Step 5: Run focused Workbench tests** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_record_authoring.py tests/test_research_record_authoring_ui.py tests/test_dashboard_helpers.py -k 'authoring or research_thesis' -q +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_render_smoke.py -q +``` + +Expected: all focused tests pass, Public and Operator routes contain no composer, and no repository ledger changes. + +- [ ] **Step 6: Commit Task 4** + +Run: + +```bash +git add -- src/dashboard.py src/research_record_authoring_ui.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py tests/test_research_record_authoring_ui.py +git diff --cached --check +make staged-hygiene-check +git commit -m "Integrate research authoring in Company Workbench" +``` + +### Task 5: Responsive Workflow Evidence And Release Documentation + +**Files:** +- Modify: `src/browser_qa_evidence.py` +- Modify: `tests/test_browser_qa_evidence.py` +- Modify: `README.md` +- Modify: `docs/PRODUCT_SPEC.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `docs/superpowers/specs/2026-07-22-in-app-research-record-authoring-design.md` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Produces: truthful release and responsive evidence for the completed Priority 3 implementation. +- Consumes: current runtime results only; screenshots remain route-marker evidence, not proof that a record was persisted. +- Invariant: Priority 4 becomes the earliest incomplete lane only after every Priority 3 acceptance criterion has direct current evidence. + +- [ ] **Step 1: Add failing browser and documentation contracts** + +Require the Research Company Workbench route to include `Add a reviewed research record` under its details boundary and a stop rule that rejects any visible confirmation before preview. Require README/Product Spec/Roadmap/continuation docs to state: + +- all four record types are available; +- exact preview and explicit confirmation are required; +- drafts are untrusted and preview receipts are session-only; +- production tests do not append repository ledgers; +- saves cannot change readiness, forecasts, probabilities, recommendations, or another ledger; and +- Priority 3 is complete locally while Priority 4 is next and still requires a permitted point-in-time dataset for exit. + +Use exact assertions in `tests/test_public_v1_release_docs.py`; do not use broad keyword-count assertions. + +- [ ] **Step 2: Run and verify documentation failures** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_browser_qa_evidence.py tests/test_public_v1_release_docs.py -q +``` + +Expected: failures name the missing implemented-status and responsive-authoring contracts. + +- [ ] **Step 3: Update evidence and documentation** + +Update the Company Workbench browser-QA row to keep the composer below the primary answer and collapsed by default. State explicitly that route-marker screenshots do not prove validation, confirmation, or persistence; the temporary-ledger AppTest and direct persistence tests provide that evidence. + +Update the design `Status` and add an `Implementation Evidence` section containing commit identifiers only after those commits exist. Update `ROADMAP.md` and the continuation prompt to mark Priority 3 complete locally only after desktop/phone route review and all automated acceptance tests pass. + +- [ ] **Step 4: Run the complete required verification matrix** + +Run each command and record the actual result: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_record_authoring.py tests/test_research_record_authoring_ui.py tests/test_research_thesis_journal.py tests/test_catalyst_evidence_timeline.py tests/test_research_outcome_review.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py tests/test_browser_qa_evidence.py tests/test_public_v1_release_docs.py -q +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make linkedin-share-check +make browser-qa-evidence +make pilot-readiness-check TOP_N=10 +make commercial-beta-release-check +make diff-hygiene-summary +git diff --check +``` + +Expected: focused and full tests pass; all read-only release gates pass; pilot readiness may remain truthfully blocked on external/source-depth gates; diff hygiene reports only intentional product files plus the same excluded generated churn. + +- [ ] **Step 5: Perform live desktop and phone review** + +Open `http://localhost:8501/?mode=research&page=company-workbench&ticker=NVDA&open=1` at `1280x720` and `390x844`. Verify: + +1. The primary selected-company answer precedes the collapsed composer. +2. The composer opens without horizontal overflow. +3. Locked profile/ticker and the four record types are understandable. +4. Invalid source/timestamp input shows a field-level rejection and writes nothing. +5. Editing after preview removes or blocks confirmation until re-preview. +6. A successful persistence rehearsal uses only the test fixture with temporary ledgers, never the production NVDA ledger. +7. Advanced evidence stays collapsed and the Research Conclusion/Next Research Task remain below the journal workflow. + +Record measurements and text findings in the design's `Implementation Evidence` section. Keep screenshots ephemeral and unstaged. + +- [ ] **Step 6: Stage exact files and commit the release slice** + +Run `git status --short`, review every intentional path, then stage only the implementation/docs/tests named by `git diff --name-only`: + +```bash +git add -- src/research_record_authoring.py src/research_record_authoring_ui.py src/dashboard.py src/browser_qa_evidence.py tests/test_research_record_authoring.py tests/test_research_record_authoring_ui.py tests/fixtures/research_record_authoring_app.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py tests/test_browser_qa_evidence.py tests/test_public_v1_release_docs.py README.md docs/PRODUCT_SPEC.md ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md docs/superpowers/specs/2026-07-22-in-app-research-record-authoring-design.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Complete in-app research record authoring" +``` + +If a listed file is unchanged, omit it from the exact `git add --` command rather than staging a broader path. + +- [ ] **Step 7: Push, update draft PR #113, and require exact-head CI** + +Run: + +```bash +git push origin codex/personal-research-mode-mvp +gh pr view 113 --json state,isDraft,mergeable,headRefOid,statusCheckRollup,url +gh pr checks 113 --watch --interval 20 +``` + +Update the draft PR with the exact commit, test counts, runtime evidence, generated-artifact exclusion, and remaining external gates. Confirm the PR remains open and draft and the successful hosted check's head SHA equals local `HEAD`. + +- [ ] **Step 8: Rescan Priorities 1-10** + +Run the current roadmap/status/hygiene checks. If Priority 3 has direct evidence for every acceptance criterion, select Priority 4's earliest safe local contract slice. Do not call Priority 4 complete without one bounded permitted real point-in-time dataset that passes rights, identity, corporate-action, delisting, survivorship, cutoff, reproduction, and leakage gates. diff --git a/docs/superpowers/plans/2026-07-22-legacy-research-utility-quarantine.md b/docs/superpowers/plans/2026-07-22-legacy-research-utility-quarantine.md new file mode 100644 index 000000000..9847e68ae --- /dev/null +++ b/docs/superpowers/plans/2026-07-22-legacy-research-utility-quarantine.md @@ -0,0 +1,126 @@ +# Legacy Research Utility Quarantine Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make legacy portfolio, ranking, and action-language surfaces unmistakably operator-only compatibility utilities while preserving the supported Personal Research workflow. + +**Architecture:** Add a pure navigation contract for quarantined canonical page titles, safe mode-specific route resolution, and operator display labels. Reuse the dashboard renderers only inside a shared collapsed compatibility shell. Keep all calculations and historical filenames unchanged, then lock isolation with route, source, wording, documentation, and release tests. + +**Tech Stack:** Python 3.12, Streamlit, pytest, Streamlit render smoke tests, Markdown contract tests. + +## Global Constraints + +- Preserve Research Desk -> Discover -> Company Workbench -> Monitor as the supported workflow. +- Keep legacy calculations only for compatibility; do not promote them into current product capability. +- Do not change readiness, canonical data, forecasts, calibration, sources, or generated report schemas. +- Do not run readiness rebuilds, broad refreshes, or generated CSV/JSON/report/screenshot/timing commands. +- Never stage generated working-data churn and never use `git add -A`. +- Keep draft PR #113 draft; push only `codex/personal-research-mode-mvp`; do not merge or deploy. + +--- + +### Task 1: Lock the quarantine and route contract + +**Files:** +- Modify: `src/dashboard_navigation.py` +- Modify: `src/dashboard.py` +- Modify: `tests/test_dashboard_navigation.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` + +- [ ] **Step 1: Write failing tests for the exact quarantine set, labels, and mode-specific deep-link resolution** + +Assert that Public resolves every quarantined page to `Home`, Personal Research resolves it to `Research Desk`, and Operator preserves it. Assert historical aliases still parse to canonical titles. + +- [ ] **Step 2: Run focused tests and verify RED** + +Run: `python3 -m pytest tests/test_dashboard_navigation.py tests/test_research_mode_dashboard_contract.py -q` + +- [ ] **Step 3: Implement the pure quarantine helpers and use them before route-rail selection** + +Add immutable `LEGACY_RESEARCH_UTILITY_PAGES`, `legacy_research_utility_label(...)`, and `workspace_page_for_mode(...)`. Keep Streamlit out of the navigation module. + +- [ ] **Step 4: Run focused tests and verify GREEN** + +Run: `python3 -m pytest tests/test_dashboard_navigation.py tests/test_research_mode_dashboard_contract.py -q` + +--- + +### Task 2: Quarantine rendered legacy output + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` + +- [ ] **Step 1: Write failing source and render-helper tests** + +Require the exact boundary copy, one collapsed `Advanced: legacy compatibility output` disclosure, and a separate load checkbox. Prove legacy page renderers do not render details until the Operator deliberately uses both controls. + +- [ ] **Step 2: Run focused tests and verify RED** + +Run: `python3 -m pytest tests/test_dashboard_helpers.py tests/test_research_mode_dashboard_contract.py -q` + +- [ ] **Step 3: Implement the shared legacy shell** + +Render the compatibility boundary first. Gate Monthly Picks details and the five quarantined output-tab branches behind the collapsed disclosure and explicit load checkbox. Render after confirmation rather than nesting existing Streamlit expanders. Preserve explicit missing-output states without generating data. + +- [ ] **Step 4: Add and pass isolation tests** + +Prove `src/research_decision_lab.py` and the Company Workbench composition path do not import or consume `portfolio_review`, `monthly_picks`, `final_watchlist`, or legacy dashboard output frames. + +--- + +### Task 3: Reconcile product documentation and roadmap truth + +**Files:** +- Modify: `README.md` +- Modify: `PRODUCT_SPEC.md` +- Modify: `READINESS_MODEL.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +- [ ] **Step 1: Write failing documentation-contract tests** + +Require the supported workflow, exact legacy boundary, operator-only compatibility classification, and explicit exclusion from readiness, Decision Lab, recommendations, sizing, and transactions. + +- [ ] **Step 2: Update the documents without rewriting historical evidence** + +Mark Priority 1 complete only after code and direct tests pass. Set Priority 2 Stage B as the next local executable lane. + +- [ ] **Step 3: Run focused documentation and action-language checks** + +Run: `python3 -m pytest tests/test_public_v1_release_docs.py tests/test_public_wording.py tests/test_action_language_contract.py -q` + +--- + +### Task 4: Verify, stage exactly, synchronize the draft PR, and require exact-head CI + +- [ ] **Step 1: Run the complete verification matrix** + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make pr-range-hygiene-check +make diff-hygiene-summary +git diff --check +``` + +Pilot readiness may remain truthfully blocked by stale saved readiness or external evidence; it must not be regenerated in this slice. + +- [ ] **Step 2: Review and stage exact intentional paths only** + +Run `git diff --stat`, inspect every intentional diff, stage only the code/test/docs paths listed above, then run `make staged-hygiene-check` and `git diff --cached --check`. + +- [ ] **Step 3: Commit and push the coherent slice** + +Commit with `Quarantine legacy research utilities`, push only the existing feature branch, update draft PR #113 with evidence and remaining gates, keep it draft, and wait for exact-head `local-engineering-gate` success. + +- [ ] **Step 4: Continue automatically** + +Re-scan the ordered maturity program. Begin Priority 2 Stage B if a safe local task remains; otherwise classify its exact external unblock condition once and move to the next executable priority. diff --git a/docs/superpowers/plans/2026-07-22-research-decision-lab.md b/docs/superpowers/plans/2026-07-22-research-decision-lab.md new file mode 100644 index 000000000..fbbbed15d --- /dev/null +++ b/docs/superpowers/plans/2026-07-22-research-decision-lab.md @@ -0,0 +1,391 @@ +# Research Decision Lab Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a read-only six-lane Research Decision Lab to Company Workbench and a no-ranking Research Discipline Review to Monitor. + +**Architecture:** A new pure-Python `src/research_decision_lab.py` composes existing immutable journal, decision-process, outcome, change-review, and valuation results into immutable display contracts. `src/dashboard.py` only loads those existing results and renders cards/tables; it adds no persistence, readiness field, route, provider, or generated artifact. + +**Tech Stack:** Python 3.12, frozen dataclasses, pandas display frames, Streamlit, pytest, Streamlit AppTest, Chrome performance gate. + +## Global Constraints + +- Research-only; no investment advice, recommendation, transaction direction, position size, allocation, stop-loss, take-profit, expected return, company score, or broker action. +- Do not import or surface `src/portfolio_review.py` action states. +- Candidate context cannot populate a trusted lane or change forecasts, DCF, readiness, conclusions, or process states. +- Empty, blocked, excluded, invalid, and commercially blocked inputs remain explicit and independent. +- Add no ledger, readiness state, route, provider, generated report, CSV, JSON, screenshot, or timing artifact. +- Preserve the Company Workbench first handoff and uppercase `USE NOW` performance marker. +- Preserve Weekly Research Summary as Monitor's first useful answer and stable focused-cohort order without ranking. +- Keep technical identities, source rows, timestamps, and raw evidence under collapsed Advanced sections. +- Never run `make readiness`, broad refreshes, imports, or apply commands for this feature. +- Never use `git add -A`; stage exact reviewed code, test, and documentation paths only. +- Keep draft PR #113 draft; push only `codex/personal-research-mode-mvp`; do not merge or deploy. + +--- + +### Task 1: Read-only Decision Lab composition contract + +**Files:** +- Create: `src/research_decision_lab.py` +- Create: `tests/test_research_decision_lab.py` + +**Interfaces:** +- Consumes: `JournalState`, `DecisionProcessScorecard`, `OutcomeStatus`, and an iterable of research-review items. +- Produces: `DecisionLabLane`, `ResearchDecisionLabState`, `ResearchDisciplineRow`, `build_research_decision_lab_state(...)`, `unavailable_research_decision_lab_state(...)`, `build_research_discipline_rows(...)`, `decision_lab_cards(...)`, and `research_discipline_rows(...)`. + +- [ ] **Step 1: Write failing immutable-contract and lane-order tests** + +```python +def test_empty_history_keeps_six_lanes_independent_and_not_started(): + state = build_research_decision_lab_state( + profile_key="demo", + journal_state=empty_journal(), + scorecard=empty_scorecard(), + outcome_status=OutcomeStatus("not_started", 0, "", "", "Review later.", 0, ()), + review_items=(), + ) + assert [lane.key for lane in state.lanes] == [ + "plan", "evidence", "invalidation", "scenario", "review_trigger", "learning" + ] + assert {lane.key: lane.state for lane in state.lanes} == { + "plan": "not_started", "evidence": "not_started", "invalidation": "missing", + "scenario": "blocked", "review_trigger": "not_started", "learning": "not_started", + } +``` + +- [ ] **Step 2: Run the new module tests and verify RED** + +Run: `python3 -m pytest tests/test_research_decision_lab.py -q` + +Expected: collection fails because `src.research_decision_lab` does not exist. + +- [ ] **Step 3: Implement frozen contracts and deterministic mapping** + +```python +@dataclass(frozen=True) +class DecisionLabLane: + key: str + label: str + state: str + answer: str + evidence: str + next_step: str + + +@dataclass(frozen=True) +class ResearchDecisionLabState: + profile_key: str + ticker: str + status: str + lanes: tuple[DecisionLabLane, ...] + next_process_step: str + boundary: str + identity: str + + +@dataclass(frozen=True) +class ResearchDisciplineRow: + cohort_order: int + ticker: str + status: str + due_lanes: tuple[str, ...] + next_process_step: str + identity: str +``` + +Use the existing scorecard checks by key. Map Plan from `thesis_documented`; Evidence from `evidence_recorded` plus `conflicting_evidence_reviewed`; Invalidation from `invalidation_documented`; Scenario from `dcf_assumptions_visible`; Review trigger from matching open review items plus the journal due state; and Learning from `OutcomeStatus`. Hash the normalized contract with SHA-256 for deterministic identity. + +- [ ] **Step 4: Run lane tests and verify GREEN** + +Run: `python3 -m pytest tests/test_research_decision_lab.py -q` + +Expected: the empty-state and immutable-contract tests pass. + +- [ ] **Step 5: Add failing priority, fail-closed, and language tests** + +```python +@pytest.mark.parametrize( + ("mutation", "expected"), + [ + ("conflict", "Review recorded conflicting evidence"), + ("overdue", "Review the overdue thesis"), + ("missing_thesis", "Record a current reviewer-authored thesis"), + ("missing_invalidation", "Record a source-backed invalidation condition"), + ("missing_evidence", "Record source-backed research evidence"), + ("unscheduled", "Schedule the next evidence review"), + ("missing_dcf_assumptions", "Restore visible DCF assumptions"), + ], +) +def test_next_process_step_uses_approved_priority(mutation, expected): + assert build_state(mutation).next_process_step.startswith(expected) + + +def test_contract_contains_no_transaction_or_allocation_language(): + rendered = str(build_state("complete")).lower() + for forbidden in ("buy", "sell", "position size", "allocation", "stop loss", "take profit"): + assert forbidden not in rendered +``` + +Also cover later review clearing only the conflict gap, DCF blocked/excluded/ready, commercial learning blocker, mismatch `ValueError`, invalid-journal unavailable state, lane independence, deterministic identity, stable cohort ordering, and no severity sorting. + +- [ ] **Step 6: Implement the approved priority and cohort composition** + +Select the first applicable gap in this exact order: conflict without later review; overdue thesis; missing thesis; missing invalidation; missing evidence; unscheduled review; DCF ready without visible assumptions; otherwise continue monitoring. `build_research_discipline_rows` must preserve the caller's focused-cohort order and use ticker only as a deterministic tie-break, never process severity or market value. + +- [ ] **Step 7: Verify the complete composition slice** + +Run: + +```bash +python3 -m pytest tests/test_research_decision_lab.py tests/test_decision_process_scorecard.py tests/test_research_thesis_journal.py tests/test_research_outcome_review.py -q +python3 -m pytest tests -q +make dashboard-smoke research-dashboard-render-smoke public-wording-check public-check +make pilot-readiness-check TOP_N=10 +make pr-range-hygiene-check diff-hygiene-summary +git diff --check +``` + +Expected: all commands exit zero except pilot readiness may remain truthfully blocked by excluded generated churn or external gates; no generated artifact is added. + +- [ ] **Step 8: Stage, verify, commit, push, update PR, and require exact-head CI** + +```bash +git add -- src/research_decision_lab.py tests/test_research_decision_lab.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add research decision lab composition" +git push origin codex/personal-research-mode-mvp +``` + +Update draft PR #113 with the slice evidence and wait for the `local-engineering-gate` check on the pushed SHA to pass before Task 2. + +--- + +### Task 2: Company Workbench integration + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_dashboard_render_smoke.py` +- Modify: `tests/test_public_performance_gate.py` + +**Interfaces:** +- Consumes: `build_research_decision_lab_state`, `unavailable_research_decision_lab_state`, and `decision_lab_cards` from Task 1. +- Produces: one Workbench `Research Decision Lab` section and one collapsed `Advanced: Decision Lab evidence` disclosure. + +- [ ] **Step 1: Write failing source-order and render tests** + +```python +def test_workbench_places_one_decision_lab_after_what_changed_before_business_trend(): + render = report_renderer_source() + what_changed = render.index('st.markdown("### What Changed")') + decision_lab = render.index('st.markdown("### Research Decision Lab")') + business_trend = render.index('st.markdown("### Business Trend")') + assert what_changed < decision_lab < business_trend + assert render.count('st.markdown("### Research Decision Lab")') == 1 +``` + +Add AppTest assertions that the route contains all six lane labels, `Next process step`, Research Conclusion, and Next Research Task; that `USE NOW` occurs first; and that technical identity is inside a collapsed Advanced disclosure. + +- [ ] **Step 2: Run focused tests and verify RED** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py -q` + +Expected: failures show the missing Workbench section and markers. + +- [ ] **Step 3: Compose state once and render the compact summary** + +Build the existing `DecisionProcessScorecard` once after the journal/outcome payloads load. If the journal is invalid, build one compact unavailable Decision Lab state using the existing verification error. After `What Changed`, render six compact cards and the next process step. Reuse the same scorecard in the existing collapsed scorecard section; do not replace Research Conclusion or Next Research Task. + +- [ ] **Step 4: Keep technical evidence collapsed** + +```python +with st.expander("Advanced: Decision Lab evidence", expanded=False): + st.dataframe(decision_lab_frame(decision_lab_state), width="stretch", hide_index=True) + st.caption(f"Decision Lab identity: {decision_lab_state.identity}") + st.caption(decision_lab_state.boundary) +``` + +- [ ] **Step 5: Verify Workbench behavior and responsive performance** + +Run: + +```bash +python3 -m pytest tests/test_research_decision_lab.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py -q +python3 -m pytest tests -q +make dashboard-smoke research-dashboard-render-smoke public-wording-check public-check browser-qa-evidence +make commercial-beta-performance-gate commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make pr-range-hygiene-check diff-hygiene-summary +git diff --check +``` + +Expected: desktop and `390x844` Workbench cases pass, uppercase `USE NOW` remains first useful evidence under three seconds, and no horizontal-overflow or duplicate-section failure appears. + +- [ ] **Step 6: Stage, verify, commit, push, update PR, and require exact-head CI** + +```bash +git add -- src/dashboard.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Integrate decision lab into company workbench" +git push origin codex/personal-research-mode-mvp +``` + +Update draft PR #113 with Workbench placement, responsive, and performance evidence; require exact-head CI before Task 3. + +--- + +### Task 3: Monitor Research Discipline Review + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_dashboard_render_smoke.py` +- Modify: `tests/test_public_performance_gate.py` + +**Interfaces:** +- Consumes: `FocusedCohort`, existing saved journal/outcome ledgers, existing research-review items, and Task 1 cohort rows. +- Produces: `load_dashboard_research_discipline_rows(...)`, a compact `Research Discipline Review`, and collapsed Advanced evidence. + +- [ ] **Step 1: Write failing cohort-load, order, empty-state, and placement tests** + +```python +def test_monitor_places_discipline_review_after_weekly_summary_before_change_monitor(): + monitor = monitor_renderer_source() + weekly = monitor.index("weekly_summary_cards(weekly_summary)") + discipline = monitor.index('st.markdown("### Research Discipline Review")') + changes = monitor.index('st.markdown("### Research change monitor")') + assert weekly < discipline < changes + + +def test_monitor_discipline_rows_preserve_focused_cohort_order_without_rank(): + assert [row.ticker for row in rows] == ["BBB", "AAA"] + assert "rank" not in str(rows).lower() +``` + +Add an empty-state assertion that says no process item is due from saved reviewer-authored evidence and does not claim no risk, no research need, or no market event. + +- [ ] **Step 2: Run focused tests and verify RED** + +Run: `python3 -m pytest tests/test_research_decision_lab.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q` + +Expected: failures show the missing Monitor loader and section. + +- [ ] **Step 3: Implement one read-only cohort loader** + +Load journal and outcome ledgers once, derive each focused ticker independently, construct only saved readiness context needed by the existing scorecard, and convert failures for one ticker to an unavailable state without promoting another ticker. Do not fetch, refresh, build generated reports, or pad the cohort. + +- [ ] **Step 4: Render the review after Weekly Research Summary** + +Render a compact table with `Ticker`, `Process state`, `Due lanes`, and `Next process step`. Keep identity and evidence rows under `Advanced: Research Discipline evidence`. Render the existing Research change monitor afterward so market/source-change state stays independent. + +- [ ] **Step 5: Verify Monitor behavior and responsive performance** + +Run: + +```bash +python3 -m pytest tests/test_research_decision_lab.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py -q +python3 -m pytest tests -q +make dashboard-smoke research-dashboard-render-smoke public-wording-check public-check browser-qa-evidence +make commercial-beta-performance-gate commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make pr-range-hygiene-check diff-hygiene-summary +git diff --check +``` + +Expected: Monitor remains within desktop/phone thresholds, Weekly Research Summary remains first useful evidence, stable cohort order is preserved, and empty process evidence creates no market or risk claim. + +- [ ] **Step 6: Stage, verify, commit, push, update PR, and require exact-head CI** + +```bash +git add -- src/dashboard.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add monitor research discipline review" +git push origin codex/personal-research-mode-mvp +``` + +Update draft PR #113 with Monitor ordering, empty-state, and no-ranking evidence; require exact-head CI before Task 4. + +--- + +### Task 4: Documentation and release evidence + +**Files:** +- Modify: `README.md` only if the current public feature summary materially omits the implemented workflow +- Modify: `ROADMAP.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PROVENANCE_CONTRACT.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `docs/internal/RESEARCH_DECISION_LAB_CONTINUATION_GOAL_PROMPT.md` +- Modify: browser-QA contract source and its focused tests only if current markers require Decision Lab coverage +- Test: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes: verified Task 1-3 behavior and exact current command outputs. +- Produces: acceptance-criterion evidence map, current roadmap state, continuation contract, public-safe product explanation, and exact-head draft-PR evidence. + +- [ ] **Step 1: Write failing documentation contract tests** + +```python +def test_personal_research_docs_define_decision_lab_without_trading_scope(): + text = Path("docs/PERSONAL_RESEARCH_MODE.md").read_text(encoding="utf-8") + assert "Research Decision Lab" in text + assert "Research Discipline Review" in text + assert "no new ledger" in text.lower() +``` + +Add assertions for six lane names, independent fail-closed states, Company Workbench and Monitor placement, and the broader external maturity boundary. + +- [ ] **Step 2: Run docs tests and verify RED** + +Run: `python3 -m pytest tests/test_public_v1_release_docs.py tests/test_launchers.py -q` + +Expected: failures identify missing implemented-status and methodology/provenance wording. + +- [ ] **Step 3: Update docs and roadmap with verified evidence only** + +Mark the Decision Lab local implementation complete only after Tasks 1-3 are committed and pushed. Document that it composes saved evidence read-only; it does not prove source coverage, predictive accuracy, investment performance, independent adoption, hosted reliability, commercial demand, competitive superiority, or product-market fit. Update the dedicated continuation prompt to route to the next locally executable or external gate without repeating completed Decision Lab work. + +- [ ] **Step 4: Audit README and browser-QA markers** + +Change README only if the answer-first public workflow materially benefits from one concise Decision Lab sentence. Update browser-QA markers without replacing screenshots; existing screenshots remain route/product evidence only until explicitly recaptured and reviewed. + +- [ ] **Step 5: Run the complete release matrix** + +Run: + +```bash +python3 -m pytest tests/test_public_v1_release_docs.py tests/test_launchers.py tests/test_research_decision_lab.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_public_performance_gate.py -q +python3 -m pytest tests -q +make dashboard-smoke research-dashboard-render-smoke public-wording-check public-check browser-qa-evidence +make commercial-beta-performance-gate commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make pr-range-hygiene-check diff-hygiene-summary +git diff --check +``` + +Expected: local code, full tests, route renders, wording, browser contract, performance, release, and hygiene pass; pilot or external gates remain blocked only where current evidence says so. + +- [ ] **Step 6: Map every acceptance criterion to current evidence** + +Record each design criterion as `proven`, `contradicted`, `incomplete`, `indirect`, or `missing`. Evidence must name the exact test/gate and current revision; screenshot-only, fixture-only, local-only, stale, or indirect evidence cannot prove a broader gate. + +- [ ] **Step 7: Stage exact docs/tests, verify, commit, and push** + +```bash +git add -- ROADMAP.md docs/METHODOLOGY.md docs/PROVENANCE_CONTRACT.md docs/PERSONAL_RESEARCH_MODE.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md docs/internal/RESEARCH_DECISION_LAB_CONTINUATION_GOAL_PROMPT.md tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Document research decision lab evidence" +git push origin codex/personal-research-mode-mvp +``` + +Add README or browser-QA files to the exact staging command only if Step 4 required and verified them. + +- [ ] **Step 8: Update draft PR and require exact-head CI** + +Update PR #113 with the final acceptance map, exact local verification, generated-artifact exclusion, and external dependency classifications. Keep it draft. Confirm `headRefOid` equals local HEAD and `local-engineering-gate` succeeds on that exact SHA. Do not merge or deploy. diff --git a/docs/superpowers/plans/2026-07-23-point-in-time-universe-foundation.md b/docs/superpowers/plans/2026-07-23-point-in-time-universe-foundation.md new file mode 100644 index 000000000..14c0682f5 --- /dev/null +++ b/docs/superpowers/plans/2026-07-23-point-in-time-universe-foundation.md @@ -0,0 +1,2362 @@ +# Point-in-Time Benchmark and Universe Foundation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build the approved provider-neutral, read-only validator for immutable point-in-time security identity, benchmark/research-universe membership, corporate actions, delistings, source rights, cutoffs, reproduction, and leakage. + +**Architecture:** Keep the new validator isolated from the ticker-centric current-universe merge path. A manifest loader proves package identity, typed parsers preserve ordered source rows, lineage and temporal validators classify evidence independently, and one composer emits status/preview packets without writing, fetching, applying, rebuilding readiness, or activating analysis. + +**Tech Stack:** Python 3.12, frozen dataclasses, `csv`, `hashlib`, `json`, `pathlib`, existing `src.commercial_source_rights`, pytest, Make. + +## Global Constraints + +- Follow `docs/superpowers/specs/2026-07-23-point-in-time-universe-foundation-design.md` exactly. +- Research-only; no ranking, recommendations, investment advice, position sizing, broker integration, order routing, auto-trading, or price prediction. +- Do not modify or call the apply paths for `data/universe.csv`, `data/universe_master.csv`, or `data/universe_active.csv`. +- Do not fetch data, call providers, add keys, alter source rights, rebuild readiness, or write normalized/rejected/report artifacts. +- Stable `security_id` and `issuer_id` are required; ticker must never become permanent identity. +- Technical, temporal, identity, membership, corporate-action, delisting, source-rights, reproduction, and leakage states remain independent. +- Synthetic evidence exists only in pytest temporary directories and cannot complete Priority 4. +- Every production behavior starts with a failing focused test. +- Stage exact intentional product/code/docs/test files only; never use `git add -A`. +- Keep the 18 existing generated CSV/JSON/report files unstaged. +- Keep PR #113 open and draft; do not merge or deploy. + +--- + +## File Structure + +- Create `src/point_in_time_universe_manifest.py`: immutable manifest types, safe path resolution, hashes, row counts, and package loading. +- Create `src/point_in_time_universe_contracts.py`: schema constants, typed records, timestamp/date/enum parsing, and ordered raw/normalized rows. +- Create `src/point_in_time_universe_lineage.py`: generic exact-parent lineage validation and cutoff leaf selection. +- Create `src/point_in_time_universe.py`: independent decisions, identity/membership/action/delisting/rights/leakage composition, reproduction digests, rendering, and CLI. +- Create `tests/point_in_time_universe_fixture.py`: test-only package builder that writes exclusively beneath pytest `tmp_path`. +- Create `tests/test_point_in_time_universe_manifest.py`: manifest-integrity and no-write tests. +- Create `tests/test_point_in_time_universe_contracts.py`: schema, type, enum, and timestamp tests. +- Create `tests/test_point_in_time_universe_lineage.py`: root/leaf, fork, cycle, cross-scope, and cutoff tests. +- Create `tests/test_point_in_time_universe.py`: composed state, action, delisting, rights, leakage, reproduction, and no-current-fallback tests. +- Create `tests/test_point_in_time_universe_cli.py`: CLI/Make, status/preview, invocation, blocked-package, and no-write tests. +- Modify `Makefile`: add the two read-only entry points. +- Modify `ROADMAP.md`, `docs/METHODOLOGY.md`, `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, and `tests/test_public_v1_release_docs.py`: record implemented local scope without claiming Priority 4 completion. + +--- + +### Task 1: Test-Only Package Builder and Immutable Manifest Loader + +**Files:** +- Create: `tests/point_in_time_universe_fixture.py` +- Create: `tests/test_point_in_time_universe_manifest.py` +- Create: `src/point_in_time_universe_manifest.py` + +**Interfaces:** +- Produces `ManifestFile`, `UniverseManifest`, and `LoadedUniversePackage`. +- Produces `load_universe_package(manifest_path: Path, registry_path: Path) -> LoadedUniversePackage`. +- Produces `build_valid_package(root: Path) -> tuple[Path, Path]` for tests only. + +- [ ] **Step 1: Write the test-only package builder** + +```python +# tests/point_in_time_universe_fixture.py +from __future__ import annotations + +import csv +import hashlib +import json +from pathlib import Path + + +def _write_csv(path: Path, rows: list[dict[str, str]]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w", encoding="utf-8", newline="") as handle: + writer = csv.DictWriter(handle, fieldnames=list(rows[0])) + writer.writeheader() + writer.writerows(rows) + + +def _sha256(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def build_valid_package(root: Path) -> tuple[Path, Path]: + package = root / "package" + package.mkdir() + identity = [{ + "identity_row_id": "id-1", "security_id": "sec-1", "issuer_id": "issuer-1", + "ticker": "AAA", "exchange": "XNYS", "security_type": "common_stock", + "currency": "USD", "valid_from": "2020-01-01T00:00:00Z", "valid_to": "", + "source_id": "fixture_source", "source_ref": "fixture://identity/id-1", + "source_published_at": "2020-01-01T00:00:00Z", + "retrieved_at": "2020-01-02T00:00:00Z", + "supersedes_identity_row_id": "", + }] + membership = [ + { + "membership_row_id": f"member-{universe}", "universe_id": universe, + "universe_kind": kind, "security_id": "sec-1", + "membership_state": "included", "effective_from": "2020-01-01T00:00:00Z", + "effective_to": "", "observation_at": "2020-01-01T00:00:00Z", + "source_id": "fixture_source", "source_ref": f"fixture://membership/{universe}", + "source_published_at": "2020-01-01T00:00:00Z", + "retrieved_at": "2020-01-02T00:00:00Z", + "supersedes_membership_row_id": "", + } + for universe, kind in (("bench-1", "benchmark"), ("research-1", "research_universe")) + ] + events = [{ + "event_row_id": "event-1", "security_id": "sec-1", "event_type": "listing", + "effective_at": "2020-01-01T00:00:00Z", "successor_security_id": "", + "ratio_numerator": "", "ratio_denominator": "", "listing_state_after": "active", + "source_id": "fixture_source", "source_ref": "fixture://event/event-1", + "source_published_at": "2020-01-01T00:00:00Z", + "retrieved_at": "2020-01-02T00:00:00Z", "supersedes_event_row_id": "", + }] + evaluations = [ + { + "evaluation_row_id": f"eval-{universe}", "universe_id": universe, + "evaluation_at": "2021-01-01T00:00:00Z", + "available_at": "2021-01-01T00:00:00Z", "partition": "walk_forward", + "source_ref": f"fixture://evaluation/{universe}", + } + for universe in ("bench-1", "research-1") + ] + payloads = { + "identity.csv": identity, + "membership.csv": membership, + "events.csv": events, + "evaluations.csv": evaluations, + } + files = [] + contract_by_name = { + "identity.csv": "security_identity", + "membership.csv": "membership", + "events.csv": "events", + "evaluations.csv": "evaluations", + } + for name, rows in payloads.items(): + path = package / name + _write_csv(path, rows) + files.append({ + "path": name, "contract": contract_by_name[name], + "sha256": _sha256(path), "row_count": len(rows), + }) + registry = root / "source_rights.yml" + registry.write_text( + "sources:\n" + " - source_id: fixture_source\n" + " display_name: Test-only source\n" + " permitted_use: test_only\n" + " commercial_use: approved\n" + " redistribution: test_only\n" + " storage_limits: pytest temporary directory only\n" + " attribution: synthetic fixture\n" + " rate_limits: not_applicable\n" + " authentication: none\n" + " expected_freshness: point in time\n" + " supported_fields: [security_identity, universe_membership, corporate_actions, delistings]\n" + " fallback_priority: 1\n", + encoding="utf-8", + ) + manifest = { + "schema_version": "point_in_time_universe_v1", + "dataset_id": "fixture-dataset", "manifest_id": "fixture-manifest", + "manifest_created_at": "2021-01-02T00:00:00Z", + "observation_cutoff_at": "2021-01-01T00:00:00Z", + "coverage_semantics": "complete_snapshot", + "declared_universes": [ + {"universe_id": "bench-1", "universe_kind": "benchmark"}, + {"universe_id": "research-1", "universe_kind": "research_universe"}, + ], + "allowed_source_ids": ["fixture_source"], + "source_rights_registry_sha256": _sha256(registry), + "files": files, + "evaluation_policy": {"kind": "walk_forward", "minimum_history_count": 1}, + "corporate_action_policy": { + "listing": "required", "ticker_change": "not_applicable", + "exchange_change": "not_applicable", "split": "not_applicable", + "reverse_split": "not_applicable", "merger": "not_applicable", + "acquisition": "not_applicable", "spinoff": "not_applicable", + "delisting": "not_applicable", "suspension": "not_applicable", + "reactivation": "not_applicable", + }, + "delisting_policy": { + "retain_historical_members": True, + "missing_evidence": "blocked", + }, + "survivorship_policy": { + "filter_by_current_listing_state": False, + }, + "reproduction_contract": "membership_count_and_sha256_at_cutoff_v1", + } + manifest_path = package / "manifest.json" + manifest_path.write_text(json.dumps(manifest, sort_keys=True), encoding="utf-8") + return manifest_path, registry +``` + +- [ ] **Step 2: Write failing manifest tests** + +```python +# tests/test_point_in_time_universe_manifest.py +from pathlib import Path +import json +import pytest + +from tests.point_in_time_universe_fixture import build_valid_package + + +def _file_bytes(root: Path) -> dict[str, bytes]: + return {p.relative_to(root).as_posix(): p.read_bytes() for p in root.rglob("*") if p.is_file()} + + +def test_loads_hash_bound_manifest_without_writing(tmp_path): + from src.point_in_time_universe_manifest import load_universe_package + manifest, registry = build_valid_package(tmp_path) + before = _file_bytes(tmp_path) + loaded = load_universe_package(manifest, registry) + assert loaded.manifest.schema_version == "point_in_time_universe_v1" + assert set(loaded.files) == {"security_identity", "membership", "events", "evaluations"} + assert _file_bytes(tmp_path) == before + + +@pytest.mark.parametrize("mutation,match", [ + ("hash", "manifest_hash_mismatch"), + ("row_count", "manifest_row_count_mismatch"), + ("schema", "manifest_schema_unsupported"), + ("registry", "manifest_registry_digest_mismatch"), +]) +def test_manifest_integrity_fails_closed(tmp_path, mutation, match): + from src.point_in_time_universe_manifest import load_universe_package + manifest, registry = build_valid_package(tmp_path) + raw = json.loads(manifest.read_text()) + if mutation == "hash": + raw["files"][0]["sha256"] = "0" * 64 + elif mutation == "row_count": + raw["files"][0]["row_count"] += 1 + elif mutation == "schema": + raw["schema_version"] = "unknown" + else: + registry.write_text(registry.read_text() + "\n", encoding="utf-8") + manifest.write_text(json.dumps(raw), encoding="utf-8") + with pytest.raises(ValueError, match=match): + load_universe_package(manifest, registry) +``` + +- [ ] **Step 3: Run the manifest tests and observe RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe_manifest.py -q +``` + +Expected: collection fails because `src.point_in_time_universe_manifest` does not exist. + +- [ ] **Step 4: Implement the immutable manifest loader** + +```python +# src/point_in_time_universe_manifest.py +from __future__ import annotations + +import csv +import hashlib +import json +from dataclasses import dataclass +from pathlib import Path +from types import MappingProxyType +from typing import Any, Mapping + + +REQUIRED_CONTRACTS = frozenset({"security_identity", "membership", "events", "evaluations"}) + + +@dataclass(frozen=True) +class ManifestFile: + path: str + contract: str + sha256: str + row_count: int + + +@dataclass(frozen=True) +class UniverseManifest: + schema_version: str + dataset_id: str + manifest_id: str + manifest_created_at: str + observation_cutoff_at: str + coverage_semantics: str + declared_universes: tuple[Mapping[str, str], ...] + allowed_source_ids: tuple[str, ...] + source_rights_registry_sha256: str + files: tuple[ManifestFile, ...] + evaluation_policy: Mapping[str, Any] + corporate_action_policy: Mapping[str, str] + delisting_policy: Mapping[str, Any] + survivorship_policy: Mapping[str, Any] + reproduction_contract: str + + +@dataclass(frozen=True) +class LoadedUniversePackage: + manifest_path: Path + registry_path: Path + manifest: UniverseManifest + files: Mapping[str, Path] + + +def _sha256(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def _csv_row_count(path: Path) -> int: + with path.open(encoding="utf-8", newline="") as handle: + return sum(1 for _ in csv.DictReader(handle)) + + +def _safe_child(base: Path, relative: str) -> Path: + if not relative or Path(relative).is_absolute(): + raise ValueError("manifest_path_unsafe") + resolved_base = base.resolve() + resolved = (base / relative).resolve() + if resolved == resolved_base or resolved_base not in resolved.parents: + raise ValueError("manifest_path_unsafe") + return resolved + + +def load_universe_package(manifest_path: Path, registry_path: Path) -> LoadedUniversePackage: + manifest_path = Path(manifest_path) + registry_path = Path(registry_path) + try: + raw = json.loads(manifest_path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as exc: + raise ValueError("manifest_unreadable") from exc + if raw.get("schema_version") != "point_in_time_universe_v1": + raise ValueError("manifest_schema_unsupported") + file_records = tuple(ManifestFile(**item) for item in raw.get("files", [])) + contracts = [item.contract for item in file_records] + if set(contracts) != REQUIRED_CONTRACTS or len(contracts) != len(set(contracts)): + raise ValueError("manifest_contract_set_invalid") + resolved: dict[str, Path] = {} + for item in file_records: + path = _safe_child(manifest_path.parent, item.path) + if _sha256(path) != item.sha256: + raise ValueError("manifest_hash_mismatch") + if _csv_row_count(path) != item.row_count: + raise ValueError("manifest_row_count_mismatch") + resolved[item.contract] = path + if _sha256(registry_path) != raw.get("source_rights_registry_sha256"): + raise ValueError("manifest_registry_digest_mismatch") + manifest = UniverseManifest( + schema_version=raw["schema_version"], + dataset_id=raw["dataset_id"], + manifest_id=raw["manifest_id"], + manifest_created_at=raw["manifest_created_at"], + observation_cutoff_at=raw["observation_cutoff_at"], + coverage_semantics=raw["coverage_semantics"], + declared_universes=tuple(MappingProxyType(dict(item)) for item in raw["declared_universes"]), + allowed_source_ids=tuple(raw["allowed_source_ids"]), + source_rights_registry_sha256=raw["source_rights_registry_sha256"], + files=file_records, + evaluation_policy=MappingProxyType(dict(raw["evaluation_policy"])), + corporate_action_policy=MappingProxyType(dict(raw["corporate_action_policy"])), + delisting_policy=MappingProxyType(dict(raw["delisting_policy"])), + survivorship_policy=MappingProxyType(dict(raw["survivorship_policy"])), + reproduction_contract=raw["reproduction_contract"], + ) + return LoadedUniversePackage( + manifest_path=manifest_path.resolve(), + registry_path=registry_path.resolve(), + manifest=manifest, + files=MappingProxyType(resolved), + ) +``` + +- [ ] **Step 5: Add path traversal and symlink-escape tests** + +```python +def test_manifest_rejects_parent_traversal(tmp_path): + from src.point_in_time_universe_manifest import load_universe_package + manifest, registry = build_valid_package(tmp_path) + raw = json.loads(manifest.read_text()) + raw["files"][0]["path"] = "../outside.csv" + manifest.write_text(json.dumps(raw), encoding="utf-8") + with pytest.raises(ValueError, match="manifest_path_unsafe"): + load_universe_package(manifest, registry) + + +def test_manifest_rejects_symlink_escape(tmp_path): + from src.point_in_time_universe_manifest import load_universe_package + manifest, registry = build_valid_package(tmp_path) + outside = tmp_path / "outside.csv" + outside.write_text("identity_row_id\nid-1\n", encoding="utf-8") + link = manifest.parent / "escape.csv" + link.symlink_to(outside) + raw = json.loads(manifest.read_text()) + raw["files"][0].update(path="escape.csv", sha256=__import__("hashlib").sha256(outside.read_bytes()).hexdigest(), row_count=1) + manifest.write_text(json.dumps(raw), encoding="utf-8") + with pytest.raises(ValueError, match="manifest_path_unsafe"): + load_universe_package(manifest, registry) +``` + +- [ ] **Step 6: Run focused tests and commit** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe_manifest.py -q +git diff --check +git add -- src/point_in_time_universe_manifest.py tests/point_in_time_universe_fixture.py tests/test_point_in_time_universe_manifest.py +make staged-hygiene-check +git commit -m "Add immutable universe package loader" +``` + +Expected: all manifest tests pass; staged generated artifacts are `0`. + +--- + +### Task 2: Typed Evidence Contracts and Technical Validation + +**Files:** +- Create: `src/point_in_time_universe_contracts.py` +- Create: `tests/test_point_in_time_universe_contracts.py` + +**Interfaces:** +- Consumes `LoadedUniversePackage`. +- Produces `RawEvidenceRow`, `IdentityObservation`, `MembershipObservation`, `UniverseEvent`, `EvaluationObservation`, `ContractFinding`, and `ParsedUniverseEvidence`. +- Produces `parse_universe_evidence(package: LoadedUniversePackage) -> ParsedUniverseEvidence`. + +- [ ] **Step 1: Write failing schema and type tests** + +```python +# tests/test_point_in_time_universe_contracts.py +import csv +import json +import math +import pytest + +from tests.point_in_time_universe_fixture import build_valid_package + + +def _rewrite_csv_and_manifest(manifest, contract, mutate): + raw = json.loads(manifest.read_text()) + entry = next(item for item in raw["files"] if item["contract"] == contract) + path = manifest.parent / entry["path"] + with path.open(encoding="utf-8", newline="") as handle: + rows = list(csv.DictReader(handle)) + mutate(rows) + with path.open("w", encoding="utf-8", newline="") as handle: + writer = csv.DictWriter(handle, fieldnames=list(rows[0])) + writer.writeheader() + writer.writerows(rows) + entry["sha256"] = __import__("hashlib").sha256(path.read_bytes()).hexdigest() + entry["row_count"] = len(rows) + manifest.write_text(json.dumps(raw), encoding="utf-8") + + +def test_parser_preserves_raw_order_and_normalizes_display_ticker(tmp_path): + from src.point_in_time_universe_manifest import load_universe_package + from src.point_in_time_universe_contracts import parse_universe_evidence + manifest, registry = build_valid_package(tmp_path) + _rewrite_csv_and_manifest(manifest, "security_identity", lambda rows: rows[0].update(ticker=" aaa ")) + parsed = parse_universe_evidence(load_universe_package(manifest, registry)) + assert parsed.identities[0].ticker == "AAA" + assert parsed.raw[0].contract == "security_identity" + assert parsed.raw[0].source_row == 2 + + +@pytest.mark.parametrize("contract,column,value,reason", [ + ("security_identity", "valid_from", "2020-01-01", "schema_timestamp_invalid"), + ("membership", "membership_state", "maybe", "schema_enum_invalid"), + ("membership", "universe_kind", "portfolio", "schema_enum_invalid"), + ("events", "event_type", "dividend", "schema_enum_invalid"), + ("events", "ratio_numerator", "nan", "schema_ratio_invalid"), + ("evaluations", "partition", "future", "schema_enum_invalid"), +]) +def test_invalid_values_become_stable_technical_findings(tmp_path, contract, column, value, reason): + from src.point_in_time_universe_manifest import load_universe_package + from src.point_in_time_universe_contracts import parse_universe_evidence + manifest, registry = build_valid_package(tmp_path) + _rewrite_csv_and_manifest(manifest, contract, lambda rows: rows[0].update({column: value})) + parsed = parse_universe_evidence(load_universe_package(manifest, registry)) + assert reason in {code for finding in parsed.findings for code in finding.reason_codes} +``` + +- [ ] **Step 2: Run tests and observe RED** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe_contracts.py -q +``` + +Expected: collection fails because `src.point_in_time_universe_contracts` does not exist. + +- [ ] **Step 3: Implement frozen record types and parsers** + +```python +# src/point_in_time_universe_contracts.py +from __future__ import annotations + +import csv +import math +from dataclasses import dataclass +from datetime import datetime, timezone +from pathlib import Path +from types import MappingProxyType +from typing import Any, Mapping + +from src.point_in_time_universe_manifest import LoadedUniversePackage + + +EVENT_TYPES = frozenset({"listing", "ticker_change", "exchange_change", "split", "reverse_split", "merger", "acquisition", "spinoff", "delisting", "suspension", "reactivation"}) +LISTING_STATES = frozenset({"", "active", "delisted", "suspended"}) +PARTITIONS = frozenset({"train", "validation", "test", "walk_forward"}) + + +@dataclass(frozen=True) +class RawEvidenceRow: + contract: str + source_file: str + source_row: int + values: Mapping[str, str] + + +@dataclass(frozen=True) +class ContractFinding: + contract: str + source_row: int + row_id: str + reason_codes: tuple[str, ...] + + +@dataclass(frozen=True) +class IdentityObservation: + identity_row_id: str; security_id: str; issuer_id: str; ticker: str + exchange: str; security_type: str; currency: str + valid_from: datetime; valid_to: datetime | None + source_id: str; source_ref: str; source_published_at: datetime; retrieved_at: datetime + supersedes_identity_row_id: str + + +@dataclass(frozen=True) +class MembershipObservation: + membership_row_id: str; universe_id: str; universe_kind: str; security_id: str + membership_state: str; effective_from: datetime; effective_to: datetime | None + observation_at: datetime; source_id: str; source_ref: str + source_published_at: datetime; retrieved_at: datetime + supersedes_membership_row_id: str + + +@dataclass(frozen=True) +class UniverseEvent: + event_row_id: str; security_id: str; event_type: str; effective_at: datetime + successor_security_id: str; ratio_numerator: float | None; ratio_denominator: float | None + listing_state_after: str; source_id: str; source_ref: str + source_published_at: datetime; retrieved_at: datetime; supersedes_event_row_id: str + + +@dataclass(frozen=True) +class EvaluationObservation: + evaluation_row_id: str; universe_id: str; evaluation_at: datetime + available_at: datetime; partition: str; source_ref: str + + +@dataclass(frozen=True) +class ParsedUniverseEvidence: + raw: tuple[RawEvidenceRow, ...] + identities: tuple[IdentityObservation, ...] + memberships: tuple[MembershipObservation, ...] + events: tuple[UniverseEvent, ...] + evaluations: tuple[EvaluationObservation, ...] + findings: tuple[ContractFinding, ...] + + +def parse_utc(value: str) -> datetime: + text = str(value or "").strip() + if not text.endswith("Z"): + raise ValueError("schema_timestamp_invalid") + parsed = datetime.fromisoformat(text[:-1] + "+00:00") + if parsed.tzinfo != timezone.utc: + raise ValueError("schema_timestamp_invalid") + return parsed + + +def optional_utc(value: str) -> datetime | None: + return None if not str(value or "").strip() else parse_utc(value) + + +def optional_positive_float(value: str) -> float | None: + if not str(value or "").strip(): + return None + parsed = float(value) + if not math.isfinite(parsed) or parsed <= 0: + raise ValueError("schema_ratio_invalid") + return parsed + + +IDENTITY_COLUMNS = ( + "identity_row_id", "security_id", "issuer_id", "ticker", "exchange", + "security_type", "currency", "valid_from", "valid_to", "source_id", + "source_ref", "source_published_at", "retrieved_at", + "supersedes_identity_row_id", +) +MEMBERSHIP_COLUMNS = ( + "membership_row_id", "universe_id", "universe_kind", "security_id", + "membership_state", "effective_from", "effective_to", "observation_at", + "source_id", "source_ref", "source_published_at", "retrieved_at", + "supersedes_membership_row_id", +) +EVENT_COLUMNS = ( + "event_row_id", "security_id", "event_type", "effective_at", + "successor_security_id", "ratio_numerator", "ratio_denominator", + "listing_state_after", "source_id", "source_ref", + "source_published_at", "retrieved_at", "supersedes_event_row_id", +) +EVALUATION_COLUMNS = ( + "evaluation_row_id", "universe_id", "evaluation_at", "available_at", + "partition", "source_ref", +) +COLUMNS = { + "security_identity": IDENTITY_COLUMNS, + "membership": MEMBERSHIP_COLUMNS, + "events": EVENT_COLUMNS, + "evaluations": EVALUATION_COLUMNS, +} +ROW_ID_FIELDS = { + "security_identity": "identity_row_id", + "membership": "membership_row_id", + "events": "event_row_id", + "evaluations": "evaluation_row_id", +} + + +def _required(row, *names): + values = tuple(str(row.get(name, "") or "").strip() for name in names) + if any(not value for value in values): + raise ValueError("schema_required_field_missing") + return values + + +def _parse_identity(row): + required = _required( + row, "identity_row_id", "security_id", "issuer_id", "ticker", + "exchange", "security_type", "currency", "valid_from", "source_id", + "source_ref", "source_published_at", "retrieved_at", + ) + return IdentityObservation( + identity_row_id=required[0], security_id=required[1], + issuer_id=required[2], ticker=required[3].upper(), + exchange=required[4], security_type=required[5], currency=required[6], + valid_from=parse_utc(required[7]), valid_to=optional_utc(row["valid_to"]), + source_id=required[8], source_ref=required[9], + source_published_at=parse_utc(required[10]), + retrieved_at=parse_utc(required[11]), + supersedes_identity_row_id=row["supersedes_identity_row_id"].strip(), + ) + + +def _parse_membership(row): + required = _required( + row, "membership_row_id", "universe_id", "universe_kind", + "security_id", "membership_state", "effective_from", + "observation_at", "source_id", "source_ref", + "source_published_at", "retrieved_at", + ) + if required[2] not in {"benchmark", "research_universe"}: + raise ValueError("schema_enum_invalid") + if required[4] not in {"included", "excluded"}: + raise ValueError("schema_enum_invalid") + return MembershipObservation( + membership_row_id=required[0], universe_id=required[1], + universe_kind=required[2], security_id=required[3], + membership_state=required[4], effective_from=parse_utc(required[5]), + effective_to=optional_utc(row["effective_to"]), + observation_at=parse_utc(required[6]), source_id=required[7], + source_ref=required[8], source_published_at=parse_utc(required[9]), + retrieved_at=parse_utc(required[10]), + supersedes_membership_row_id=row["supersedes_membership_row_id"].strip(), + ) + + +def _parse_event(row): + required = _required( + row, "event_row_id", "security_id", "event_type", "effective_at", + "source_id", "source_ref", "source_published_at", "retrieved_at", + ) + if required[2] not in EVENT_TYPES or row["listing_state_after"].strip() not in LISTING_STATES: + raise ValueError("schema_enum_invalid") + try: + numerator = optional_positive_float(row["ratio_numerator"]) + denominator = optional_positive_float(row["ratio_denominator"]) + except (TypeError, ValueError) as exc: + raise ValueError("schema_ratio_invalid") from exc + if (numerator is None) != (denominator is None): + raise ValueError("schema_ratio_pair_required") + return UniverseEvent( + event_row_id=required[0], security_id=required[1], + event_type=required[2], effective_at=parse_utc(required[3]), + successor_security_id=row["successor_security_id"].strip(), + ratio_numerator=numerator, ratio_denominator=denominator, + listing_state_after=row["listing_state_after"].strip(), + source_id=required[4], source_ref=required[5], + source_published_at=parse_utc(required[6]), + retrieved_at=parse_utc(required[7]), + supersedes_event_row_id=row["supersedes_event_row_id"].strip(), + ) + + +def _parse_evaluation(row): + required = _required( + row, "evaluation_row_id", "universe_id", "evaluation_at", + "available_at", "partition", "source_ref", + ) + if required[4] not in PARTITIONS: + raise ValueError("schema_enum_invalid") + return EvaluationObservation( + evaluation_row_id=required[0], universe_id=required[1], + evaluation_at=parse_utc(required[2]), available_at=parse_utc(required[3]), + partition=required[4], source_ref=required[5], + ) + + +PARSERS = { + "security_identity": _parse_identity, + "membership": _parse_membership, + "events": _parse_event, + "evaluations": _parse_evaluation, +} + + +def parse_universe_evidence(package: LoadedUniversePackage) -> ParsedUniverseEvidence: + raw_rows: list[RawEvidenceRow] = [] + parsed: dict[str, list] = {name: [] for name in COLUMNS} + findings: list[ContractFinding] = [] + for contract in ("security_identity", "membership", "events", "evaluations"): + path = package.files[contract] + with path.open(encoding="utf-8", newline="") as handle: + reader = csv.DictReader(handle) + if tuple(reader.fieldnames or ()) != COLUMNS[contract]: + findings.append(ContractFinding( + contract, 1, "", ("schema_columns_invalid",), + )) + continue + for source_row, values in enumerate(reader, start=2): + clean = MappingProxyType({ + key: str(value or "") for key, value in values.items() + }) + raw_rows.append(RawEvidenceRow( + contract, path.name, source_row, clean, + )) + row_id = clean.get(ROW_ID_FIELDS[contract], "").strip() + try: + parsed[contract].append(PARSERS[contract](clean)) + except (KeyError, TypeError, ValueError) as exc: + reason = str(exc) + if not reason.startswith("schema_"): + reason = "schema_value_invalid" + findings.append(ContractFinding( + contract, source_row, row_id, (reason,), + )) + return ParsedUniverseEvidence( + raw=tuple(raw_rows), + identities=tuple(parsed["security_identity"]), + memberships=tuple(parsed["membership"]), + events=tuple(parsed["events"]), + evaluations=tuple(parsed["evaluations"]), + findings=tuple(findings), + ) +``` + +- [ ] **Step 4: Add ratio-pair and exact-column tests** + +```python +def test_split_requires_both_positive_ratio_values(tmp_path): + from src.point_in_time_universe_manifest import load_universe_package + from src.point_in_time_universe_contracts import parse_universe_evidence + manifest, registry = build_valid_package(tmp_path) + def mutate(rows): + rows[0].update(event_type="split", ratio_numerator="2", ratio_denominator="") + _rewrite_csv_and_manifest(manifest, "events", mutate) + parsed = parse_universe_evidence(load_universe_package(manifest, registry)) + assert "schema_ratio_pair_required" in {c for f in parsed.findings for c in f.reason_codes} + + +def test_unexpected_or_missing_columns_block_contract(tmp_path): + from src.point_in_time_universe_manifest import load_universe_package + from src.point_in_time_universe_contracts import parse_universe_evidence + manifest, registry = build_valid_package(tmp_path) + path = manifest.parent / "identity.csv" + text = path.read_text().replace("issuer_id,", "") + path.write_text(text, encoding="utf-8") + raw = json.loads(manifest.read_text()) + entry = next(i for i in raw["files"] if i["contract"] == "security_identity") + entry["sha256"] = __import__("hashlib").sha256(path.read_bytes()).hexdigest() + manifest.write_text(json.dumps(raw), encoding="utf-8") + parsed = parse_universe_evidence(load_universe_package(manifest, registry)) + assert "schema_columns_invalid" in {c for f in parsed.findings for c in f.reason_codes} +``` + +- [ ] **Step 5: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe_contracts.py tests/test_point_in_time_universe_manifest.py -q +git diff --check +git add -- src/point_in_time_universe_contracts.py tests/test_point_in_time_universe_contracts.py +make staged-hygiene-check +git commit -m "Add point-in-time universe contracts" +``` + +Expected: contract and manifest tests pass; no repository CSV/JSON is staged. + +--- + +### Task 3: Exact Revision Lineage, Stable Identity, and Historical Membership + +**Files:** +- Create: `src/point_in_time_universe_lineage.py` +- Create: `tests/test_point_in_time_universe_lineage.py` +- Create: `src/point_in_time_universe.py` +- Create: `tests/test_point_in_time_universe.py` + +**Interfaces:** +- Produces `LineageResult[T]` and `resolve_lineage(records, row_id, parent_id, scope, available_at, cutoff)`. +- Produces `Decision`, `ExcludedRow`, `MembershipDigest`, and `PointInTimeUniversePacket`. +- Produces `validate_point_in_time_universe(manifest_path, registry_path, top_n=20)`. + +- [ ] **Step 1: Write failing generic lineage tests** + +```python +# tests/test_point_in_time_universe_lineage.py +from dataclasses import dataclass +from datetime import datetime, timezone + +from src.point_in_time_universe_lineage import resolve_lineage + + +@dataclass(frozen=True) +class Row: + row_id: str + parent_id: str + scope: str + available_at: datetime + + +T0 = datetime(2020, 1, 1, tzinfo=timezone.utc) +T1 = datetime(2021, 1, 1, tzinfo=timezone.utc) + + +def test_selects_latest_unambiguous_leaf_available_by_cutoff(): + rows = (Row("a", "", "scope", T0), Row("b", "a", "scope", T1)) + result = resolve_lineage( + rows, row_id=lambda r: r.row_id, parent_id=lambda r: r.parent_id, + scope=lambda r: r.scope, available_at=lambda r: r.available_at, cutoff=T1, + ) + assert result.leaves == (rows[1],) + assert result.reason_codes == () + + +def test_fork_blocks_scope_instead_of_picking_a_leaf(): + rows = (Row("a", "", "scope", T0), Row("b", "a", "scope", T1), Row("c", "a", "scope", T1)) + result = resolve_lineage( + rows, row_id=lambda r: r.row_id, parent_id=lambda r: r.parent_id, + scope=lambda r: r.scope, available_at=lambda r: r.available_at, cutoff=T1, + ) + assert result.leaves == () + assert "lineage_fork" in result.reason_codes +``` + +Add the remaining lineage cases with explicit rows: + +```python +def _resolve(rows, cutoff=T1): + return resolve_lineage( + rows, row_id=lambda r: r.row_id, parent_id=lambda r: r.parent_id, + scope=lambda r: r.scope, available_at=lambda r: r.available_at, + cutoff=cutoff, + ) + + +@pytest.mark.parametrize("rows,reason", [ + ((Row("a", "", "s", T0), Row("a", "", "s", T1)), "lineage_duplicate_id"), + ((Row("b", "missing", "s", T1),), "lineage_missing_parent"), + ((Row("a", "", "s1", T0), Row("b", "a", "s2", T1)), "lineage_cross_scope_parent"), + ((Row("a", "", "s", T0), Row("b", "", "s", T1)), "lineage_multiple_roots"), + ((Row("a", "b", "s", T0), Row("b", "a", "s", T1)), "lineage_cycle"), + ((Row("a", "", "s", T1), Row("b", "a", "s", T0)), "lineage_order_reversed"), +]) +def test_invalid_lineage_is_blocked(rows, reason): + assert reason in _resolve(rows).reason_codes + + +def test_post_cutoff_revision_is_not_selected(): + rows = (Row("a", "", "scope", T0), Row("b", "a", "scope", T1)) + result = _resolve(rows, cutoff=T0) + assert tuple(row.row_id for row in result.leaves) == ("a",) +``` + +- [ ] **Step 2: Run lineage tests and observe RED** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe_lineage.py -q +``` + +Expected: collection fails because `src.point_in_time_universe_lineage` does not exist. + +- [ ] **Step 3: Implement generic lineage resolution** + +```python +# src/point_in_time_universe_lineage.py +from __future__ import annotations + +from dataclasses import dataclass +from datetime import datetime +from typing import Callable, Generic, Iterable, TypeVar + +T = TypeVar("T") + + +@dataclass(frozen=True) +class LineageResult(Generic[T]): + leaves: tuple[T, ...] + excluded: tuple[T, ...] + reason_codes: tuple[str, ...] + + +def resolve_lineage( + records: Iterable[T], *, row_id: Callable[[T], str], + parent_id: Callable[[T], str], scope: Callable[[T], str], + available_at: Callable[[T], datetime], cutoff: datetime, +) -> LineageResult[T]: + eligible = tuple(record for record in records if available_at(record) <= cutoff) + ids = [row_id(record) for record in eligible] + reasons: set[str] = set() + if len(ids) != len(set(ids)): + reasons.add("lineage_duplicate_id") + by_id = {row_id(record): record for record in eligible} + children: dict[str, list[T]] = {} + roots: dict[str, list[T]] = {} + for record in eligible: + parent = parent_id(record) + record_scope = scope(record) + if not parent: + roots.setdefault(record_scope, []).append(record) + continue + prior = by_id.get(parent) + if prior is None: + reasons.add("lineage_missing_parent") + continue + if scope(prior) != record_scope: + reasons.add("lineage_cross_scope_parent") + if available_at(record) <= available_at(prior): + reasons.add("lineage_order_reversed") + children.setdefault(parent, []).append(record) + if any(len(value) > 1 for value in roots.values()): + reasons.add("lineage_multiple_roots") + if any(len(value) > 1 for value in children.values()): + reasons.add("lineage_fork") + for start in eligible: + seen: set[str] = set() + current = start + while parent_id(current): + current_id = row_id(current) + if current_id in seen: + reasons.add("lineage_cycle") + break + seen.add(current_id) + parent = by_id.get(parent_id(current)) + if parent is None: + break + current = parent + if reasons: + return LineageResult((), eligible, tuple(sorted(reasons))) + leaves = tuple(record for record in eligible if row_id(record) not in children) + return LineageResult(leaves, tuple(record for record in eligible if record not in leaves), ()) +``` + +- [ ] **Step 4: Write failing identity and membership tests** + +```python +# tests/test_point_in_time_universe.py +import csv +import json + +from tests.point_in_time_universe_fixture import build_valid_package +from tests.test_point_in_time_universe_contracts import _rewrite_csv_and_manifest + + +def test_ticker_change_preserves_security_identity_without_current_ticker_fallback(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + def mutate(rows): + prior = dict(rows[0]) + rows[0]["valid_to"] = "2020-06-01T00:00:00Z" + rows.append({ + **prior, "identity_row_id": "id-2", "ticker": "BBB", + "valid_from": "2020-06-01T00:00:00Z", "valid_to": "", + "source_ref": "fixture://identity/id-2", + "source_published_at": "2020-06-01T00:00:00Z", + "retrieved_at": "2020-06-02T00:00:00Z", + "supersedes_identity_row_id": "id-1", + }) + _rewrite_csv_and_manifest(manifest, "security_identity", mutate) + packet = validate_point_in_time_universe(manifest, registry) + assert packet.decisions["identity_coverage"].status == "passed" + assert packet.display_tickers["sec-1"] == "BBB" + + +def test_same_ticker_for_two_security_ids_does_not_merge_membership(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + def mutate(rows): + rows.append({**rows[0], "identity_row_id": "id-2", "security_id": "sec-2", "issuer_id": "issuer-2", "source_ref": "fixture://identity/id-2"}) + _rewrite_csv_and_manifest(manifest, "security_identity", mutate) + packet = validate_point_in_time_universe(manifest, registry) + assert packet.membership_digests[0].member_count == 1 +``` + +Add explicit identity and membership failure cases: + +```python +@pytest.mark.parametrize("case,reason", [ + ("overlapping_identity", "identity_interval_overlap"), + ("missing_identity", "identity_missing"), + ("membership_outside_interval", "membership_interval_inactive"), + ("undeclared_universe", "membership_universe_undeclared"), + ("kind_mismatch", "membership_universe_kind_mismatch"), +]) +def test_identity_and_membership_fail_closed(tmp_path, case, reason): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + mutate_identity_membership_case(manifest, case) + packet = validate_point_in_time_universe(manifest, registry) + assert reason in { + code for decision in packet.decisions.values() + for code in decision.reason_codes + } + assert packet.analysis_eligible is False + + +def mutate_identity_membership_case(manifest, case): + if case == "overlapping_identity": + def mutate(rows): + rows.append({ + **rows[0], "identity_row_id": "id-overlap", + "source_ref": "fixture://identity/id-overlap", + "source_published_at": "2020-02-01T00:00:00Z", + "retrieved_at": "2020-02-02T00:00:00Z", + "supersedes_identity_row_id": "id-1", + }) + _rewrite_csv_and_manifest(manifest, "security_identity", mutate) + elif case == "missing_identity": + _rewrite_csv_and_manifest( + manifest, "membership", + lambda rows: rows[0].update(security_id="sec-missing"), + ) + elif case == "membership_outside_interval": + _rewrite_csv_and_manifest( + manifest, "membership", + lambda rows: rows[0].update(effective_to="2020-06-01T00:00:00Z"), + ) + elif case == "undeclared_universe": + _rewrite_csv_and_manifest( + manifest, "membership", + lambda rows: rows[0].update(universe_id="unknown"), + ) + elif case == "kind_mismatch": + _rewrite_csv_and_manifest( + manifest, "membership", + lambda rows: rows[0].update(universe_kind="research_universe"), + ) +``` + +- [ ] **Step 5: Implement packet types and identity/membership composition** + +```python +# src/point_in_time_universe.py +from __future__ import annotations + +import hashlib +from dataclasses import dataclass +from pathlib import Path +from types import MappingProxyType +from typing import Mapping + +from src.point_in_time_universe_contracts import parse_universe_evidence, parse_utc +from src.point_in_time_universe_lineage import resolve_lineage +from src.point_in_time_universe_manifest import load_universe_package + + +@dataclass(frozen=True) +class Decision: + area: str + status: str + reason_codes: tuple[str, ...] + + +@dataclass(frozen=True) +class ExcludedRow: + contract: str + source_row: int + row_id: str + reason_codes: tuple[str, ...] + + +@dataclass(frozen=True) +class MembershipDigest: + universe_id: str + evaluation_at: str + member_count: int + sha256: str + + +@dataclass(frozen=True) +class PointInTimeUniversePacket: + dataset_id: str + manifest_id: str + analysis_eligible: bool + decisions: Mapping[str, Decision] + raw_count: int + normalized_count: int + excluded: tuple[ExcludedRow, ...] + membership_digests: tuple[MembershipDigest, ...] + display_tickers: Mapping[str, str] + boundary: str + + +def _membership_digest(universe_id: str, evaluation_at: str, members: set[str]) -> MembershipDigest: + payload = "\n".join(sorted(members)).encode("utf-8") + return MembershipDigest(universe_id, evaluation_at, len(members), hashlib.sha256(payload).hexdigest()) + + +def _contains(start, end, at): + return start <= at and (end is None or at < end) + + +def _row_number(parsed, contract, row_id): + id_field = { + "security_identity": "identity_row_id", + "membership": "membership_row_id", + }[contract] + return next( + (row.source_row for row in parsed.raw if row.contract == contract and row.values.get(id_field) == row_id), + 0, + ) + + +def _identity_membership_decisions(manifest, parsed): + declared = { + item["universe_id"]: item["universe_kind"] + for item in manifest.declared_universes + } + identity_reasons: set[str] = set() + membership_reasons: set[str] = set() + excluded: list[ExcludedRow] = [] + digests: list[MembershipDigest] = [] + display: dict[str, str] = {} + cutoff = parse_utc(manifest.observation_cutoff_at) + evaluations = tuple(item for item in parsed.evaluations if item.evaluation_at <= cutoff) + for evaluation in evaluations: + expected_kind = declared.get(evaluation.universe_id) + if expected_kind is None: + membership_reasons.add("membership_universe_undeclared") + continue + grouped: dict[str, list] = {} + for row in parsed.memberships: + if row.universe_id != evaluation.universe_id: + continue + if row.universe_kind != expected_kind: + membership_reasons.add("membership_universe_kind_mismatch") + continue + grouped.setdefault(row.security_id, []).append(row) + members: set[str] = set() + for security_id, rows in grouped.items(): + lineage = resolve_lineage( + rows, + row_id=lambda row: row.membership_row_id, + parent_id=lambda row: row.supersedes_membership_row_id, + scope=lambda row: f"{row.universe_id}:{row.security_id}", + available_at=lambda row: max(row.observation_at, row.source_published_at, row.retrieved_at), + cutoff=evaluation.evaluation_at, + ) + membership_reasons.update(lineage.reason_codes) + for leaf in lineage.leaves: + if not _contains(leaf.effective_from, leaf.effective_to, evaluation.evaluation_at): + membership_reasons.add("membership_interval_inactive") + excluded.append(ExcludedRow( + "membership", _row_number(parsed, "membership", leaf.membership_row_id), + leaf.membership_row_id, ("membership_interval_inactive",), + )) + continue + if leaf.membership_state == "excluded": + continue + identity_rows = tuple(row for row in parsed.identities if row.security_id == security_id) + identity_lineage = resolve_lineage( + identity_rows, + row_id=lambda row: row.identity_row_id, + parent_id=lambda row: row.supersedes_identity_row_id, + scope=lambda row: row.security_id, + available_at=lambda row: max(row.source_published_at, row.retrieved_at), + cutoff=evaluation.evaluation_at, + ) + identity_reasons.update(identity_lineage.reason_codes) + active = tuple( + row for row in identity_lineage.leaves + if _contains(row.valid_from, row.valid_to, evaluation.evaluation_at) + ) + if not active: + identity_reasons.add("identity_missing") + excluded.append(ExcludedRow( + "membership", _row_number(parsed, "membership", leaf.membership_row_id), + leaf.membership_row_id, ("identity_missing",), + )) + continue + if len(active) != 1: + identity_reasons.add("identity_interval_overlap") + continue + members.add(security_id) + display[security_id] = active[0].ticker + if not members: + membership_reasons.add("membership_no_eligible_members") + digests.append(_membership_digest( + evaluation.universe_id, + evaluation.evaluation_at.isoformat().replace("+00:00", "Z"), + members, + )) + kinds = set(declared.values()) + if "benchmark" not in kinds: + membership_reasons.add("membership_benchmark_missing") + if "research_universe" not in kinds: + membership_reasons.add("membership_research_universe_missing") + if not evaluations: + membership_reasons.add("membership_no_evaluation") + return ( + Decision("identity_coverage", "blocked" if identity_reasons else "passed", tuple(sorted(identity_reasons))), + Decision("membership_coverage", "blocked" if membership_reasons else "passed", tuple(sorted(membership_reasons))), + tuple(digests), + MappingProxyType(display), + tuple(excluded), + ) + + +def validate_point_in_time_universe(manifest_path: Path, registry_path: Path, *, top_n: int = 20) -> PointInTimeUniversePacket: + package = load_universe_package(manifest_path, registry_path) + parsed = parse_universe_evidence(package) + cutoff = parse_utc(package.manifest.observation_cutoff_at) + decisions: dict[str, Decision] = {} + excluded: list[ExcludedRow] = [ + ExcludedRow(f.contract, f.source_row, f.row_id, f.reason_codes) + for f in parsed.findings + ] + decisions["technical_validity"] = Decision( + "technical_validity", "blocked" if parsed.findings else "passed", + tuple(sorted({code for finding in parsed.findings for code in finding.reason_codes})), + ) + identity, membership, digests, display, composed_excluded = ( + _identity_membership_decisions(package.manifest, parsed) + ) + decisions[identity.area] = identity + decisions[membership.area] = membership + excluded.extend(composed_excluded) + return PointInTimeUniversePacket( + dataset_id=package.manifest.dataset_id, + manifest_id=package.manifest.manifest_id, + analysis_eligible=False, + decisions=MappingProxyType(decisions), + raw_count=len(parsed.raw), + normalized_count=sum((len(parsed.identities), len(parsed.memberships), len(parsed.events), len(parsed.evaluations))), + excluded=tuple(excluded[:top_n]), + membership_digests=digests, + display_tickers=display, + boundary="Local evidence eligibility only; no readiness, backtest, probability, recommendation, or trading activation.", + ) +``` + +- [ ] **Step 6: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_point_in_time_universe_lineage.py \ + tests/test_point_in_time_universe.py \ + tests/test_point_in_time_universe_contracts.py \ + tests/test_point_in_time_universe_manifest.py -q +git diff --check +git add -- src/point_in_time_universe_lineage.py src/point_in_time_universe.py tests/test_point_in_time_universe_lineage.py tests/test_point_in_time_universe.py +make staged-hygiene-check +git commit -m "Validate historical universe identity and membership" +``` + +Expected: lineage, identity, membership, contracts, and manifest tests pass. + +--- + +### Task 4: Corporate Actions, Delistings, and Exact-Source Rights + +**Files:** +- Modify: `src/point_in_time_universe.py` +- Modify: `tests/test_point_in_time_universe.py` + +**Interfaces:** +- Adds `corporate_action_coverage`, `delisting_coverage`, and `source_rights_eligibility` decisions. +- Uses `load_source_rights_registry` and `review_commercial_field_scope`. + +- [ ] **Step 0: Add the existing source-rights imports** + +```python +# src/point_in_time_universe.py +from src.commercial_source_rights import ( + load_source_rights_registry, + review_commercial_field_scope, +) +``` + +- [ ] **Step 1: Write failing action and delisting tests** + +```python +def test_split_requires_positive_explicit_ratio_and_does_not_rewrite_membership(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + def mutate(rows): + rows[0].update(event_type="split", ratio_numerator="2", ratio_denominator="1") + _rewrite_csv_and_manifest(manifest, "events", mutate) + packet = validate_point_in_time_universe(manifest, registry) + assert packet.decisions["corporate_action_coverage"].status == "passed" + assert packet.membership_digests[0].member_count == 1 + + +def test_delisted_historical_member_is_retained_and_not_filtered_by_current_state(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + def mutate(rows): + rows.append({ + **rows[0], "event_row_id": "event-2", "event_type": "delisting", + "effective_at": "2022-01-01T00:00:00Z", "listing_state_after": "delisted", + "source_ref": "fixture://event/event-2", + "source_published_at": "2022-01-01T00:00:00Z", + "retrieved_at": "2022-01-02T00:00:00Z", + "supersedes_event_row_id": "", + }) + _rewrite_csv_and_manifest(manifest, "events", mutate) + raw = json.loads(manifest.read_text()) + raw["corporate_action_policy"]["delisting"] = "required" + manifest.write_text(json.dumps(raw), encoding="utf-8") + packet = validate_point_in_time_universe(manifest, registry) + assert all(d.member_count == 1 for d in packet.membership_digests) + assert packet.decisions["delisting_coverage"].status == "passed" +``` + +Add the remaining explicit action-policy cases: + +```python +@pytest.mark.parametrize("event_type,updates,reason", [ + ("merger", {"successor_security_id": ""}, "corporate_action_successor_required"), + ("acquisition", {"successor_security_id": ""}, "corporate_action_successor_required"), + ("spinoff", {"successor_security_id": ""}, "corporate_action_successor_required"), + ("delisting", {"listing_state_after": "active"}, "delisting_state_invalid"), + ("reactivation", {"listing_state_after": "active"}, "delisting_transition_invalid"), +]) +def test_invalid_action_or_listing_transition_is_blocked(tmp_path, event_type, updates, reason): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + def mutate(rows): + rows[0].update(event_type=event_type, **updates) + _rewrite_csv_and_manifest(manifest, "events", mutate) + raw = json.loads(manifest.read_text()) + raw["corporate_action_policy"][event_type] = "required" + manifest.write_text(json.dumps(raw), encoding="utf-8") + packet = validate_point_in_time_universe(manifest, registry) + assert reason in { + code for decision in packet.decisions.values() + for code in decision.reason_codes + } + + +def test_present_event_marked_unsupported_is_blocked(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + raw = json.loads(manifest.read_text()) + raw["corporate_action_policy"]["listing"] = "unsupported" + manifest.write_text(json.dumps(raw), encoding="utf-8") + packet = validate_point_in_time_universe(manifest, registry) + assert "corporate_action_policy_unsupported" in packet.decisions["corporate_action_coverage"].reason_codes +``` + +- [ ] **Step 2: Write failing independent rights tests** + +```python +def test_technical_pass_does_not_promote_unverified_rights(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + registry.write_text(registry.read_text().replace("commercial_use: approved", "commercial_use: unverified"), encoding="utf-8") + raw = json.loads(manifest.read_text()) + raw["source_rights_registry_sha256"] = __import__("hashlib").sha256(registry.read_bytes()).hexdigest() + manifest.write_text(json.dumps(raw), encoding="utf-8") + packet = validate_point_in_time_universe(manifest, registry) + assert packet.decisions["technical_validity"].status == "passed" + assert packet.decisions["source_rights_eligibility"].status == "blocked" + assert packet.analysis_eligible is False + + +def test_missing_registered_delisting_scope_blocks_only_rights_state(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + _rewrite_csv_and_manifest( + manifest, "events", + lambda rows: rows[0].update( + event_type="delisting", listing_state_after="delisted", + ), + ) + raw = json.loads(manifest.read_text()) + raw["corporate_action_policy"]["listing"] = "not_applicable" + raw["corporate_action_policy"]["delisting"] = "required" + manifest.write_text(json.dumps(raw), encoding="utf-8") + registry.write_text(registry.read_text().replace(", delistings", ""), encoding="utf-8") + raw = json.loads(manifest.read_text()) + raw["source_rights_registry_sha256"] = __import__("hashlib").sha256(registry.read_bytes()).hexdigest() + manifest.write_text(json.dumps(raw), encoding="utf-8") + packet = validate_point_in_time_universe(manifest, registry) + assert packet.decisions["technical_validity"].status == "passed" + assert "source_rights_field_scope_missing" in packet.decisions["source_rights_eligibility"].reason_codes +``` + +- [ ] **Step 3: Run focused tests and observe RED** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe.py -q +``` + +Expected: new assertions fail because the three independent decisions are absent. + +- [ ] **Step 4: Implement actions, delistings, and rights composition** + +Add these exact helpers: + +```python +def _event_decisions(manifest, parsed) -> tuple[Decision, Decision, tuple[ExcludedRow, ...]]: + action_reasons: set[str] = set() + delisting_reasons: set[str] = set() + excluded: list[ExcludedRow] = [] + events_by_type: dict[str, list] = {} + for event in parsed.events: + events_by_type.setdefault(event.event_type, []).append(event) + reasons: set[str] = set() + policy = manifest.corporate_action_policy.get(event.event_type) + if policy == "unsupported": + reasons.add("corporate_action_policy_unsupported") + if event.event_type in {"split", "reverse_split"} and ( + event.ratio_numerator is None or event.ratio_denominator is None + ): + reasons.add("corporate_action_ratio_required") + if event.event_type in {"merger", "acquisition", "spinoff"} and not event.successor_security_id: + reasons.add("corporate_action_successor_required") + if event.event_type == "delisting" and event.listing_state_after != "delisted": + reasons.add("delisting_state_invalid") + if event.event_type == "suspension" and event.listing_state_after != "suspended": + reasons.add("delisting_transition_invalid") + if event.event_type == "reactivation": + prior_suspension = any( + prior.security_id == event.security_id + and prior.event_type == "suspension" + and prior.effective_at < event.effective_at + for prior in parsed.events + ) + if event.listing_state_after != "active" or not prior_suspension: + reasons.add("delisting_transition_invalid") + if reasons: + target = delisting_reasons if event.event_type in {"delisting", "suspension", "reactivation"} else action_reasons + target.update(reasons) + source_row = next( + (row.source_row for row in parsed.raw if row.contract == "events" and row.values.get("event_row_id") == event.event_row_id), + 0, + ) + excluded.append(ExcludedRow("events", source_row, event.event_row_id, tuple(sorted(reasons)))) + for event_type, state in manifest.corporate_action_policy.items(): + if state == "required" and not events_by_type.get(event_type): + if event_type == "delisting": + delisting_reasons.add("delisting_evidence_missing") + else: + action_reasons.add("corporate_action_evidence_missing") + if manifest.delisting_policy.get("retain_historical_members") is not True: + delisting_reasons.add("delisting_survivorship_policy_invalid") + if manifest.survivorship_policy.get("filter_by_current_listing_state") is not False: + delisting_reasons.add("delisting_survivorship_policy_invalid") + delisting_applicable = ( + manifest.corporate_action_policy.get("delisting") == "required" + or any(event.event_type in {"delisting", "suspension", "reactivation"} for event in parsed.events) + ) + return ( + Decision("corporate_action_coverage", "blocked" if action_reasons else "passed", tuple(sorted(action_reasons))), + Decision( + "delisting_coverage", + "blocked" if delisting_reasons else "passed" if delisting_applicable else "not_applicable", + tuple(sorted(delisting_reasons)), + ), + tuple(excluded), + ) + + +def _rights_decision(manifest, parsed, registry) -> Decision: + blockers: set[str] = set() + for source_id in sorted({row.source_id for rows in (parsed.identities, parsed.memberships, parsed.events) for row in rows}): + if source_id not in manifest.allowed_source_ids: + blockers.add("source_rights_source_not_allowed") + required: set[str] = set() + if any(row.source_id == source_id for row in parsed.identities): + required.add("security_identity") + if any(row.source_id == source_id for row in parsed.memberships): + required.add("universe_membership") + source_events = tuple(row for row in parsed.events if row.source_id == source_id) + if any(row.event_type != "delisting" for row in source_events): + required.add("corporate_actions") + if any(row.event_type == "delisting" for row in source_events): + required.add("delistings") + review = review_commercial_field_scope(registry, source_id, tuple(sorted(required))) + if not review.commercial_rights_approved: + blockers.add(f"source_rights_{review.rights_status}") + if review.missing_supported_fields: + blockers.add("source_rights_field_scope_missing") + return Decision("source_rights_eligibility", "blocked" if blockers else "passed", tuple(sorted(blockers))) +``` + +Call `_event_decisions(package.manifest, parsed)` and +`_rights_decision(package.manifest, parsed, +load_source_rights_registry(package.registry_path))` from the packet composer, +then append their decisions and exclusions without changing the already +computed identity or membership decisions. + +- [ ] **Step 5: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe.py tests/test_commercial_source_rights.py -q +git diff --check +git add -- src/point_in_time_universe.py tests/test_point_in_time_universe.py +make staged-hygiene-check +git commit -m "Enforce universe action and rights gates" +``` + +Expected: action, delisting, rights, and existing commercial-rights tests pass. + +--- + +### Task 5: Evaluation Cutoffs, Leakage Safety, and Deterministic Reproduction + +**Files:** +- Modify: `src/point_in_time_universe.py` +- Modify: `tests/test_point_in_time_universe.py` + +**Interfaces:** +- Adds `temporal_validity`, `reproduction_ready`, and `leakage_safe`. +- Completes `analysis_eligible` as the conjunction of all applicable independent states plus one eligible benchmark and one eligible research universe. + +- [ ] **Step 1: Write failing cutoff and leakage tests** + +```python +import pytest + + +@pytest.mark.parametrize("contract,column", [ + ("security_identity", "source_published_at"), + ("security_identity", "retrieved_at"), + ("membership", "observation_at"), + ("membership", "source_published_at"), + ("membership", "retrieved_at"), + ("events", "effective_at"), + ("events", "source_published_at"), + ("events", "retrieved_at"), +]) +def test_post_cutoff_evidence_is_excluded_without_poisoning_independent_states(tmp_path, contract, column): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + _rewrite_csv_and_manifest(manifest, contract, lambda rows: rows[0].update({column: "2022-01-01T00:00:00Z"})) + packet = validate_point_in_time_universe(manifest, registry) + assert packet.decisions["leakage_safe"].status == "blocked" + assert any(code.startswith("leakage_") or code.startswith("cutoff_") for row in packet.excluded for code in row.reason_codes) + assert packet.analysis_eligible is False + + +def test_later_revision_is_invisible_at_earlier_evaluation(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + # Add a post-evaluation membership revision that excludes sec-1. + def mutate(rows): + rows.append({ + **rows[0], "membership_row_id": "member-late", + "membership_state": "excluded", + "source_ref": "fixture://membership/late", + "source_published_at": "2022-01-01T00:00:00Z", + "retrieved_at": "2022-01-02T00:00:00Z", + "supersedes_membership_row_id": rows[0]["membership_row_id"], + }) + _rewrite_csv_and_manifest(manifest, "membership", mutate) + packet = validate_point_in_time_universe(manifest, registry) + assert packet.membership_digests[0].member_count == 1 +``` + +- [ ] **Step 2: Write failing partition and reproduction tests** + +```python +def test_repeated_validation_reproduces_counts_digests_and_reasons(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + first = validate_point_in_time_universe(manifest, registry) + second = validate_point_in_time_universe(manifest, registry) + assert first.membership_digests == second.membership_digests + assert first.decisions == second.decisions + assert first.excluded == second.excluded + assert {d.universe_id: d.member_count for d in first.membership_digests} == {"bench-1": 1, "research-1": 1} + + +def test_partition_policy_rejects_overlap_and_post_hoc_order(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + raw = json.loads(manifest.read_text()) + raw["evaluation_policy"] = { + "kind": "partitioned", + "train": {"start": "2020-01-01T00:00:00Z", "end": "2021-01-01T00:00:00Z"}, + "validation": {"start": "2020-12-01T00:00:00Z", "end": "2021-06-01T00:00:00Z"}, + "test": {"start": "2021-06-01T00:00:00Z", "end": "2022-01-01T00:00:00Z"}, + } + manifest.write_text(json.dumps(raw), encoding="utf-8") + packet = validate_point_in_time_universe(manifest, registry) + assert "partition_overlap" in packet.decisions["leakage_safe"].reason_codes +``` + +- [ ] **Step 3: Run focused tests and observe RED** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe.py -q +``` + +Expected: cutoff, partition, reproduction, or final eligibility assertions fail. + +- [ ] **Step 4: Implement temporal, partition, and reproduction helpers** + +```python +def _temporal_decision(parsed) -> tuple[Decision, tuple[str, ...], tuple[ExcludedRow, ...]]: + reasons: set[str] = set() + leakage_reasons: set[str] = set() + excluded: list[ExcludedRow] = [] + for evaluation in parsed.evaluations: + if evaluation.available_at > evaluation.evaluation_at: + reasons.add("cutoff_evaluation_unavailable") + leakage_reasons.add("leakage_evaluation_available_late") + excluded.append(ExcludedRow( + "evaluations", 0, evaluation.evaluation_row_id, + ("cutoff_evaluation_unavailable", "leakage_evaluation_available_late"), + )) + scoped = ( + ( + "security_identity", parsed.identities, + lambda row: row.security_id, + lambda row: max(row.source_published_at, row.retrieved_at), + lambda row: row.identity_row_id, + ), + ( + "membership", + tuple(row for row in parsed.memberships if row.universe_id == evaluation.universe_id), + lambda row: f"{row.universe_id}:{row.security_id}", + lambda row: max(row.observation_at, row.source_published_at, row.retrieved_at), + lambda row: row.membership_row_id, + ), + ( + "events", parsed.events, + lambda row: f"{row.security_id}:{row.event_type}", + lambda row: max(row.effective_at, row.source_published_at, row.retrieved_at), + lambda row: row.event_row_id, + ), + ) + for contract, rows, scope, available_at, row_id in scoped: + groups: dict[str, list] = {} + for row in rows: + groups.setdefault(scope(row), []).append(row) + for group in groups.values(): + if any(available_at(row) <= evaluation.evaluation_at for row in group): + continue + reasons.add("cutoff_required_scope_unavailable") + leakage_reasons.add("leakage_post_cutoff_evidence") + for row in group: + source_row = next( + ( + raw.source_row for raw in parsed.raw + if raw.contract == contract + and row_id(row) in raw.values.values() + ), + 0, + ) + excluded.append(ExcludedRow( + contract, source_row, row_id(row), + ("cutoff_required_scope_unavailable", "leakage_post_cutoff_evidence"), + )) + return ( + Decision("temporal_validity", "blocked" if reasons else "passed", tuple(sorted(reasons))), + tuple(sorted(leakage_reasons)), + tuple(excluded), + ) + + +def _partition_decision(manifest, evaluations, extra_reasons=()) -> Decision: + policy = manifest.evaluation_policy + reasons = set(extra_reasons) + if policy.get("kind") == "walk_forward": + minimum = policy.get("minimum_history_count") + if not isinstance(minimum, int) or isinstance(minimum, bool) or minimum <= 0: + reasons.add("partition_minimum_history_invalid") + elif policy.get("kind") == "partitioned": + try: + train_start = parse_utc(policy["train"]["start"]) + train_end = parse_utc(policy["train"]["end"]) + validation_start = parse_utc(policy["validation"]["start"]) + validation_end = parse_utc(policy["validation"]["end"]) + test_start = parse_utc(policy["test"]["start"]) + test_end = parse_utc(policy["test"]["end"]) + except (KeyError, TypeError, ValueError): + reasons.add("partition_schema_invalid") + else: + if not all(( + train_start < train_end, + validation_start < validation_end, + test_start < test_end, + )): + reasons.add("partition_order_invalid") + if train_end > validation_start or validation_end > test_start: + reasons.add("partition_overlap") + else: + reasons.add("partition_policy_invalid") + return Decision("leakage_safe", "blocked" if reasons else "passed", tuple(sorted(reasons))) + + +def _reproduction_decision(manifest, digests) -> Decision: + reasons: set[str] = set() + if manifest.reproduction_contract != "membership_count_and_sha256_at_cutoff_v1": + reasons.add("reproduction_contract_unsupported") + keys = [(item.universe_id, item.evaluation_at) for item in digests] + if len(keys) != len(set(keys)): + reasons.add("reproduction_duplicate_evaluation") + if any(len(item.sha256) != 64 for item in digests): + reasons.add("reproduction_digest_invalid") + return Decision("reproduction_ready", "blocked" if reasons else "passed", tuple(sorted(reasons))) + + +def _final_eligibility(decisions, digests, declared_universes) -> bool: + applicable_pass = all(decision.status in {"passed", "not_applicable"} for decision in decisions.values()) + kinds = {item["universe_id"]: item["universe_kind"] for item in declared_universes} + eligible_ids = {digest.universe_id for digest in digests} + return ( + applicable_pass + and any(kinds.get(item) == "benchmark" for item in eligible_ids) + and any(kinds.get(item) == "research_universe" for item in eligible_ids) + ) +``` + +Compose the final packet in this fixed decision order: + +```python +DECISION_ORDER = ( + "manifest_integrity", "technical_validity", "temporal_validity", + "identity_coverage", "membership_coverage", "corporate_action_coverage", + "delisting_coverage", "source_rights_eligibility", + "reproduction_ready", "leakage_safe", +) +``` + +Update `validate_point_in_time_universe` after identity/membership composition +with these exact calls: + +```python +decisions["manifest_integrity"] = Decision("manifest_integrity", "passed", ()) +temporal, cutoff_leakage, temporal_excluded = _temporal_decision(parsed) +decisions[temporal.area] = temporal +excluded.extend(temporal_excluded) +action, delisting, event_excluded = _event_decisions(package.manifest, parsed) +decisions[action.area] = action +decisions[delisting.area] = delisting +excluded.extend(event_excluded) +registry = load_source_rights_registry(package.registry_path) +rights = _rights_decision(package.manifest, parsed, registry) +decisions[rights.area] = rights +reproduction = _reproduction_decision(package.manifest, digests) +decisions[reproduction.area] = reproduction +leakage = _partition_decision( + package.manifest, parsed.evaluations, cutoff_leakage, +) +decisions[leakage.area] = leakage +ordered_decisions = MappingProxyType({ + name: decisions[name] for name in DECISION_ORDER +}) +analysis_eligible = _final_eligibility( + ordered_decisions, digests, package.manifest.declared_universes, +) +``` + +Pass `ordered_decisions` and `analysis_eligible` into the final packet. A +missing decision key is an implementation error and must fail tests rather +than be defaulted to passed. + +- [ ] **Step 5: Add empty, all-excluded, missing-benchmark, and missing-research-universe tests** + +```python +@pytest.mark.parametrize("mutation,reason", [ + ("no_evaluations", "membership_no_evaluation"), + ("benchmark_only", "membership_research_universe_missing"), + ("research_only", "membership_benchmark_missing"), + ("all_excluded", "membership_no_eligible_members"), +]) +def test_empty_or_one_sided_packages_fail_closed(tmp_path, mutation, reason): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + mutate_package_for_empty_case(manifest, mutation) + packet = validate_point_in_time_universe(manifest, registry) + assert packet.analysis_eligible is False + assert reason in {code for decision in packet.decisions.values() for code in decision.reason_codes} +``` + +Add these exact test helpers above the parametrized test: + +```python +def _replace_contract_rows(manifest, contract, rows): + raw = json.loads(manifest.read_text()) + entry = next(item for item in raw["files"] if item["contract"] == contract) + path = manifest.parent / entry["path"] + with path.open(encoding="utf-8", newline="") as handle: + fieldnames = next(csv.reader(handle)) + with path.open("w", encoding="utf-8", newline="") as handle: + writer = csv.DictWriter(handle, fieldnames=fieldnames) + writer.writeheader() + writer.writerows(rows) + entry["sha256"] = __import__("hashlib").sha256(path.read_bytes()).hexdigest() + entry["row_count"] = len(rows) + manifest.write_text(json.dumps(raw), encoding="utf-8") + + +def _read_contract_rows(manifest, contract): + raw = json.loads(manifest.read_text()) + entry = next(item for item in raw["files"] if item["contract"] == contract) + with (manifest.parent / entry["path"]).open(encoding="utf-8", newline="") as handle: + return list(csv.DictReader(handle)) + + +def mutate_package_for_empty_case(manifest, mutation): + memberships = _read_contract_rows(manifest, "membership") + evaluations = _read_contract_rows(manifest, "evaluations") + raw = json.loads(manifest.read_text()) + if mutation == "no_evaluations": + _replace_contract_rows(manifest, "evaluations", []) + return + if mutation == "benchmark_only": + memberships = [row for row in memberships if row["universe_kind"] == "benchmark"] + evaluations = [row for row in evaluations if row["universe_id"] == "bench-1"] + raw["declared_universes"] = [ + item for item in raw["declared_universes"] + if item["universe_kind"] == "benchmark" + ] + elif mutation == "research_only": + memberships = [row for row in memberships if row["universe_kind"] == "research_universe"] + evaluations = [row for row in evaluations if row["universe_id"] == "research-1"] + raw["declared_universes"] = [ + item for item in raw["declared_universes"] + if item["universe_kind"] == "research_universe" + ] + elif mutation == "all_excluded": + for row in memberships: + row["membership_state"] = "excluded" + manifest.write_text(json.dumps(raw), encoding="utf-8") + _replace_contract_rows(manifest, "membership", memberships) + _replace_contract_rows(manifest, "evaluations", evaluations) +``` + +- [ ] **Step 6: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe.py tests/test_point_in_time_universe_lineage.py -q +git diff --check +git add -- src/point_in_time_universe.py tests/test_point_in_time_universe.py +make staged-hygiene-check +git commit -m "Add leakage-safe universe reproduction" +``` + +Expected: all composed validator and lineage tests pass. + +--- + +### Task 6: Read-Only CLI and Make Interfaces + +**Files:** +- Create: `tests/test_point_in_time_universe_cli.py` +- Modify: `src/point_in_time_universe.py` +- Modify: `Makefile` + +**Interfaces:** +- Produces `render_status(packet) -> str` and `render_preview(packet, top_n=20) -> str`. +- Produces `main(argv: list[str] | None = None) -> int`. +- Adds `make point-in-time-universe-status MANIFEST= [REGISTRY=]`. +- Adds `make point-in-time-universe-preview MANIFEST= [REGISTRY=] [TOP_N=20]`. + +- [ ] **Step 1: Write failing CLI and Make tests** + +```python +# tests/test_point_in_time_universe_cli.py +from pathlib import Path +import subprocess +import sys + +from tests.point_in_time_universe_fixture import build_valid_package + + +def _bytes(root: Path) -> dict[str, bytes]: + return {p.relative_to(root).as_posix(): p.read_bytes() for p in root.rglob("*") if p.is_file()} + + +def test_status_and_preview_are_read_only_and_truthful(tmp_path): + manifest, registry = build_valid_package(tmp_path) + before = _bytes(tmp_path) + for mode in ("status", "preview"): + result = subprocess.run( + [sys.executable, "-m", "src.point_in_time_universe", mode, + "--manifest", str(manifest), "--registry", str(registry), "--top-n", "5"], + capture_output=True, text=True, check=False, + ) + assert result.returncode == 0 + assert "Research-only" in result.stdout + assert "does not activate readiness, backtesting, calibration, or probability" in result.stdout + assert "analysis_eligible:" in result.stdout + assert _bytes(tmp_path) == before + + +def test_invalid_invocation_is_nonzero_without_traceback(): + result = subprocess.run( + [sys.executable, "-m", "src.point_in_time_universe", "status"], + capture_output=True, text=True, check=False, + ) + assert result.returncode == 2 + assert "MANIFEST is required" in result.stderr + assert "Traceback" not in result.stderr + + +def test_readable_blocked_package_returns_zero_with_blocked_states(tmp_path): + manifest, registry = build_valid_package(tmp_path) + registry.write_text(registry.read_text().replace("commercial_use: approved", "commercial_use: unverified"), encoding="utf-8") + raw = __import__("json").loads(manifest.read_text()) + raw["source_rights_registry_sha256"] = __import__("hashlib").sha256(registry.read_bytes()).hexdigest() + manifest.write_text(__import__("json").dumps(raw), encoding="utf-8") + result = subprocess.run( + [sys.executable, "-m", "src.point_in_time_universe", "status", + "--manifest", str(manifest), "--registry", str(registry)], + capture_output=True, text=True, check=False, + ) + assert result.returncode == 0 + assert "source_rights_eligibility: blocked" in result.stdout +``` + +- [ ] **Step 2: Run CLI tests and observe RED** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_point_in_time_universe_cli.py -q +``` + +Expected: CLI and Make assertions fail because rendering and targets are absent. + +- [ ] **Step 3: Implement rendering and CLI** + +```python +def render_status(packet: PointInTimeUniversePacket) -> str: + lines = [ + "Point-in-Time Universe Status", + "Read-only: validates one supplied immutable package; it does not fetch, write, apply, refresh, or rebuild data.", + "Research-only: this does not activate readiness, backtesting, calibration, or probability and is not investment advice.", + f"dataset_id: {packet.dataset_id}", + f"manifest_id: {packet.manifest_id}", + f"analysis_eligible: {str(packet.analysis_eligible).lower()}", + ] + lines.extend( + f"{name}: {packet.decisions[name].status}; reasons={','.join(packet.decisions[name].reason_codes) or 'none'}" + for name in DECISION_ORDER + ) + lines.append(f"boundary: {packet.boundary}") + return "\n".join(lines) + + +def render_preview(packet: PointInTimeUniversePacket, *, top_n: int = 20) -> str: + lines = [render_status(packet), "", "Membership reproduction:"] + lines.extend( + f"- {item.universe_id} @ {item.evaluation_at}: members={item.member_count}; sha256={item.sha256}" + for item in packet.membership_digests + ) + lines.append("Excluded sample:") + lines.extend( + f"- {item.contract}:{item.source_row}:{item.row_id}; reasons={','.join(item.reason_codes)}" + for item in packet.excluded[:top_n] + ) + return "\n".join(lines) + + +def main(argv: list[str] | None = None) -> int: + import argparse + parser = argparse.ArgumentParser(description="Validate one immutable point-in-time universe package.") + parser.add_argument("mode", choices=("status", "preview")) + parser.add_argument("--manifest") + parser.add_argument("--registry", default="config/source_rights.yml") + parser.add_argument("--top-n", type=int, default=20) + args = parser.parse_args(argv) + if not args.manifest: + parser.error("MANIFEST is required") + try: + packet = validate_point_in_time_universe(Path(args.manifest), Path(args.registry), top_n=args.top_n) + except ValueError as exc: + parser.error(str(exc)) + print(render_preview(packet, top_n=args.top_n) if args.mode == "preview" else render_status(packet)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) +``` + +- [ ] **Step 4: Add exact Make targets** + +```make +.PHONY: point-in-time-universe-status point-in-time-universe-preview + +point-in-time-universe-status: + @test -n "$(MANIFEST)" || (echo "MANIFEST is required" >&2; exit 2) + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.point_in_time_universe status \ + --manifest "$(MANIFEST)" \ + --registry "$(or $(REGISTRY),config/source_rights.yml)" \ + --top-n "$(or $(TOP_N),20)" + +point-in-time-universe-preview: + @test -n "$(MANIFEST)" || (echo "MANIFEST is required" >&2; exit 2) + @PYTHONDONTWRITEBYTECODE=1 python3 -m src.point_in_time_universe preview \ + --manifest "$(MANIFEST)" \ + --registry "$(or $(REGISTRY),config/source_rights.yml)" \ + --top-n "$(or $(TOP_N),20)" +``` + +Add both names to the top-level `.PHONY` declaration and help output. Do not +add stage, apply, record, refresh, or output parameters. + +- [ ] **Step 5: Add Make no-write test** + +```python +def test_make_status_and_preview_use_exact_read_only_contract(tmp_path): + manifest, registry = build_valid_package(tmp_path) + before = _bytes(tmp_path) + for target in ("point-in-time-universe-status", "point-in-time-universe-preview"): + result = subprocess.run( + ["make", "--no-print-directory", target, f"MANIFEST={manifest}", + f"REGISTRY={registry}", "TOP_N=5"], + capture_output=True, text=True, check=False, + ) + assert result.returncode == 0 + assert "Read-only" in result.stdout + assert _bytes(tmp_path) == before +``` + +- [ ] **Step 6: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_point_in_time_universe_cli.py \ + tests/test_point_in_time_universe.py \ + tests/test_makefile_test_targets.py -q +git diff --check +git add -- Makefile src/point_in_time_universe.py tests/test_point_in_time_universe_cli.py +make staged-hygiene-check +git commit -m "Add read-only universe validation commands" +``` + +Expected: CLI, Make, composed validator, and existing Make target tests pass. + +--- + +### Task 7: Full Acceptance Matrix and No-Write Proof + +**Files:** +- Modify: `tests/test_point_in_time_universe_manifest.py` +- Modify: `tests/test_point_in_time_universe_contracts.py` +- Modify: `tests/test_point_in_time_universe_lineage.py` +- Modify: `tests/test_point_in_time_universe.py` +- Modify: `tests/test_point_in_time_universe_cli.py` + +**Interfaces:** +- No new production interface. +- Completes every test group named in the approved specification. + +- [ ] **Step 1: Add the complete stable reason-code matrix** + +```python +EXPECTED_REASON_PREFIXES = { + "manifest_", "schema_", "lineage_", "identity_", "membership_", + "corporate_action_", "delisting_", "source_rights_", "cutoff_", + "leakage_", "partition_", "reproduction_", +} + + +def test_every_exclusion_uses_an_approved_stable_reason_family(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + _rewrite_csv_and_manifest(manifest, "membership", lambda rows: rows[0].update(retrieved_at="2022-01-01T00:00:00Z")) + packet = validate_point_in_time_universe(manifest, registry) + for item in packet.excluded: + for code in item.reason_codes: + assert any(code.startswith(prefix) for prefix in EXPECTED_REASON_PREFIXES) +``` + +- [ ] **Step 2: Add the whole-root no-write test** + +```python +def test_library_cli_and_make_leave_entire_root_byte_identical(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + before = _bytes(tmp_path) + validate_point_in_time_universe(manifest, registry) + subprocess.run( + [sys.executable, "-m", "src.point_in_time_universe", "preview", + "--manifest", str(manifest), "--registry", str(registry)], + check=True, capture_output=True, text=True, + ) + subprocess.run( + ["make", "--no-print-directory", "point-in-time-universe-preview", + f"MANIFEST={manifest}", f"REGISTRY={registry}"], + check=True, capture_output=True, text=True, + ) + assert _bytes(tmp_path) == before +``` + +- [ ] **Step 3: Add explicit no-current-universe dependency test** + +```python +def test_validator_result_is_unchanged_when_current_universe_files_change(tmp_path): + from src.point_in_time_universe import validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + first = validate_point_in_time_universe(manifest, registry) + data = tmp_path / "data" + data.mkdir() + (data / "universe.csv").write_text("ticker\nZZZ\n", encoding="utf-8") + (data / "universe_master.csv").write_text("ticker,is_active_listing\nAAA,false\n", encoding="utf-8") + second = validate_point_in_time_universe(manifest, registry) + assert first == second +``` + +- [ ] **Step 4: Add test-only and no-completion assertions** + +```python +def test_synthetic_package_never_claims_priority_four_completion(tmp_path): + from src.point_in_time_universe import render_status, validate_point_in_time_universe + manifest, registry = build_valid_package(tmp_path) + output = render_status(validate_point_in_time_universe(manifest, registry)) + assert "Priority 4 complete" not in output + assert "real permitted dataset" in output + assert "synthetic" in output.lower() +``` + +Update `render_status` boundary text so it explicitly says a synthetic or +technically valid package is local software evidence only and Priority 4 still +requires one independently reviewed permitted real dataset. + +- [ ] **Step 5: Run all focused tests and fix only proven gaps** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_point_in_time_universe_manifest.py \ + tests/test_point_in_time_universe_contracts.py \ + tests/test_point_in_time_universe_lineage.py \ + tests/test_point_in_time_universe.py \ + tests/test_point_in_time_universe_cli.py \ + tests/test_commercial_source_rights.py \ + tests/test_makefile_test_targets.py -q +``` + +Expected: all focused tests pass. If a test fails, use +`superpowers:systematic-debugging`; do not weaken the assertion to match an +unsafe implementation. + +- [ ] **Step 6: Commit the acceptance hardening** + +```bash +git diff --check +git add -- src/point_in_time_universe.py tests/test_point_in_time_universe_manifest.py tests/test_point_in_time_universe_contracts.py tests/test_point_in_time_universe_lineage.py tests/test_point_in_time_universe.py tests/test_point_in_time_universe_cli.py +make staged-hygiene-check +git commit -m "Complete universe foundation acceptance matrix" +``` + +Expected: only Python product/test files are staged; generated artifacts are `0`. + +--- + +### Task 8: Documentation, Release Gates, PR, and Exact-Head CI + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Records local implementation truth and exact external exit condition. +- Does not change public product claims or call Priority 4 complete. + +- [ ] **Step 1: Write failing documentation-contract test** + +```python +def test_priority_four_local_validator_is_documented_without_claiming_real_data_completion(): + roadmap = _read("ROADMAP.md") + methodology = _read("docs/METHODOLOGY.md") + prompt = _read("docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md") + for text in (roadmap, methodology, prompt): + assert "make point-in-time-universe-status MANIFEST=" in text + assert "make point-in-time-universe-preview MANIFEST= TOP_N=20" in text + assert "membership_count_and_sha256_at_cutoff_v1" in text + assert "Synthetic fixtures remain test-only" in text + assert "one bounded permitted real dataset" in text + assert "Priority 4 is complete" not in roadmap +``` + +- [ ] **Step 2: Run the documentation test and observe RED** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_public_v1_release_docs.py::test_priority_four_local_validator_is_documented_without_claiming_real_data_completion -q +``` + +Expected: fails because implementation evidence is not yet recorded in all three documents. + +- [ ] **Step 3: Update documentation with exact boundaries** + +Add these facts to all required documents: + +```text +Implemented locally: read-only immutable package status/preview with independent +manifest, technical, temporal, identity, membership, corporate-action, +delisting, source-rights, reproduction, and leakage states. + +Reproduction contract: membership_count_and_sha256_at_cutoff_v1. + +Synthetic fixtures remain test-only. Priority 4 remains incomplete until one +bounded permitted real dataset reproduces the independently reviewed expected +membership count and digest and passes rights, identity, corporate-action, +delisting, survivorship, cutoff, and leakage gates. +``` + +Document only the two read-only commands. Do not document an apply, record, +refresh, readiness, or broad-source command. + +- [ ] **Step 4: Run focused and full verification** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest \ + tests/test_point_in_time_universe_manifest.py \ + tests/test_point_in_time_universe_contracts.py \ + tests/test_point_in_time_universe_lineage.py \ + tests/test_point_in_time_universe.py \ + tests/test_point_in_time_universe_cli.py \ + tests/test_public_v1_release_docs.py -q +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: + +- focused tests pass; +- full suite passes; +- dashboard and all six Personal Research routes pass; +- public and commercial-beta gates pass; +- pilot is `pilot-ready with manual gates` after commit; +- diff hygiene lists only intentional product files plus the 18 excluded generated files. + +- [ ] **Step 5: Stage exact files and verify hygiene** + +```bash +git add -- ROADMAP.md docs/METHODOLOGY.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md tests/test_public_v1_release_docs.py +make staged-hygiene-check +git diff --cached --check +``` + +Expected: staged generated CSV/JSON/report churn is `0`. + +- [ ] **Step 6: Commit documentation** + +```bash +git commit -m "Document point-in-time universe validator" +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: no product candidate remains; 18 generated files remain local and unstaged. + +- [ ] **Step 7: Push only the approved branch** + +```bash +git status --short --branch +git rev-list --left-right --count origin/codex/personal-research-mode-mvp...HEAD +git push origin codex/personal-research-mode-mvp +``` + +Expected: push succeeds only to `codex/personal-research-mode-mvp`. + +- [ ] **Step 8: Update draft PR #113 and require exact-head CI** + +Use the PR comment to report: + +```text +Priority 4 local validator implemented: +- immutable manifest and file integrity +- stable identity and timestamped membership +- corporate-action and delisting policy +- exact-source rights and registered field scope +- cutoff/leakage and deterministic membership digest +- status/preview only; no write/apply/readiness activation +- synthetic fixtures test-only; real dataset gate remains open +- focused/full/release/hygiene evidence +``` + +Then: + +```bash +gh pr view 113 --repo YuzeJ21/Stock-Analysis --json state,isDraft,mergeable,headRefOid,statusCheckRollup +gh run list --repo YuzeJ21/Stock-Analysis --branch codex/personal-research-mode-mvp --event pull_request --limit 5 +gh run watch --repo YuzeJ21/Stock-Analysis --exit-status +git fetch origin codex/personal-research-mode-mvp +git rev-list --left-right --count origin/codex/personal-research-mode-mvp...HEAD +``` + +Expected: + +- PR #113 remains open and draft; +- exact-head `Commercial Research Beta / local-engineering-gate` succeeds; +- branch divergence is `0 0`; +- no merge or deployment occurs. + +--- + +## Plan Completion Audit + +Before calling the local implementation slice complete, verify each approved +spec requirement against direct evidence: + +| Requirement | Direct evidence | +| --- | --- | +| Immutable package identity | Manifest hash, row-count, registry-digest, path, and schema tests | +| Stable security identity | Ticker-change, ticker-reuse, overlap, and missing-identity tests | +| Historical membership | Complete-snapshot/event-history, effective interval, declared universe, and kind tests | +| Revision lineage | Duplicate, parent, scope, root, fork, cycle, order, and cutoff tests | +| Corporate actions | Split ratio, successor, policy, suspension/reactivation tests | +| Delistings/survivorship | Explicit delisting state and historical-member retention tests | +| Source rights | Approved/unverified/unknown/missing-scope independent-state tests | +| Cutoff/leakage | Publication, retrieval, observation, event, evaluation, and later-revision tests | +| Reproduction | Repeated count/digest/state/exclusion equality tests | +| Raw/normalized/excluded/eligible | Packet counts, exclusions, reason families, and eligibility tests | +| Read-only operation | Whole-root byte snapshots across library, CLI, and Make | +| No current fallback | Result unchanged after current universe files are created or changed | +| Synthetic test-only boundary | Rendered boundary and documentation tests | +| Release safety | Focused/full/release/hygiene/exact-head CI | +| Real-data exit gate remains open | ROADMAP, methodology, prompt, PR, and rendered boundary | + +Do not mark Priority 4 complete unless one independently reviewed permitted +real dataset later passes every applicable gate and reproduces the expected +membership count and digest. Completing this plan proves only the local +validator implementation. diff --git a/docs/superpowers/plans/2026-07-26-provider-neutral-workspace-authorization.md b/docs/superpowers/plans/2026-07-26-provider-neutral-workspace-authorization.md new file mode 100644 index 000000000..3ef6399ff --- /dev/null +++ b/docs/superpowers/plans/2026-07-26-provider-neutral-workspace-authorization.md @@ -0,0 +1,1154 @@ +# Provider-Neutral Workspace Authorization Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build one pure, provider-neutral, deny-by-default workspace authorization decision that enforces principal/workspace isolation, least privilege, append-only research records, and privacy-safe audit obligations. + +**Architecture:** Add a single standard-library policy module whose frozen domain types form the supported trust boundary and whose evaluator returns an immutable allow/deny decision for every expected request. Keep the first slice isolated from Streamlit, ledgers, readiness, providers, persistence, and hosted infrastructure; update only truthful architecture and roadmap evidence after the policy tests pass. + +**Tech Stack:** Python 3, standard-library `dataclasses`, `enum`, and `unicodedata`; pytest; Markdown contract tests; existing Make release and hygiene gates. + +## Global Constraints + +- The authoritative design is `docs/superpowers/specs/2026-07-26-provider-neutral-workspace-authorization-design.md`. +- The evaluator must be pure, provider-neutral, deterministic, and deny by default. +- The module must depend only on the Python standard library. +- Identifiers must be strings of 1–128 Unicode scalar values with no leading/trailing whitespace, C0/C1 controls, Unicode line/paragraph separators, or surrogate code points. +- `authenticated` and `active` must be exact booleans; role, action, and resource must be members of their declared enums. +- Expected authorization failures return immutable denied decisions and do not raise. +- Research records are append-only: `update` and `delete` are denied for every role. +- Every decision carries a privacy-safe audit obligation; the evaluator never persists an audit event. +- No dashboard, ledger, readiness, source-rights, provider, file, environment, network, or generated-data integration is permitted in this slice. +- Do not claim runtime authentication, hosted isolation, persistence, audit storage, retention, monitoring, rollback, incident response, operated capacity, or market validation. +- Do not run readiness rebuilds, broad refreshes, or generated CSV/JSON/report commands. +- Keep the 18 existing generated CSV/report changes unstaged and preserve their diff fingerprint `a2c2f428b489dbb291dd54fd8a6e1e7f4ad9481414320ca248c54be89f4062b9`. +- Stage or synchronize exact intentional paths only; never use `git add -A`. +- Push only to `codex/personal-research-mode-mvp`; keep PR #113 open and draft; do not merge or deploy. + +## File Map + +- Create `src/hosted_access_control.py`: frozen authorization domain types, structural validation, static policy matrix, audit-obligation factory, and pure evaluator. +- Create `tests/test_hosted_access_control.py`: exhaustive policy, denial-order, malformed-input, Unicode, immutability, audit, privacy, and side-effect tests. +- Modify `docs/PRIVATE_BETA_ARCHITECTURE.md`: describe the local policy contract without promoting hosted readiness. +- Modify `ROADMAP.md`: record the completed local Priority 6 slice and preserve the external hosted exit gate. +- Modify `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`: add the exact local lineage and next safe lane while preserving external blockers. +- Modify `tests/test_public_v1_release_docs.py`: lock the provider-neutral, deny-by-default, append-only, audit, and non-claim language. + +--- + +### Task 1: Define The Frozen Trust-Boundary Types And Structural Validation + +**Files:** +- Create: `src/hosted_access_control.py` +- Create: `tests/test_hosted_access_control.py` + +**Interfaces:** +- Consumes: no project modules; Python standard library only. +- Produces: `WorkspaceRole`, `WorkspaceAction`, `WorkspaceResource`, `PrincipalContext`, `WorkspaceMembership`, `WorkspaceAccessRequest`, `AuditObligation`, `WorkspaceAccessDecision`, and `evaluate_workspace_access(principal: object, membership: object, request: object) -> WorkspaceAccessDecision`. + +- [ ] **Step 1: Write failing tests for malformed trust-boundary inputs** + +Create `tests/test_hosted_access_control.py` with the imports, valid fixture factory, and structural-denial cases: + +```python +from dataclasses import FrozenInstanceError + +import pytest + +from src.hosted_access_control import ( + AuditObligation, + PrincipalContext, + WorkspaceAccessDecision, + WorkspaceAccessRequest, + WorkspaceAction, + WorkspaceMembership, + WorkspaceResource, + WorkspaceRole, + evaluate_workspace_access, +) + + +def _valid_inputs( + *, + principal_id: object = "principal-1", + membership_principal_id: object = "principal-1", + membership_workspace_id: object = "workspace-1", + request_workspace_id: object = "workspace-1", + request_id: object = "request-1", + authenticated: object = True, + active: object = True, + role: object = WorkspaceRole.EDITOR, + action: object = WorkspaceAction.READ, + resource: object = WorkspaceResource.RESEARCH_RECORD, +) -> tuple[PrincipalContext, WorkspaceMembership, WorkspaceAccessRequest]: + return ( + PrincipalContext( + principal_id=principal_id, + authenticated=authenticated, + ), + WorkspaceMembership( + principal_id=membership_principal_id, + workspace_id=membership_workspace_id, + role=role, + active=active, + ), + WorkspaceAccessRequest( + request_id=request_id, + workspace_id=request_workspace_id, + action=action, + resource=resource, + ), + ) + + +@pytest.mark.parametrize( + ("principal", "membership", "request"), + [ + (object(),) + _valid_inputs()[1:], + (_valid_inputs()[0], object(), _valid_inputs()[2]), + _valid_inputs()[:2] + (object(),), + ], +) +def test_malformed_boundary_objects_fail_closed(principal, membership, request): + decision = evaluate_workspace_access(principal, membership, request) + + assert decision.allowed is False + assert decision.reason_code == "invalid_request" + + +@pytest.mark.parametrize( + ("principal", "membership", "request"), + [ + (object.__new__(PrincipalContext),) + _valid_inputs()[1:], + ( + _valid_inputs()[0], + object.__new__(WorkspaceMembership), + _valid_inputs()[2], + ), + _valid_inputs()[:2] + (object.__new__(WorkspaceAccessRequest),), + ], +) +def test_incomplete_exact_type_objects_fail_closed( + principal, membership, request +): + decision = evaluate_workspace_access(principal, membership, request) + + assert decision.allowed is False + assert decision.reason_code == "invalid_request" + + +@pytest.mark.parametrize( + ("field", "value"), + [ + ("principal_id", None), + ("principal_id", ""), + ("principal_id", " leading"), + ("principal_id", "trailing "), + ("principal_id", "line\nbreak"), + ("principal_id", "c1\u0085control"), + ("principal_id", "line\u2028separator"), + ("principal_id", "paragraph\u2029separator"), + ("principal_id", "\ud800"), + ("principal_id", "x" * 129), + ("membership_principal_id", 7), + ("membership_workspace_id", "\x00"), + ("request_workspace_id", "\tworkspace"), + ("request_id", False), + ("authenticated", 1), + ("active", "true"), + ("role", "editor"), + ("action", "read"), + ("resource", "research_record"), + ], +) +def test_malformed_fields_fail_closed(field, value): + decision = evaluate_workspace_access(*_valid_inputs(**{field: value})) + + assert decision == WorkspaceAccessDecision( + allowed=False, + reason_code="invalid_request", + audit=AuditObligation( + event_type="workspace_access_decision", + outcome="denied", + reason_code="invalid_request", + required_fields=( + "request_id", + "principal_id", + "workspace_id", + "resource", + "action", + "outcome", + "reason_code", + "occurred_at", + ), + ), + ) + + +def test_frozen_inputs_and_decisions_cannot_be_mutated(): + principal, membership, request = _valid_inputs() + decision = evaluate_workspace_access(principal, membership, request) + + with pytest.raises(FrozenInstanceError): + principal.principal_id = "other" + with pytest.raises(FrozenInstanceError): + membership.role = WorkspaceRole.OWNER + with pytest.raises(FrozenInstanceError): + request.workspace_id = "other" + with pytest.raises(FrozenInstanceError): + decision.allowed = True +``` + +- [ ] **Step 2: Run the new structural tests and confirm the missing-module failure** + +Run: + +```bash +python3 -m pytest tests/test_hosted_access_control.py -q +``` + +Expected: test collection fails with `ModuleNotFoundError: No module named 'src.hosted_access_control'`. + +- [ ] **Step 3: Implement enums, frozen dataclasses, identifier validation, and invalid-request decisions** + +Create `src/hosted_access_control.py`: + +```python +"""Provider-neutral, deny-by-default workspace authorization policy.""" + +from __future__ import annotations + +import unicodedata +from dataclasses import dataclass +from enum import Enum + + +class WorkspaceRole(str, Enum): + VIEWER = "viewer" + EDITOR = "editor" + OWNER = "owner" + + +class WorkspaceAction(str, Enum): + READ = "read" + APPEND = "append" + UPDATE = "update" + DELETE = "delete" + EXPORT = "export" + MANAGE = "manage" + + +class WorkspaceResource(str, Enum): + RESEARCH_RECORD = "research_record" + SAVED_WORKSPACE_STATE = "saved_workspace_state" + WORKSPACE_MEMBERSHIP = "workspace_membership" + WORKSPACE_AUDIT = "workspace_audit" + + +@dataclass(frozen=True) +class PrincipalContext: + principal_id: object + authenticated: object + + +@dataclass(frozen=True) +class WorkspaceMembership: + principal_id: object + workspace_id: object + role: object + active: object + + +@dataclass(frozen=True) +class WorkspaceAccessRequest: + request_id: object + workspace_id: object + action: object + resource: object + + +@dataclass(frozen=True) +class AuditObligation: + event_type: str + outcome: str + reason_code: str + required_fields: tuple[str, ...] + + +@dataclass(frozen=True) +class WorkspaceAccessDecision: + allowed: bool + reason_code: str + audit: AuditObligation + + +_AUDIT_FIELDS = ( + "request_id", + "principal_id", + "workspace_id", + "resource", + "action", + "outcome", + "reason_code", + "occurred_at", +) + + +def _valid_identifier(value: object) -> bool: + if not isinstance(value, str) or not 1 <= len(value) <= 128: + return False + if value != value.strip(): + return False + return all( + unicodedata.category(character) not in {"Cc", "Cs", "Zl", "Zp"} + for character in value + ) + + +def _decision(*, allowed: bool, reason_code: str) -> WorkspaceAccessDecision: + outcome = "allowed" if allowed else "denied" + return WorkspaceAccessDecision( + allowed=allowed, + reason_code=reason_code, + audit=AuditObligation( + event_type="workspace_access_decision", + outcome=outcome, + reason_code=reason_code, + required_fields=_AUDIT_FIELDS, + ), + ) + + +_MISSING_FIELD = object() + + +def _structurally_valid( + principal: object, + membership: object, + request: object, +) -> bool: + if ( + type(principal) is not PrincipalContext + or type(membership) is not WorkspaceMembership + or type(request) is not WorkspaceAccessRequest + ): + return False + + principal_id = getattr(principal, "principal_id", _MISSING_FIELD) + authenticated = getattr(principal, "authenticated", _MISSING_FIELD) + membership_principal_id = getattr( + membership, + "principal_id", + _MISSING_FIELD, + ) + membership_workspace_id = getattr( + membership, + "workspace_id", + _MISSING_FIELD, + ) + role = getattr(membership, "role", _MISSING_FIELD) + active = getattr(membership, "active", _MISSING_FIELD) + request_id = getattr(request, "request_id", _MISSING_FIELD) + request_workspace_id = getattr(request, "workspace_id", _MISSING_FIELD) + action = getattr(request, "action", _MISSING_FIELD) + resource = getattr(request, "resource", _MISSING_FIELD) + + return ( + _valid_identifier(principal_id) + and type(authenticated) is bool + and _valid_identifier(membership_principal_id) + and _valid_identifier(membership_workspace_id) + and isinstance(role, WorkspaceRole) + and type(active) is bool + and _valid_identifier(request_id) + and _valid_identifier(request_workspace_id) + and isinstance(action, WorkspaceAction) + and isinstance(resource, WorkspaceResource) + ) + + +def evaluate_workspace_access( + principal: object, + membership: object, + request: object, +) -> WorkspaceAccessDecision: + """Return one immutable policy decision without side effects.""" + if not _structurally_valid(principal, membership, request): + return _decision(allowed=False, reason_code="invalid_request") + return _decision(allowed=False, reason_code="role_action_denied") +``` + +- [ ] **Step 4: Run the structural tests** + +Run: + +```bash +python3 -m pytest tests/test_hosted_access_control.py -q +``` + +Expected: all structural and immutability tests pass. + +- [ ] **Step 5: Run a focused style and whitespace check** + +Run: + +```bash +python3 -m ruff check src/hosted_access_control.py tests/test_hosted_access_control.py +git diff --check -- src/hosted_access_control.py tests/test_hosted_access_control.py +``` + +Expected: both commands exit 0. + +- [ ] **Step 6: Synchronize the exact Task 1 paths as one coherent commit** + +Because the original worktree index is read-only and stale, do not use it to recommit the already synchronized Priority 4 package. Create a controlled writable clone from the current remote head or use the GitHub Git Data API, include only: + +```text +src/hosted_access_control.py +tests/test_hosted_access_control.py +``` + +Commit message: + +```text +Add workspace authorization trust boundary +``` + +Verify the remote compare shows exactly those two paths and zero generated artifacts. + +--- + +### Task 2: Implement Deterministic Isolation, Least Privilege, And Audit Decisions + +**Files:** +- Modify: `src/hosted_access_control.py` +- Modify: `tests/test_hosted_access_control.py` + +**Interfaces:** +- Consumes: the Task 1 enums, frozen dataclasses, `_decision`, and `_structurally_valid`. +- Produces: the complete `evaluate_workspace_access(principal: object, membership: object, request: object) -> WorkspaceAccessDecision` policy. + +- [ ] **Step 1: Add failing tests for the exact allow matrix** + +Append: + +```python +_ALLOWED_CASES = ( + (WorkspaceRole.VIEWER, WorkspaceResource.RESEARCH_RECORD, WorkspaceAction.READ), + (WorkspaceRole.EDITOR, WorkspaceResource.RESEARCH_RECORD, WorkspaceAction.READ), + (WorkspaceRole.EDITOR, WorkspaceResource.RESEARCH_RECORD, WorkspaceAction.APPEND), + (WorkspaceRole.OWNER, WorkspaceResource.RESEARCH_RECORD, WorkspaceAction.READ), + (WorkspaceRole.OWNER, WorkspaceResource.RESEARCH_RECORD, WorkspaceAction.APPEND), + (WorkspaceRole.OWNER, WorkspaceResource.RESEARCH_RECORD, WorkspaceAction.EXPORT), + (WorkspaceRole.VIEWER, WorkspaceResource.SAVED_WORKSPACE_STATE, WorkspaceAction.READ), + (WorkspaceRole.EDITOR, WorkspaceResource.SAVED_WORKSPACE_STATE, WorkspaceAction.READ), + (WorkspaceRole.EDITOR, WorkspaceResource.SAVED_WORKSPACE_STATE, WorkspaceAction.APPEND), + (WorkspaceRole.EDITOR, WorkspaceResource.SAVED_WORKSPACE_STATE, WorkspaceAction.UPDATE), + (WorkspaceRole.OWNER, WorkspaceResource.SAVED_WORKSPACE_STATE, WorkspaceAction.READ), + (WorkspaceRole.OWNER, WorkspaceResource.SAVED_WORKSPACE_STATE, WorkspaceAction.APPEND), + (WorkspaceRole.OWNER, WorkspaceResource.SAVED_WORKSPACE_STATE, WorkspaceAction.UPDATE), + (WorkspaceRole.OWNER, WorkspaceResource.SAVED_WORKSPACE_STATE, WorkspaceAction.DELETE), + (WorkspaceRole.OWNER, WorkspaceResource.SAVED_WORKSPACE_STATE, WorkspaceAction.EXPORT), + (WorkspaceRole.OWNER, WorkspaceResource.WORKSPACE_MEMBERSHIP, WorkspaceAction.READ), + (WorkspaceRole.OWNER, WorkspaceResource.WORKSPACE_MEMBERSHIP, WorkspaceAction.MANAGE), + (WorkspaceRole.OWNER, WorkspaceResource.WORKSPACE_AUDIT, WorkspaceAction.READ), + (WorkspaceRole.OWNER, WorkspaceResource.WORKSPACE_AUDIT, WorkspaceAction.EXPORT), +) + + +@pytest.mark.parametrize(("role", "resource", "action"), _ALLOWED_CASES) +def test_exact_policy_matrix_allows_only_declared_entries(role, resource, action): + decision = evaluate_workspace_access( + *_valid_inputs(role=role, resource=resource, action=action) + ) + + assert decision.allowed is True + assert decision.reason_code == "allowed" + assert decision.audit.outcome == "allowed" + assert decision.audit.reason_code == "allowed" +``` + +- [ ] **Step 2: Add failing exhaustive deny-matrix and append-only tests** + +Append: + +```python +_ALLOWED_SET = set(_ALLOWED_CASES) +_DENIED_CASES = tuple( + (role, resource, action) + for role in WorkspaceRole + for resource in WorkspaceResource + for action in WorkspaceAction + if (role, resource, action) not in _ALLOWED_SET +) + + +@pytest.mark.parametrize(("role", "resource", "action"), _DENIED_CASES) +def test_every_absent_policy_entry_is_denied(role, resource, action): + decision = evaluate_workspace_access( + *_valid_inputs(role=role, resource=resource, action=action) + ) + + expected = ( + "append_only_mutation_denied" + if resource is WorkspaceResource.RESEARCH_RECORD + and action in {WorkspaceAction.UPDATE, WorkspaceAction.DELETE} + else "role_action_denied" + ) + assert decision.allowed is False + assert decision.reason_code == expected + assert decision.audit.outcome == "denied" + assert decision.audit.reason_code == expected + + +@pytest.mark.parametrize("role", tuple(WorkspaceRole)) +@pytest.mark.parametrize( + "action", + (WorkspaceAction.UPDATE, WorkspaceAction.DELETE), +) +def test_no_role_can_mutate_append_only_research_records(role, action): + decision = evaluate_workspace_access( + *_valid_inputs( + role=role, + resource=WorkspaceResource.RESEARCH_RECORD, + action=action, + ) + ) + + assert decision.allowed is False + assert decision.reason_code == "append_only_mutation_denied" +``` + +- [ ] **Step 3: Add failing tests for deterministic denial precedence** + +Append: + +```python +@pytest.mark.parametrize( + ("changes", "reason_code"), + [ + ({"authenticated": False}, "authentication_required"), + ({"membership_principal_id": "principal-2"}, "principal_mismatch"), + ({"active": False}, "membership_inactive"), + ({"request_workspace_id": "workspace-2"}, "workspace_mismatch"), + ], +) +def test_isolation_denials_follow_declared_order(changes, reason_code): + decision = evaluate_workspace_access(*_valid_inputs(**changes)) + + assert decision.allowed is False + assert decision.reason_code == reason_code + + +def test_earlier_denial_cannot_be_overridden_by_later_allow_rule(): + decision = evaluate_workspace_access( + *_valid_inputs( + authenticated=False, + membership_principal_id="principal-2", + active=False, + request_workspace_id="workspace-2", + role=WorkspaceRole.OWNER, + resource=WorkspaceResource.SAVED_WORKSPACE_STATE, + action=WorkspaceAction.DELETE, + ) + ) + + assert decision.reason_code == "authentication_required" + + +def test_workspace_mismatch_precedes_append_only_policy_denial(): + decision = evaluate_workspace_access( + *_valid_inputs( + request_workspace_id="workspace-2", + role=WorkspaceRole.OWNER, + resource=WorkspaceResource.RESEARCH_RECORD, + action=WorkspaceAction.DELETE, + ) + ) + + assert decision.reason_code == "workspace_mismatch" +``` + +- [ ] **Step 4: Run the policy tests and confirm they fail against the Task 1 fallback** + +Run: + +```bash +python3 -m pytest tests/test_hosted_access_control.py -q +``` + +Expected: allow-matrix and ordered-denial assertions fail because the evaluator still returns `role_action_denied` for all structurally valid inputs. + +- [ ] **Step 5: Implement the static matrix and deterministic decision order** + +Insert above the evaluator: + +```python +_ALLOWED_ACTIONS = { + WorkspaceResource.RESEARCH_RECORD: { + WorkspaceRole.VIEWER: frozenset({WorkspaceAction.READ}), + WorkspaceRole.EDITOR: frozenset( + {WorkspaceAction.READ, WorkspaceAction.APPEND} + ), + WorkspaceRole.OWNER: frozenset( + { + WorkspaceAction.READ, + WorkspaceAction.APPEND, + WorkspaceAction.EXPORT, + } + ), + }, + WorkspaceResource.SAVED_WORKSPACE_STATE: { + WorkspaceRole.VIEWER: frozenset({WorkspaceAction.READ}), + WorkspaceRole.EDITOR: frozenset( + { + WorkspaceAction.READ, + WorkspaceAction.APPEND, + WorkspaceAction.UPDATE, + } + ), + WorkspaceRole.OWNER: frozenset( + { + WorkspaceAction.READ, + WorkspaceAction.APPEND, + WorkspaceAction.UPDATE, + WorkspaceAction.DELETE, + WorkspaceAction.EXPORT, + } + ), + }, + WorkspaceResource.WORKSPACE_MEMBERSHIP: { + WorkspaceRole.VIEWER: frozenset(), + WorkspaceRole.EDITOR: frozenset(), + WorkspaceRole.OWNER: frozenset( + {WorkspaceAction.READ, WorkspaceAction.MANAGE} + ), + }, + WorkspaceResource.WORKSPACE_AUDIT: { + WorkspaceRole.VIEWER: frozenset(), + WorkspaceRole.EDITOR: frozenset(), + WorkspaceRole.OWNER: frozenset( + {WorkspaceAction.READ, WorkspaceAction.EXPORT} + ), + }, +} +``` + +Replace the evaluator body with: + +```python + if not _structurally_valid(principal, membership, request): + return _decision(allowed=False, reason_code="invalid_request") + if principal.authenticated is not True: + return _decision( + allowed=False, + reason_code="authentication_required", + ) + if membership.principal_id != principal.principal_id: + return _decision(allowed=False, reason_code="principal_mismatch") + if membership.active is not True: + return _decision(allowed=False, reason_code="membership_inactive") + if request.workspace_id != membership.workspace_id: + return _decision(allowed=False, reason_code="workspace_mismatch") + if ( + request.resource is WorkspaceResource.RESEARCH_RECORD + and request.action in {WorkspaceAction.UPDATE, WorkspaceAction.DELETE} + ): + return _decision( + allowed=False, + reason_code="append_only_mutation_denied", + ) + allowed_actions = _ALLOWED_ACTIONS[request.resource][membership.role] + if request.action not in allowed_actions: + return _decision(allowed=False, reason_code="role_action_denied") + return _decision(allowed=True, reason_code="allowed") +``` + +- [ ] **Step 6: Run the complete focused authorization suite** + +Run: + +```bash +python3 -m pytest tests/test_hosted_access_control.py -q +``` + +Expected: all tests pass. + +- [ ] **Step 7: Add audit privacy, Unicode, determinism, and side-effect tests** + +Append: + +```python +def test_valid_unicode_scalar_identifiers_are_accepted_deterministically(): + inputs = _valid_inputs( + principal_id="研究员-😀", + membership_principal_id="研究员-😀", + membership_workspace_id="工作区-α", + request_workspace_id="工作区-α", + request_id="请求-1", + ) + + first = evaluate_workspace_access(*inputs) + second = evaluate_workspace_access(*inputs) + + assert first == second + assert first.allowed is True + + +@pytest.mark.parametrize( + "changes", + [ + {}, + {"authenticated": False}, + {"membership_principal_id": "principal-2"}, + {"active": False}, + {"request_workspace_id": "workspace-2"}, + {"action": WorkspaceAction.MANAGE}, + { + "resource": WorkspaceResource.RESEARCH_RECORD, + "action": WorkspaceAction.DELETE, + "role": WorkspaceRole.OWNER, + }, + ], +) +def test_every_decision_has_the_exact_privacy_safe_audit_obligation(changes): + decision = evaluate_workspace_access(*_valid_inputs(**changes)) + + assert decision.audit.event_type == "workspace_access_decision" + assert decision.audit.outcome == ("allowed" if decision.allowed else "denied") + assert decision.audit.reason_code == decision.reason_code + assert decision.audit.required_fields == ( + "request_id", + "principal_id", + "workspace_id", + "resource", + "action", + "outcome", + "reason_code", + "occurred_at", + ) + assert "principal-1" not in decision.reason_code + assert "workspace-1" not in decision.reason_code + + +def test_public_contract_contains_no_secret_or_research_content_fields(): + public_fields = { + *PrincipalContext.__dataclass_fields__, + *WorkspaceMembership.__dataclass_fields__, + *WorkspaceAccessRequest.__dataclass_fields__, + *AuditObligation.__dataclass_fields__, + *WorkspaceAccessDecision.__dataclass_fields__, + } + forbidden = { + "credential", + "token", + "cookie", + "api_key", + "session_secret", + "thesis", + "evidence", + "catalyst", + "research_outcome", + "forecast", + "probability", + "recommendation", + "position", + "transaction", + } + + assert public_fields.isdisjoint(forbidden) + + +def test_evaluation_does_not_mutate_inputs_or_write_files(tmp_path, monkeypatch): + monkeypatch.chdir(tmp_path) + principal, membership, request = _valid_inputs() + before = (principal, membership, request) + + evaluate_workspace_access(principal, membership, request) + + assert (principal, membership, request) == before + assert list(tmp_path.iterdir()) == [] +``` + +- [ ] **Step 8: Add an import-boundary test** + +Append: + +```python +def test_module_uses_only_the_approved_standard_library_imports(): + import ast + import inspect + + import src.hosted_access_control as module + + tree = ast.parse(inspect.getsource(module)) + imported_roots = { + alias.name.split(".", 1)[0] + for node in ast.walk(tree) + if isinstance(node, ast.Import) + for alias in node.names + } + imported_roots.update( + node.module.split(".", 1)[0] + for node in ast.walk(tree) + if isinstance(node, ast.ImportFrom) and node.module != "__future__" + ) + + assert imported_roots == {"dataclasses", "enum", "unicodedata"} +``` + +- [ ] **Step 9: Run focused authorization and existing private-beta tests** + +Run: + +```bash +python3 -m pytest \ + tests/test_hosted_access_control.py \ + tests/test_private_beta_readiness.py \ + -q +python3 -m ruff check src/hosted_access_control.py tests/test_hosted_access_control.py +git diff --check -- src/hosted_access_control.py tests/test_hosted_access_control.py +``` + +Expected: all tests and both static checks pass. Existing authentication, workspace, separation, audit, retention, entitlements, monitoring, health-check, incident, rollback, and owner-capacity states remain external or manually verified; none becomes hosted-ready. + +- [ ] **Step 10: Synchronize the exact Task 2 paths as one coherent commit** + +Include only: + +```text +src/hosted_access_control.py +tests/test_hosted_access_control.py +``` + +Commit message: + +```text +Enforce fail-closed workspace authorization +``` + +Verify the remote compare contains no dashboard, ledger, provider, readiness artifact, generated CSV/JSON/report, screenshot, or timing path. + +--- + +### Task 3: Reconcile Truthful Priority 6 Documentation And Regression Evidence + +**Files:** +- Modify: `docs/PRIVATE_BETA_ARCHITECTURE.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `tests/test_public_v1_release_docs.py` + +**Interfaces:** +- Consumes: the implemented `src.hosted_access_control.evaluate_workspace_access` contract and passing Task 2 tests. +- Produces: truthful local-evidence documentation that leaves all real hosted and operated states external. + +- [ ] **Step 1: Write the failing documentation regression test** + +Append to `tests/test_public_v1_release_docs.py`: + +```python +def test_provider_neutral_workspace_authorization_is_documented_without_hosted_claims(): + architecture = _read("docs/PRIVATE_BETA_ARCHITECTURE.md") + roadmap = _read("ROADMAP.md") + continuation = _read( + "docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md" + ) + + for document in (architecture, roadmap, continuation): + normalized = " ".join(document.split()) + assert "provider-neutral" in normalized + assert "deny-by-default" in normalized + assert "append-only" in normalized + assert "privacy-safe audit obligation" in normalized + assert "does not prove hosted authentication" in normalized + + assert "src.hosted_access_control.evaluate_workspace_access" in architecture + assert "actual hosted environment" in roadmap + assert "Do not create or change hosted accounts" in continuation +``` + +- [ ] **Step 2: Run the documentation test and confirm the missing-language failure** + +Run: + +```bash +python3 -m pytest \ + tests/test_public_v1_release_docs.py::test_provider_neutral_workspace_authorization_is_documented_without_hosted_claims \ + -q +``` + +Expected: FAIL because the three documents do not yet contain the exact local contract and non-claim language. + +- [ ] **Step 3: Add the local contract section to the private-beta architecture** + +In `docs/PRIVATE_BETA_ARCHITECTURE.md`, after `## Local Contract`, add: + +```markdown +### Provider-Neutral Authorization Policy + +`src.hosted_access_control.evaluate_workspace_access()` is a local, +provider-neutral, deny-by-default policy contract. It requires exact +authenticated-principal, active-membership, and workspace matches before an +explicit role/resource/action rule can allow a request. Thesis, evidence, +catalyst, and outcome research records remain append-only, and every allow or +deny result carries a privacy-safe audit obligation that a future approved +adapter must record. + +The evaluator performs no authentication, persistence, audit storage, +retention, monitoring, network, provider, dashboard, ledger, readiness, or +generated-artifact operation. This local contract does not prove hosted +authentication, private-workspace isolation in a deployed service, audit +storage, retention execution, monitoring, rollback, incident response, or +operated capacity. All such states remain external until directly verified in +the actual approved environment. +``` + +- [ ] **Step 4: Record the completed local slice and unchanged exit gate in ROADMAP** + +Under `### Priority 6 — Controlled hosted operating boundary`, insert after the current-lane paragraph: + +```markdown +**Implemented locally:** `src.hosted_access_control.evaluate_workspace_access` +now provides one pure, provider-neutral, deny-by-default policy decision. +Structural validation, authenticated-principal matching, active membership, +exact workspace matching, least-privilege role/resource/action rules, +append-only research-record protection, stable privacy-safe reasons, and a +privacy-safe audit obligation are independently tested. The module has no +dashboard, ledger, readiness, provider, persistence, environment, network, or +generated-artifact integration. This local contract does not prove hosted +authentication, deployed isolation, audit storage, retention, monitoring, +rollback, incident response, operated capacity, or market validation. +``` + +Retain the existing sentence: + +```markdown +**Exit gate:** the actual hosted environment directly proves every claimed control, including an observed rollback rehearsal and named owner. Local code, configuration, or a URL alone is insufficient. +``` + +- [ ] **Step 5: Reconcile the continuation contract with exact lineage and next lane** + +Under `Priority 6 — Controlled hosted operating boundary` in `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, add: + +```markdown +- The first local slice implements + `src.hosted_access_control.evaluate_workspace_access`, a pure, + provider-neutral, deny-by-default contract for authenticated-principal, + active-membership, workspace, role/resource/action, append-only + research-record, stable reason, and privacy-safe audit-obligation decisions. + It does not prove hosted authentication, deployed isolation, persistence, + audit storage, retention, monitoring, rollback, incident response, operated + capacity, or market validation. +- Reverify the exact implementation commit and exact-head CI before relying on + the local contract. Do not create or change hosted accounts, choose a + provider, use credentials, deploy, or publish without explicit approval. +- After this local policy slice passes, the next provider-neutral executable + lane is a separately reviewed retention/deletion or append-only audit-event + interface design. Provider-specific integration remains blocked until the + exact identity, storage, logging, host, and operating environment are + explicitly approved. +``` + +During the synchronization step, replace “the exact implementation commit” with the actual remote SHA rather than a guessed or local stale SHA. + +- [ ] **Step 6: Run the focused documentation and private-beta regression suite** + +Run: + +```bash +python3 -m pytest \ + tests/test_public_v1_release_docs.py \ + tests/test_private_beta_readiness.py \ + tests/test_hosted_access_control.py \ + -q +git diff --check -- \ + docs/PRIVATE_BETA_ARCHITECTURE.md \ + ROADMAP.md \ + docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md \ + tests/test_public_v1_release_docs.py +``` + +Expected: all tests pass and whitespace is clean. + +- [ ] **Step 7: Synchronize the exact Task 3 documentation commit** + +Include only: + +```text +docs/PRIVATE_BETA_ARCHITECTURE.md +ROADMAP.md +docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md +tests/test_public_v1_release_docs.py +``` + +Commit message: + +```text +Document workspace authorization boundary +``` + +Verify the remote compare contains exactly those four files and zero generated artifacts. + +--- + +### Task 4: Run Full Release Verification And Update Draft PR Evidence + +**Files:** +- Verify only: all six Task 1–3 intentional paths plus this reviewed plan, + for seven paths total. +- Update externally: draft PR #113 description. +- Do not modify: generated CSV/JSON/report/sample-report/screenshot/timing artifacts. + +**Interfaces:** +- Consumes: the synchronized authorization module, tests, and documentation. +- Produces: current local and exact-head CI evidence for the isolated Priority 6 slice. + +- [ ] **Step 1: Run the focused authorization and documentation suites** + +Run: + +```bash +python3 -m pytest \ + tests/test_hosted_access_control.py \ + tests/test_private_beta_readiness.py \ + tests/test_public_v1_release_docs.py \ + -q +``` + +Expected: all focused tests pass. + +- [ ] **Step 2: Run the complete automated test suite** + +Run: + +```bash +python3 -m pytest tests -q +``` + +Expected: the complete suite passes, except only an already classified environment-limited skip or existing dependency warning may remain. + +- [ ] **Step 3: Run the required product and release gates** + +Run each command independently: + +```bash +make dashboard-smoke +make research-dashboard-render +make commercial-beta-check +make commercial-beta-performance-check +make public-wording-check +make browser-qa-evidence-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: + +- dashboard, render, commercial-beta, performance, public wording, browser evidence, diff hygiene, and whitespace gates pass; +- pilot readiness may remain truthfully blocked because saved readiness is stale; +- no command writes or promotes readiness artifacts. + +- [ ] **Step 4: Confirm generated-artifact exclusion** + +Run: + +```bash +git diff -- \ + data/analyst_estimates_readiness.csv \ + data/dcf_readiness.csv \ + data/earnings_readiness.csv \ + data/price_coverage_report.csv \ + data/reports/analyst_estimates_readiness_report.csv \ + data/reports/data_source_status.csv \ + data/reports/dcf_readiness_report.csv \ + data/reports/earnings_readiness_report.csv \ + data/reports/feature_readiness_summary.csv \ + data/reports/fundamentals_coverage_report.csv \ + data/reports/peer_readiness_report.csv \ + data/reports/peer_unlock_worklist.csv \ + data/reports/price_coverage_report.csv \ + data/reports/ticker_readiness_report.csv \ + data/reports/universe_coverage_report.csv \ + data/universe_master.csv \ + outputs/feature_readiness_summary.csv \ + outputs/peer_unlock_worklist.csv | shasum -a 256 +``` + +Expected: + +```text +a2c2f428b489dbb291dd54fd8a6e1e7f4ad9481414320ca248c54be89f4062b9 - +``` + +- [ ] **Step 5: Verify the final remote compare and draft PR state** + +Confirm: + +```text +branch: codex/personal-research-mode-mvp +PR: #113 +state: open +draft: true +merged: false +generated paths in this slice: 0 +``` + +The authoritative reviewed slice from +`3c5a2bc8fc47ac144290087e9dad513bb683252c` must contain only these seven +paths: + +```text +src/hosted_access_control.py +tests/test_hosted_access_control.py +docs/PRIVATE_BETA_ARCHITECTURE.md +ROADMAP.md +docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md +tests/test_public_v1_release_docs.py +docs/superpowers/plans/2026-07-26-provider-neutral-workspace-authorization.md +``` + +- [ ] **Step 6: Require exact-head GitHub Actions success** + +Wait for the current PR head’s `Commercial Research Beta` workflow to complete. Require successful: + +```text +full test suite +dashboard startup +Personal Research route render +public wording +generated-artifact hygiene +whitespace +``` + +Do not reuse an earlier commit’s CI result. + +- [ ] **Step 7: Update draft PR #113 with truthful evidence** + +Record: + +- the exact implementation and documentation commit SHAs; +- focused and full test counts; +- each release-gate result; +- unchanged generated-artifact fingerprint; +- the provider-neutral, deny-by-default, cross-principal, cross-workspace, append-only, and audit-obligation behavior; +- that authentication, deployed isolation, audit storage, retention, monitoring, rollback, incident response, operated capacity, external validation, and market maturity remain unproven; +- the exact next safe executable lane. + +Keep the PR open and draft. Do not merge or deploy. + +- [ ] **Step 8: Hand off the verified slice** + +Report: + +- authoritative remote branch and PR status; +- product stage; +- files and behavior added; +- focused/full/release/CI evidence; +- generated artifacts excluded; +- external dependencies and exact unblock conditions; +- remaining maturity gates; +- exact next executable step; +- whether the branch is safe for review; +- whether the overall goal remains active. diff --git a/docs/superpowers/plans/2026-07-27-accessibility-narrow-remediation.md b/docs/superpowers/plans/2026-07-27-accessibility-narrow-remediation.md new file mode 100644 index 000000000..601867e4f --- /dev/null +++ b/docs/superpowers/plans/2026-07-27-accessibility-narrow-remediation.md @@ -0,0 +1,287 @@ +# Accessibility Narrow Remediation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Repair the five reproduced keyboard, narrow-navigation, link-name, disclosure-focus, and authoring-error defects without changing research behavior. + +**Architecture:** Pure HTML/CSS helpers provide deterministic navigation and link labels. A fixed local accessibility bridge may apply field ARIA attributes and focus only after rejected user-triggered validation; it never reads field values, invokes actions, transmits data, or changes draft and ledger state. + +**Tech Stack:** Python 3, Streamlit, fixed same-origin JavaScript component, HTML/CSS, pytest, Streamlit AppTest, Playwright browser verification. + +## Global Constraints + +- This slice does not claim WCAG conformance or screen-reader success. +- The skip link is the first application-owned focus target and preserves the full current route. +- Narrow navigation contains exactly Research Desk, Discover, Company Workbench when a ticker exists, and Monitor; it never exposes Operator routes. +- Discover action names include the exact ticker without changing ordering or destinations. +- Field-error binding is presentation-only and cannot change draft digest, preview receipt, confirmation, append engine, or ledger. +- The bridge can set accessibility attributes and focus only; it cannot read or transmit field values or invoke application actions. +- No readiness, evidence, forecast, ledger, generated-data, or route-boundary change. +- Generated CSV, JSON, report, sample-report, screenshot, timing, canonical-data, and manual-review churn remain unstaged. + +--- + +### Task 1: Deterministic Workflow Navigation, Link Names, and Focus CSS + +**Files:** +- Modify: `src/research_workspace.py` +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_workspace.py` +- Modify: `tests/test_dashboard_helpers.py` + +**Interfaces:** +- Produces: `research_workflow_navigation_html(*, active_page: str, ticker: str = "") -> str`. +- Produces: `discover_review_action_label(ticker: str) -> str`. +- Preserves: current route query format and readiness ordering. + +- [ ] **Step 1: Write failing HTML and CSS contract tests** + +```python +def test_mobile_workflow_navigation_is_labelled_and_has_one_current_page(): + rendered = research_workflow_navigation_html(active_page="discover", ticker="AVGO") + assert "aria-label='Personal research workflow'" in rendered + assert rendered.count("aria-current='page'") == 1 + assert all(label in rendered for label in ("Research Desk", "Discover", "Company Workbench", "Monitor")) + assert "ticker=AVGO" in rendered + + +def test_discover_action_label_is_ticker_specific(): + assert discover_review_action_label("avgo") == "Open AVGO review" +``` + +Add a source-contract assertion that shared theme CSS contains `summary:focus-visible`. + +- [ ] **Step 2: Run the focused tests and confirm failure** + +Run: `python3 -m pytest tests/test_research_workspace.py tests/test_dashboard_helpers.py -q` + +Expected: FAIL because the new helpers and summary focus rule do not exist. + +- [ ] **Step 3: Implement escaped route HTML and the shared focus rule** + +Use the existing research query parameters. Omit Company Workbench when `ticker.strip()` is empty. Apply `aria-current="page"` only to the active link. Render visible action text as `Open {TICKER} review` at both Discover action locations. Extend the existing visible focus selector group with: + +```css +summary:focus-visible { + outline: 3px solid #0f766e; + outline-offset: 3px; +} +``` + +- [ ] **Step 4: Render the navigation on all four Personal Research routes** + +Place the navigation directly after the skip link and before the route answer. Add narrow-width styling that keeps it visible, wraps links, preserves 44px interactive height, and produces no horizontal overflow at `390x844`. + +- [ ] **Step 5: Run focused tests** + +Run: `python3 -m pytest tests/test_research_workspace.py tests/test_dashboard_helpers.py tests/test_research_mode_dashboard_contract.py -q` + +Expected: PASS. + +- [ ] **Step 6: Commit the deterministic UI repairs** + +```bash +git add src/research_workspace.py src/dashboard.py tests/test_research_workspace.py tests/test_dashboard_helpers.py tests/test_research_mode_dashboard_contract.py +git commit -m "Improve research workflow accessibility" +``` + +### Task 2: First Application-Owned Skip Focus + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_dashboard_render_smoke.py` + +**Interfaces:** +- Consumes: existing `public_workflow_skip_link_html("#public-page-answer")`. +- Preserves: `mode`, `page`, `ticker`, and `open` through the fragment-only target. + +- [ ] **Step 1: Add a failing render-order contract test** + +```python +def test_skip_link_renders_before_application_sidebar_widgets(): + source = Path("src/dashboard.py").read_text(encoding="utf-8") + skip_call = source.index("render_public_workflow_skip_link(") + sidebar_call = source.index("render_sidebar(") + assert skip_call < sidebar_call +``` + +Use the actual sidebar entrypoint name found in `main()` when writing the final test; assert the exact call order, not a broad string count. + +- [ ] **Step 2: Run the test and confirm it fails against current order** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py -q` + +Expected: FAIL with the sidebar call before the skip-link call. + +- [ ] **Step 3: Move the skip link before application-owned sidebar rendering** + +Keep exactly one skip link. Do not move route content, change query parameters, or create a second destination. Ensure hidden project-controlled controls use `display: none` or are not focusable rather than merely visually hidden. + +- [ ] **Step 4: Run focused contract and render tests** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q` + +Expected: PASS. + +- [ ] **Step 5: Commit the focus-entry repair** + +```bash +git add src/dashboard.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py +git commit -m "Place skip link before application controls" +``` + +### Task 3: Field-Level Authoring Error Association + +**Files:** +- Create: `src/accessibility_bridge.py` +- Create: `tests/test_accessibility_bridge.py` +- Modify: `src/research_record_authoring_ui.py` +- Modify: `tests/test_research_record_authoring_ui.py` + +**Interfaces:** +- Produces: `AuthoringFieldError(field_name: str, field_label: str, message: str, error_id: str)`. +- Produces: `authoring_field_error(reason: str, *, profile_key: str, ticker: str, kind: str) -> AuthoringFieldError | None`. +- Produces: `render_authoring_error_binding(component_html: Callable[..., Any], error: AuthoringFieldError) -> None`. +- Bridge input contains field label and fixed generated error ID only; it contains no field value. + +- [ ] **Step 1: Write failing deterministic-mapping and security tests** + +```python +def test_required_thesis_id_maps_to_stable_field_error(): + error = authoring_field_error( + "thesis_id is required", profile_key="personal", ticker="AVGO", kind="thesis" + ) + assert error is not None + assert error.field_name == "thesis_id" + assert error.field_label == "Thesis Id" + assert error.error_id == "research-authoring-personal-avgo-thesis-thesis-id-error" + + +def test_accessibility_bridge_has_no_network_or_value_reading(): + source = Path("src/accessibility_bridge.py").read_text(encoding="utf-8").lower() + for forbidden in ("fetch(", "xmlhttprequest", "websocket", "localstorage", "sessionstorage", ".value"): + assert forbidden not in source +``` + +- [ ] **Step 2: Run focused tests and confirm failure** + +Run: `python3 -m pytest tests/test_accessibility_bridge.py tests/test_research_record_authoring_ui.py -q` + +Expected: FAIL because the mapping and bridge do not exist. + +- [ ] **Step 3: Implement exact rejected-reason mapping** + +Parse only the existing validation form `" is required"` and accept the result only when `field_name` is in `authoring_field_contract(kind)`. Unknown reasons return `None`; do not guess. Build the stable ID from normalized profile, ticker, kind, and field name. + +- [ ] **Step 4: Implement the bounded presentation bridge** + +Render one bounded local component script through `streamlit.components.v1.html`. It finds exactly one Streamlit widget by its accessible label, sets `aria-invalid="true"` and `aria-describedby=error.error_id`, focuses it after the validation button attempt, and creates one stable text error immediately beside that widget inside the authoring expander. Escape configuration through `json.dumps`; pass only field label, fixed error ID, and deterministic validation message. Retain `st_api.error(preview.reason)` for alert announcement. If zero or multiple matching fields exist, change no field. + +- [ ] **Step 5: Prove preview and ledger semantics remain unchanged** + +Extend AppTest coverage to submit an empty thesis, assert one field-bound error and one global alert, and assert the journal, catalyst, and outcome ledger bytes are unchanged. Retain all existing preview-digest and confirm-before-save tests. + +- [ ] **Step 6: Run focused tests** + +Run: `python3 -m pytest tests/test_accessibility_bridge.py tests/test_research_record_authoring_ui.py -q` + +Expected: PASS. + +- [ ] **Step 7: Commit authoring accessibility** + +```bash +git add src/accessibility_bridge.py src/research_record_authoring_ui.py tests/test_accessibility_bridge.py tests/test_research_record_authoring_ui.py +git commit -m "Associate authoring errors with fields" +``` + +### Task 4: Direct Browser Retest and Durable Evidence + +**Files:** +- Create: `src/research_accessibility_browser_gate.py` +- Create: `tests/test_research_accessibility_browser_gate.py` +- Modify: `Makefile` +- Modify: `docs/ACCESSIBILITY_EVIDENCE.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: running local Streamlit dashboard and the five repaired contracts. +- Produces: `run_research_accessibility_browser_gate(base_dir: Path | str, *, base_url: str = "", chrome_executable: Path | None = None) -> dict[str, object]`. +- Produces: `python3 -m src.research_accessibility_browser_gate` and `make research-accessibility-browser-check`. +- Produces: reproducible engineering evidence only, explicitly not a conformance claim. + +- [ ] **Step 1: Write failing gate-contract tests** + +```python +def test_accessibility_browser_gate_covers_both_viewports_and_research_routes(): + assert VIEWPORTS == ((1280, 720), (390, 844)) + assert [route.name for route in RESEARCH_ROUTES[:4]] == [ + "Research Desk", "Discover", "Company Workbench", "Monitor" + ] + + +def test_makefile_exposes_non_writing_browser_gate(): + makefile = Path("Makefile").read_text(encoding="utf-8") + assert "research-accessibility-browser-check:" in makefile + assert "python3 -m src.research_accessibility_browser_gate" in makefile +``` + +- [ ] **Step 2: Run the new contract tests and confirm failure** + +Run: `python3 -m pytest tests/test_research_accessibility_browser_gate.py -q` + +Expected: FAIL because the gate module and Make target do not exist. + +- [ ] **Step 3: Implement a read-only local browser harness** + +Reuse the Chrome discovery and local Streamlit server lifecycle patterns from `src/public_performance_gate.py`, but record no timing or report file. Return a structured in-memory payload with one result per route, viewport, and assertion. Exit nonzero when any required assertion fails or Chrome/Playwright is unavailable. + +- [ ] **Step 4: Add browser assertions for the five defects** + +At desktop and `390x844`, assert: the skip link precedes application controls in Tab order; activation preserves the URL and focuses `#public-page-answer`; labelled mobile navigation is visible; ten Discover actions have unique ticker-specific names; `summary` shows a non-none focus outline; empty thesis preview binds one error to Thesis Id; document width does not exceed viewport; no traceback appears. + +- [ ] **Step 5: Run unit and direct browser verification** + +Run: + +```bash +python3 -m pytest tests/test_research_accessibility_browser_gate.py -q +make research-accessibility-browser-check +``` + +Expected: PASS on all five direct retests at both viewports. + +- [ ] **Step 6: Update evidence and roadmap truth** + +Record the commands, viewports, exact five results, current commit, and explicit limitation: zoom, forced colors, reduced motion, screen-reader navigation, and independent human testing remain incomplete. + +- [ ] **Step 7: Run full release and hygiene gates** + +Run: + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all pass without new generated artifact churn. + +- [ ] **Step 8: Stage exact files, check hygiene, commit, push, and require exact-head CI** + +```bash +git add src/dashboard.py src/research_workspace.py src/accessibility_bridge.py src/research_record_authoring_ui.py src/research_accessibility_browser_gate.py tests/test_dashboard_helpers.py tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_accessibility_bridge.py tests/test_research_record_authoring_ui.py tests/test_research_accessibility_browser_gate.py Makefile docs/ACCESSIBILITY_EVIDENCE.md ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md +make staged-hygiene-check +git commit -m "Verify narrow accessibility remediations" +git push origin codex/personal-research-mode-mvp +gh pr checks 113 --watch +``` + +Expected: PR #113 remains open and draft; exact-head CI passes; generated working-data churn remains unstaged. diff --git a/docs/superpowers/plans/2026-07-27-framework-safe-semantic-main.md b/docs/superpowers/plans/2026-07-27-framework-safe-semantic-main.md new file mode 100644 index 000000000..6c3c00f48 --- /dev/null +++ b/docs/superpowers/plans/2026-07-27-framework-safe-semantic-main.md @@ -0,0 +1,250 @@ +# Framework-Safe Semantic Main Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make the actual Streamlit primary content container the single stable `main` landmark on every Personal Research route. + +**Architecture:** Extend the fixed local accessibility bridge with an idempotent same-origin script targeting exactly one `[data-testid="stMain"]`. The script sets metadata on the existing container, replaces its own prior observer on rerender, and never creates, moves, reads, transmits, or persists research content. + +**Tech Stack:** Python 3, Streamlit components, fixed JavaScript, MutationObserver, pytest, Streamlit AppTest, Playwright browser verification. + +## Global Constraints + +- Target exactly one `[data-testid="stMain"]`; never attach the landmark to a fallback node. +- Set `role="main"`, `id="research-main"`, and `aria-label="Stock research workspace"` on the existing primary container. +- Reuse an existing native or role-based main and never create a second landmark. +- The helper is called exactly once per dashboard run after theme initialization and before page content. +- The bridge uses no user-provided interpolation, network, telemetry, cookies, storage, clipboard, form values, callbacks, navigation, or application actions. +- Absence or ambiguity produces no false landmark and no user-visible traceback; browser verification must fail until exactly one target resolves. +- No readiness, evidence, authoring, routing, generated-data, or research-result change. +- Automated DOM evidence is not screen-reader usability or WCAG conformance. + +--- + +### Task 1: Fixed Idempotent Landmark Bridge + +**Files:** +- Modify: `src/accessibility_bridge.py` +- Modify: `tests/test_accessibility_bridge.py` + +**Interfaces:** +- Produces: `SEMANTIC_MAIN_BRIDGE_HTML: str`. +- Produces: `render_semantic_main_bridge(component_html: Callable[..., Any] = streamlit.components.v1.html) -> None`. +- JavaScript status attribute: `data-research-main-bridge-status`, with exact values `applied`, `missing`, or `ambiguous`. +- Reuses: the fixed local component-rendering pattern introduced by the narrow accessibility slice. + +- [ ] **Step 1: Write failing source-contract tests** + +```python +def test_semantic_main_bridge_is_fixed_idempotent_and_non_networked(): + source = SEMANTIC_MAIN_BRIDGE_HTML.lower() + assert '[data-testid="stmain"]' in source + assert 'role' in source and 'main' in source + assert 'research-main' in source + assert 'stock research workspace' in source + assert 'mutationobserver' in source + assert 'disconnect()' in source + for forbidden in ("fetch(", "xmlhttprequest", "websocket", "localstorage", "sessionstorage", "clipboard", ".value"): + assert forbidden not in source +``` + +- [ ] **Step 2: Run the focused test and confirm failure** + +Run: `python3 -m pytest tests/test_accessibility_bridge.py -q` + +Expected: FAIL because the semantic-main bridge contract is absent. + +- [ ] **Step 3: Implement exact-target application and observer replacement** + +The fixed script must: + +```javascript +const host = window.parent.document; +const observerKey = "__stockResearchMainObserver"; +const targetKey = "__stockResearchMainTarget"; +if (window.parent[observerKey]) { + window.parent[observerKey].disconnect(); +} +function applyMainLandmark() { + const nodes = host.querySelectorAll('[data-testid="stMain"]'); + const status = nodes.length === 1 ? "applied" : (nodes.length === 0 ? "missing" : "ambiguous"); + host.documentElement.setAttribute("data-research-main-bridge-status", status); + const previous = window.parent[targetKey]; + if (previous && (nodes.length !== 1 || previous !== nodes[0])) { + if (previous.getAttribute("data-research-main-bridge-owned") === "true") { + previous.removeAttribute("role"); + previous.removeAttribute("id"); + previous.removeAttribute("aria-label"); + previous.removeAttribute("data-research-main-bridge-owned"); + } + window.parent[targetKey] = null; + } + if (nodes.length !== 1) return; + const target = nodes[0]; + if (target.tagName.toLowerCase() !== "main" && target.getAttribute("role") !== "main") { + target.setAttribute("data-research-main-bridge-owned", "true"); + } + target.setAttribute("role", "main"); + target.setAttribute("id", "research-main"); + target.setAttribute("aria-label", "Stock research workspace"); + window.parent[targetKey] = target; +} +applyMainLandmark(); +window.parent[observerKey] = new MutationObserver(applyMainLandmark); +window.parent[observerKey].observe(host.body, {childList: true, subtree: true}); +``` + +Before setting attributes, remove no native semantics and create no element. On a missing or ambiguous rerender, remove only attributes previously owned by this bridge; never remove attributes from a native or pre-existing main. The Python helper renders only this constant and accepts no research-content arguments. + +- [ ] **Step 4: Add missing and ambiguous behavior tests** + +Use a small JavaScript-capable DOM test if already available; otherwise source-contract tests must prove `nodes.length !== 1` returns before any target mutation and that all three deterministic statuses exist. + +- [ ] **Step 5: Run focused tests** + +Run: `python3 -m pytest tests/test_accessibility_bridge.py -q` + +Expected: PASS. + +- [ ] **Step 6: Commit the bridge** + +```bash +git add src/accessibility_bridge.py tests/test_accessibility_bridge.py +git commit -m "Add idempotent semantic main bridge" +``` + +### Task 2: Exactly-Once Dashboard Integration + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_dashboard_render_smoke.py` + +**Interfaces:** +- Consumes: `render_semantic_main_bridge(st_api: Any) -> None`. +- Produces: exactly one call in the main dashboard execution path after `apply_dashboard_theme()` and before sidebar or route content. + +- [ ] **Step 1: Write a failing call-order and call-count test** + +```python +def test_semantic_main_bridge_runs_once_after_theme_before_route_content(): + source = Path("src/dashboard.py").read_text(encoding="utf-8") + assert source.count("render_semantic_main_bridge()") == 1 + theme_index = source.index("apply_dashboard_theme()") + bridge_index = source.index("render_semantic_main_bridge()") + route_index = source.index("render_research_desk(") + assert theme_index < bridge_index < route_index +``` + +Anchor the final route-order assertion to the actual route dispatch block so function definitions earlier in the file cannot create a false result. + +- [ ] **Step 2: Run focused tests and confirm failure** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q` + +Expected: FAIL because the bridge is not called. + +- [ ] **Step 3: Import and call the bridge once** + +Add one top-level import and one execution-path call. Do not call it inside individual route renderers. Preserve the existing `#public-page-answer` target and all sidebar, routing, authoring, and readiness behavior. + +- [ ] **Step 4: Run focused contract and smoke tests** + +Run: `python3 -m pytest tests/test_accessibility_bridge.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q` + +Expected: PASS. + +- [ ] **Step 5: Commit the integration** + +```bash +git add src/dashboard.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py +git commit -m "Apply semantic main landmark once" +``` + +### Task 3: Six-Route Desktop and Phone Verification + +**Files:** +- Modify: `src/research_accessibility_browser_gate.py` +- Modify: `tests/test_research_accessibility_browser_gate.py` +- Modify: `docs/ACCESSIBILITY_EVIDENCE.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: Research Desk, Discover, ticker-bound Company Workbench, Monitor, Data Health, and Proof History. +- Extends: `run_research_accessibility_browser_gate(...)` with the landmark assertions; it remains in-memory and non-writing. +- Produces: direct current DOM evidence at desktop and `390x844`. + +- [ ] **Step 1: Add browser assertions for the complete landmark contract** + +For each of the six routes and both viewports, assert: + +```javascript +const mains = page.getByRole("main"); +await expect(mains).toHaveCount(1); +await expect(mains).toHaveAttribute("id", "research-main"); +await expect(mains).toHaveAttribute("aria-label", "Stock research workspace"); +await expect(mains.locator("#public-page-answer")).toHaveCount(1); +await expect(mains.getByRole("heading", {level: 1})).toHaveCount(1); +``` + +Navigate away and back once to force rerender, then repeat the count. Assert `data-research-main-bridge-status="applied"`, no console error, and no traceback. Activate the skip link and assert the focused target is inside the main landmark. + +- [ ] **Step 2: Extend the route contract unit test** + +```python +def test_accessibility_browser_gate_covers_all_six_personal_research_routes(): + assert [route.name for route in RESEARCH_ROUTES] == [ + "Research Desk", + "Discover", + "Company Workbench", + "Monitor", + "Data Health", + "Proof History", + ] +``` + +- [ ] **Step 3: Run the six-route browser verification** + +Run: + +```bash +python3 -m pytest tests/test_research_accessibility_browser_gate.py -q +make research-accessibility-browser-check +``` + +Expected: exactly one main landmark on all twelve route/viewport combinations, including after rerender. + +- [ ] **Step 4: Update evidence and roadmap boundaries** + +Record current commit, routes, viewports, commands, and outcomes. State explicitly that automated DOM verification does not prove screen-reader landmark navigation, WCAG conformance, hosted behavior, or independent-human accessibility validation. + +- [ ] **Step 5: Run full release and hygiene gates** + +Run: + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all pass without new generated artifact churn. + +- [ ] **Step 6: Stage exact files, run staged hygiene, commit, push, and require exact-head CI** + +```bash +git add src/accessibility_bridge.py src/dashboard.py src/research_accessibility_browser_gate.py tests/test_accessibility_bridge.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py tests/test_research_accessibility_browser_gate.py docs/ACCESSIBILITY_EVIDENCE.md ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md +make staged-hygiene-check +git commit -m "Verify semantic main landmark" +git push origin codex/personal-research-mode-mvp +gh pr checks 113 --watch +``` + +Expected: PR #113 remains open and draft; exact-head CI passes; generated working-data churn remains unstaged. diff --git a/docs/superpowers/plans/2026-07-27-observation-recency.md b/docs/superpowers/plans/2026-07-27-observation-recency.md new file mode 100644 index 000000000..17facb0ca --- /dev/null +++ b/docs/superpowers/plans/2026-07-27-observation-recency.md @@ -0,0 +1,404 @@ +# Observation Recency Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add an independent, read-only market-observation recency contract that keeps saved readiness separate from the selected ticker, profile price lane, SPY, and QQQ dates. + +**Architecture:** A focused `src/observation_recency.py` module parses caller-supplied CSV rows or one explicitly selected `prices.csv` without fallback or writes. Dashboard presentation consumes immutable results and shows exact dates in the primary answer while keeping policy, path, and excluded-row diagnostics under Advanced evidence. + +**Tech Stack:** Python 3, standard-library `csv`, frozen dataclasses, Streamlit HTML helpers, pytest, Streamlit AppTest. + +## Global Constraints + +- Seven calendar days is the exact local threshold: `current` when `age_days <= 7`, otherwise `stale_review_only`. +- Valid observations must be on or before the explicit review date; malformed and future dates are excluded and counted. +- Missing files, unreadable files, and absent tickers fail closed as `unavailable`. +- `ProfileContext.freshness_state` remains saved-readiness state and must not be derived from observation recency. +- No provider fetch, refresh, import, readiness rebuild, canonical write, generated artifact, or ledger mutation. +- No forecast, probability, ranking, expected-return score, recommendation, allocation, or transaction direction. +- Primary copy shows exact dates and state; file path, threshold, and excluded counts stay under Advanced evidence. +- Generated CSV, JSON, report, sample-report, screenshot, timing, canonical-data, and manual-review churn remain unstaged. + +--- + +### Task 1: Pure Observation-Recency Evaluator + +**Files:** +- Create: `src/observation_recency.py` +- Create: `tests/test_observation_recency.py` + +**Interfaces:** +- Consumes: iterable CSV-shaped mappings with `ticker` and `date`; explicit `as_of: datetime.date`. +- Produces: `ObservationRecency(scope: str, through_date: str, age_days: int | None, state: str, message: str, excluded_date_count: int)`. +- Produces: `ObservationRecencySet(selected_ticker: ObservationRecency, profile_price_lane: ObservationRecency, benchmarks: tuple[ObservationRecency, ...], policy_days: int, source_path: str, as_of: str)`. +- Produces: `evaluate_observation_rows(rows: Iterable[Mapping[str, object]], *, selected_ticker: str, benchmark_tickers: tuple[str, ...] = ("SPY", "QQQ"), as_of: date, source_path: str = "") -> ObservationRecencySet`. +- Produces: `load_observation_recency(prices_path: Path, *, selected_ticker: str, benchmark_tickers: tuple[str, ...] = ("SPY", "QQQ"), as_of: date) -> ObservationRecencySet`. + +- [ ] **Step 1: Write evaluator tests that fail before the module exists** + +```python +from datetime import date + +from src.observation_recency import evaluate_observation_rows + + +def test_observation_recency_keeps_scopes_independent_and_excludes_bad_dates(): + result = evaluate_observation_rows( + [ + {"ticker": "AVGO", "date": "2026-07-20"}, + {"ticker": "AVGO", "date": "2026-08-01"}, + {"ticker": "SPY", "date": "2026-07-19"}, + {"ticker": "QQQ", "date": "not-a-date"}, + ], + selected_ticker="AVGO", + as_of=date(2026, 7, 27), + ) + assert (result.selected_ticker.state, result.selected_ticker.age_days) == ("current", 7) + assert result.selected_ticker.excluded_date_count == 1 + assert result.benchmarks[0].state == "stale_review_only" + assert result.benchmarks[1].state == "unavailable" + assert result.profile_price_lane.through_date == "2026-07-20" +``` + +- [ ] **Step 2: Run the new focused test and confirm the expected import failure** + +Run: `python3 -m pytest tests/test_observation_recency.py -q` + +Expected: FAIL because `src.observation_recency` does not exist. + +- [ ] **Step 3: Implement frozen result types, strict date parsing, and pure scope evaluation** + +```python +CURRENT_MAX_AGE_DAYS = 7 + + +@dataclass(frozen=True) +class ObservationRecency: + scope: str + through_date: str + age_days: int | None + state: str + message: str + excluded_date_count: int = 0 + + +def _result(scope: str, dates: list[date], excluded: int, as_of: date) -> ObservationRecency: + if not dates: + return ObservationRecency( + scope, "", None, "unavailable", + "No valid observation is available on or before the review date.", excluded + ) + latest = max(dates) + age_days = (as_of - latest).days + state = "current" if age_days <= CURRENT_MAX_AGE_DAYS else "stale_review_only" + message = ( + "Observation is within the seven-calendar-day local review policy." + if state == "current" + else "Historical context only; do not use for a current-market interpretation." + ) + return ObservationRecency(scope, latest.isoformat(), age_days, state, message, excluded) +``` + +Complete `evaluate_observation_rows` by normalizing tickers to uppercase, evaluating the selected ticker, the entire profile lane, and each exact benchmark separately. Complete `load_observation_recency` using `csv.DictReader`; catch `OSError`, `UnicodeError`, and `csv.Error` and return unavailable results instead of raising. + +- [ ] **Step 4: Add boundary, file-failure, missing-scope, and current-fixture tests** + +```python +def test_existing_profile_prices_are_stale_or_unavailable_without_writes(): + project_root = Path(__file__).resolve().parents[1] + result = load_observation_recency( + project_root / "data" / "prices.csv", + selected_ticker="AVGO", + as_of=date(2026, 7, 27), + ) + assert result.selected_ticker.state in {"stale_review_only", "unavailable"} + assert {row.scope: row.state for row in result.benchmarks}["SPY"] == "stale_review_only" + assert {row.scope: row.state for row in result.benchmarks}["QQQ"] == "stale_review_only" +``` + +Also assert an exactly seven-day-old value is current, an eight-day-old value is stale, a missing file returns all scopes unavailable, and one missing benchmark does not alter the selected ticker. + +- [ ] **Step 5: Run focused tests** + +Run: `python3 -m pytest tests/test_observation_recency.py -q` + +Expected: PASS. + +- [ ] **Step 6: Commit the evaluator** + +```bash +git add src/observation_recency.py tests/test_observation_recency.py +git commit -m "Add independent observation recency evaluator" +``` + +### Task 2: Truthful Dashboard Presentation + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `src/research_workspace.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_research_workspace.py` + +**Interfaces:** +- Consumes: `ObservationRecencySet` from Task 1. +- Produces: `observation_recency_strip_html(result: ObservationRecencySet, *, include_selected: bool) -> str`. +- Produces: `observation_recency_advanced_details(result: ObservationRecencySet) -> dict[str, object]`. +- Preserves: existing `research_workspace_header_html(...)` call compatibility and `ProfileContext.freshness_state`. + +- [ ] **Step 1: Write failing helper tests for exact labels and withheld wording** + +```python +def test_profile_strip_labels_saved_readiness_not_generic_freshness(profile_context): + rendered = profile_trust_strip_html(profile_context) + assert "Saved readiness" in rendered + assert "Freshness" not in rendered + + +def test_stale_observation_strip_exposes_date_without_current_market_claim(stale_recency): + rendered = observation_recency_strip_html(stale_recency, include_selected=True) + assert "2026-05-22" in rendered + assert "Historical context only" in rendered + assert "current-market" not in rendered.lower().replace("no current-market", "") +``` + +- [ ] **Step 2: Run the helper tests and confirm they fail** + +Run: `python3 -m pytest tests/test_dashboard_helpers.py tests/test_research_workspace.py -q` + +Expected: FAIL on the old `Freshness` label and missing observation helpers. + +- [ ] **Step 3: Implement one shared HTML renderer and Advanced diagnostics** + +Render the profile lane on all four Personal Research routes. On Company Workbench, also render the selected ticker, SPY, and QQQ. Each item must contain scope, exact `through_date` or `Unavailable`, state, and the approved fail-closed message. Escape every value with `html.escape`. + +Advanced details must use these exact keys: + +```python +{ + "Policy threshold": "7 calendar days", + "Source path": result.source_path or "Unavailable", + "Review date": result.as_of, + "Excluded dates": { + row.scope: row.excluded_date_count + for row in (result.selected_ticker, result.profile_price_lane, *result.benchmarks) + }, +} +``` + +Relabel profile trust-strip copy to `Saved readiness`, its `aria-label` to `Selected data profile and saved readiness`, and Advanced `Freshness detail` to `Saved readiness detail`. + +- [ ] **Step 4: Run the focused helper tests** + +Run: `python3 -m pytest tests/test_dashboard_helpers.py tests/test_research_workspace.py -q` + +Expected: PASS. + +- [ ] **Step 5: Commit the presentation helpers** + +```bash +git add src/dashboard.py src/research_workspace.py tests/test_dashboard_helpers.py tests/test_research_workspace.py +git commit -m "Show truthful market observation recency" +``` + +### Task 3: Four-Route Integration and Evidence Boundaries + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_dashboard_render_smoke.py` +- Modify: `ROADMAP.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: `load_observation_recency(context.data_dir / "prices.csv", selected_ticker=ticker, as_of=review_date)`. +- Produces: one recency evaluation per dashboard run, passed read-only into Research Desk, Discover, Company Workbench, and Monitor renderers. + +- [ ] **Step 1: Add failing route and no-write contract tests** + +```python +def test_personal_research_routes_do_not_render_ambiguous_freshness_label(): + source = Path("src/dashboard.py").read_text(encoding="utf-8") + assert "Freshness" not in source + assert "Saved readiness" in source + assert "load_observation_recency" in source +``` + +Extend render-smoke fixtures so Company Workbench includes AVGO, SPY, and QQQ independently and every route contains the profile-lane observation state. + +- [ ] **Step 2: Run the focused contract tests and confirm failure** + +Run: `python3 -m pytest tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q` + +Expected: FAIL until all four routes receive the new result. + +- [ ] **Step 3: Integrate the evaluator once per dashboard run** + +Use one explicit review date derived from the existing dashboard clock, never file modification time. Do not add a fallback data profile. Pass the result through render function parameters rather than recomputing it inside each route. Put policy, source path, and excluded counts in the existing Advanced evidence expander. + +- [ ] **Step 4: Update durable truth documentation** + +Record that observation recency is locally implemented and independently fail-closed, while permitted market-data source rights and hosted freshness remain external gates. Record the exact seven-calendar-day policy and state that it is not an exchange-session SLA. + +- [ ] **Step 5: Run focused and full verification** + +Run: + +```bash +python3 -m pytest tests/test_observation_recency.py tests/test_dashboard_helpers.py tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py -q +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all checks pass without creating or staging new CSV, JSON, report, sample-report, screenshot, or timing churn. + +- [ ] **Step 6: Stage exact files, run staged hygiene, commit, push, and verify exact-head CI** + +```bash +git add src/observation_recency.py src/dashboard.py src/research_workspace.py tests/test_observation_recency.py tests/test_dashboard_helpers.py tests/test_research_workspace.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_render_smoke.py ROADMAP.md docs/METHODOLOGY.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md +make staged-hygiene-check +git commit -m "Integrate observation recency across research routes" +git push origin codex/personal-research-mode-mvp +gh pr checks 113 --watch +``` + +Expected: PR #113 remains open and draft; exact-head CI passes; pre-existing generated working-data churn remains unstaged. + +### Task 4: Concise Primary Interpretation and Responsive Advanced Evidence + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_dashboard_render_smoke.py` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: the existing immutable `ObservationRecencySet`. +- Produces: `observation_recency_summary_html(result: ObservationRecencySet, *, include_selected: bool) -> str`. +- Produces: `observation_recency_evidence_html(result: ObservationRecencySet) -> str`. +- Retires: the obsolete full-detail `observation_recency_strip_html(...)` + renderer after all production consumers move to the summary/evidence split. +- Preserves: `observation_recency_advanced_details(...)`, the evaluator, the + seven-calendar-day policy, and every independent readiness state. + +- [ ] **Step 1: Write failing helper tests for primary/Advanced separation** + +```python +def test_observation_primary_summary_is_one_route_relevant_interpretation(stale_recency): + rendered = dashboard.observation_recency_summary_html( + stale_recency, + include_selected=True, + ) + assert "Stale" in rendered + assert "Historical context only" in rendered + assert "AVGO" not in rendered + assert "2026-05-22" not in rendered + assert "stale_review_only" not in rendered + assert "SPY" not in rendered + assert "QQQ" not in rendered + + +def test_observation_advanced_evidence_keeps_all_scopes_and_dates(stale_recency): + rendered = dashboard.observation_recency_evidence_html(stale_recency) + for value in ("AVGO", "profile_price_lane", "SPY", "QQQ"): + assert value in rendered + for value in ("2026-05-22", "2026-05-21", "2026-05-20"): + assert value in rendered + assert rendered.count("stale_review_only") == 4 +``` + +- [ ] **Step 2: Run the focused tests and verify the intended failure** + +Run: + +```bash +python3 -m pytest \ + tests/test_dashboard_helpers.py::test_observation_primary_summary_is_one_route_relevant_interpretation \ + tests/test_dashboard_helpers.py::test_observation_advanced_evidence_keeps_all_scopes_and_dates -q +``` + +Expected: FAIL because the two presentation helpers do not yet exist. + +- [ ] **Step 3: Implement the minimal presentation split** + +Add two pure, escaped HTML helpers in `src/dashboard.py`. The summary chooses +`selected_ticker` when `include_selected=True`, otherwise +`profile_price_lane`, and maps states to the exact labels `Current`, `Stale`, +and `Unavailable`. The evidence helper renders all four scopes with scope, +through date, machine state, and fail-closed message, then renders review date, +policy, source path, and excluded counts. `render_observation_recency(...)` +renders the summary before the collapsed Advanced expander and the evidence +inside it. + +- [ ] **Step 4: Add and pass responsive-style contract tests** + +```python +def test_observation_evidence_has_responsive_non_concatenating_styles(): + source = Path(dashboard.__file__).read_text(encoding="utf-8") + evidence_css = source[ + source.index(".observation-recency-evidence {"): + source.index(".research-workspace-heading span,") + ] + assert "display: grid" in evidence_css + assert "grid-template-columns: repeat(auto-fit, minmax(13rem, 1fr))" in evidence_css + assert "overflow-wrap: anywhere" in evidence_css + assert "@media (max-width: 640px)" in evidence_css + assert "grid-template-columns: 1fr" in evidence_css +``` + +Run: + +```bash +python3 -m pytest tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py -q +``` + +Expected: PASS. + +- [ ] **Step 5: Update durable routing truth** + +Mark the bounded observation-recency UX repair complete in `ROADMAP.md` and +move the active local queue to shared quant provenance/recency eligibility. +Update the continuation prompt with the exact implementation evidence and keep +all external source-rights and hosted-freshness gates open. + +- [ ] **Step 6: Run focused, full, release, and hygiene verification** + +Run: + +```bash +python3 -m pytest tests/test_observation_recency.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py tests/test_research_mode_dashboard_contract.py -q +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all commands pass without producing or staging CSV, JSON, report, +sample-report, screenshot, timing, canonical-data, or research-ledger churn. + +- [ ] **Step 7: Stage exact files, verify staged hygiene, commit, push, and require exact-head CI** + +```bash +git add src/dashboard.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md docs/superpowers/specs/2026-07-27-observation-recency-design.md docs/superpowers/plans/2026-07-27-observation-recency.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Clarify market observation recency" +git push origin codex/personal-research-mode-mvp +gh pr checks 113 --watch +``` + +Expected: PR #113 remains open and draft, exact-head CI passes, and all +pre-existing generated working-data differences remain unstaged. diff --git a/docs/superpowers/plans/2026-07-27-roadmap-truth-reconciliation.md b/docs/superpowers/plans/2026-07-27-roadmap-truth-reconciliation.md new file mode 100644 index 000000000..865c021c1 --- /dev/null +++ b/docs/superpowers/plans/2026-07-27-roadmap-truth-reconciliation.md @@ -0,0 +1,125 @@ +# Roadmap Truth Reconciliation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Turn `ROADMAP.md` into a concise current-decision index while preserving every fail-closed boundary and routing detailed completion evidence to its canonical document. + +**Architecture:** Keep current commands, product stage, ordered executable work, external dependencies, later work, completion evidence, success gates, and permanent exclusions in `ROADMAP.md`. Remove duplicated implementation chronology when the same evidence already lives in `docs/COMPLETED_MILESTONES.md`, `docs/ACCESSIBILITY_EVIDENCE.md`, `docs/internal/POINT_IN_TIME_UNIVERSE_REVIEW_HISTORY.md`, or the continuation contract. Correct the stale next-stage conclusion in the capability audit. + +**Tech Stack:** Markdown, Python/pytest documentation contracts, existing Make release and hygiene gates. + +## Global Constraints + +- Start from current repository truth. +- Preserve independent readiness and every research-only, source-rights, explicit-Q4, EPS split-basis, synthetic-fixture, calibration, and no-fabrication boundary. +- Do not run readiness rebuilds or generate CSV, JSON, report, screenshot, or timing artifacts. +- Never stage generated working-data churn. +- Keep PR #113 open and draft. + +--- + +### Task 1: Define the active-roadmap contract + +**Files:** +- Modify: `tests/test_public_v1_release_docs.py` +- Modify: `tests/test_launchers.py` + +**Interfaces:** +- Consumes: the user-approved `Now / Next / Externally blocked / Later / Completed with evidence` structure. +- Produces: a behavioral documentation contract that rejects a missing section, stale current-priority statement, or oversized active roadmap. + +- [ ] **Step 1: Write the failing contract test** + +```python +def test_active_roadmap_is_a_concise_current_decision_index(): + roadmap = _read("ROADMAP.md") + for heading in ( + "## Now", + "## Next", + "## Externally blocked", + "## Later", + "## Completed with evidence", + ): + assert heading in roadmap + assert len(roadmap.splitlines()) <= 320 +``` + +- [ ] **Step 2: Run the contract test and verify the current roadmap fails** + +Run: `python3 -m pytest tests/test_public_v1_release_docs.py::test_active_roadmap_is_a_concise_current_decision_index -q` + +Expected: failure because the exact new headings and line budget are not yet satisfied. + +- [ ] **Step 3: Remove obsolete assertions that require completed performance work to remain in the active-priority order** + +Replace the old performance-before-hosting index assertion with checks that the performance release evidence is linked under `Completed with evidence` and that hosted validation stays external. + +- [ ] **Step 4: Run the focused documentation tests** + +Run: `python3 -m pytest tests/test_public_v1_release_docs.py tests/test_launchers.py -q` + +Expected: failures name only roadmap phrases that must be preserved, intentionally rerouted, or updated. + +### Task 2: Reconcile the active roadmap and stale capability audit + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/analysis_capability_audit.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: current HEAD, exact-head CI evidence, roadmap priorities 1–10, and the external-dependency classifications already recorded in the continuation contract. +- Produces: a concise active roadmap, a current capability conclusion, and a continuation contract pointing to the reconciled roadmap. + +- [ ] **Step 1: Rewrite `ROADMAP.md` as the current decision index** + +Keep: + +```text +Current Truth +Now +Next +Externally blocked +Later +Completed with evidence +Success gates +Permanently out of scope +``` + +Preserve exact unblock conditions for point-in-time universe data, consensus, reviewed peers, hosted controls, accessibility environments, independent reviewers, and calibration. + +- [ ] **Step 2: Correct the stale capability-audit conclusion** + +Replace the completed performance-release recommendation with observation-recency truth, accessibility remediation, permitted point-in-time data, and independent beta validation as the next maturity work. + +- [ ] **Step 3: Update the continuation contract** + +Record that Stage 0 roadmap reconciliation is complete only after the focused/full suite, release checks, hygiene, commit, push, PR update, and exact-head CI pass. + +- [ ] **Step 4: Run focused tests and correct only truthful compatibility failures** + +Run: `python3 -m pytest tests/test_public_v1_release_docs.py tests/test_launchers.py tests/test_pilot_review_feedback_template.py -q` + +Expected: pass with no stale priority assertion. + +- [ ] **Step 5: Run the complete release contract** + +Run: + +```text +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all executable gates pass; generated churn remains excluded. + +- [ ] **Step 6: Stage, verify, commit, push, and update the draft PR** + +Stage only the intentional plan, roadmap, capability-audit, continuation-contract, and test files. Run `make staged-hygiene-check` and `git diff --cached --check`, commit coherently, push only `codex/personal-research-mode-mvp`, update PR #113, and require exact-head CI. diff --git a/docs/superpowers/plans/2026-07-28-discover-monitor-workflow-maturity.md b/docs/superpowers/plans/2026-07-28-discover-monitor-workflow-maturity.md new file mode 100644 index 000000000..e2f910bab --- /dev/null +++ b/docs/superpowers/plans/2026-07-28-discover-monitor-workflow-maturity.md @@ -0,0 +1,268 @@ +# Discover And Monitor Workflow Maturity Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make Discover explain why a company is reviewable and make Monitor expose deterministic research-process attention without company ranking. + +**Architecture:** Discover renders three existing saved-evidence fields with truthful fallbacks. A pure attention adapter composes the immutable six-lane Decision Lab state with optional validated catalyst evidence, preserves focused-cohort order, and gives the dashboard ready-to-render process labels without embedding logic in Streamlit. + +**Tech Stack:** Python 3.12, frozen dataclasses, pandas, existing catalyst/outcome/Decision Lab contracts, pytest, Streamlit AppTest, Playwright browser gate. + +## Global Constraints + +- Discover uses only existing `Why Included`, `Supported Now`, and `Blocked / Missing` saved fields. +- Monitor attention precedence is exactly unresolved evidence change, conflicting evidence, overdue review, invalidation follow-up, outcome evidence follow-up, scheduled catalyst, scheduled review, then monitor. +- Attention labels describe research-process timing only; they are not company rank, attractiveness, severity, expected return, or investment priority. +- Preserve focused-cohort order and the immutable six-lane Decision Lab contract. +- Price, volatility, technical indicators, valuation upside/downside, market capitalization, and candidate context cannot affect attention. +- Empty catalyst/outcome ledgers remain neutral and fabricate no due state. +- Upcoming catalyst evidence is scheduled context only, never urgent or price-predictive. +- Technical identity, exact source metadata, rights blockers, and raw lane detail remain under Advanced. +- No recommendation, score, allocation, holdings, broker, order-routing, auto-trading, or transaction behavior. +- No generated data, broad refresh, readiness rebuild, or ledger write. + +--- + +### Task 1: Discover Three-Question Rows + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` + +**Interfaces:** +- Preserves: `stock_selector_result_table_html(...) -> str`. +- Adds pure helper: `discover_research_answer(row: Mapping[str, object]) -> dict[str, str]` with keys `why_reviewable`, `usable_now`, and `principal_blocker`. + +- [ ] **Step 1: Write failing literal row tests** + +```python +def test_discover_row_answers_three_saved_research_questions(): + rendered = stock_selector_result_table_html( + pd.DataFrame( + [{ + "Ticker": "NVDA", + "Readiness": "partial", + "Why Included": "Core company data is ready.", + "Supported Now": "Price and DCF review.", + "Blocked / Missing": "Peer evidence remains unavailable.", + }] + ), + total_count=1, + target_mode="research", + target_page="company-workbench", + ) + assert "Why reviewable" in rendered + assert "Core company data is ready." in rendered + assert "Usable now" in rendered + assert "Price and DCF review." in rendered + assert "Principal blocker" in rendered + assert "Peer evidence remains unavailable." in rendered + assert rendered.count("Open NVDA review") == 1 +``` + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py tests/test_research_mode_dashboard_contract.py -q` + +Expected: FAIL because current compact rows omit why and blocker. + +- [ ] **Step 2: Implement escaped fallbacks and compact labelled stack** + +Fallbacks: + +```python +{ + "why_reviewable": "Saved readiness does not record why this company is reviewable.", + "usable_now": "No usable research lane is recorded in saved readiness.", + "principal_blocker": "No principal blocker is recorded in saved readiness; this does not mean no risk or external research need exists.", +} +``` + +Map blank/NaN values to fallbacks. Normalize exact `no blocker` to the principal-blocker fallback. Escape all values, keep one action, and do not add source commands or scores. + +- [ ] **Step 3: Add responsive style assertions** + +Require visible labels, text wrapping, one-column phone layout, no clipped text, and a minimum 44-pixel action target. Do not hide any of the three answers on phone. + +- [ ] **Step 4: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py tests/test_research_mode_dashboard_contract.py -q +git diff --check +git add -- src/dashboard.py tests/test_dashboard_helpers.py tests/test_research_mode_dashboard_contract.py +make staged-hygiene-check +git commit -m "Clarify Discover research routing" +``` + +### Task 2: Pure Process-Attention Contract + +**Files:** +- Modify: `src/research_decision_lab.py` +- Modify: `tests/test_research_decision_lab.py` + +**Interfaces:** +- Produces: `ResearchProcessAttention(state: str, label: str, reason: str, source: str)`. +- Produces: `derive_research_process_attention(state: ResearchDecisionLabState, catalyst_timeline: CatalystTimeline | None = None, catalyst_error: str = "") -> ResearchProcessAttention`. +- Extends: `ResearchDisciplineRow` with `attention_state`, `attention_label`, `attention_reason`, and `attention_source`. +- Preserves: `cohort_order`, `ticker`, `status`, `due_lanes`, `next_process_step`, and `identity`. + +- [ ] **Step 1: Write failing precedence tests with hand-built lane literals** + +```python +def test_unresolved_change_precedes_overdue_and_invalidation(): + state = _decision_state( + evidence_state="current", + invalidation_state="missing", + review_trigger_state="evidence_change_due", + ) + attention = derive_research_process_attention(state) + assert attention.state == "evidence_change_due" + assert attention.label == "Needs review" + assert attention.source == "review_trigger" +``` + +Add separate literal tests for conflict, overdue, invalidation, commercial outcome blocker, scheduled catalyst, scheduled review, monitor, and unavailable. + +- [ ] **Step 2: Run focused tests and verify missing contract** + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_decision_lab.py -q` + +Expected: FAIL because the attention types and helper do not exist. + +- [ ] **Step 3: Implement fixed precedence without sorting** + +Read exact lane keys/states. For `scheduled_catalyst`, require +`CatalystTimeline.upcoming` and use the first validated event's exact +`effective_at`; do not use recent events or candidate context to claim an +outstanding review. Apply valid journal/outcome/source-change precedence first; +if none matches and `catalyst_error` is non-empty, return `unavailable` rather +than `monitor`. Never inspect metric or price fields. + +- [ ] **Step 4: Extend discipline rows while preserving order and identity** + +Pass optional `catalyst_timelines_by_ticker: Mapping[str, CatalystTimeline] | None` into `build_research_discipline_rows`. Do not add attention to Decision Lab identity or mutate its six lanes. Assert input order is unchanged even when later rows need review. + +- [ ] **Step 5: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_decision_lab.py -q +git diff --check +git add -- src/research_decision_lab.py tests/test_research_decision_lab.py +make staged-hygiene-check +git commit -m "Add deterministic research process attention" +``` + +### Task 3: Monitor Read-Only Catalyst Composition and Presentation + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_dashboard_render_smoke.py` + +**Interfaces:** +- Extends: `load_dashboard_research_discipline_rows(...)` with read-only catalyst loading. +- Produces: `research_discipline_summary_cards(rows: Iterable[ResearchDisciplineRow]) -> list[dict[str, object]]`. +- Preserves: saved focused-cohort order in `research_discipline_rows(...)`. + +- [ ] **Step 1: Write failing empty-ledger and scheduled-catalyst tests** + +```python +def test_empty_catalyst_and_outcome_ledgers_do_not_create_attention(tmp_path, monkeypatch): + rows = load_dashboard_research_discipline_rows( + _context(tmp_path), + _cohort("NVDA"), + (), + as_of="2026-07-28T12:00:00Z", + ) + assert rows[0].attention_state in {"monitor", "invalidation_follow_up", "scheduled_review"} + assert "catalyst" not in rows[0].attention_reason.lower() + assert "outcome" not in rows[0].attention_reason.lower() +``` + +Add a validated upcoming catalyst fixture and assert `Scheduled`, exact effective date, no `urgent`, no price language, and no state for a different ticker. + +- [ ] **Step 2: Run focused tests and confirm current loader ignores catalyst evidence** + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py -q` + +Expected: FAIL because catalyst timelines are not passed into discipline composition. + +- [ ] **Step 3: Load existing catalyst ledger once and fail per scope** + +Use `load_catalyst_events(DATA_DIR / "catalyst_evidence.csv")` once. Build each +timeline with the explicit Monitor `as_of`, selected profile, selected ticker, +and `commercial_mode=True`. A missing ledger is empty. If the shared ledger is +malformed, pass one deterministic `catalyst_error` to every cohort attention +derivation; earlier valid journal/outcome/source-change states still win, but a +row with no earlier state becomes `unavailable`, never `monitor`. + +- [ ] **Step 4: Render one summary and preserved-order table** + +Place count cards for `Needs review`, `Scheduled`, and `Monitor` above the table. Add `Process attention` and `Why` columns. Keep exact source metadata, catalyst details, rights blockers, identity, and raw lanes under `Advanced: Research Discipline evidence`. + +- [ ] **Step 5: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_decision_lab.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py -q +git diff --check +git add -- src/dashboard.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py +make staged-hygiene-check +git commit -m "Improve Monitor process attention" +``` + +### Task 4: Responsive Workflow Evidence and Closeout + +**Files:** +- Modify: `src/research_accessibility_browser_gate.py` +- Modify: `tests/test_research_accessibility_browser_gate.py` +- Modify: `ROADMAP.md` +- Modify: `docs/NEXT_STAGE_ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `docs/superpowers/specs/2026-07-28-discover-monitor-workflow-maturity-design.md` + +**Interfaces:** +- Extends direct Research Discover and Monitor browser assertions without screenshots or repository writes. + +- [ ] **Step 1: Add failing direct workflow assertions** + +For every actual Discover row, require three labels, non-empty values, a unique ticker-bound action, and a 44-pixel target. For Monitor, require preserved displayed cohort order, one process-attention label/reason per row, no rank/score/return columns, and Advanced identity separation. + +- [ ] **Step 2: Run desktop and phone browser evidence** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_accessibility_browser_gate.py -q +make research-accessibility-browser-check +``` + +Require Discover and Monitor at `1280x720` and `390x844`, exact route/query retention, zero overflow, zero console/page error, zero traceback, and no fabricated ledger content. + +- [ ] **Step 3: Update roadmap and design evidence** + +Record exact implementation commit and direct evidence. Keep market validation, independent sessions, source rights, hosted controls, assistive technology, and calibration open. + +- [ ] **Step 4: Run full release and hygiene gates** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make research-accessibility-browser-check +make diff-hygiene-summary +git diff --check +``` + +- [ ] **Step 5: Stage exact files, commit, push, update PR, and require exact-head CI** + +```bash +git add -- src/research_accessibility_browser_gate.py tests/test_research_accessibility_browser_gate.py ROADMAP.md docs/NEXT_STAGE_ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md docs/superpowers/specs/2026-07-28-discover-monitor-workflow-maturity-design.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Verify Discover and Monitor workflow maturity" +git push origin codex/personal-research-mode-mvp +gh pr checks 113 --watch +``` + +Expected: PR #113 remains open/draft, exact-head CI passes, and the 18 existing generated differences remain unstaged. diff --git a/docs/superpowers/plans/2026-07-28-quant-interpretation-eligibility.md b/docs/superpowers/plans/2026-07-28-quant-interpretation-eligibility.md new file mode 100644 index 000000000..2537aa8df --- /dev/null +++ b/docs/superpowers/plans/2026-07-28-quant-interpretation-eligibility.md @@ -0,0 +1,364 @@ +# Quant Interpretation Eligibility Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add one provider-neutral, fail-closed interpretation overlay for valuation, indicators, and review/risk metrics without changing their calculations or readiness. + +**Architecture:** A new pure module owns immutable evidence and decision contracts. Thin adapters in the three existing calculation modules translate existing statuses plus explicitly supplied recency, provenance, rights, and field-scope states; stock-report and dashboard presentation consume the decisions without mutating calculation results. + +**Tech Stack:** Python 3.12, frozen dataclasses, pandas, existing observation-recency contracts, pytest, Streamlit AppTest. + +## Global Constraints + +- Keep calculation, readiness, observation recency, provenance, rights, field scope, display eligibility, and commercial eligibility independent. +- Allowed interpretation states are exactly `current_context_eligible`, `historical_review_only`, and `withheld`. +- `current_context_eligible` requires an available calculation, current accepted observation, verified provenance, and permitted or genuinely not-applicable rights and field scope. +- Partial calculations never become current-context eligible. +- Stale or incompletely proven calculations may remain historical/review-only only when provenance is not invalid and rights/field scope are not restricted. +- Missing, malformed, future, invalid, restricted, or unavailable evidence fails closed. +- Do not infer `verified` or `permitted` from non-empty free text or provider class names. +- Do not modify formulas, readiness, forecasts, scenarios, consensus, Revenue, EPS, peers, catalysts, outcomes, backtesting, calibration, or ledgers. +- Do not expose a nowcast, probability, ranking, recommendation, allocation, transaction direction, or expected-return score. +- Tests use synthetic fixtures and write no repository CSV, JSON, report, screenshot, timing, canonical-data, or ledger artifact. + +--- + +### Task 1: Pure Interpretation Evaluator + +**Files:** +- Create: `src/quant_interpretation_eligibility.py` +- Create: `tests/test_quant_interpretation_eligibility.py` + +**Interfaces:** +- Produces: `QuantEvidenceAssessment(family: str, scope: str, calculation_state: str, observation_state: str, observation_through_date: str, provenance_state: str, rights_state: str, field_scope_state: str, evidence_notes: tuple[str, ...])`. +- Produces: `QuantInterpretationEligibility(family: str, scope: str, interpretation_state: str, commercial_eligible: bool, reasons: tuple[str, ...], summary: str, boundary: str)`. +- Produces: `evaluate_quant_interpretation(assessment: QuantEvidenceAssessment) -> QuantInterpretationEligibility`. + +- [ ] **Step 1: Write the failing literal decision-table tests** + +```python +import pytest + +from src.quant_interpretation_eligibility import ( + QuantEvidenceAssessment, + evaluate_quant_interpretation, +) + + +@pytest.mark.parametrize( + ("overrides", "state", "commercial", "reasons"), + [ + ({}, "current_context_eligible", True, ()), + ({"observation_state": "stale_review_only"}, "historical_review_only", False, ("observation_stale",)), + ({"provenance_state": "unverified"}, "historical_review_only", False, ("provenance_unverified",)), + ({"rights_state": "unverified"}, "historical_review_only", False, ("rights_unverified",)), + ({"calculation_state": "partial"}, "historical_review_only", False, ("calculation_partial",)), + ({"observation_state": "unavailable"}, "withheld", False, ("observation_unavailable",)), + ({"provenance_state": "invalid"}, "withheld", False, ("provenance_invalid",)), + ({"rights_state": "restricted"}, "withheld", False, ("rights_restricted",)), + ], +) +def test_interpretation_table(overrides, state, commercial, reasons): + values = { + "family": "valuation", + "scope": "NVDA:dcf", + "calculation_state": "available", + "observation_state": "current", + "observation_through_date": "2026-07-27", + "provenance_state": "verified", + "rights_state": "permitted", + "field_scope_state": "permitted", + "evidence_notes": (), + } + result = evaluate_quant_interpretation( + QuantEvidenceAssessment(**(values | overrides)) + ) + assert (result.interpretation_state, result.commercial_eligible) == ( + state, + commercial, + ) + assert result.reasons == reasons +``` + +- [ ] **Step 2: Run the new test and verify the expected import failure** + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_quant_interpretation_eligibility.py -q` + +Expected: FAIL because the module does not exist. + +- [ ] **Step 3: Implement closed vocabularies, validation, and ordered additive reasons** + +```python +INTERPRETATION_BOUNDARY = ( + "Research interpretation only; this does not change readiness, create a " + "forecast or probability, rank a company, or provide an investment action." +) + + +def evaluate_quant_interpretation( + assessment: QuantEvidenceAssessment, +) -> QuantInterpretationEligibility: + assessment.validate() + reasons = _ordered_reasons(assessment) + if _must_withhold(assessment): + state = "withheld" + elif _can_be_current(assessment): + state = "current_context_eligible" + else: + state = "historical_review_only" + return QuantInterpretationEligibility( + family=assessment.family, + scope=assessment.scope, + interpretation_state=state, + commercial_eligible=state == "current_context_eligible" + and assessment.rights_state == "permitted" + and assessment.field_scope_state == "permitted", + reasons=reasons, + summary=_summary(state, assessment.observation_through_date), + boundary=INTERPRETATION_BOUNDARY, + ) +``` + +Reject unknown or empty family/scope/state values with `ValueError`. Deduplicate reasons in a fixed order without dropping independent blockers. Treat `not_applicable` as sufficient only for a genuinely non-applicable rights or field-scope dimension; it does not make `commercial_eligible=True`. + +- [ ] **Step 4: Add constructor and multi-blocker boundary tests** + +Add literal cases proving unknown tokens raise, future/malformed observations are supplied as unavailable and withheld, two blockers remain two ordered reasons, `not_applicable` cannot make a commercial result, and original tuples/objects are not mutated. + +- [ ] **Step 5: Run focused tests and commit** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_quant_interpretation_eligibility.py -q +git diff --check +git add -- src/quant_interpretation_eligibility.py tests/test_quant_interpretation_eligibility.py +make staged-hygiene-check +git commit -m "Add quant interpretation eligibility contract" +``` + +Expected: focused tests and staged hygiene pass; only the new module and tests are committed. + +### Task 2: Three Thin Family Adapters + +**Files:** +- Modify: `src/valuation.py` +- Modify: `src/indicators.py` +- Modify: `src/review_metrics.py` +- Modify: `tests/test_valuation.py` +- Modify: `tests/test_indicators.py` +- Modify: `tests/test_review_metrics.py` + +**Interfaces:** +- Consumes: `ObservationRecency` from `src.observation_recency`. +- Produces: `valuation_quant_assessment(result: ValuationResult | DCFResult | RelativeValuationResult, *, scope: str, observation: ObservationRecency, provenance_state: str, rights_state: str, field_scope_state: str) -> QuantEvidenceAssessment`. +- Produces: `indicator_quant_assessment(row: Mapping[str, object], *, metric_name: str, observation: ObservationRecency, benchmark_observation: ObservationRecency | None, provenance_state: str, rights_state: str, field_scope_state: str) -> QuantEvidenceAssessment`. +- Produces: `review_metric_quant_assessment(metric: ReviewMetric, *, ticker: str, observation: ObservationRecency, benchmark_observation: ObservationRecency | None, provenance_state: str, rights_state: str, field_scope_state: str) -> QuantEvidenceAssessment`. + +- [ ] **Step 1: Write failing valuation adapter tests** + +```python +def test_calculated_valuation_with_stale_observation_is_historical_only(): + result = build_valuation_result(_complete_valuation_input()) + assessment = valuation_quant_assessment( + result, + scope="NVDA:valuation", + observation=_recency("NVDA", "stale_review_only", "2026-05-22"), + provenance_state="verified", + rights_state="permitted", + field_scope_state="permitted", + ) + decision = evaluate_quant_interpretation(assessment) + assert result.status == "calculated" + assert decision.interpretation_state == "historical_review_only" +``` + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_valuation.py -q` + +Expected: FAIL because `valuation_quant_assessment` does not exist. + +- [ ] **Step 2: Implement valuation status mapping without touching result objects** + +Map `calculated` to `available`, `partial` and `peer_data_unavailable` to `partial`, `not_applicable` to `excluded`, and every insufficient/unknown result to `unavailable`. Require a non-empty matching scope and copy only deterministic notes; do not parse free-text source metadata into proof states. + +- [ ] **Step 3: Write failing indicator dependency tests** + +```python +def test_relative_indicator_requires_both_ticker_and_benchmark_observations(): + assessment = indicator_quant_assessment( + {"ticker": "NVDA", "relative_return_vs_spy": 0.12}, + metric_name="relative_return_vs_spy", + observation=_recency("NVDA", "current", "2026-07-27"), + benchmark_observation=_recency("SPY", "stale_review_only", "2026-06-01"), + provenance_state="verified", + rights_state="permitted", + field_scope_state="permitted", + ) + assert evaluate_quant_interpretation( + assessment + ).interpretation_state == "historical_review_only" +``` + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_indicators.py -q` + +Expected: FAIL because the adapter does not exist. + +- [ ] **Step 4: Implement indicator mapping and benchmark composition** + +Use `pd.to_numeric(..., errors="coerce")` and `math.isfinite` for the selected metric. Combine recency by the strictest state: unavailable beats stale, stale beats current. Require a benchmark observation for `relative_return_vs_spy`, `relative_return_vs_qqq`, and `relative_return_vs_sector_etf`; reject ticker or benchmark scope mismatches. + +- [ ] **Step 5: Write failing review-metric tests and implement its adapter** + +```python +def test_ready_review_metric_does_not_imply_current_context(): + metric = ReviewMetric("max_drawdown", "ready", -0.22, "percent") + assessment = review_metric_quant_assessment( + metric, + ticker="NVDA", + observation=_recency("NVDA", "current", "2026-07-27"), + benchmark_observation=None, + provenance_state="unverified", + rights_state="unverified", + field_scope_state="unverified", + ) + assert assessment.calculation_state == "available" + assert evaluate_quant_interpretation( + assessment + ).interpretation_state == "historical_review_only" +``` + +Map `ready`, `partial`, `blocked`, and `excluded` exactly. Require benchmark evidence only when `metric.benchmark` is non-empty. Do not treat `source_context` as structured provenance. + +- [ ] **Step 6: Run all adapter tests and commit** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_valuation.py tests/test_indicators.py tests/test_review_metrics.py tests/test_quant_interpretation_eligibility.py -q +git diff --check +git add -- src/valuation.py src/indicators.py src/review_metrics.py tests/test_valuation.py tests/test_indicators.py tests/test_review_metrics.py +make staged-hygiene-check +git commit -m "Adapt quant results to evidence eligibility" +``` + +Expected: focused tests pass and existing numerical outputs remain unchanged. + +### Task 3: Stock-Report and Dashboard Consumption + +**Files:** +- Modify: `src/stock_report.py` +- Modify: `src/dashboard.py` +- Modify: `tests/test_stock_report.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `tests/test_dashboard_render_smoke.py` + +**Interfaces:** +- Produces: `StockReport.quant_interpretation: dict[str, object]` with `valuation`, `indicators`, and `review_metrics` keys. +- Produces: `stock_report_quant_interpretation_cards(report_payload: Mapping[str, object]) -> list[dict[str, object]]`. +- Consumes: explicit selected-ticker/SPY/QQQ `ObservationRecency` results; explicit unverified proof states remain unverified until a caller supplies structured proof. + +- [ ] **Step 1: Write failing stock-report contract tests** + +```python +def test_stock_report_keeps_quant_values_but_adds_independent_eligibility(provider): + report = build_stock_report("NVDA", provider) + payload = report.to_dict() + assert payload["valuation_snapshot"]["status"] == "calculated" + assert payload["quant_interpretation"]["valuation"]["interpretation_state"] in { + "historical_review_only", + "withheld", + } + assert payload["quant_interpretation"]["valuation"]["commercial_eligible"] is False +``` + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_stock_report.py -q` + +Expected: FAIL because `quant_interpretation` is absent. + +- [ ] **Step 2: Compose decisions without inferring proof** + +Add a default-empty `quant_interpretation` field to `StockReport`. Build exact observation results from the already loaded price histories and explicit report cutoff. Until structured source proof is supplied, pass `provenance_state="unverified"`, `rights_state="unverified"`, and `field_scope_state="unverified"`; do not inspect provider class names or source prose. + +Persist only decision dictionaries in the in-memory report object. Do not write a JSON report or regenerate samples in tests. + +- [ ] **Step 3: Write failing presentation tests** + +```python +def test_quant_cards_label_historical_values_and_hide_technical_detail(): + cards = stock_report_quant_interpretation_cards(_historical_payload()) + rendered = str(cards) + assert "Historical review only" in rendered + assert "Current market" not in rendered + assert "provenance_unverified" not in rendered +``` + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py -q` + +Expected: FAIL because the cards and Advanced detail do not exist. + +- [ ] **Step 4: Add one concise answer plus Advanced evidence** + +Show one primary limitation only when an otherwise visible quant result is historical/review-only or withheld. Put family, scope, calculation state, observation date/state, provenance, rights, field scope, and reason codes in the existing Advanced quantitative evidence section. Escape every value and preserve empty states. + +- [ ] **Step 5: Run focused render tests and commit** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_stock_report.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py tests/test_quant_interpretation_eligibility.py -q +git diff --check +git add -- src/stock_report.py src/dashboard.py tests/test_stock_report.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py +make staged-hygiene-check +git commit -m "Expose quant interpretation boundaries" +``` + +Expected: focused tests pass; no current-market claim appears from stale or unverified evidence. + +### Task 4: Release Evidence and Exact-Head Closeout + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/analysis_capability_audit.md` +- Modify: `docs/NEXT_STAGE_ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `docs/superpowers/specs/2026-07-28-quant-interpretation-eligibility-design.md` + +**Interfaces:** +- Records the exact implementation commit, verification commands, limitations, and remaining external proof gates. + +- [ ] **Step 1: Update documentation after implementation evidence exists** + +Record that the shared overlay is implemented, not that external provenance or source rights are complete. State that current local results remain historical/review-only or withheld wherever structured proof is absent. + +- [ ] **Step 2: Run focused and full verification** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_quant_interpretation_eligibility.py tests/test_valuation.py tests/test_indicators.py tests/test_review_metrics.py tests/test_stock_report.py tests/test_dashboard_helpers.py tests/test_dashboard_render_smoke.py -q +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +Expected: all automated gates pass; pilot may remain truthfully blocked on external or uncommitted-package gates. + +- [ ] **Step 3: Stage exact files, commit, push, update PR, and require exact-head CI** + +```bash +git add -- ROADMAP.md docs/METHODOLOGY.md docs/analysis_capability_audit.md docs/NEXT_STAGE_ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md docs/superpowers/specs/2026-07-28-quant-interpretation-eligibility-design.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Document quant interpretation eligibility" +git push origin codex/personal-research-mode-mvp +gh pr checks 113 --watch +``` + +Expected: PR #113 stays open and draft, exact-head CI succeeds, and all generated working-data churn remains unstaged. diff --git a/docs/superpowers/plans/2026-07-28-research-state-accessibility.md b/docs/superpowers/plans/2026-07-28-research-state-accessibility.md new file mode 100644 index 000000000..872fe437f --- /dev/null +++ b/docs/superpowers/plans/2026-07-28-research-state-accessibility.md @@ -0,0 +1,215 @@ +# Research State Accessibility Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Give user-triggered research-authoring transitions one deterministic visible announcement while keeping initial static states readable and non-noisy. + +**Architecture:** A pure message module maps closed state tokens to visible alert/status HTML. The authoring UI owns transition identity and deduplication; a synthetic test-only Streamlit harness and the real Company Workbench verify semantics without writing production ledgers. + +**Tech Stack:** Python 3.12, frozen dataclasses, escaped HTML, Streamlit session state, pytest, Streamlit AppTest, Playwright browser gate. + +## Global Constraints + +- Live announcements are limited to `validation_rejected`, `preview_ready`, `draft_changed`, `save_reloaded`, and `save_reload_unverified`. +- Initial loading, empty, withheld, stale, blocked, and failure content remains ordinary accessible content unless it changes after a user action. +- `validation_rejected` and `save_reload_unverified` use `role="alert"` and assertive semantics; other transitions use `role="status"` and polite semantics. +- Every live message is visible, atomic, scope-specific, and rendered once per exact transition identity. +- Preserve exact required-field association, focus, global-alert count, preview receipt, save, and reload semantics. +- No message claims source rights, readiness, evidence quality, forecast, commercial eligibility, or investment action changed. +- Synthetic browser fixtures are test-only and are not screen-reader, human, hosted, or WCAG evidence. +- No production ledger, generated CSV/JSON/report, screenshot, timing, or canonical-data write. + +--- + +### Task 1: Pure State Message Contract + +**Files:** +- Create: `src/research_state_accessibility.py` +- Create: `tests/test_research_state_accessibility.py` + +**Interfaces:** +- Produces: `ResearchStateMessage(state: str, title: str, detail: str, role: str, live: str, message_id: str)`. +- Produces: `research_state_message(state: str, *, scope: str, title: str, detail: str, identity: str) -> ResearchStateMessage`. +- Produces: `research_state_message_html(message: ResearchStateMessage, *, announce: bool = True) -> str`. +- Produces: `research_state_transition_key(message: ResearchStateMessage) -> str`. + +- [ ] **Step 1: Write failing literal semantic tests** + +```python +@pytest.mark.parametrize( + ("state", "role", "live"), + [ + ("validation_rejected", "alert", "assertive"), + ("preview_ready", "status", "polite"), + ("draft_changed", "status", "polite"), + ("save_reloaded", "status", "polite"), + ("save_reload_unverified", "alert", "assertive"), + ], +) +def test_state_message_semantics(state, role, live): + message = research_state_message( + state, + scope="demo:NVDA:thesis", + title="State changed", + detail="Review the visible next step.", + identity="receipt-1", + ) + assert (message.role, message.live) == (role, live) + rendered = research_state_message_html(message, announce=True) + assert f"role='{role}'" in rendered + assert f"aria-live='{live}'" in rendered + assert "aria-atomic='true'" in rendered +``` + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_state_accessibility.py -q` + +Expected: FAIL because the module does not exist. + +- [ ] **Step 2: Implement closed mapping, escaped visible HTML, and stable identity** + +Use `html.escape` on all visible and attribute values. Reject empty scope/identity/title and unknown states. Generate `message_id` from normalized scope/state plus a SHA-256 prefix of the supplied identity; never place receipt contents in the DOM. + +- [ ] **Step 3: Add duplicate/unsafe-content tests** + +Prove identical inputs produce the same transition key, different receipt +identity produces a different key, HTML characters are escaped, and the +rendered element contains exactly one title/detail with no visually hidden +duplicate. Also prove `announce=False` renders visible text with +`role="group"` and no `aria-live` attribute. + +- [ ] **Step 4: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_state_accessibility.py -q +git diff --check +git add -- src/research_state_accessibility.py tests/test_research_state_accessibility.py +make staged-hygiene-check +git commit -m "Add research state announcement contract" +``` + +### Task 2: Authoring Transition Integration + +**Files:** +- Modify: `src/research_record_authoring_ui.py` +- Modify: `tests/test_research_record_authoring_ui.py` +- Modify: `tests/fixtures/research_record_authoring_app.py` + +**Interfaces:** +- Produces: `_render_authoring_state_message(st_api, message, *, session_key: str) -> None`. +- Consumes: exact preview digest/receipt, draft digest, and persisted record ID already present in the authoring flow. + +- [ ] **Step 1: Write failing AppTest transition tests** + +```python +def test_valid_preview_announces_unsaved_state_once(tmp_path, monkeypatch): + app = _enter_valid_thesis(_app(tmp_path, monkeypatch)) + app.button(key=_validate_key()).click().run() + statuses = app.get("html") + assert _messages(statuses, role="status") == [ + "Preview ready This exact record is ready for review and is not saved." + ] + app.run() + assert len(_messages(app.get("html"), role="status")) == 1 +``` + +Add separate tests for rejected validation, changed draft, reloaded save, and reload-unverified receipt. Assert one message, correct role, exact recovery wording, and no production-ledger write. + +- [ ] **Step 2: Run focused tests and confirm missing announcements** + +Run: `PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_record_authoring_ui.py -q` + +Expected: FAIL because transitions use existing Streamlit alerts/captions without the shared contract. + +- [ ] **Step 3: Integrate one visible message per exact transition** + +Use the pure helper to replace, not duplicate, the corresponding `st.error`, +`st.warning`, and `st.success` messages. Store the last rendered transition key +under the existing profile/ticker/kind authoring namespace. A normal rerun +with the same transition key calls +`research_state_message_html(message, announce=False)` so the state remains +visible but does not insert another live node. + +Keep field-error binding called once. Validation rejection remains one global alert and focuses only exact supported required-field errors. + +- [ ] **Step 4: Verify save recovery language** + +`save_reloaded` includes the exact record ID and append-only correction rule. `save_reload_unverified` says verification is incomplete and directs ledger inspection; it must not display `Retry save`, `Save again`, or any equivalent duplicate-write invitation. + +- [ ] **Step 5: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_state_accessibility.py tests/test_research_record_authoring_ui.py tests/test_research_record_authoring.py -q +git diff --check +git add -- src/research_record_authoring_ui.py tests/test_research_record_authoring_ui.py tests/fixtures/research_record_authoring_app.py +make staged-hygiene-check +git commit -m "Announce research authoring transitions" +``` + +### Task 3: Static-State and Direct Browser Evidence + +**Files:** +- Create: `tests/fixtures/research_state_accessibility_app.py` +- Modify: `src/research_accessibility_browser_gate.py` +- Modify: `tests/test_research_accessibility_browser_gate.py` +- Modify: `tests/test_dashboard_helpers.py` +- Modify: `docs/ACCESSIBILITY_EVIDENCE.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `docs/superpowers/specs/2026-07-28-research-state-accessibility-design.md` + +**Interfaces:** +- Adds a loopback, synthetic, in-memory/stdout-only state harness to the existing browser gate. +- Produces per-state assertions for role, live policy, atomicity, visible text, duplicates, overflow, console errors, and traceback. + +- [ ] **Step 1: Write failing harness contract tests** + +Assert the harness exposes exactly five transition controls and six static cases: loading, empty, withheld, stale, failure, and validation. Assert the gate rejects duplicate live nodes, hidden-only messages, wrong role, missing atomicity, and any repository write. + +- [ ] **Step 2: Implement the synthetic harness using production render helpers** + +The fixture imports `research_state_message` and `research_state_message_html`. It uses synthetic labels such as `TEST1` and contains no real company evidence, forecast, probability, or ledger. The gate starts it on an isolated loopback port and records results only in memory/stdout. + +- [ ] **Step 3: Add static-state semantic assertions** + +Loading retains readable text and `aria-busy=true`. Empty, withheld, stale, and failure initial content has no `aria-live` attribute. Validation has one alert and, in the real Company Workbench regression, exact field association and focus. + +- [ ] **Step 4: Run direct desktop and phone evidence** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_accessibility_browser_gate.py tests/test_dashboard_helpers.py -q +make research-accessibility-browser-check +``` + +Require `1280x720` and `390x844`, zero overflow, zero duplicate announcement, zero console/page error, zero traceback, and unchanged production ledger hashes. + +- [ ] **Step 5: Update bounded evidence and run full gates** + +Document automation-only evidence and keep zoom, forced colors, reduced motion, screen-reader, and independent-human gates open. + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make research-accessibility-browser-check +make diff-hygiene-summary +git diff --check +``` + +- [ ] **Step 6: Stage exact files, commit, push, update PR, and require exact-head CI** + +```bash +git add -- tests/fixtures/research_state_accessibility_app.py src/research_accessibility_browser_gate.py tests/test_research_accessibility_browser_gate.py tests/test_dashboard_helpers.py docs/ACCESSIBILITY_EVIDENCE.md ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md docs/superpowers/specs/2026-07-28-research-state-accessibility-design.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Verify accessible research state transitions" +git push origin codex/personal-research-mode-mvp +gh pr checks 113 --watch +``` + +Expected: PR remains draft, exact-head CI succeeds, and generated working data remains unstaged. diff --git a/docs/superpowers/plans/2026-07-28-semantic-main-transport-migration.md b/docs/superpowers/plans/2026-07-28-semantic-main-transport-migration.md new file mode 100644 index 000000000..c7925996c --- /dev/null +++ b/docs/superpowers/plans/2026-07-28-semantic-main-transport-migration.md @@ -0,0 +1,223 @@ +# Semantic Main Transport Migration Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Remove the deprecated Streamlit component transport while preserving same-document semantic-main ownership and authoring field-error binding. + +**Architecture:** Both fixed accessibility scripts move from `st.components.v1.html` to `st.html(..., unsafe_allow_javascript=True)`. The scripts and their ownership rules remain unchanged; dependency tests enforce `streamlit>=1.52,<2`, and direct browser gates prove landmark, validation, route, rerun, and layout behavior. + +**Tech Stack:** Python 3.12, Streamlit `st.html`, fixed JavaScript, pytest, Streamlit AppTest, Playwright/Chrome browser gate. + +## Global Constraints + +- Supported Streamlit range is exactly `streamlit>=1.52,<2`. +- No product path calls `st.components.v1.html` or substitutes `st.iframe`. +- Only fixed local script constants may be rendered with `unsafe_allow_javascript=True`. +- Never interpolate user, URL, source, research, or form-value content into executable HTML. +- Preserve exactly one actual main landmark, answer containment, skip behavior, route/query retention, rerun recovery, and bridge ownership cleanup. +- Preserve exact authoring required-field association, cleanup, and focus behavior. +- Unsupported runtime APIs fail closed; no deprecated fallback. +- The script-only HTML creates no visible box, focus target, blank gap, or horizontal overflow. +- No readiness, research result, persistence, ledger, generated-data, or routing change. +- Automated browser evidence is not screen-reader, WCAG, hosted, or cross-major-version proof. + +--- + +### Task 1: Dependency Compatibility Boundary + +**Files:** +- Modify: `requirements.txt` +- Modify: `pyproject.toml` +- Modify: `tests/test_public_v1_release_docs.py` +- Modify: `tests/test_hosted_demo_readiness.py` + +**Interfaces:** +- Produces one exact dependency literal: `streamlit>=1.52,<2`. + +- [ ] **Step 1: Write failing dependency-consistency tests** + +```python +def test_streamlit_range_supports_same_document_javascript_transport(): + requirements = Path("requirements.txt").read_text(encoding="utf-8") + pyproject = Path("pyproject.toml").read_text(encoding="utf-8") + assert "streamlit>=1.52,<2" in requirements + assert '"streamlit>=1.52,<2"' in pyproject + assert "streamlit>=1.44" not in requirements + pyproject +``` + +Update hosted-readiness fixtures to expect the same literal, then run: + +`PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_public_v1_release_docs.py tests/test_hosted_demo_readiness.py -q` + +Expected: FAIL on the current `>=1.44` range. + +- [ ] **Step 2: Change both dependency declarations and fixtures** + +Edit only the Streamlit requirement. Do not update unrelated packages or lock files. + +- [ ] **Step 3: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_public_v1_release_docs.py tests/test_hosted_demo_readiness.py -q +git diff --check +git add -- requirements.txt pyproject.toml tests/test_public_v1_release_docs.py tests/test_hosted_demo_readiness.py +make staged-hygiene-check +git commit -m "Require Streamlit same-document HTML support" +``` + +### Task 2: Migrate Both Fixed Accessibility Renderers + +**Files:** +- Modify: `src/accessibility_bridge.py` +- Modify: `src/research_record_authoring_ui.py` +- Modify: `tests/test_accessibility_bridge.py` +- Modify: `tests/test_research_record_authoring_ui.py` +- Modify: `tests/test_dashboard_render_smoke.py` + +**Interfaces:** +- Produces: `render_semantic_main_bridge(*, html_renderer: Callable[..., Any] | None = None) -> None`. +- Produces: `render_authoring_error_binding(error: AuthoringFieldError | None, *, html_renderer: Callable[..., Any] | None = None) -> None`. +- Default renderer: `streamlit.html`. +- Exact call: `renderer(fixed_document, unsafe_allow_javascript=True)`. + +- [ ] **Step 1: Write failing renderer-contract tests** + +```python +def test_semantic_bridge_uses_same_document_html_renderer(): + calls = [] + render_semantic_main_bridge( + html_renderer=lambda *args, **kwargs: calls.append((args, kwargs)) + ) + assert calls == [ + ((SEMANTIC_MAIN_BRIDGE_HTML,), {"unsafe_allow_javascript": True}) + ] + + +def test_authoring_binding_uses_same_document_html_renderer(): + calls = [] + render_authoring_error_binding( + _field_error(), + html_renderer=lambda *args, **kwargs: calls.append((args, kwargs)), + ) + assert len(calls) == 1 + assert calls[0][1] == {"unsafe_allow_javascript": True} +``` + +Run: + +`PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_accessibility_bridge.py tests/test_research_record_authoring_ui.py -q` + +Expected: FAIL because current calls use `height=0`, `scrolling=False`, and the deprecated component. + +- [ ] **Step 2: Implement renderer resolution with an explicit compatibility error** + +```python +def _same_document_html_renderer( + renderer: Callable[..., Any] | None, +) -> Callable[..., Any]: + resolved = renderer if renderer is not None else getattr(streamlit, "html", None) + if not callable(resolved): + raise RuntimeError( + "Streamlit >=1.52,<2 with st.html JavaScript support is required." + ) + return resolved +``` + +Call the resolved renderer with only the fixed document and `unsafe_allow_javascript=True`. Remove `streamlit.components.v1` imports. Update authoring call sites to pass the error first and use renderer injection only in tests. Keep every JavaScript byte unchanged except transport-neutral comments if tests require them. + +- [ ] **Step 3: Update AppTest inspection to `html` elements** + +Replace iframe `srcdoc` lookups with the actual Streamlit AppTest element type exposed by the installed 1.59.2 runtime. Characterize it first in a focused test and assert the exact fixed document or authoring payload; do not mock away Streamlit rendering. + +- [ ] **Step 4: Preserve authoring error binding behavior** + +Run required-field tests proving Thesis Id and Effective At association, stale-error cleanup, exact focus, one global alert, and no ledger write. The renderer migration must not change the error contract or allow arbitrary validation reasons to bind a field. + +- [ ] **Step 5: Run focused tests and commit** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_accessibility_bridge.py tests/test_research_record_authoring_ui.py tests/test_dashboard_render_smoke.py -q +git diff --check +git add -- src/accessibility_bridge.py src/research_record_authoring_ui.py tests/test_accessibility_bridge.py tests/test_research_record_authoring_ui.py tests/test_dashboard_render_smoke.py +make staged-hygiene-check +git commit -m "Migrate accessibility bridges to st html" +``` + +### Task 3: Direct Runtime Regression and Warning Removal + +**Files:** +- Modify: `src/research_accessibility_browser_gate.py` +- Modify: `tests/test_research_accessibility_browser_gate.py` +- Modify: `docs/ACCESSIBILITY_EVIDENCE.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Modify: `docs/superpowers/specs/2026-07-28-semantic-main-transport-migration-design.md` + +**Interfaces:** +- Extends the existing six-route/two-viewport gate with bridge layout and deprecation-warning assertions. + +- [ ] **Step 1: Add failing browser-result assertions** + +Add result fields for: + +```python +{ + "deprecated_component_warning_count": 0, + "bridge_iframe_count": 0, + "bridge_focusable_count": 0, + "bridge_height": 0, +} +``` + +The gate must fail when stderr/console contains `st.components.v1.html`, when an accessibility bridge iframe exists, or when the script-only HTML has positive visible height or a focusable descendant. + +- [ ] **Step 2: Run the focused gate tests and confirm the old transport fails** + +Run: + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/test_research_accessibility_browser_gate.py -q +make research-dashboard-render-smoke +``` + +Expected before migration: the render smoke reproduces the deprecation warning. + +- [ ] **Step 3: Run direct desktop and phone evidence after migration** + +Run: + +`make research-accessibility-browser-check` + +Require all six Research routes at `1280x720` and `390x844` to retain exactly one main, one contained answer, route H1, skip focus, exact query, same-document rerun recovery, mutation recovery, zero overflow, zero browser error, zero deprecated warning, zero bridge iframe, and zero visible bridge footprint. + +- [ ] **Step 4: Update evidence and roadmap with exact tested boundaries** + +Record the exact commit, installed Streamlit version, dependency range, routes, viewports, authoring validation regression, and warning count. Keep manual assistive-technology and hosted evidence open. + +- [ ] **Step 5: Run full release gates** + +```bash +PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make research-accessibility-browser-check +make diff-hygiene-summary +git diff --check +``` + +- [ ] **Step 6: Stage exact files, commit, push, update PR, and require exact-head CI** + +```bash +git add -- src/research_accessibility_browser_gate.py tests/test_research_accessibility_browser_gate.py docs/ACCESSIBILITY_EVIDENCE.md ROADMAP.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md docs/superpowers/specs/2026-07-28-semantic-main-transport-migration-design.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Verify same-document accessibility transport" +git push origin codex/personal-research-mode-mvp +gh pr checks 113 --watch +``` + +Expected: PR #113 remains draft, exact-head CI passes, and the 18 existing generated differences remain unstaged. diff --git a/docs/superpowers/plans/2026-07-31-accessibility-media-preferences.md b/docs/superpowers/plans/2026-07-31-accessibility-media-preferences.md new file mode 100644 index 000000000..581cb9974 --- /dev/null +++ b/docs/superpowers/plans/2026-07-31-accessibility-media-preferences.md @@ -0,0 +1,880 @@ +# Accessibility Media Preferences Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add explicit forced-colors and reduced-motion behavior plus read-only automated regression evidence across the existing six-route, two-viewport Research accessibility matrix. + +**Architecture:** A pure CSS helper in `src/research_workspace.py` owns the Research media-preference rules and `src/dashboard.py` injects them after the existing Research styles. Pure evaluators and bounded browser-observation helpers in `src/research_accessibility_browser_gate.py` exercise both preferences sequentially in each existing page context, restore ordinary media settings, and retain every current landmark, navigation, state, runtime, and repository-write check. + +**Tech Stack:** Python 3.12, Streamlit, HTML/CSS, Playwright sync API, pytest, Make, GitHub Actions. + +## Global Constraints + +- Automated forced-colors and reduced-motion results are engineering evidence only; they are not manual task completion, assistive-technology validation, independent-human review, hosted validation, or WCAG conformance. +- Keep the existing six routes and `VIEWPORTS == ((1280, 720), (390, 844))`; add no route, browser context, server process, screenshot, timing file, or report artifact. +- Preserve Research Desk -> Discover -> Company Workbench -> Monitor, Data Health, Proof History, independent readiness states, research-only copy, and all fail-closed source and quantitative boundaries. +- Add no source, dataset, readiness promotion, forecast, probability, recommendation, research record, ledger mutation, or generated CSV/JSON/report/sample-report/screenshot/timing artifact. +- Ordinary media preferences must be restored even when one emulated-mode observation raises. +- Never run `make readiness`, broad refreshes, provider-wide imports, or generated report commands. +- Never use `git add -A`; stage only the exact files named by each task and keep the existing 18 generated paths unstaged. +- Keep PR #113 open and draft; push only `codex/personal-research-mode-mvp`; do not merge or deploy. + +--- + +### Task 1: Shared Research Media-Preference CSS + +**Files:** +- Modify: `src/research_workspace.py` +- Modify: `src/dashboard.py` +- Test: `tests/test_research_workspace.py` +- Test: `tests/test_research_mode_dashboard_contract.py` + +**Interfaces:** +- Produces: `research_accessibility_media_preferences_css() -> str`. +- Consumes: the existing `render_research_workspace_styles()` composition point. +- Preserves: every normal-theme selector and all existing Research HTML. + +- [ ] **Step 1: Write the failing CSS-helper contract test** + +Add the helper to the import list in `tests/test_research_workspace.py`, then add: + +```python +def test_research_accessibility_media_preferences_css_declares_bounded_fallbacks(): + css = research_accessibility_media_preferences_css() + + assert "@media (forced-colors: active)" in css + assert ".research-workflow-link[aria-current='page']" in css + assert ".research-workspace-boundary" in css + assert "outline: 3px solid Highlight !important" in css + assert "border-color: CanvasText !important" in css + assert "box-shadow: none !important" in css + assert "@media (prefers-reduced-motion: reduce)" in css + assert ".stApp *::before" in css + assert ".stApp *::after" in css + assert "animation-duration: 0.01ms !important" in css + assert "animation-iteration-count: 1 !important" in css + assert "transition-duration: 0.01ms !important" in css + assert "transition-delay: 0ms !important" in css + assert "scroll-behavior: auto !important" in css + assert "forced-color-adjust: none" not in css +``` + +Add this render-composition contract to `tests/test_research_mode_dashboard_contract.py`: + +```python +def test_research_workspace_styles_inject_media_preferences_after_normal_styles(): + source = dashboard.Path(dashboard.__file__).read_text(encoding="utf-8") + start = source.index("def render_research_workspace_styles()") + end = source.index("\ndef render_research_workspace_header(", start) + styles = source[start:end] + + normal_styles = styles.index("st.markdown(") + preferences = styles.index("research_accessibility_media_preferences_css()") + assert normal_styles < preferences + assert "unsafe_allow_html=True" in styles[preferences:] +``` + +- [ ] **Step 2: Run the focused tests and verify RED** + +Run: + +```bash +python3 -m pytest \ + tests/test_research_workspace.py::test_research_accessibility_media_preferences_css_declares_bounded_fallbacks \ + tests/test_research_mode_dashboard_contract.py::test_research_workspace_styles_inject_media_preferences_after_normal_styles \ + -q +``` + +Expected: collection or assertion failure because `research_accessibility_media_preferences_css` does not exist and the renderer does not inject it. + +- [ ] **Step 3: Implement the pure CSS helper** + +Add this function near the existing Research HTML helpers in `src/research_workspace.py`: + +```python +def research_accessibility_media_preferences_css() -> str: + """Return Research-only media preference fallbacks without changing data.""" + + return """ +@media (forced-colors: active) { + .stApp a:focus-visible, + .stApp button:focus-visible, + .stApp input:focus-visible, + .stApp select:focus-visible, + .stApp textarea:focus-visible, + .stApp [role="button"]:focus-visible, + .stApp [role="radio"]:focus-visible, + .stApp [role="tab"]:focus-visible, + .stApp summary:focus-visible, + .stApp [tabindex]:not([tabindex="-1"]):focus-visible { + outline: 3px solid Highlight !important; + outline-offset: 3px !important; + box-shadow: none !important; + } + .research-workflow-link[aria-current='page'] { + border: 2px solid Highlight !important; + outline: 1px solid CanvasText !important; + outline-offset: -4px !important; + } + .research-workspace-boundary, + .observation-recency-summary, + .research-state-message, + .signal-card { + border-color: CanvasText !important; + } + .research-workspace-boundary { + border-style: solid !important; + border-width: 1px !important; + border-radius: 4px; + padding: 0.2rem 0.35rem; + } +} +@media (prefers-reduced-motion: reduce) { + .stApp, + .stApp *, + .stApp *::before, + .stApp *::after { + animation-duration: 0.01ms !important; + animation-iteration-count: 1 !important; + transition-duration: 0.01ms !important; + transition-delay: 0ms !important; + scroll-behavior: auto !important; + } +} +""".strip() +``` + +- [ ] **Step 4: Inject the helper after existing Research styles** + +Import `research_accessibility_media_preferences_css` in `src/dashboard.py`. At the end of `render_research_workspace_styles()`, after the existing normal style block, add: + +```python + st.markdown( + "", + unsafe_allow_html=True, + ) +``` + +- [ ] **Step 5: Run focused CSS and render tests** + +Run: + +```bash +python3 -m pytest \ + tests/test_research_workspace.py \ + tests/test_research_mode_dashboard_contract.py \ + tests/test_dashboard_render_smoke.py \ + -q +``` + +Expected: PASS with no generated files changed. + +- [ ] **Step 6: Commit the CSS contract** + +```bash +git add -- \ + src/research_workspace.py \ + src/dashboard.py \ + tests/test_research_workspace.py \ + tests/test_research_mode_dashboard_contract.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add research media preference styles" +``` + +--- + +### Task 2: Pure Fail-Closed Media Observation Evaluators + +**Files:** +- Modify: `src/research_accessibility_browser_gate.py` +- Test: `tests/test_research_accessibility_browser_gate.py` + +**Interfaces:** +- Produces: `evaluate_forced_colors_observation(observation: dict[str, object], *, primary_route: bool) -> list[dict[str, object]]`. +- Produces: `evaluate_reduced_motion_observation(observation: dict[str, object]) -> list[dict[str, object]]`. +- Consumes: literal browser observations only; neither function receives a page object. + +- [ ] **Step 1: Write the failing forced-colors evaluator test** + +Add: + +```python +def test_forced_colors_observation_fails_closed_for_each_required_signal(): + from src.research_accessibility_browser_gate import ( + evaluate_forced_colors_observation, + ) + + passing = { + "media_active": True, + "skip_count": 1, + "skip_focused": True, + "skip_outline_style": "solid", + "skip_outline_width_px": 3.0, + "current_route_count": 1, + "current_route_value": "page", + "current_route_marker_width_px": 2.0, + "boundary_count": 1, + "boundary_visible": True, + "boundary_border_width_px": 1.0, + "heading_visible": True, + "boundary_text_visible": True, + "overflow_px": 0.0, + "traceback_visible": False, + } + assertions = evaluate_forced_colors_observation(passing, primary_route=True) + assert assertions and all(item["passed"] for item in assertions) + + mutations = ( + ("forced_colors_media_active", {"media_active": False}), + ("forced_colors_skip_focus", {"skip_focused": False}), + ("forced_colors_focus_outline", {"skip_outline_width_px": 0.0}), + ("forced_colors_current_route", {"current_route_value": ""}), + ("forced_colors_current_route_marker", {"current_route_marker_width_px": 0.0}), + ("forced_colors_boundary", {"boundary_visible": False}), + ("forced_colors_boundary_border", {"boundary_border_width_px": 0.0}), + ("forced_colors_required_text", {"heading_visible": False}), + ("forced_colors_no_overflow", {"overflow_px": 2.0}), + ("forced_colors_no_traceback", {"traceback_visible": True}), + ) + for name, changed in mutations: + failed = evaluate_forced_colors_observation( + {**passing, **changed}, + primary_route=True, + ) + assert next(item for item in failed if item["name"] == name)["passed"] is False + + secondary = evaluate_forced_colors_observation( + {**passing, "current_route_count": 0, "current_route_value": "", "current_route_marker_width_px": 0.0}, + primary_route=False, + ) + assert all(item["passed"] for item in secondary) +``` + +- [ ] **Step 2: Write the failing reduced-motion evaluator test** + +Add: + +```python +def test_reduced_motion_observation_fails_closed_for_each_required_signal(): + from src.research_accessibility_browser_gate import ( + evaluate_reduced_motion_observation, + ) + + passing = { + "media_active": True, + "target_count": 3, + "max_animation_duration_ms": 0.01, + "max_transition_duration_ms": 0.01, + "max_animation_iterations": 1.0, + "scroll_behavior": "auto", + "heading_visible": True, + "boundary_visible": True, + "overflow_px": 0.0, + "traceback_visible": False, + } + assertions = evaluate_reduced_motion_observation(passing) + assert assertions and all(item["passed"] for item in assertions) + + mutations = ( + ("reduced_motion_media_active", {"media_active": False}), + ("reduced_motion_targets", {"target_count": 0}), + ("reduced_motion_animation_duration", {"max_animation_duration_ms": 250.0}), + ("reduced_motion_transition_duration", {"max_transition_duration_ms": 250.0}), + ("reduced_motion_animation_iterations", {"max_animation_iterations": 2.0}), + ("reduced_motion_scroll_behavior", {"scroll_behavior": "smooth"}), + ("reduced_motion_required_text", {"boundary_visible": False}), + ("reduced_motion_no_overflow", {"overflow_px": 2.0}), + ("reduced_motion_no_traceback", {"traceback_visible": True}), + ) + for name, changed in mutations: + failed = evaluate_reduced_motion_observation({**passing, **changed}) + assert next(item for item in failed if item["name"] == name)["passed"] is False +``` + +- [ ] **Step 3: Run the evaluator tests and verify RED** + +Run: + +```bash +python3 -m pytest \ + tests/test_research_accessibility_browser_gate.py::test_forced_colors_observation_fails_closed_for_each_required_signal \ + tests/test_research_accessibility_browser_gate.py::test_reduced_motion_observation_fails_closed_for_each_required_signal \ + -q +``` + +Expected: collection failure because both evaluator functions are absent. + +- [ ] **Step 4: Implement both pure evaluators** + +Add the functions after `_assertion` in `src/research_accessibility_browser_gate.py`. Use these exact assertions and thresholds: + +```python +def evaluate_forced_colors_observation( + observation: dict[str, object], + *, + primary_route: bool, +) -> list[dict[str, object]]: + current_route_passed = ( + int(observation.get("current_route_count", 0)) == 1 + and str(observation.get("current_route_value") or "") == "page" + ) if primary_route else int(observation.get("current_route_count", 0)) == 0 + marker_passed = ( + float(observation.get("current_route_marker_width_px", 0)) > 0 + if primary_route + else True + ) + return [ + _assertion("forced_colors_media_active", observation.get("media_active") is True, "forced-colors media query active"), + _assertion("forced_colors_skip_focus", int(observation.get("skip_count", 0)) == 1 and observation.get("skip_focused") is True, "one physical Tab focused the sole skip link"), + _assertion("forced_colors_focus_outline", str(observation.get("skip_outline_style") or "") != "none" and float(observation.get("skip_outline_width_px", 0)) > 0, "focused skip link retains a visible outline"), + _assertion("forced_colors_current_route", current_route_passed, "current-route semantic state preserved"), + _assertion("forced_colors_current_route_marker", marker_passed, "current route retains a non-color marker"), + _assertion("forced_colors_boundary", int(observation.get("boundary_count", 0)) == 1 and observation.get("boundary_visible") is True, "one research boundary remains visible"), + _assertion("forced_colors_boundary_border", float(observation.get("boundary_border_width_px", 0)) > 0, "research boundary retains a visible border"), + _assertion("forced_colors_required_text", observation.get("heading_visible") is True and observation.get("boundary_text_visible") is True, "heading and research-only text remain visible"), + _assertion("forced_colors_no_overflow", float(observation.get("overflow_px", math.inf)) <= 1, f"horizontal overflow={observation.get('overflow_px')}px"), + _assertion("forced_colors_no_traceback", observation.get("traceback_visible") is False, "no traceback rendered"), + ] + + +def evaluate_reduced_motion_observation( + observation: dict[str, object], +) -> list[dict[str, object]]: + return [ + _assertion("reduced_motion_media_active", observation.get("media_active") is True, "reduced-motion media query active"), + _assertion("reduced_motion_targets", int(observation.get("target_count", 0)) > 0, "application-owned motion targets observed"), + _assertion("reduced_motion_animation_duration", float(observation.get("max_animation_duration_ms", math.inf)) <= 0.1, f"max animation duration={observation.get('max_animation_duration_ms')}ms"), + _assertion("reduced_motion_transition_duration", float(observation.get("max_transition_duration_ms", math.inf)) <= 0.1, f"max transition duration={observation.get('max_transition_duration_ms')}ms"), + _assertion("reduced_motion_animation_iterations", float(observation.get("max_animation_iterations", math.inf)) <= 1, f"max animation iterations={observation.get('max_animation_iterations')}"), + _assertion("reduced_motion_scroll_behavior", str(observation.get("scroll_behavior") or "") != "smooth", f"scroll behavior={observation.get('scroll_behavior')!r}"), + _assertion("reduced_motion_required_text", observation.get("heading_visible") is True and observation.get("boundary_visible") is True, "heading and research boundary remain visible"), + _assertion("reduced_motion_no_overflow", float(observation.get("overflow_px", math.inf)) <= 1, f"horizontal overflow={observation.get('overflow_px')}px"), + _assertion("reduced_motion_no_traceback", observation.get("traceback_visible") is False, "no traceback rendered"), + ] +``` + +- [ ] **Step 5: Run focused evaluator and existing gate-contract tests** + +```bash +python3 -m pytest tests/test_research_accessibility_browser_gate.py -q +``` + +Expected: PASS. + +- [ ] **Step 6: Commit the pure contracts** + +```bash +git add -- src/research_accessibility_browser_gate.py tests/test_research_accessibility_browser_gate.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add accessibility media evaluators" +``` + +--- + +### Task 3: Integrate Media Emulation Into Every Route Measurement + +**Files:** +- Modify: `src/research_accessibility_browser_gate.py` +- Test: `tests/test_research_accessibility_browser_gate.py` + +**Interfaces:** +- Produces: `_forced_colors_observation(page: Any, route: ResearchRoute) -> dict[str, object]`. +- Produces: `_reduced_motion_observation(page: Any, route: ResearchRoute) -> dict[str, object]`. +- Produces: `_media_preference_assertions(page: Any, route: ResearchRoute) -> list[dict[str, object]]`. +- Consumes: the Task 2 evaluator functions and existing `_horizontal_overflow_pixels` helper. +- Preserves: `_measure_route(...) -> dict[str, object]` and its current result schema. + +- [ ] **Step 1: Write the failing emulation-order and restoration test** + +Add: + +```python +def test_media_preference_assertions_emulate_both_modes_and_restore_each(monkeypatch): + import src.research_accessibility_browser_gate as gate + + class FakePage: + def __init__(self): + self.calls = [] + + def emulate_media(self, **kwargs): + self.calls.append(kwargs) + + page = FakePage() + monkeypatch.setattr(gate, "_forced_colors_observation", lambda page, route: {}) + monkeypatch.setattr(gate, "_reduced_motion_observation", lambda page, route: {}) + monkeypatch.setattr(gate, "evaluate_forced_colors_observation", lambda observation, *, primary_route: [{"name": "forced", "passed": True, "detail": "ok"}]) + monkeypatch.setattr(gate, "evaluate_reduced_motion_observation", lambda observation: [{"name": "motion", "passed": True, "detail": "ok"}]) + + assertions = gate._media_preference_assertions(page, gate.RESEARCH_ROUTES[0]) + + assert all(item["passed"] for item in assertions) + assert page.calls == [ + {"forced_colors": "active", "reduced_motion": "no-preference"}, + {"forced_colors": "none", "reduced_motion": "no-preference"}, + {"forced_colors": "none", "reduced_motion": "reduce"}, + {"forced_colors": "none", "reduced_motion": "no-preference"}, + ] +``` + +Add this restoration regression: + +```python +def test_media_preference_assertions_restore_and_continue_after_probe_failure(monkeypatch): + import src.research_accessibility_browser_gate as gate + + class FakePage: + def __init__(self): + self.calls = [] + + def emulate_media(self, **kwargs): + self.calls.append(kwargs) + + def fail_forced(page, route): + raise RuntimeError("forced probe") + + page = FakePage() + monkeypatch.setattr(gate, "_forced_colors_observation", fail_forced) + monkeypatch.setattr(gate, "_reduced_motion_observation", lambda page, route: {}) + monkeypatch.setattr(gate, "evaluate_reduced_motion_observation", lambda observation: [{"name": "motion", "passed": True, "detail": "ok"}]) + + assertions = gate._media_preference_assertions(page, gate.RESEARCH_ROUTES[0]) + + forced = next(item for item in assertions if item["name"] == "forced_colors_execution") + assert forced["passed"] is False + assert "RuntimeError: forced probe" in forced["detail"] + assert next(item for item in assertions if item["name"] == "motion")["passed"] is True + assert page.calls == [ + {"forced_colors": "active", "reduced_motion": "no-preference"}, + {"forced_colors": "none", "reduced_motion": "no-preference"}, + {"forced_colors": "none", "reduced_motion": "reduce"}, + {"forced_colors": "none", "reduced_motion": "no-preference"}, + ] +``` + +- [ ] **Step 2: Run the integration tests and verify RED** + +Run: + +```bash +python3 -m pytest tests/test_research_accessibility_browser_gate.py -q +``` + +Expected: failure because `_media_preference_assertions` and both observation helpers are absent. + +- [ ] **Step 3: Implement bounded forced-colors observation** + +Implement `_forced_colors_observation` with one bounded `page.evaluate` call that returns: + +```python +{ + "media_active": bool(window.matchMedia("(forced-colors: active)").matches), + "skip_count": int, + "skip_focused": bool, + "skip_outline_style": str, + "skip_outline_width_px": float, + "current_route_count": int, + "current_route_value": str, + "current_route_marker_width_px": float, + "boundary_count": int, + "boundary_visible": bool, + "boundary_border_width_px": float, + "heading_visible": bool, + "boundary_text_visible": bool, + "overflow_px": float, + "traceback_visible": bool, +} +``` + +Before reading computed focus styles, clear application focus with the existing body-focus pattern and send one physical `page.keyboard.press("Tab")`. Require the focused element to be the sole `a.public-skip-link[href='#public-page-answer']`. For primary routes, inspect exactly one `.research-workflow-link[aria-current='page']`; for secondary routes, record zero. Inspect exactly one `.research-workspace-boundary` and the exact route H1. + +Use this implementation shape so the browser returns only bounded literal +observations: + +```python +def _forced_colors_observation( + page: Any, + route: ResearchRoute, +) -> dict[str, object]: + page.evaluate( + """ +() => { + if (document.activeElement && document.activeElement !== document.body) { + document.activeElement.blur(); + } + document.body.setAttribute("tabindex", "-1"); + document.body.focus({preventScroll: true}); +} +""" + ) + page.keyboard.press("Tab") + page.evaluate("document.body.removeAttribute('tabindex')") + return page.evaluate( + """ +(primaryRoute) => { + const visible = (node) => { + if (!node) return false; + const box = node.getBoundingClientRect(); + const style = getComputedStyle(node); + return box.width > 0 && box.height > 0 && + style.display !== "none" && style.visibility !== "hidden"; + }; + const width = (style, names) => Math.max( + ...names.map((name) => Number.parseFloat(style[name]) || 0) + ); + const skips = [...document.querySelectorAll( + "a.public-skip-link[href='#public-page-answer']" + )]; + const currents = [...document.querySelectorAll( + ".research-workflow-link[aria-current='page']" + )]; + const boundaries = [...document.querySelectorAll( + ".research-workspace-boundary" + )]; + const skipStyle = skips.length === 1 ? getComputedStyle(skips[0]) : null; + const currentStyle = currents.length === 1 ? getComputedStyle(currents[0]) : null; + const boundaryStyle = boundaries.length === 1 ? getComputedStyle(boundaries[0]) : null; + const heading = document.querySelector("[role='main'] h1"); + return { + media_active: matchMedia("(forced-colors: active)").matches, + skip_count: skips.length, + skip_focused: skips.length === 1 && document.activeElement === skips[0], + skip_outline_style: skipStyle ? skipStyle.outlineStyle : "", + skip_outline_width_px: skipStyle ? Number.parseFloat(skipStyle.outlineWidth) || 0 : 0, + current_route_count: currents.length, + current_route_value: currents.length === 1 ? currents[0].getAttribute("aria-current") || "" : "", + current_route_marker_width_px: currentStyle ? width(currentStyle, ["borderTopWidth", "borderRightWidth", "borderBottomWidth", "borderLeftWidth", "outlineWidth"]) : 0, + boundary_count: boundaries.length, + boundary_visible: boundaries.length === 1 && visible(boundaries[0]), + boundary_border_width_px: boundaryStyle ? width(boundaryStyle, ["borderTopWidth", "borderRightWidth", "borderBottomWidth", "borderLeftWidth"]) : 0, + heading_visible: visible(heading), + boundary_text_visible: boundaries.length === 1 && visible(boundaries[0]) && boundaries[0].innerText.includes("Research-only"), + overflow_px: Math.max(0, document.documentElement.scrollWidth - window.innerWidth), + traceback_visible: document.body.innerText.includes("Traceback (most recent call last)"), + primary_route: primaryRoute, + }; +} +""", + route.requires_primary_navigation, + ) +``` + +- [ ] **Step 4: Implement bounded reduced-motion observation** + +Use one `page.evaluate` call over `.stApp`, `.research-workflow-link`, `.research-workspace-boundary`, and `.research-state-message`. Parse every comma-separated CSS duration as milliseconds in the browser function: values ending in `ms` keep their numeric value; values ending in `s` multiply by `1000`; missing or unparsable values become the JSON-safe fail-closed sentinel `Number.MAX_SAFE_INTEGER`. Return the maximum animation duration, transition duration, and finite animation iteration count, plus media match, target count, computed `.stApp` scroll behavior, heading/boundary visibility, overflow, and traceback state. + +Use this exact observation shape: + +```python +def _reduced_motion_observation( + page: Any, + route: ResearchRoute, +) -> dict[str, object]: + return page.evaluate( + """ +() => { + const app = document.querySelector(".stApp"); + const visible = (node) => { + if (!node) return false; + const box = node.getBoundingClientRect(); + const style = getComputedStyle(node); + return box.width > 0 && box.height > 0 && + style.display !== "none" && style.visibility !== "hidden"; + }; + const toMilliseconds = (value) => value.split(",").map((part) => { + const token = part.trim(); + const amount = Number.parseFloat(token); + if (!Number.isFinite(amount)) return Number.MAX_SAFE_INTEGER; + if (token.endsWith("ms")) return amount; + if (token.endsWith("s")) return amount * 1000; + return Number.MAX_SAFE_INTEGER; + }); + const toIterations = (value) => value.split(",").map((part) => { + const token = part.trim(); + if (token === "infinite") return Number.MAX_SAFE_INTEGER; + const amount = Number.parseFloat(token); + return Number.isFinite(amount) ? amount : Number.MAX_SAFE_INTEGER; + }); + const targets = [...new Set([ + app, + ...document.querySelectorAll(".research-workflow-link"), + ...document.querySelectorAll(".research-workspace-boundary"), + ...document.querySelectorAll(".research-state-message"), + ].filter(Boolean))]; + const styles = targets.map((node) => getComputedStyle(node)); + const animationDurations = styles.flatMap((style) => toMilliseconds(style.animationDuration)); + const transitionDurations = styles.flatMap((style) => toMilliseconds(style.transitionDuration)); + const iterations = styles.flatMap((style) => toIterations(style.animationIterationCount)); + const boundary = document.querySelector(".research-workspace-boundary"); + const heading = document.querySelector("[role='main'] h1"); + return { + media_active: matchMedia("(prefers-reduced-motion: reduce)").matches, + target_count: targets.length, + max_animation_duration_ms: animationDurations.length ? Math.max(...animationDurations) : Number.MAX_SAFE_INTEGER, + max_transition_duration_ms: transitionDurations.length ? Math.max(...transitionDurations) : Number.MAX_SAFE_INTEGER, + max_animation_iterations: iterations.length ? Math.max(...iterations) : Number.MAX_SAFE_INTEGER, + scroll_behavior: app ? getComputedStyle(app).scrollBehavior : "", + heading_visible: visible(heading), + boundary_visible: visible(boundary), + overflow_px: Math.max(0, document.documentElement.scrollWidth - window.innerWidth), + traceback_visible: document.body.innerText.includes("Traceback (most recent call last)"), + }; +} +""" + ) +``` + +- [ ] **Step 5: Implement restoration and failure isolation** + +Implement `_media_preference_assertions` with separate phase blocks and a +fail-closed restore helper: + +```python +def _media_preference_assertions( + page: Any, + route: ResearchRoute, +) -> list[dict[str, object]]: + assertions: list[dict[str, object]] = [] + + def restore() -> None: + try: + page.emulate_media( + forced_colors="none", + reduced_motion="no-preference", + ) + except Exception as exc: + assertions.append( + _assertion( + "media_preferences_restore", + False, + f"{type(exc).__name__}: {exc}", + ) + ) + + try: + page.emulate_media(forced_colors="active", reduced_motion="no-preference") + assertions.extend( + evaluate_forced_colors_observation( + _forced_colors_observation(page, route), + primary_route=route.requires_primary_navigation, + ) + ) + except Exception as exc: + assertions.append(_assertion("forced_colors_execution", False, f"{type(exc).__name__}: {exc}")) + finally: + restore() + + try: + page.emulate_media(forced_colors="none", reduced_motion="reduce") + assertions.extend( + evaluate_reduced_motion_observation( + _reduced_motion_observation(page, route) + ) + ) + except Exception as exc: + assertions.append(_assertion("reduced_motion_execution", False, f"{type(exc).__name__}: {exc}")) + finally: + restore() + return assertions +``` + +Add this restore-failure case so a reset failure cannot disappear into the +route-level exception: + +```python +def test_media_preference_assertions_report_each_restore_failure(monkeypatch): + import src.research_accessibility_browser_gate as gate + + class FakePage: + def __init__(self): + self.calls = [] + + def emulate_media(self, **kwargs): + self.calls.append(kwargs) + if kwargs == {"forced_colors": "none", "reduced_motion": "no-preference"}: + raise RuntimeError("restore failed") + + page = FakePage() + monkeypatch.setattr(gate, "_forced_colors_observation", lambda page, route: {}) + monkeypatch.setattr(gate, "_reduced_motion_observation", lambda page, route: {}) + monkeypatch.setattr(gate, "evaluate_forced_colors_observation", lambda observation, *, primary_route: [{"name": "forced", "passed": True, "detail": "ok"}]) + monkeypatch.setattr(gate, "evaluate_reduced_motion_observation", lambda observation: [{"name": "motion", "passed": True, "detail": "ok"}]) + + assertions = gate._media_preference_assertions(page, gate.RESEARCH_ROUTES[0]) + + restores = [item for item in assertions if item["name"] == "media_preferences_restore"] + assert len(restores) == 2 + assert all(item["passed"] is False for item in restores) + assert all("RuntimeError: restore failed" in item["detail"] for item in restores) +``` + +- [ ] **Step 6: Insert the media checks into `_measure_route`** + +Immediately after the route-specific initial assertions and before `_same_document_streamlit_rerun_assertions`, add: + +```python + assertions.extend(_media_preference_assertions(page, route)) +``` + +Update `test_route_result_includes_fail_closed_bridge_transport_fields` to monkeypatch `_media_preference_assertions` to `[]`; do not add `emulate_media` to the unrelated fake page. + +- [ ] **Step 7: Run focused gate and workflow tests** + +```bash +python3 -m pytest \ + tests/test_research_accessibility_browser_gate.py \ + tests/test_research_workspace.py \ + tests/test_research_mode_dashboard_contract.py \ + tests/test_dashboard_render_smoke.py \ + -q +``` + +Expected: PASS. + +- [ ] **Step 8: Commit the complete implementation candidate** + +```bash +git add -- src/research_accessibility_browser_gate.py tests/test_research_accessibility_browser_gate.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Verify accessibility media preferences" +``` + +- [ ] **Step 9: Run the clean-tree direct browser gate** + +```bash +make research-accessibility-browser-check TIMEOUT_SECONDS=90 +``` + +Expected: `verdict=passed`; all 12 route/viewport results contain passing forced-colors and reduced-motion assertions; the state harness remains green; the repository fingerprint is unchanged; only the existing 18 generated paths are excluded. + +Record the implementation anchor: + +```bash +git rev-parse HEAD +``` + +Do not create a JSON, timing, screenshot, or report artifact from this run. + +--- + +### Task 4: Durable Evidence, Full Verification, and Draft-PR Synchronization + +**Files:** +- Modify: `docs/ACCESSIBILITY_EVIDENCE.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Consumes: the exact Task 3 implementation SHA and direct browser output. +- Produces: durable, bounded engineering evidence and truthful next-stage routing. +- Preserves: `accessibility_manual_review_environment_required` and every other external dependency classification. + +- [ ] **Step 1: Record direct evidence without upgrading manual tasks** + +Append a dated section to `docs/ACCESSIBILITY_EVIDENCE.md` containing: + +- the exact implementation SHA; +- `make research-accessibility-browser-check TIMEOUT_SECONDS=90`; +- all six routes and both viewports; +- passing automated forced-colors and reduced-motion emulation; +- unchanged landmark, focus, navigation, authoring, dynamic-state, runtime, overflow, and repository-write checks; and +- the explicit statement that C01, C02, M01, zoom, screen-reader, assistive-technology, independent-human, hosted, and WCAG evidence remain incomplete. + +- [ ] **Step 2: Update roadmap and continuation routing** + +In Priority 7 of `ROADMAP.md`, add the implementation anchor and automated media-preference result. Keep Priority 7 incomplete and keep its external unblock condition unchanged except to clarify that automated emulation now exists but direct platform/human evidence does not. + +In `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, mark the local media-preference engineering slice complete and route the next local scan to the previously reproduced Discover primary-copy containment gap. Do not mark independent accessibility, hosted controls, source evidence, reviewers, or calibration complete. + +- [ ] **Step 3: Run documentation and full local verification before staging** + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make pilot-readiness-check TOP_N=10 +make commercial-beta-release-check +make diff-hygiene-summary +git diff --check +``` + +Expected: all commands pass; diff hygiene reports only the intentional product/docs/test files plus the same 18 unstaged generated artifacts. + +- [ ] **Step 4: Stage only durable documentation and commit** + +```bash +git add -- \ + docs/ACCESSIBILITY_EVIDENCE.md \ + ROADMAP.md \ + docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Document accessibility media evidence" +``` + +- [ ] **Step 5: Verify the exact final HEAD** + +Run the full required gate set again at the final documentation commit: + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make research-accessibility-browser-check TIMEOUT_SECONDS=90 +make public-wording-check +make public-check +make pilot-readiness-check TOP_N=10 +make commercial-beta-release-check +make diff-hygiene-summary +git diff --check +``` + +Expected: all pass; the browser gate reports all 12 route/viewport media contracts and the state harness green; generated churn remains exactly excluded and unstaged. + +- [ ] **Step 6: Push only the approved branch** + +```bash +git status --short --branch +git rev-list --left-right --count HEAD...origin/codex/personal-research-mode-mvp +git push origin codex/personal-research-mode-mvp +``` + +Expected: push succeeds and local/upstream divergence becomes `0 0`. + +- [ ] **Step 7: Update draft PR #113 without changing its draft state** + +Post a concise PR comment containing the final HEAD, implementation anchor, direct browser result, focused/full test totals, release-gate results, 18 excluded generated paths, and the manual accessibility boundary. Confirm: + +```bash +gh pr view 113 --json state,isDraft,mergeable,headRefOid,url +``` + +Expected: `state=OPEN`, `isDraft=true`, and `headRefOid` equals local `HEAD`. + +- [ ] **Step 8: Require exact-head GitHub CI** + +Identify the Commercial Research Beta run whose `headSha` equals local `HEAD`, then watch it: + +```bash +EXACT_HEAD_SHA="$(git rev-parse HEAD)" +EXACT_CI_RUN_ID="$(gh run list --branch codex/personal-research-mode-mvp --workflow "Commercial Research Beta" --limit 10 --json databaseId,headSha --jq ".[] | select(.headSha == \"$EXACT_HEAD_SHA\") | .databaseId" | head -1)" +test -n "$EXACT_CI_RUN_ID" +gh run watch "$EXACT_CI_RUN_ID" --exit-status +``` + +Expected: the exact-head run concludes `success`. Do not use an older green run as evidence. + +- [ ] **Step 9: Final hygiene and next-lane scan** + +```bash +git status --short --branch +git rev-list --left-right --count HEAD...origin/codex/personal-research-mode-mvp +make diff-hygiene-summary +git diff --check +``` + +Expected: product/code/docs/tests clean, divergence `0 0`, and exactly the known generated paths remain unstaged. Rescan the ordered roadmap and begin the first safe executable item; do not retry unchanged external dependencies. diff --git a/docs/superpowers/plans/2026-07-31-daily-momentum-valuation-research-queue.md b/docs/superpowers/plans/2026-07-31-daily-momentum-valuation-research-queue.md new file mode 100644 index 000000000..fc85f1c9c --- /dev/null +++ b/docs/superpowers/plans/2026-07-31-daily-momentum-valuation-research-queue.md @@ -0,0 +1,377 @@ +# Daily Momentum And Valuation Research Queue Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a read-only, fail-closed Discover queue for companies that satisfy the approved momentum, own-history valuation, and minimum fundamental-safeguard intersection. + +**Architecture:** A new pure module owns immutable evidence/result contracts, deterministic gate evaluation, comparison, and presentation payloads. A narrow dashboard adapter reads the selected profile, builds existing indicators and valuation packets in memory, and renders an answer-first Discover section without reading legacy ranking outputs or writing artifacts. + +**Tech Stack:** Python 3, dataclasses, pandas, existing indicator/valuation/source-rights/observation-recency modules, Streamlit, pytest. + +## Global Constraints + +- Research-only; no investment advice, recommendation, company rank, composite score, target price, probability, expected return, position sizing, transaction direction, broker integration, order routing, or auto-trading. +- Evaluate only saved `momentum_ready=true` rows. +- Require `close > SMA50 > SMA200`, positive three- and six-month returns, and positive SPY-relative return. +- Require a commercial-eligible, current Historical Valuation Regime at or below percentile `40.0`. +- Require positive free cash flow, non-negative revenue growth, and debt to equity at or below the configured quality-value threshold. +- Missing, non-finite, stale, unverified, restricted, malformed, or source-rights-ineligible evidence fails closed. +- Do not read legacy Monthly Picks, Momentum Leaders, Final Watchlist, portfolio, or action-language outputs. +- Do not write or generate CSV, JSON, reports, sample reports, screenshots, timing files, readiness, canonical data, or ledgers. +- Synthetic eligible evidence is test-only. +- Preserve independent actuals, consensus, Revenue, EPS, valuation, catalysts, outcomes, backtesting, calibration, indicator, review-metric, and readiness states. +- Stage exact intentional files only; never use `git add -A`. + +--- + +### Task 1: Pure Daily Queue Contract + +**Files:** +- Create: `src/daily_research_queue.py` +- Create: `tests/test_daily_research_queue.py` + +**Interfaces:** +- Consumes: caller-supplied `DailyQueueEvidence` records and optional prior eligible ticker strings. +- Produces: `DailyQueuePolicy`, `DailyQueueEvidence`, `DailyQueueItem`, `DailyQueueResult`, `DailyQueueComparison`, `evaluate_daily_queue(...)`, `compare_daily_queues(...)`, `daily_queue_display_rows(...)`, and `daily_queue_summary_cards(...)`. + +- [ ] **Step 1: Write failing tests for the exact approved intersection** + +Create fixture evidence with finite values and assert that one row is eligible +only when every momentum, valuation, fundamental, recency, provenance, rights, +and field-scope condition passes. Parameterize each individual false, missing, +non-finite, stale, or restricted input and assert the stable blocker code. + +```python +def test_exact_intersection_requires_every_approved_gate(): + result = evaluate_daily_queue((_eligible_evidence(),)) + assert result.eligible[0].ticker == "ALFA" + assert result.eligible[0].blockers == () + + +@pytest.mark.parametrize( + ("changes", "blocker"), + [ + ({"momentum_ready": False}, "momentum_not_ready"), + ({"close": 99.0}, "price_not_above_sma50"), + ({"sma_50": 89.0, "sma_200": 90.0}, "sma50_not_above_sma200"), + ({"return_3m": 0.0}, "three_month_return_not_positive"), + ({"return_6m": float("nan")}, "six_month_return_missing"), + ({"relative_return_vs_spy": -0.01}, "spy_relative_return_not_positive"), + ({"valuation_percentile": 40.01}, "valuation_percentile_above_threshold"), + ({"free_cash_flow": 0.0}, "free_cash_flow_not_positive"), + ({"revenue_growth": -0.001}, "revenue_growth_negative"), + ({"debt_to_equity": 2.01}, "debt_above_threshold"), + ({"current_market_eligible": False}, "current_market_evidence_ineligible"), + ({"price_provenance_eligible": False}, "price_provenance_ineligible"), + ({"price_rights_eligible": False}, "price_rights_ineligible"), + ], +) +def test_each_failed_gate_withholds(changes, blocker): + item = evaluate_daily_queue((replace(_eligible_evidence(), **changes),)).withheld[0] + assert blocker in item.blockers +``` + +- [ ] **Step 2: Run the new tests and verify red** + +Run: `python3 -m pytest tests/test_daily_research_queue.py -q` + +Expected: FAIL because `src.daily_research_queue` does not exist. + +- [ ] **Step 3: Implement immutable contracts and deterministic evaluation** + +Use frozen dataclasses. Keep blocker order fixed in code rather than deriving it +from sets. Normalize tickers to uppercase, reject duplicate tickers +deterministically, and sort output by ticker. + +```python +@dataclass(frozen=True) +class DailyQueuePolicy: + valuation_percentile_max: float = 40.0 + maximum_debt_to_equity: float = 2.0 + + +def evaluate_daily_queue( + evidence: Iterable[DailyQueueEvidence], + *, + policy: DailyQueuePolicy = DailyQueuePolicy(), +) -> DailyQueueResult: + items = tuple(sorted((_evaluate(row, policy) for row in evidence), key=lambda row: row.ticker)) + return DailyQueueResult( + status="eligible" if any(row.state == "eligible" for row in items) else "withheld", + eligible=tuple(row for row in items if row.state == "eligible"), + withheld=tuple(row for row in items if row.state == "withheld"), + boundary=QUEUE_BOUNDARY, + ) +``` + +- [ ] **Step 4: Add comparison, display, and prohibited-field tests** + +Assert `baseline_missing`, `new_today`, `still_qualifies`, and `exited_today` +behavior; alphabetic ordering; exact ticker-bound Company Workbench URLs; and +the absence of score/rank/recommendation/probability/action keys and copy. + +- [ ] **Step 5: Run focused tests** + +Run: `python3 -m pytest tests/test_daily_research_queue.py -q` + +Expected: PASS. + +- [ ] **Step 6: Commit the pure contract** + +```bash +git add -- src/daily_research_queue.py tests/test_daily_research_queue.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add fail-closed daily research queue" +``` + +### Task 2: Selected-Profile Evidence Adapter + +**Files:** +- Create: `src/daily_research_queue_adapter.py` +- Create: `tests/test_daily_research_queue_adapter.py` + +**Interfaces:** +- Consumes: explicit `project_root: Path`, `data_dir: Path`, `as_of: date`, and optional `DailyQueuePolicy`. +- Produces: `build_daily_research_queue_from_files(...) -> DailyQueueResult` and `DailyQueueBuildStatus` with source/coverage diagnostics. +- Uses from Task 1: `DailyQueueEvidence`, `DailyQueuePolicy`, and `evaluate_daily_queue`. + +- [ ] **Step 1: Write failing adapter tests with temporary files** + +Build temporary readiness, prices, fundamentals, universe, historical +valuation, config, and source-rights inputs. Assert the adapter: + +- reads only `momentum_ready=true` tickers; +- uses `build_indicator_snapshot`, not legacy outputs; +- creates eligible synthetic evidence only when every lineage/right is present; +- returns an empty/withheld result for an absent valuation ledger; +- withholds stale ticker, profile, or SPY observations; +- withholds missing price lineage fields; +- withholds unsupported fundamental field scope; +- isolates malformed ticker evidence; and +- leaves a byte snapshot of the temporary tree unchanged. + +- [ ] **Step 2: Run the adapter tests and verify red** + +Run: `python3 -m pytest tests/test_daily_research_queue_adapter.py -q` + +Expected: FAIL because the adapter does not exist. + +- [ ] **Step 3: Implement defensive read-only loading** + +Read CSVs with one `optional_csv()` helper that returns an empty frame on +missing, malformed, encoding, or OS errors. Normalize tickers and numeric +fields. Build the indicator snapshot once. Group valuation observations by +ticker before building packets. Evaluate recency from explicit `as_of`; never +use file mtimes. + +Price commercial eligibility requires explicit row-level `source`, +`source_ref`, and `retrieved_at`, an approved source-rights entry, and supported +`prices` scope. Fundamental eligibility requires explicit source/reference +evidence and approved registered scope for every required fundamental input. + +- [ ] **Step 4: Run adapter tests** + +Run: `python3 -m pytest tests/test_daily_research_queue_adapter.py tests/test_daily_research_queue.py -q` + +Expected: PASS. + +- [ ] **Step 5: Commit the adapter** + +```bash +git add -- src/daily_research_queue_adapter.py tests/test_daily_research_queue_adapter.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Build daily queue from reviewed evidence" +``` + +### Task 3: Discover Integration + +**Files:** +- Modify: `src/dashboard.py` +- Modify: `tests/test_research_mode_dashboard_contract.py` +- Modify: `tests/test_dashboard_helpers.py` + +**Interfaces:** +- Consumes from Task 2: `build_daily_research_queue_from_files(...)`. +- Consumes from Task 1: `daily_queue_display_rows(...)`, `daily_queue_summary_cards(...)`. +- Produces: `load_dashboard_daily_research_queue(...)` and `render_daily_research_queue(...)`. + +- [ ] **Step 1: Write failing dashboard contract tests** + +Assert that Discover: + +- renders `Daily Momentum & Valuation Research Queue` before the general stock + selector; +- renders current eligible rows or a truthful empty/withheld state; +- places blocker detail under `Advanced: daily queue evidence`; +- uses `?mode=research&page=company-workbench&ticker=`; +- contains no rank, score, recommendation, target, probability, expected + return, buy/sell, or portfolio-action language; and +- does not call a writer or legacy output loader. + +- [ ] **Step 2: Run dashboard tests and verify red** + +Run: + +```bash +python3 -m pytest \ + tests/test_research_mode_dashboard_contract.py \ + tests/test_dashboard_helpers.py \ + -q +``` + +Expected: FAIL because the Discover queue functions and markers are absent. + +- [ ] **Step 3: Add the dashboard loader and renderer** + +The loader passes `review_date` explicitly and returns a fail-closed result on +adapter exceptions. The renderer leads with current eligibility, uses +alphabetical rows, limits first-screen copy, and keeps technical blockers in a +collapsed Advanced expander. It renders one ticker-bound Workbench action per +eligible row and does not expose a company score. + +- [ ] **Step 4: Integrate before the stock selector** + +Call the queue loader and renderer immediately after +`## Which stock can I review?`, then retain the existing general selector and +cohort context unchanged. + +- [ ] **Step 5: Run focused dashboard and queue tests** + +Run: + +```bash +python3 -m pytest \ + tests/test_daily_research_queue.py \ + tests/test_daily_research_queue_adapter.py \ + tests/test_research_mode_dashboard_contract.py \ + tests/test_dashboard_helpers.py \ + -q +``` + +Expected: PASS. + +- [ ] **Step 6: Commit Discover integration** + +```bash +git add -- src/dashboard.py tests/test_research_mode_dashboard_contract.py tests/test_dashboard_helpers.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Show daily research queue in Discover" +``` + +### Task 4: Methodology, Roadmap, And Continuation Contract + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `docs/METHODOLOGY.md` +- Modify: `docs/PERSONAL_RESEARCH_MODE.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` + +**Interfaces:** +- Documents the exact contract and current empty/withheld real-data state. +- Does not change runtime behavior or generated data. + +- [ ] **Step 1: Update methodology and user workflow documentation** + +Document the strict intersection, alphabetical non-ranking, no-baseline +behavior, exact-source/current-observation gates, empty-state truth, and +Company Workbench handoff. + +- [ ] **Step 2: Update roadmap truth** + +Record the local feature implementation separately from its external +operational gates. State that an absent historical valuation ledger and +unapproved commercial price history prevent real daily candidates. + +- [ ] **Step 3: Update continuation instructions** + +Set the next executable lane to direct browser verification and one bounded +permitted price/valuation evidence activation path. Require skipping unavailable +providers once and forbid generated data churn. + +- [ ] **Step 4: Run wording and whitespace checks** + +Run: + +```bash +make public-wording-check +git diff --check +``` + +Expected: PASS. + +- [ ] **Step 5: Commit documentation** + +```bash +git add -- ROADMAP.md docs/METHODOLOGY.md docs/PERSONAL_RESEARCH_MODE.md docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md +make staged-hygiene-check +git diff --cached --check +git commit -m "Document daily research queue boundaries" +``` + +### Task 5: Full Verification And PR Synchronization + +**Files:** +- Modify only if a verified defect is found in the intentional feature scope. + +**Interfaces:** +- Produces exact-tree local and GitHub evidence. + +- [ ] **Step 1: Run focused verification** + +```bash +python3 -m pytest \ + tests/test_daily_research_queue.py \ + tests/test_daily_research_queue_adapter.py \ + tests/test_research_mode_dashboard_contract.py \ + tests/test_dashboard_helpers.py \ + -q +``` + +- [ ] **Step 2: Run the complete suite and product gates** + +```bash +python3 -m pytest tests -q +make dashboard-smoke +make research-dashboard-render-smoke +make public-wording-check +make public-check +make commercial-beta-check +make commercial-beta-performance-gate +make commercial-beta-release-check +make pilot-readiness-check TOP_N=10 +make diff-hygiene-summary +git diff --check +``` + +- [ ] **Step 3: Run direct browser verification** + +Run: + +```bash +make research-accessibility-browser-check TIMEOUT_SECONDS=90 +``` + +The queue must render at desktop and phone widths without traceback, +horizontal overflow, duplicate ticker actions, or non-ticker-bound Workbench +links. If the managed browser cannot launch, classify that environment once; +do not claim browser completion. + +- [ ] **Step 4: Verify staged and generated-artifact hygiene** + +Confirm only the pre-existing 18 generated CSV/output paths remain unstaged. +Run `make staged-hygiene-check` if anything is staged. Never stage them. + +- [ ] **Step 5: Push and update draft PR** + +Push only `codex/personal-research-mode-mvp`, update PR #113 with the feature, +truthful withheld-data state, verification, and external unblock conditions, +keep it draft, and require exact-head CI. + +- [ ] **Step 6: Record the handoff** + +Report repository/PR state, product stage, feature behavior, current real-data +result, tests, commit/push status, generated exclusions, external dependencies, +remaining gates, exact next step, and review safety. diff --git a/docs/superpowers/plans/2026-07-31-performance-sampling-reconciliation.md b/docs/superpowers/plans/2026-07-31-performance-sampling-reconciliation.md new file mode 100644 index 000000000..e67b58347 --- /dev/null +++ b/docs/superpowers/plans/2026-07-31-performance-sampling-reconciliation.md @@ -0,0 +1,100 @@ +# Commercial Beta Performance Sampling Reconciliation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make Commercial Research shell and first-useful performance evidence category-correct by independently measuring and enforcing warm p90 and cold maximum at the unchanged one-second and three-second thresholds. + +**Architecture:** Keep the existing browser runner and raw samples. Partition shell and first-useful timings by `run_kind` in the summary, then make the evaluator enforce all category-specific fields while preserving existing sample-count and full-settle rules. + +**Tech Stack:** Python 3.12, pytest, Playwright, Streamlit, Make. + +## Global Constraints + +- Keep `shell_seconds = 1.0` and `first_useful_seconds = 3.0`; do not weaken any threshold. +- Do not add retries, fastest-run selection, outlier deletion, or discarded required samples. +- Do not change route markers, readiness, research calculations, or source-rights behavior. +- Keep generated CSV, JSON, report, sample-report, screenshot, timing, canonical-data, and manual-review churn unstaged. +- Stage exact paths only; never use `git add -A`. +- Keep PR #113 open and draft; do not merge or deploy. + +--- + +### Task 1: Separate and enforce first-useful evidence + +**Files:** +- Modify: `src/public_performance_gate.py` +- Test: `tests/test_public_performance_gate.py` + +**Interfaces:** +- Consumes: `RouteTimingSample`, `_timing_values()`, `nearest_rank_percentile()`, `PerformanceThresholds.first_useful_seconds`. +- Produces: summary fields `warm_shell_p90_seconds: float | None`, `cold_shell_max_seconds: float | None`, `warm_first_useful_p90_seconds: float | None`, and `cold_first_useful_max_seconds: float | None` consumed by `evaluate_performance_gate()` and the JSON payload. + +- [x] **Step 1: Write the failing aggregation and evaluation test** + +Add one test with one cold shell value of `1.1` seconds, one cold first-useful value of `3.1` seconds, and five warm values below both limits. Assert the summary exposes `warm_shell_p90_seconds == 0.3`, `cold_shell_max_seconds == 1.1`, `warm_first_useful_p90_seconds == 1.5`, and `cold_first_useful_max_seconds == 3.1`, with neither ambiguous combined field. Assert the unchanged thresholds fail specifically with `cold shell max 1.100s exceeds 1.000s` and `cold first-useful max 3.100s exceeds 3.000s`. + +- [x] **Step 2: Run the focused test and confirm the red state** + +Run `python3 -m pytest tests/test_public_performance_gate.py -q`. + +Expected: fail because the two category-specific summary fields do not yet exist. + +- [x] **Step 3: Implement the minimal category split** + +In `summarize_route_timings()`, collect warm and cold shell and first-useful values independently. Emit nearest-rank p90 for warm values and maximum for cold values. In `evaluate_performance_gate()`, replace both combined checks with independent warm-p90 and cold-max checks using the existing shell and first-useful thresholds. + +- [x] **Step 4: Run focused tests green** + +Run `python3 -m pytest tests/test_public_performance_gate.py -q`. + +Expected: all focused tests pass. + +- [x] **Step 5: Run a controlled browser performance result** + +Run `make commercial-beta-performance-gate TIMEOUT_SECONDS=90` once from the verified worktree. Inspect all category-specific failures. Do not rerun an unchanged failure. + +Expected: the result truthfully passes or identifies a reproducible warm or cold category for later optimization; the temporary JSON remains outside the repository. + +Verified result: commit `6328c8cea` recorded 48 successful category-separated samples with zero failures on local Chrome. The temporary JSON stayed under `/tmp` and out of Git. + +### Task 2: Reconcile documentation and release evidence + +**Files:** +- Modify: `docs/PERFORMANCE_RELEASE_GATE.md` +- Modify: `ROADMAP.md` +- Modify: `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md` +- Test: `tests/test_public_performance_gate.py` + +**Interfaces:** +- Consumes: the Task 1 metric names and current controlled-browser verdict. +- Produces: one unambiguous operator contract and truthful roadmap/continuation state. + +- [x] **Step 1: Protect documentation terminology** + +Update the existing documentation-contract test to require `warm first-useful p90`, `cold first-useful max`, and the unchanged three-second threshold in `docs/PERFORMANCE_RELEASE_GATE.md`. + +- [x] **Step 2: Run the documentation test red** + +Run `python3 -m pytest tests/test_public_performance_gate.py -q`. + +Expected: fail until the performance document uses the new terms. + +- [x] **Step 3: Update current truth** + +Document the category split, the controlled-browser result, and its exact limits in `docs/PERFORMANCE_RELEASE_GATE.md`. Update `ROADMAP.md` and the continuation contract with the verified performance status. Preserve all external data, reviewer, hosted, calibration, and operating blockers. + +- [x] **Step 4: Run complete verification** + +Run focused tests, `python3 -m pytest tests -q`, dashboard and research render checks, public wording and package checks, commercial-beta performance/release checks, pilot readiness, accessibility checks, `make diff-hygiene-summary`, and `git diff --check`. + +Expected: all applicable local gates pass, or a category-specific product performance failure remains explicitly open without retry loops. + +Verified result: the focused contract, 4,474-test full suite, dashboard startup and render checks, research render checks, public wording/package checks, category-separated performance gate, aggregate Commercial Research Beta release check, pilot classification, six-route/two-viewport accessibility browser gate, state harness, diff hygiene, and whitespace checks passed. Exactly 18 generated differences remained excluded. + +- [x] **Step 5: Stage exact paths, verify hygiene, commit, and push** + +Stage only the intentional source, test, specification, plan, roadmap, and documentation paths. Run `make staged-hygiene-check` and `git diff --cached --check`. Commit one coherent slice, push only `codex/personal-research-mode-mvp`, update draft PR #113, and require exact-head CI. + +Expected: the branch is aligned with origin, PR #113 remains draft, exact-head CI passes, and all 18 pre-existing generated differences remain unstaged. + +Verified result: exact staging and hygiene passed, commit `6328c8cea` was pushed only to `codex/personal-research-mode-mvp`, PR #113 remained open/draft/mergeable, exact-head GitHub Actions run `30634355602` passed, and all 18 generated differences remained unstaged. diff --git a/docs/superpowers/plans/2026-08-01-company-workbench-html-research-brief.md b/docs/superpowers/plans/2026-08-01-company-workbench-html-research-brief.md new file mode 100644 index 000000000..08bdd9b05 --- /dev/null +++ b/docs/superpowers/plans/2026-08-01-company-workbench-html-research-brief.md @@ -0,0 +1,844 @@ +# Company Workbench HTML Research Brief Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a secure, evidence-gated Company Workbench HTML Research Brief that previews and downloads from memory without creating repository artifacts or duplicating the Python valuation engine. + +**Architecture:** Extend the authoritative Python DCF result with its already-computed explicit-period subtotal. Build one immutable, sanitized presentation snapshot from existing Workbench objects. Render that snapshot as either a scoped Streamlit fragment or a complete offline document. Prepare one fail-closed Scenario Lab session result before the brief and reuse it in the existing detailed controls. + +**Tech Stack:** Python 3.10+, frozen dataclasses, standard-library escaping/hashing/URL parsing, pandas, Streamlit 1.52 to less than 2, pytest 8+, Playwright with the repository's existing Chrome discovery. + +## Non-Negotiable Boundaries + +- Start from `docs/superpowers/specs/2026-07-31-company-workbench-html-research-brief-design.md` and current branch truth. +- Keep Streamlit, the four-route workflow, saved-data architecture, readiness model, and Python calculations authoritative. +- The HTML layer performs no DCF, sensitivity, momentum, historical-valuation, readiness, ranking, expected-return, or probability calculation. +- Use `Scenario value/share`; never produce target price, recommendation, ranking, transaction, position, allocation, upside/downside, margin-of-safety, or action language. +- Preserve independent actuals, consensus, Revenue, EPS, valuation, peer, historical-valuation, catalyst, outcome, backtesting, and calibration states. +- A calculated DCF status does not unlock equity or per-share output without the corresponding existing bridge inputs and results. +- Display `shares_outstanding` only as `Shares outstanding used by existing model`; disclose its basis as unverified unless explicit proof exists. +- Candidate context cannot alter deterministic scenarios or become trusted evidence. Q4 and EPS split-basis boundaries remain unchanged. Synthetic fixtures stay test-only. +- The renderer accepts no output path and performs no file, network, refresh, readiness, import, apply, report, ledger, screenshot, timing, or external-service write. +- No executable JavaScript, form, iframe, image, font, external stylesheet, analytics, or runtime network request. +- Preserve the 18 pre-existing dirty generated CSV/report paths byte-for-byte and keep them unstaged. Never use `git add -A`. +- Do not run `make readiness`, `make pipeline`, `make verify`, `make validate-all`, broad refresh/import/apply commands, report writers, screenshot writers, or timing writers. +- Keep PR #113 open and draft. Do not merge or deploy. + +## File Map + +- Modify `src/valuation.py`, `tests/test_valuation.py`. +- Create `src/company_workbench_html.py`, `tests/test_company_workbench_html.py`. +- Create `src/scenario_lab_session.py`, `tests/test_scenario_lab_session.py`. +- Modify `src/dashboard.py`, `tests/test_dashboard_helpers.py`, `tests/test_dashboard_render_smoke.py`, `tests/test_research_mode_dashboard_contract.py`. +- Modify `scripts/public_wording_check.py`, `tests/test_public_wording_check.py`, `tests/test_diff_hygiene.py`. +- Create `src/company_workbench_html_browser_gate.py`, `tests/test_company_workbench_html_browser_gate.py`; modify `Makefile`. +- Modify `.github/workflows/commercial-research-beta.yml` and `tests/test_github_actions_workflow.py` so exact-head CI installs the declared browser-test dependency and Chromium before the full suite. +- Modify `README.md`, `docs/METHODOLOGY.md`, `ROADMAP.md`, `docs/ACCESSIBILITY_EVIDENCE.md`, `docs/DASHBOARD_QA.md`, `docs/internal/COMMERCIAL_RESEARCH_BETA_CONTINUATION_GOAL_PROMPT.md`, `tests/test_public_v1_release_docs.py`, and `tests/test_launchers.py`. + +## Preflight Protected-Artifact Baseline + +Before Task 1, verify branch/remote/PR truth and the exact dirty-path set. Local commits may be ahead of origin, but origin must not contain commits absent locally. Stop before editing if any assertion fails. + +```bash +git fetch origin codex/personal-research-mode-mvp +test "$(git branch --show-current)" = "codex/personal-research-mode-mvp" +test "$(git rev-list --left-only --count origin/codex/personal-research-mode-mvp...HEAD)" = "0" +test "$(gh pr view 113 --json state --jq .state)" = "OPEN" +test "$(gh pr view 113 --json isDraft --jq .isDraft)" = "true" +test "$(gh pr view 113 --json headRefName --jq .headRefName)" = "codex/personal-research-mode-mvp" +python3 - <<'PY' +import subprocess + +expected = { + "data/analyst_estimates_readiness.csv", + "data/dcf_readiness.csv", + "data/earnings_readiness.csv", + "data/price_coverage_report.csv", + "data/reports/analyst_estimates_readiness_report.csv", + "data/reports/data_source_status.csv", + "data/reports/dcf_readiness_report.csv", + "data/reports/earnings_readiness_report.csv", + "data/reports/feature_readiness_summary.csv", + "data/reports/fundamentals_coverage_report.csv", + "data/reports/peer_readiness_report.csv", + "data/reports/peer_unlock_worklist.csv", + "data/reports/price_coverage_report.csv", + "data/reports/ticker_readiness_report.csv", + "data/reports/universe_coverage_report.csv", + "data/universe_master.csv", + "outputs/feature_readiness_summary.csv", + "outputs/peer_unlock_worklist.csv", +} +rows = subprocess.check_output( + ["git", "status", "--porcelain=v1", "--untracked-files=all"], + text=True, +).splitlines() +actual = {row[3:] for row in rows} +if actual != expected: + raise SystemExit(f"unexpected preflight paths: {sorted(actual ^ expected)}") +PY +``` + +Then record the output of this read-only command in the execution handoff. Re-run it after every task before committing and after the final verification matrix; every hash must remain identical to the pre-Task-1 baseline. + +```bash +shasum -a 256 \ + data/analyst_estimates_readiness.csv \ + data/dcf_readiness.csv \ + data/earnings_readiness.csv \ + data/price_coverage_report.csv \ + data/reports/analyst_estimates_readiness_report.csv \ + data/reports/data_source_status.csv \ + data/reports/dcf_readiness_report.csv \ + data/reports/earnings_readiness_report.csv \ + data/reports/feature_readiness_summary.csv \ + data/reports/fundamentals_coverage_report.csv \ + data/reports/peer_readiness_report.csv \ + data/reports/peer_unlock_worklist.csv \ + data/reports/price_coverage_report.csv \ + data/reports/ticker_readiness_report.csv \ + data/reports/universe_coverage_report.csv \ + data/universe_master.csv \ + outputs/feature_readiness_summary.csv \ + outputs/peer_unlock_worklist.csv +git status --short --branch +``` + +After each comparison, run `make diff-hygiene-summary` to detect any new generated or manually reviewed artifact path. Stop that slice if a protected hash changes or a new artifact appears; do not absorb it into a later baseline. + +--- + +### Task 1: Authoritative DCF Subtotal And Immutable Snapshot + +**Files:** +- Modify: `src/valuation.py` +- Modify: `tests/test_valuation.py` +- Create: `src/company_workbench_html.py` +- Create: `tests/test_company_workbench_html.py` + +**Primary interface:** `build_company_workbench_html_snapshot(inputs: CompanyWorkbenchHtmlInputs) -> CompanyWorkbenchHtmlSnapshot`. + +- [ ] **Step 1: Add a failing authoritative-subtotal test** + +Add a test named `test_calculate_dcf_exposes_authoritative_discounted_explicit_total`. Calculate one complete DCF and assert: + +```python +assert result.discounted_explicit_total == pytest.approx(sum(result.discounted_fcfs)) +assert result.enterprise_value == pytest.approx( + result.discounted_explicit_total + result.discounted_terminal_value +) +``` + +Run: + +```bash +python3 -m pytest tests/test_valuation.py::test_calculate_dcf_exposes_authoritative_discounted_explicit_total -q +``` + +Expected: fail because `DCFResult` has no `discounted_explicit_total` field. + +- [ ] **Step 2: Expose the subtotal inside the authoritative calculation** + +Add `discounted_explicit_total: float | None = None` to `DCFResult`. In `calculate_dcf`, assign `sum(discounted_fcfs)` once and use that named value in `enterprise_value = discounted_explicit_total + discounted_terminal_value`. Do not change any formula, rounding, default scenario, or readiness rule. + +Run: + +```bash +python3 -m pytest tests/test_valuation.py -q -k 'discounted_explicit_total or dcf' +``` + +Expected: the new test and existing DCF tests pass. + +- [ ] **Step 3: Write failing snapshot-contract tests before creating the module** + +Create synthetic test-only fixtures in `tests/test_company_workbench_html.py`. Tests must cover: + +- exact copying of projected FCFs, discounted FCFs, explicit subtotal, terminal values, enterprise value, equity value, per-share value, and every sensitivity cell; +- enterprise value available while equity/per-share are withheld, with overall bridge state `partial`; +- a malicious payload containing equity/per-share values but neither net debt nor both cash and debt; those values remain withheld; +- a non-calculated DCF payload containing populated projected/discounted FCF, subtotal, terminal, enterprise, equity, and per-share values exposes none of them; +- missing, zero, negative, NaN, or infinite shares withhold per-share output; +- share label and unverified basis wording; +- canonical Bear/Base/Bull order; +- a modified Base accepted only when result status is calculated, ticker/profile match, input identity is present, and changed assumptions are non-empty; +- stale ticker, stale profile, empty identity, or unchanged Scenario Lab results rejected without affecting canonical scenarios; +- canonical sensitivity copied from `valuation_snapshot.sensitivity_table`; accepted modified Base sensitivity copied from `ScenarioLabResult.sensitivity_table`; +- a legacy/malicious finite sensitivity grid remains withheld whenever its owning canonical or modified Base bridge has `per_share_state != "available"`; +- exhaustive independent lane states for actuals, consensus, Revenue, EPS, valuation, peers, historical valuation, catalysts, outcomes, backtesting, and calibration; +- generic analyst-estimate availability cannot unlock consensus, backtesting, calibration, or probability without a matching source-backed point-in-time `nowcast_packet`; +- a same-ticker nowcast packet for a different fiscal period, with an invalid `as_of_timestamp`, or dated after either the report `generated_at` or the brief `review_cutoff` is rejected and withholds consensus, backtesting, calibration, and probability; +- a malicious packet with `probability_available=True`, at least 100 events, no failed gates, and an invented probability still emits no numerical Beat/Miss probability because the current packet has no separately approved probability-producing field contract; +- even a matching consensus-ready packet caps consensus, backtesting, and calibration at `partial` with `portable nowcast provenance incomplete`; reference-like `source_ids` and the forecast cutoff cannot be relabeled as source ID, retrieval time, rights, or field scope; +- every profile-bearing input/event must match `profile_context.profile_key`, every ticker-bearing input must match the report ticker, and mismatched selected answer, recency, journal, Decision Lab, catalyst, Forward View, valuation regime, quarterly trend, or nowcast evidence is withheld; +- the independent outcome lane is always `withheld` in version 1 with `portable outcome scope and provenance incomplete`; no `OutcomeStatus` text, count, or state is accepted into the snapshot, while the separately profile-and-ticker-scoped Decision Lab learning lane remains independent; +- empty valuation, catalyst, thesis, outcome, consensus, peer, and quarterly-actuals lanes shown as empty/withheld, never fixture text; +- explicit Q4, EPS split-basis, candidate-context, calibration, and synthetic-fixture boundaries; +- source references, private paths, secrets, unsafe URLs, and reviewer-authored action language fail closed; +- deterministic filename and snapshot identity; +- no loader, calculator, file, network, refresh, readiness, report, or ledger call. + +Add a concrete order test for snapshot section keys: + +```python +assert [row.key for row in snapshot.research_sections] == [ + "business-trend", + "key-drivers", + "risks", + "catalysts", + "evidence-gaps", + "valuation-regime", +] +assert [row.key for row in snapshot.decision_lanes] == [ + "plan", + "evidence", + "invalidation", + "scenario", + "review-trigger", + "learning", +] +``` + +Run: + +```bash +python3 -m pytest tests/test_company_workbench_html.py -q +``` + +Expected: collection fails because `src.company_workbench_html` does not exist. + +- [ ] **Step 4: Define exact frozen presentation contracts** + +Create these frozen dataclasses in `src/company_workbench_html.py`; every collection field is a tuple containing only frozen dataclasses, strings, booleans, integers, finite floats, or `None`: + +- `HtmlBriefSafeReference(label: str, href: str)`. +- `HtmlBriefAnswer(label: str, title: str, body: str, state: str, badges: tuple[str, ...])`. +- `HtmlBriefDcfBridge(state: str, enterprise_state: str, equity_state: str, per_share_state: str, explicit_total_state: str, projected_fcfs: tuple[float, ...], discounted_fcfs: tuple[float, ...], discounted_explicit_total: float | None, terminal_value: float | None, discounted_terminal_value: float | None, enterprise_value: float | None, cash: float | None, debt: float | None, net_debt: float | None, equity_value: float | None, shares_outstanding: float | None, shares_label: str, share_basis_state: str, scenario_value_per_share: float | None, currency: str, blockers: tuple[str, ...])`. +- `HtmlBriefScenario(name: str, state: str, modified: bool, method_name: str, revenue_growth: float | None, fcf_margin: float | None, wacc: float | None, terminal_growth: float | None, forecast_years: int | None, bridge: HtmlBriefDcfBridge)`. +- `HtmlBriefSensitivity(state: str, wacc_values: tuple[float, ...], terminal_growth_values: tuple[float, ...], value_grid: tuple[tuple[float | None, ...], ...], blockers: tuple[str, ...])`. +- `HtmlBriefSection(key: str, title: str, state: str, answer: str, facts: tuple[tuple[str, str], ...], blockers: tuple[str, ...])`. +- `HtmlBriefEvidenceRow(section: str, state: str, source_id: str, source_ref: HtmlBriefSafeReference, as_of: str, retrieved_at: str, rights_state: str, field_scope_state: str, model_identity: str, input_identity: str, blockers: tuple[str, ...])`. +- `CompanyWorkbenchHtmlInputs(report_payload: Mapping[str, object], profile_context: ProfileContext, observation_recency: ObservationRecencySet | None, selected_answer: Mapping[str, object], authoritative_task: Mapping[str, object], scenario_lab_result: ScenarioLabResult | None, nowcast_packet: Mapping[str, object] | None, decision_lab_state: ResearchDecisionLabState, quarterly_trend: QuarterlyTrendPacket, forward_view: ForwardViewPacket, journal_state: JournalState | None, valuation_regime: ValuationRegimePacket, catalyst_timeline: CatalystTimeline)`. +- `CompanyWorkbenchHtmlSnapshot(ticker: str, profile_label: str, review_cutoff: str, source_as_of: str, generated_at: str, model_version: str, freshness_state: str, rights_state: str, boundary: str, answers: tuple[HtmlBriefAnswer, ...], recency: HtmlBriefSection, readiness_lanes: tuple[HtmlBriefSection, ...], scenarios: tuple[HtmlBriefScenario, ...], sensitivity: HtmlBriefSensitivity, research_sections: tuple[HtmlBriefSection, ...], decision_lanes: tuple[HtmlBriefSection, ...], evidence_rows: tuple[HtmlBriefEvidenceRow, ...], blockers: tuple[str, ...], identity: str)`. + +Copy every consumed list/dict into the primitive tuple contracts above before returning. Add a mutation-isolation test that mutates the original report, source metadata, and reviewer object after construction and proves snapshot content and identity do not change. Do not place profile key, data/output directories, `snapshot_inputs`, observation `source_path`, raw source mappings, analyst recommendation/target fields, or hidden session state in the snapshot. + +- [ ] **Step 5: Implement exhaustive state, text, and source normalization** + +Add `normalize_html_brief_state(value: object) -> str` with this exact mapping: + +- `available`: `available`, `ready`, `calculated`, `current`, `supported`, `complete`, `usable_now`, `documented`, `reviewable`, `reviewed`, `review_current`, `evidence_recorded`, `process_documented`, `thesis_documented`, `invalidation_documented`, `baseline_ready`, `backtest_ready`, `signal_context_ready`, `probability_available`. +- `partial`: `partial`, `incomplete`, `conflict_review_needed`, `overdue_review`, `scheduled_review`, `review_now`. +- `stale`: `stale`, `stale_review_only`, `stale_or_unknown`. +- `not_recorded`: `not_recorded`, `not recorded`, `not_started`, `empty`, `missing`. +- `excluded`: `excluded`, `not_applicable`, `candidate_context_only`. +- `withheld`: `withheld`, `blocked`, `still_blocked`, `commercial_evidence_blocked`, `unavailable`, `insufficient_data`, `insufficient_history`, `not_supported`, `unverified`, `rejected`, an empty value, and every unknown value. + +Add `safe_html_brief_text(value) -> str` and `safe_html_brief_reference(value) -> HtmlBriefSafeReference`: + +- reject control characters, absolute/local/repository paths, path traversal, credential-like values, tokens, cookies, and secret-like key/value fragments; +- replace direct buy/sell/short/hold, position-size, allocation, stop-loss/take-profit, order, broker, ranking, target-price, expected-return, upside/downside, and margin-of-safety instructions with one explicit withheld message instead of echoing the text; +- preserve safe plain identifiers such as an SEC accession as escaped text with `href=""`; +- expose an href only for `https` with a hostname, no userinfo, query, fragment, control character, or secret-like path; +- reject `javascript:`, `data:`, `file:`, `vbscript:`, protocol-relative URLs, and every other active/non-HTTPS URL form. + +Apply these functions to every dynamic string before it reaches a frozen snapshot, not only source references. + +- [ ] **Step 6: Implement independent DCF bridge gates and scenario selection** + +For each supplied DCF result: + +1. No projected/discounted FCF, explicit subtotal, terminal, enterprise, equity, or per-share number is displayable unless `dcf_result.status == "calculated"`. Within a calculated result, copy only finite existing numeric fields; never recompute a missing field. +2. `enterprise_state` is available only when status is calculated and enterprise value is finite. +3. `explicit_total_state` is available only when status is calculated and the new authoritative field is finite. Never sum discounted FCFs in the snapshot or renderer. +4. The equity bridge is eligible only when assumptions contain finite net debt, or both finite cash and finite debt. `equity_state` is available only when that bridge is eligible and the supplied equity value is finite. +5. `per_share_state` is available only when equity is available, shares are finite and greater than zero, and supplied per-share value is finite. +6. Overall bridge state is available when all three stages are available, partial when at least one stage is available, otherwise withheld. +7. Keep an exact blocker for each unavailable stage. An upstream available stage remains visible. + +Accept a Scenario Lab result only under all matching conditions from Step 3. Use it only for the Base scenario. Pass both `report_payload` and the accepted result to sensitivity selection; use the accepted result's grid for modified Base and the canonical report grid otherwise. Validate dimensions and finite cells without recalculation. A sensitivity grid is available only when the owning canonical or modified Base bridge has `per_share_state == "available"`; otherwise withhold the entire grid with an exact bridge blocker. Bear and Bull always remain canonical. + +- [ ] **Step 7: Build the complete snapshot and identity** + +Implement `build_company_workbench_html_snapshot` with this deterministic precedence and no other source reads: + +- `ticker`: normalized `report_payload["ticker"]`; `selected_answer["Ticker"]`, observation selected scope, nowcast, quarterly trend, Forward View, journal, Decision Lab, valuation regime, and catalyst timeline must match. Any mismatch adds a blocker and withholds that object and its evidence rows. +- `profile scope`: journal, Decision Lab, every catalyst event, and accepted Scenario Lab result must equal `profile_context.profile_key`. Any mismatch is withheld. Profile key is used only for matching and is never copied into the portable snapshot. +- `generated_at`: valid ISO timestamp from `report_payload["generated_at"]`; otherwise `not recorded`. Never use wall-clock time. +- `review_cutoff`: valid `forward_view.source_cutoff`, else the date/time part of `generated_at`, else valid `profile_context.source_as_of`, else `not recorded`. +- `source_as_of`: `profile_context.source_as_of` only. +- `model_version`: `report_payload["method_version"]`, else `report_payload["provenance"]["method_version"]`, else `not recorded`. +- `currency`: `report_payload["financial_summary"]["currency"]`, else `price_snapshot.currency`, else `not recorded`. +- `freshness_state`: normalize the selected-ticker observation state when present; otherwise normalize `profile_context.freshness_state`. +- `rights_state`: derive only from emitted evidence rows. An empty evidence set is `withheld`. Any `restricted`, `unverified`, missing, or inconsistent rights/scope row makes the rollup `withheld`; a non-empty set in which every row is `not_applicable` makes it `excluded`; only a non-empty set whose applicable rows are all `permitted` can be `available`. This rollup never unlocks a lane. +- `share_basis_state`: `unverified` in version 1 because current report/DCF contracts contain shares outstanding but no explicit share-basis proof. Never infer or relabel diluted shares. + +Map inputs to presentation rows as follows: + +- Primary answers: selected-answer columns `Use Now` and `Still Blocked`, then the authoritative task; render labels `Usable now`, `Still withheld`, `Next research task` in that order. +- Recency: selected ticker state, through date, age/message, policy days, and evaluation as-of; omit `source_path`. +- Readiness lanes in fixed order `actuals`, `consensus`, `revenue`, `eps`, `valuation`, `peers`, `historical-valuation`, `catalysts`, `outcomes`, `backtesting`, `calibration`. Actuals/Revenue/EPS come from `QuarterlyTrendPacket`; valuation uses DCF field gates; peers use `ForwardViewPacket.peer_context`; historical valuation and catalysts use their supplied domain objects. The independent `outcomes` lane is always `withheld` in version 1 with `portable outcome scope and provenance incomplete`; no `OutcomeStatus` field is accepted or copied. +- Consensus, backtesting, and calibration use only a matching `nowcast_packet` whose `evidence_scope == "source_backed_preview_only"`. Matching requires normalized packet ticker equality, exact equality between `nowcast_packet["fiscal_period"]` and `report_payload["earnings_summary"]["fiscal_period"]`, and a valid packet `as_of_timestamp` no later than every valid applicable boundary among the report `generated_at` and computed brief `review_cutoff`; a missing earnings fiscal period, invalid timestamp, different period, or post-cutoff packet withholds all three lanes. Consensus also requires `readiness.consensus_ready is True`; analyst-estimate availability in the report is never sufficient. Backtesting may copy only packet verdict/count/blockers, and calibration may copy only state/event count/gates. Because the current packet exposes reference-like source IDs and a forecast cutoff but no distinct source ID, retrieval timestamp, rights state, or field scope, each matching lane is capped at `partial` with `portable nowcast provenance incomplete`; otherwise it is withheld. Version 1 always withholds numerical Beat/Miss probability: `probability_available`, event counts, clean gates, or any uncontracted probability-like packet key cannot expose a number. A missing or synthetic packet withholds all three lanes independently. +- Research sections in the exact Step 3 order use quarterly/Forward View/report risk/catalyst/valuation-regime objects. Empty objects produce explicit empty/withheld rows. +- Decision lanes copy the supplied six `ResearchDecisionLabState.lanes` by key in fixed order, preserving each independent state. +- Evidence rows use only sanitized `provenance.source_records`, `valuation_snapshot.source_metadata`, accepted Scenario Lab `source_metadata`/`input_identity`, journal entries, and catalyst events. Deduplicate by the full frozen row while preserving section order. Matching nowcast `source_ids` may appear only as sanitized reference-like facts on a `partial`/`withheld` lane; packet `as_of_timestamp` is a forecast cutoff, not a retrieval time. They must not populate `HtmlBriefEvidenceRow.source_id`, `retrieved_at`, `rights_state`, or `field_scope_state`. Never infer a source ID, reference, retrieval time, rights status, or field scope. +- Current `QuarterlyTrendPacket` retains a latest source reference but not a complete source ID/retrieval/rights tuple, and `ValuationRegimePacket` retains references but not the complete tuple. Their research summaries may render only as `partial` or `withheld` with an exact `portable provenance incomplete` blocker unless a complete evidence row is explicitly linked to the exact same scoped source record in another supplied object. They cannot contribute an `available` evidence row by themselves. `OutcomeStatus` is not an input in version 1 because it retains neither profile/ticker scope nor portable source identity; do not infer or self-attest that linkage. + +Build primary answers in this order: `Usable now`, `Still withheld`, `Next research task`. Keep technical evidence and input identity in `evidence_rows` for Advanced only. + +Construct the entire sanitized snapshot with `identity=""`, then compute SHA-256 over `json.dumps(asdict(snapshot_without_identity), sort_keys=True, separators=(",", ":"), ensure_ascii=False)` and return a copy containing that identity. This makes dates, recency, readiness lanes, freshness, rights, boundary, answers, scenarios, sensitivity, research sections, Decision Lab lanes, evidence, input identities, and blockers identity-bearing. + +Add `company_workbench_html_filename(snapshot) -> str` using only uppercase ticker characters `[A-Z0-9.-]`, the first valid ISO date from `review_cutoff`, then `generated_at`, and suffix `-research-brief.html`; use `undated` when neither date is valid. The helper accepts no path. + +- [ ] **Step 8: Run focused tests and commit Task 1** + +```bash +python3 -m pytest tests/test_valuation.py tests/test_company_workbench_html.py -q +git diff --check +git add -- src/valuation.py src/company_workbench_html.py tests/test_valuation.py tests/test_company_workbench_html.py +make staged-hygiene-check +git diff --cached --check +git commit -m "Add HTML research brief snapshot contract" +``` + +Expected: all focused tests pass; only the four named files are staged; the 18 generated paths remain unstaged. + +--- + +### Task 2: Scoped Fragment, Secure Offline Document, And Download Contract + +**Files:** +- Modify: `src/company_workbench_html.py` +- Modify: `tests/test_company_workbench_html.py` + +- [ ] **Step 1: Write all renderer tests before implementation** + +Add failing tests for: + +- exact section order: overview, answers, scenarios, DCF bridge, sensitivity, business/forward view, Decision Lab, Advanced evidence; +- one H1 plus semantic header/main/section/table/caption/footer in the full document; +- a fragment rooted at one `
` with an H2, and no ``, ``, ``, `
`, `
`, `