diff --git a/CONTEXT.md b/CONTEXT.md index 41fc2d1..6207b42 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -61,28 +61,48 @@ Income and cash-flow statements use SimFin **TTM**; balance sheet uses the lates _Avoid_: mixing balance-sheet TTM into ROC denominators, using annual income for ranking between rebalance dates **Return on capital (ROC)**: -`EBIT / (Net Working Capital + Net Fixed Assets)`. Quality factor; higher is better; cross-sectional **ROC rank** (1 = best). Spec: `spec/features/007-high-quality-stocks/spec.md`. -_Avoid_: ROE, ROIC (unless explicitly that metric) +`EBIT / (Net Working Capital + Net Fixed Assets)`. Quality factor for the **Magic Formula benchmark**; higher is better; cross-sectional **ROC rank** (1 = best). Spec: `spec/features/007-high-quality-stocks/spec.md`. +_Avoid_: using ROC rank alone as production quality in Phase 2 — see **FS-Score** **Net working capital (NWC)**: `max(Current Assets − Excess Cash − Current Liabilities + Short-Term Debt, 0)` per versioned Greenblatt-style config. **v1:** excess cash uses curated `cash` (SimFin cash + cash equivalents + short-term investments — same field as EV). _Avoid_: total working capital without the excess-cash adjustment **Earnings yield (EY)**: -`EBIT / Enterprise Value`. Cheapness factor; higher is cheaper; cross-sectional **EY rank** (1 = cheapest). Spec: `spec/features/003-cheap-stocks/spec.md`. -_Avoid_: dividend yield, earnings/price without EV +`EBIT / Enterprise Value`. Cheapness factor for the **Magic Formula benchmark**; higher is cheaper; cross-sectional **EY rank** (1 = cheapest). Spec: `spec/features/003-cheap-stocks/spec.md`. +_Avoid_: using EY rank alone as production cheapness in Phase 2 — see **value pool** **Enterprise value (EV)**: `Market Cap + Total Debt + Preferred Equity + Minority Interest − Cash`. **v1:** cash is SimFin cash + cash equivalents + short-term investments (curated `cash`). _Avoid_: market cap alone as “value” **Combined rank**: -Sum of ROC rank and EY rank; lower is better. Used for portfolio selection and sell-watch opportunity cost. +Sum of ROC rank and EY rank; lower is better. Used for **Magic Formula replica** benchmark portfolio selection and sell-watch opportunity cost in the demo path. **Not** production ranking after Phase 2 — see **QV funnel rank**. _Avoid_: average of ranks, z-score blend (not MVP) +**Quantitative Value (QV)**: +Production scoring methodology (Gray/Carlisle): forensic hard exclusion → EBIT/TEV value pool → FS-Score quality screen → concentrated model portfolio. Spec: `spec/features/013-quantitative-value/spec.md`. +_Avoid_: conflating QV with the June 30 demo Magic Formula path + +**FS-Score**: +10-point Financial Strength Score (Gray/Carlisle variant): sum of binary profitability, stability, and operational-improvement signals (0–10). **Production quality** factor applied within the **value pool**. Spec: `spec/features/013-quantitative-value/spec.md`. +_Avoid_: Piotroski F-Score (different components), subjective ESG quality + +**EBIT/TEV (production value metric)**: +`EBIT / Enterprise Value` using the same EV definition as **Earnings yield**. Cross-sectional rank among forensic survivors selects the **value pool** (top decile ~10%). Distinct from MF **EY rank** used in the benchmark path. +_Avoid_: treating EY rank and value-pool membership as interchangeable + +**Value pool**: +Names surviving forensic screening that rank in the top decile (~10%) by **EBIT/TEV** on a run date. **Production cheapness** is membership in this pool, not MF **EY rank** alone. +_Avoid_: full universe, watchlist, final model portfolio + +**QV funnel rank**: +Order within the **value pool** after the FS-Score quality screen; lower rank number = higher FS-Score (with market-cap tie-break). Drives production **model portfolio** selection (~50 names). Supersedes **combined rank** for production. +_Avoid_: combined rank, ROC rank, EY rank in Phase 2 production scoring + **Magic Formula replica**: -Canonical benchmark portfolio using the same ROC, EY, combined rank, universe, and annual rebalance as production. Used to gate backtest pass vs strategy Sharpe. -_Avoid_: live Greenblatt fund, generic “value factor” +Benchmark portfolio using ROC, EY, combined rank, universe, and annual rebalance — same formulas as the June 30 demo slice. Used to gate backtest pass vs strategy Sharpe. **Not** production scoring after Phase 2. +_Avoid_: live Greenblatt fund, generic “value factor”, production scoring path **Cross-sectional rank**: Rank across all passing companies on one run date. Not comparable across dates without re-running the pipeline. @@ -93,7 +113,7 @@ Version id for ROC, EY, or filter rules so runs and backtests stay reproducible. _Avoid_: “latest formula”, implicit default **Model portfolio**: -Target long-only holdings from the pipeline; **June 30 demo:** top 30 names by combined rank, equal-weight only, market-cap tie-break on ranks. No watchlist in demo slice. Paper-traded in full MVP (phase 2). +Target long-only holdings from the pipeline. **June 30 demo:** top 30 names by combined rank, equal-weight, market-cap tie-break. **Phase 2 production:** top ~50 names by **QV funnel rank**, equal-weight (configurable cap). Paper-traded in full MVP (phase 2). _Avoid_: personal portfolio, watchlist (demo slice) **Watchlist**: @@ -130,10 +150,11 @@ _Avoid_: ad-hoc snapshot without run id ## Relationships -- A **run date** drives **universe** → **permanent loss filter** → **ROC** and **EY** ranks → **combined rank** → **model portfolio** +- **June 30 demo:** A **run date** drives **universe** → **ROC** and **EY** ranks → **combined rank** → **model portfolio** (30 names) +- **Phase 2 production:** A **run date** drives **universe** → **permanent loss filter** / forensic screen → **value pool** (EBIT/TEV decile) → **FS-Score** → **QV funnel rank** → **model portfolio** (~50 names) - **As-of date** tags each fundamental row; PIT queries filter `as_of_date <= run_date` -- **Watchlist** superset of names that may enter the **model portfolio** on rebalance -- **Magic Formula replica** is the strategy’s primary benchmark comparator for Sharpe pass/fail +- **Magic Formula replica** is the strategy’s primary benchmark comparator for Sharpe pass/fail; uses **combined rank**, not **QV funnel rank** +- **Watchlist** superset of names that may enter the **model portfolio** on rebalance (phase 2) - **Sell-watch** evaluates only the **model portfolio**, not the user’s personal portfolio ## Flagged ambiguities @@ -142,11 +163,11 @@ Resolved scope cuts (see ADRs and [`spec/constitution/roadmap.md`](spec/constitu - **June 30 demo MVP:** SimFin bulk US → raw → normalizer → **universe (US market)** → ROC/EY → combined rank → top-30 EW model portfolio → Streamlit dashboard. No permanent loss filter, backtest, sell-watch, or paper trading in this slice. - SEC ETL spike (`sec_client`, `edgartools_client`, `download-fundamentals`) is **frozen** in repo for phase 2; demo pipeline uses SimFin bulk for fundamentals and run-date prices (`shareprices/latest`). -- **Phase 2 (Quantitative Value):** will need multi-period fundamentals (not only TTM snapshots)—lake design should not block adding annual/quarterly income history later. +- **Phase 2 (Quantitative Value):** production scoring follows the QV funnel (`spec/features/013-quantitative-value/spec.md`); MF ROC/EY/combined rank remain benchmark-only. Multi-period fundamentals required for FS-Score and Beneish ([#87](https://github.com/JLaborda/SmartWealthAI/issues/87)). Terminology reminders: -- “Cheap” means high **EY**, not low P/E—use **EY rank** in issues and code names. -- “Quality” means high **ROC**, not ESG or subjective moat—use **ROC rank**. +- **June 30 demo:** “Cheap” means high **EY rank**; “quality” means high **ROC rank**; portfolio uses **combined rank**. +- **Phase 2 production:** “Cheap” means **value pool** membership (top EBIT/TEV decile); “quality” means high **FS-Score**; portfolio uses **QV funnel rank**. - “Value trap” in specs means negative EBIT routed to **review queue**, not a separate score. - MVP specs in `spec/` remain canonical until an ADR or architecture decision supersedes them; update `CONTEXT.md` when `/grill-with-docs` resolves a term conflict. diff --git a/spec/constitution/roadmap.md b/spec/constitution/roadmap.md index 7a0ff6b..6168ea8 100644 --- a/spec/constitution/roadmap.md +++ b/spec/constitution/roadmap.md @@ -94,14 +94,17 @@ flowchart LR 1. Quantitative Value feature spec + glossary ([#86](https://github.com/JLaborda/SmartWealthAI/issues/86)) 2. Multi-period fundamentals ETL + PIT history ([#87](https://github.com/JLaborda/SmartWealthAI/issues/87)) -3. Forensic evaluator + permanent loss filter ([#89](https://github.com/JLaborda/SmartWealthAI/issues/89)) -4. QV funnel orchestrator (production scoring) ([#91](https://github.com/JLaborda/SmartWealthAI/issues/91)) -5. Light backtest (5–10y QV) ([#96](https://github.com/JLaborda/SmartWealthAI/issues/96)) -6. CI/CD M2 Terraform + S3 lake ([#105](https://github.com/JLaborda/SmartWealthAI/issues/105)–[#112](https://github.com/JLaborda/SmartWealthAI/issues/112)) -7. Sell-watch + dashboard QV views ([#97](https://github.com/JLaborda/SmartWealthAI/issues/97), [#93](https://github.com/JLaborda/SmartWealthAI/issues/93)) -8. Historical S&P 500 universe + full backtest (2b) ([#99](https://github.com/JLaborda/SmartWealthAI/issues/99), [#100](https://github.com/JLaborda/SmartWealthAI/issues/100)) -9. SEC EDGAR normalizer (optional PIT upgrade) -10. Paper trading (deferred) +3. Daily price history for backtest window ([#88](https://github.com/JLaborda/SmartWealthAI/issues/88)) +4. Forensic evaluator + permanent loss filter ([#89](https://github.com/JLaborda/SmartWealthAI/issues/89)) +5. FS-Score calculator ([#90](https://github.com/JLaborda/SmartWealthAI/issues/90)) +6. QV funnel orchestrator (production scoring) ([#91](https://github.com/JLaborda/SmartWealthAI/issues/91)) +7. Magic Formula benchmark path isolation ([#92](https://github.com/JLaborda/SmartWealthAI/issues/92)) +8. Light backtest (5–10y QV) ([#96](https://github.com/JLaborda/SmartWealthAI/issues/96)) +9. CI/CD M2 Terraform + S3 lake ([#105](https://github.com/JLaborda/SmartWealthAI/issues/105)–[#112](https://github.com/JLaborda/SmartWealthAI/issues/112)) +10. Sell-watch + dashboard QV views ([#97](https://github.com/JLaborda/SmartWealthAI/issues/97), [#93](https://github.com/JLaborda/SmartWealthAI/issues/93)) +11. Historical S&P 500 universe + full backtest (2b) ([#99](https://github.com/JLaborda/SmartWealthAI/issues/99), [#100](https://github.com/JLaborda/SmartWealthAI/issues/100)) +12. SEC EDGAR normalizer (optional PIT upgrade) +13. Paper trading (deferred) ## Feature registry (stable IDs) diff --git a/spec/features/013-quantitative-value/spec.md b/spec/features/013-quantitative-value/spec.md index ab46ffd..cbda533 100644 --- a/spec/features/013-quantitative-value/spec.md +++ b/spec/features/013-quantitative-value/spec.md @@ -2,7 +2,7 @@ ## Implementation status -**planned** +**in_progress** — governance slice ([#86](https://github.com/JLaborda/SmartWealthAI/issues/86)); scoring code in downstream issues ([#87](https://github.com/JLaborda/SmartWealthAI/issues/87)–[#91](https://github.com/JLaborda/SmartWealthAI/issues/91)). ## Delivery @@ -69,13 +69,66 @@ flowchart TD 5. Select top ~50 names; equal-weight; write portfolio parquet + MLflow artifact. 6. Log stage counts: universe → forensic → value → quality → portfolio. +## Forensic stage (stage 1) + +Extends [`008-permanent-loss-filter`](../008-permanent-loss-filter/spec.md) with Phase 2a scope: + +| Mechanism | Behavior | `formula_version` | +| --- | --- | --- | +| Hard rule exclusions | Bankruptcy/distress and fraud rules fire `exclude` before any score | Per `rule_version` in permanent-loss config | +| Beneish M-Score | Computed cross-sectionally on forensic survivors; names in the **bottom 5%** (worst manipulation risk) are hard-excluded | `beneish_v1` (coefficients versioned under `config/permanent_loss/`) | +| QVAL-style percentile gate | Applied **per forensic model** that emits a continuous score (Beneish first; others may follow in 2b) | Same as parent model version | + +Each exclusion records `rule_id`, `rule_version`, `triggered_value`, `threshold`, `as_of_date`, and `explanation`. Missing inputs route to the **review queue**, not silent pass. + +## FS-Score (stage 3 quality screen) + +Gray/Carlisle 10-point **Financial Strength Score** (`fs_score_v1`). Each component is binary (0 or 1); total score is the sum (0–10). Requires multi-period PIT fundamentals ([#87](https://github.com/JLaborda/SmartWealthAI/issues/87)). + +| Component | Category | Signal (1 if true) | +| --- | --- | --- | +| `FS_ROA` | Current profitability | Return on assets > 0 | +| `FS_FCFTA` | Current profitability | Free cash flow / total assets > 0 | +| `FS_ACCRUAL` | Current profitability | Free cash flow > net income (accrual quality) | +| `FS_ΔLEVER` | Stability | Long-term debt ratio decreased YoY | +| `FS_ΔLIQUID` | Stability | Current ratio improved YoY | +| `FS_NEQISS` | Stability | Net equity issuance negative (net repurchaser) | +| `FS_ΔROA` | Recent operational improvements | ROA improved YoY | +| `FS_ΔFCFTA` | Recent operational improvements | FCFTA improved YoY | +| `FS_ΔMARGIN` | Recent operational improvements | Gross margin improved YoY | +| `FS_ΔTURN` | Recent operational improvements | Asset turnover improved YoY | + +Within the **value pool**, rank by total FS-Score descending; ties break by ascending market cap (configurable). Names below the portfolio cap are dropped. + +## Value screen (stage 2) + +| Metric | Definition | Notes | +| --- | --- | --- | +| EBIT/TEV | `EBIT / Enterprise Value` | Same EV definition as [`003-cheap-stocks`](../003-cheap-stocks/spec.md) (`formula_version = v1`) | +| Value pool | Top decile (~10%) by EBIT/TEV among forensic survivors | Configurable fraction under `config/quantitative_value/` | + +Negative EBIT rows route to the **review queue** (not the value pool). + +## Portfolio constructor (stage 4) + +| Decision | Value | +| --- | --- | +| Default size | **50** names | +| Weighting | Equal-weight long-only | +| Tie-break | Ascending market cap on FS-Score rank ties | +| Config | `config/quantitative_value/portfolio_v1.yaml` (cap configurable) | + ## Acceptance criteria -- [ ] `spec/features/013-quantitative-value/spec.md` exists with objective, scope, funnel stages, and acceptance criteria (this document). -- [ ] FS-Score components and forensic bottom-5% gate documented with formula versions. -- [ ] Portfolio size default (~50) and configurable cap recorded as a closed decision. -- [ ] `CONTEXT.md` updated with production QV terms (quality = FS-Score, cheap = value pool, QV funnel rank); MF terms marked benchmark-only. -- [ ] `spec/constitution/roadmap.md` "After the demo" section reflects QV-first production order. +### Governance ([#86](https://github.com/JLaborda/SmartWealthAI/issues/86)) + +- [x] `spec/features/013-quantitative-value/spec.md` exists with objective, scope, funnel stages, and acceptance criteria (this document). +- [x] FS-Score components and forensic bottom-5% gate documented with formula versions. +- [x] Portfolio size default (~50) and configurable cap recorded as a closed decision. +- [x] `CONTEXT.md` updated with production QV terms (quality = FS-Score, cheap = value pool, QV funnel rank); MF terms marked benchmark-only. +- [x] `spec/constitution/roadmap.md` "After the demo" section reflects QV-first production order. + +### Implementation (downstream) - [ ] `score-universe` runs full QV funnel for a `run_date` when implementation completes ([#91](https://github.com/JLaborda/SmartWealthAI/issues/91)). - [ ] Excluded names never appear in final portfolio; stage shrinkage monotonic on fixtures.