diff --git a/.agent/memory-bank/activeContext.md b/.agent/memory-bank/activeContext.md index 713f1c34..b78b511d 100644 --- a/.agent/memory-bank/activeContext.md +++ b/.agent/memory-bank/activeContext.md @@ -1,7 +1,9 @@ # Active Context ## Current Work -- **PRD follow-on:** optional `infer-schema` CLI + Postgres smoke / CI hook; Phase E **U-4** pool-bound `Session` + auto-dirty on `LifeRecord::set`. +- **lifeguard-derive `validate_fields` + Aggregate (2026-03-31):** `#[validate(custom)]` FailFast vs Aggregate branches in `life_record.rs`; struct-level `#[validation_strategy = "aggregate"|"fail_fast"]` parsed in `attributes.rs`, registered on `LifeModel`/`LifeRecord` derives, emits `ActiveModelBehavior::validation_strategy` when set (so `validate_fields` matches `run_validators` — manual second `impl ActiveModelBehavior` is invalid). Tests `validate_multi_fail_fast` / `validate_multi_aggregate` in `lifeguard-derive/tests/test_minimal.rs`; shared `name_non_empty` in `validate_attr_tests`. Verified: `cargo test -p lifeguard-derive`, `cargo clippy -p lifeguard-derive --all-targets --all-features -- -D warnings`. +- **README reorg (2026-03-28):** Landing page split — **`ROADMAP.md`**, **`COMPARISON.md`** (repository truth under **`#repository-status`** + competitive matrix + ecosystem + performance). **`STATUS.md`** merged into **`COMPARISON.md`** (2026-03-28). **`README.md`** is pitch + getting started + doc index table. **`ARCHITECTURE.md`** LifeReflector link points to **`VISION.md#the-killer-feature-lifereflector`**. PRD + SEAORM mapping reference **`COMPARISON.md`**. +- **PRD follow-on:** Phase E **insert-only** session flush for new rows (no PK); `find_related` + scopes per `docs/planning/DESIGN_FIND_RELATED_SCOPES.md`; optional `infer-schema` CLI / CI smoke. - **Dev process + Phase D v0 (2026-03-28):** `docs/planning/DEV_RUSTDOC_AND_COVERAGE.md` (rustdoc + coverage checklist); Phase D `ColumnTrait::f_*` for SeaQuery `UPDATE SET`. - **PRD Phase C v0 — scopes (2026-03-28):** `SelectQuery::scope`, `IntoScope`, `src/query/scope.rs` docs + tests; PRD §7.7; mapping doc row. Next: Phase D F() or README G6 follow-ups if requested. - **PRD Phase B v0 — validators (2026-03-28):** Landed trait-based validation pipeline + `ActiveModelError::Validation`; derive wires `run_validators` after `before_insert`/`before_update`. PRD `docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md` §6.7 + milestone §0.1. Memory bank updated with this line. @@ -15,6 +17,12 @@ - Memory Bank initialized with codegen learnings ## Recent Changes +- **Docs — COMPARISON ecosystem + PRD ValidateOp + progress Metrics (2026-03-28):** `COMPARISON.md` — factual **BRRTRouter** / **Lifeguard** / **`may`** vs **Tokio** / **`async`/`await`** / **SeaORM** / **SQLx** / **Diesel (sync)** / **Diesel-async** table; removed absolute “must”/“fundamentally incompatible” wording; performance footnote for Diesel. `PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md` — `ValidateOp` = `Insert` \| `Update` \| `Delete`; V-1 insert/update/**delete**; §5.7 index scope + §5.7a **index reconciliation** stretch; `.agent/memory-bank/progress.md` **Metrics** block labeled **historical snapshot** with pointers to **Completed**. +- **`ActiveModelTrait::take` session notify (2026-03-28):** `lifeguard-derive` `life_record.rs` — `__lg_session_notify_dirty()` only when `take` actually changes state (`#field_name.is_some()` or `__update_exprs` contained that column), matching `set`/`set_col` gating. Tests `test_take_noop_when_unset_skips_session_dirty`, `test_take_with_value_or_expr_notifies_session_dirty` in `lifeguard-derive/tests/test_minimal.rs`. `cargo test -p lifeguard-derive`, `cargo clippy -p lifeguard-derive --all-targets --all-features -- -D warnings`. +- **`SessionIdentityModelCell` / `unsafe impl Send` (2026-03-28):** Cursor review — `Rc` + `unsafe Send` is unsound if the “one thread per `Rc` clone” protocol is violated; acknowledged. Expanded `src/session/identity_model_cell.rs` module docs + `// SAFETY` block: why `ActiveModelTrait: Send` forces `Option: Send`; that `Arc>` does not restore `Send` (`RefCell: !Sync`); pointer to `SECURITY_PROMPT.md` §A.3 and `Arc>` as a type-sound long-term direction. No API change in this step. +- **`migration_db_compare_smoke` CLI assertion (2026-03-28):** `compare_schema_cli_succeeds_when_no_drift` asserted `Tables match` / `Success`, but `MigrationDbCompareReport`’s `Display` prints **`No drift`** when `!has_drift()` (`schema_migration_compare.rs`). Test now asserts `stdout.contains("No drift")`; stderr assert uses named `stderr=` placeholder. +- **Review nits (2026-03-28):** `tests/db_integration/pool_read_replica.rs` — per-test unique `pool_replica_test_{n}` / `t_pool_replica_smoke_{n}` + `setup_schema_on_primary(executor, schema, table)` to avoid parallel DDL collisions; raw `may_postgres` `query_one` uses a local `String` + `as_str()`. `SessionIdentityModelCell::replace_with` → `try_borrow_mut` + `Result<(), BorrowMutError>`; derive `__lg_session_notify_dirty` documents `to_model()` clone cost + ignores replace error. `DEVELOPMENT.md` — split `lifeguard-migrate` / optional live Postgres files / `db_integration_suite` bullets. `lifeguard-migrate` / tests — named `format!` where applicable; `src/session/uow.rs` left unchanged (already `{e}`/`{rb_err}`; explicit `err = e` trips `clippy::uninlined_format_args`). `predicates::string_utf8_chars_max` already single `chars().count()`. `ModelIdentityMap::register_pending_insert` rustdoc: `PENDING_INSERT_KEY_PREFIX`, `next_pending_id`, wrapping. Verified: `cargo clippy -p lifeguard -p lifeguard-derive -p lifeguard-migrate --all-targets --all-features -- -D warnings`, `cargo test -p lifeguard --lib session::`, `cargo test -p lifeguard-derive`. +- **ActiveModelTrait `set` / `set_col` vs `__update_exprs` (2026-03-28):** `lifeguard-derive` `life_record.rs` — match arms remove a pending `set_*_expr` only after `Value`→field conversion succeeds, so `InvalidValueType` no longer drops the scheduled F-style expression. Integration tests in `lifeguard-derive/tests/test_minimal.rs` (`test_set_invalid_type_preserves_update_expr`, `test_set_col_invalid_type_preserves_update_expr`, non-PK column). Verified: `cargo test -p lifeguard-derive`; `cargo clippy -p lifeguard -p lifeguard-derive --all-targets --all-features -- -D warnings`. - **Toxiproxy fallback test ignored (2026-03-28):** `pooled_read_falls_back_to_primary_when_replica_lagging` is `#[ignore]` (flaky CI: `is_replica_lagging` after proxy disable). Run with `cargo test -- --ignored` when debugging; TODO to stabilize `WalLagMonitor` + re-enable. - **Tiltfile — replication test resources (2026-03-28):** Targeted nextest runs for read-replica PRD work (`test-replication-pool`, `test-db-integration-replica`, `test-replication-pool-smoke`) with Kind ports 6543/6544/6545. **`test-replication-pool` / `-smoke`:** added nextest `--status-level pass` so filtered runs do not print one SKIP line per non-matching test in `db_integration_suite` (filter is intentionally narrow; full replica env suite remains `test-db-integration-replica`). - **CI Compose + Toxiproxy (2026-03-30):** [Toxiproxy](https://github.com/Shopify/toxiproxy) fronts the streaming replica on host **6547** (API **8474**) so Kind/Tilt can keep **6544** for replica-0. The integration test that PATCHes proxy `enabled` for primary fallback is **`#[ignore]`** until stabilized (see Recent Changes). **Local:** stop Tilt or avoid port collisions — `127.0.0.1:6543` may hit Kind vs Compose primary if both bind. diff --git a/.agent/memory-bank/progress.md b/.agent/memory-bank/progress.md index 76b1cb43..9df63094 100644 --- a/.agent/memory-bank/progress.md +++ b/.agent/memory-bank/progress.md @@ -1,6 +1,86 @@ # Progress Tracking +## STATUS merged into COMPARISON (2026-03-28) + +- **`STATUS.md` removed.** Former **repository truth** bullets live under **`## Repository status`** in **`COMPARISON.md`** (anchor **`#repository-status`**). **`README.md`**, **`VISION.md`**, **`ARCHITECTURE.md`**, **`ROADMAP.md`** links updated; documentation table uses a single **COMPARISON.md** row for repository truth + competitive content. + +## README badges + logo width (2026-03-28) + +- **README.md:** CI badges above hero — **Lifeguard CI** workflow badge (`ci.yaml` on `main`) + static **rustc nightly** badge linking to `.github/workflows/ci.yaml`. Logo image **`width="600"`**. + +## README blog link (2026-03-28) + +- **README.md:** “The problem” in **Why Lifeguard** links to **`LIFEGUARD_BLOG_POST.md`** at repo root (async/`Tokio` ORMs vs `may`); Documentation table row added. + +## README reorg — STATUS / ROADMAP / COMPARISON (2026-03-28) + +_Superseded for `STATUS.md`: merged into **`COMPARISON.md`** — see **STATUS merged into COMPARISON** above._ + +- **STATUS.md** (repo root): former README **“Current status (repository truth)”** bullets (verbatim relocation). +- **ROADMAP.md** (repo root): former README **Roadmap** table + story links. +- **COMPARISON.md** (repo root): competitive metrics table, legend, implementation summary, differentiators, performance tables, ecosystem compatibility, “when to use” — plus former duplicate **Performance** marketing bullets merged as **Target performance claims**. +- **README.md**: pitch-first spine — merged **Why Lifeguard** + executive intent into **Why Lifeguard (technical bet)**; links to COMPARISON/VISION; compact architecture + getting started + observability + testing; **Documentation** table indexes new files. +- **Cross-links:** `VISION.md`, `ARCHITECTURE.md` (LifeReflector anchor fixed to `VISION.md#the-killer-feature-lifereflector`), `docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md`, `docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md` now reference `COMPARISON.md` where the competitive table lives. +- **Git:** `2ee3554` on `feat/schema_validators_session_and_scopes_2` — pushed to `origin` (`farm git commit` + `farm git push`). + +## VISION.md + README CTO summary (2026-03-28) + +- **VISION.md** (repo root): moved former README **“What we’re building”** (LifeModel/LifeRecord, pool, LifeReflector, transparent caching, replica modes, feature lists) from README. +- **README.md**: new **Executive summary (CTO)** section; link to **VISION.md**; LifeReflector bullet points to **VISION.md#the-killer-feature-lifereflector**; **Documentation** list includes VISION.md; **ARCHITECTURE.md** footer links Vision. + +## OBSERVABILITY.md (repo root) + README summary (2026-03-28) + +- **OBSERVABILITY.md** (repo root): OTel-compatible / OTLP + Datadog note; moved Prometheus / tracing / LifeReflector metric bullets from README; links to **docs/OBSERVABILITY.md** (tables, Kind) and **docs/OBSERVABILITY_APP_INTEGRATION.md**. +- **README.md**: Observability § replaced with short summary; pool paragraph links root + **docs/OBSERVABILITY.md**; developer links updated. +- **docs/OBSERVABILITY.md**: top pointer to **../OBSERVABILITY.md**. + +## ARCHITECTURE.md split from README (2026-03-28) + +- **ARCHITECTURE.md** (repo root): holds full architecture content — numbered target flowchart, multi-service deployment, connection pool “300 Spartans”, LifeReflector sequence diagram, links back to README. +- **README.md**: replaced long § with **summarized** `flowchart LR` (request path vs optional/async) + pointer **[ARCHITECTURE.md](./ARCHITECTURE.md)**. Current status bullet for LifeReflector now references ARCHITECTURE.md. +- **Git:** `db5c3e2` on `feat/schema_validators_session_and_scopes_2` — pushed to `origin` (`farm git commit` + `git push`). + +## README architecture diagrams (2026-03-28) + +- **README.md** § Architecture overview: replaced flat “App → Redis” graph with **numbered** `flowchart` (ORM → pool → **primary** vs **replica**; optional Redis **5–6**; LifeReflector **7–10** background). Microservices diagram split **PostgreSQL** into **Primary** / **Replicas**, numbered **1–5** on request path, dotted optional Redis, NOTIFY/reflector as **bg**. Connection pool subsection: sentence on primary vs replica **slots**. Legend explains solid vs dotted vs async. + +## README status refresh (2026-03-28) + +- **README.md** “Current status” and competitive matrix aligned with latest shipped behavior: **`ReadPreference`** / **`with_read_preference`**; **`lifeguard-migrate`** **`infer-schema`** (composite PK `#[primary_key]`) + **`compare-schema`** column-name drift; **`find_related`** vs parent **`scope`** (chain on returned `SelectQuery`). Fixed stale rows that claimed CTEs / windows / subqueries were “not yet implemented” (now **partial** with links to `SelectQuery` APIs). Roadmap row and **Read Preferences** / **Schema Inference** / **Scopes** table cells updated. + +## Rustdoc API surface — audit snapshot (2026-03-31) + +- **Git:** `docs(rustdoc): expand SelectQuery and pool docs…` on `feat/schema_validators_session_and_scopes_2` — pushed to `origin` (upstream set). + +- **Strong:** crate root, `query` (especially `select` + execution), `pool`/`pooled`, `executor`. +- **Needs link + doctest pass:** `active_model` (private `converted_params` link, bad `super::` links, broken module example), `migration` (examples vs current API; many doctest compile failures), `session` (broken intra-doc links), `relation` (redundant links + `LazyLoader` link), `logging` (`lifeguard_log!` link), `model`/`value` (invalid_html_tags in doc text). +- **Mechanical:** `cargo doc -p lifeguard --no-deps` reports **~30** rustdoc warnings; `cargo test -p lifeguard --doc` is **not** all-green. **`lifeguard-derive`** and **`lifeguard-migrate`** are separate consumer surfaces for a full “everything the user touches” pass. + ## Completed ✅ +- **Clippy pedantic + PRD Phase C docs (`find_related` vs scopes) (2026-03-31):** `lifeguard-derive` `DeriveMigrationName` doc — backticks for `snake_case` (`clippy::doc_markdown`). `query::scope` + `FindRelated` rustdoc: parent scopes not merged into `find_related`; chain on returned `SelectQuery`. `Related` trait: fix `LazyLoader` link (`crate::relation::lazy::LazyLoader`), drop redundant `RelationDef` link targets. `DESIGN_FIND_RELATED_SCOPES.md` status + recommended default; PRD §7.7; `SEAORM_LIFEGUARD_MAPPING.md` scopes row. `cargo clippy --workspace --all-targets --all-features -- -D warnings -W clippy::pedantic`; `cargo test -p lifeguard --lib`. +- **PRD Phase A — composite PK infer + compare-schema column names (2026-03-31):** `schema_infer::emit_inferred_rust` emits `#[primary_key]` on every PK column (including composite); golden `infer_composite_pk.expected.rs` updated. `generated_migration_diff::column_map_from_merged_baseline`; `schema_migration_compare` extended with `fetch_live_table_column_names`, `TableColumnDrift`, `MigrationDbCompareReport::column_drifts`; `compare-schema` / CLI docs. PRD §5.7 + §5.7a (defer watch mode / richer CI goldens); `DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md`; `SEAORM_LIFEGUARD_MAPPING.md` parity row; `lifeguard-migrate/README.md`. `cargo test -p lifeguard-migrate`; `cargo clippy -p lifeguard-migrate --all-targets --all-features -- -D warnings`. +- **Rustdoc — opt-in SQL + read preference (2026-03-31):** `query::select` intra-doc links use `crate::query::select::SelectQuery::all` / `::one` (not `execution::SelectQuery`); advanced SQL `no_run` examples are compile-checked `sea_query` snippets plus a one-line `MyEntity::find()…` comment (avoids broken `LifeModelTrait` stubs and `dyn LifeExecutor` + `all`); `expr_window` / `expr_window_as` expanded. `ReadPreference` + `PooledLifeExecutor` rustdoc (when to use primary, `LifeExecutor` + `sea_query::Values` in `with_read_preference` example). Crate `lib.rs` **Explicit opt-in APIs**; `query/mod.rs` **Default path vs advanced SQL** + redundant `ColumnTrait` link cleanup. Verified: `cargo test -p lifeguard --lib` **435 passed**; doctests `with_cte`, `join_subquery`, `window`, `expr_window`, `expr_window_as`, `ReadPreference`, `with_read_preference` compile. +- **SelectQuery SQL builder extras — CTE + subquery join + window (2026-03-31):** `SelectQuery::with_cte` (wraps `SelectStatement::with_cte`, keeps lifeguard `all`/`one` vs raw `WithQuery`); `join_subquery`, `window`, `expr_window`, `expr_window_as`, `expr_window_name`, `expr_window_name_as`. Unit tests `with_cte_prepends_with_clause`, `join_subquery_emits_subselect_join`, `window_clause_and_expr_window_name_as`. `query/mod.rs` + `select.rs` module docs; README roadmap. `cargo test -p lifeguard --lib`; `cargo clippy -p lifeguard --all-targets --all-features -- -D warnings`. +- **`DeriveMigrationName` + `MigrationName` (2026-03-31):** `lifeguard::migration::MigrationName` trait; `#[derive(DeriveMigrationName)]` on unit structs → `MIGRATION_NAME` + `MigrationName` impl (snake_case via `lifeguard-derive` `utils::snake_case`); re-export `DeriveMigrationName` from `migration` module. `lifeguard-derive/tests/test_derive_migration_name.rs`. README competitive row + `SEAORM_LIFEGUARD_MAPPING.md`. `cargo test -p lifeguard-derive`; `cargo test -p lifeguard --lib` **432 passed**; `cargo clippy -p lifeguard -p lifeguard-derive --all-targets --all-features -- -D warnings`. +- **PRD roadmap — explicit read preference on `PooledLifeExecutor` (2026-03-31):** `ReadPreference` (`Default` \| `Primary`); `LifeguardPool::read_pool_for` + `dispatch_read_with_preference`; `PooledLifeExecutor { read_preference }` with `with_read_preference` / `read_preference()`; `query_*_values` honor preference (primary reads for read-your-writes). Unit test `read_preference_default_matches_variant`; integration `pooled_read_preference_primary_forces_primary_tier` in `tests/db_integration/pool_read_replica.rs` (needs `TEST_REPLICA_URL`). Docs: `src/pool/pooled.rs` module rustdoc, `pool/mod.rs`, README implementation summary + roadmap; `DESIGN_FIND_RELATED_SCOPES.md` bullet. `cargo clippy -p lifeguard --all-targets --all-features -- -D warnings`; `cargo test -p lifeguard --lib read_preference`. +- **PRD Phase E §9 — LifeRecord → identity-map model auto-sync (2026-03-28):** `LifeRecord::attach_session_with_model` + `__lg_session_model`; `__lg_session_notify_dirty` calls `to_model()` then updates the linked cell. **`SessionIdentityModelCell`** (`src/session/identity_model_cell.rs`) wraps `Rc>` with documented `unsafe impl Send` so `ActiveModelTrait: Send` holds; `replace_with` used by the derive. README roadmap line updated (no longer lists optional auto-sync as open). Rustdoc: `src/session/mod.rs` (no bogus `ActiveModelTrait` link). Unit test `session_identity_model_cell_replace_with_updates_rc`; integration `session_flush_dirty_after_attach_session_and_set_n_on_record` uses `attach_session_with_model`. Verified: `cargo test -p lifeguard-derive`, `cargo clippy -p lifeguard -p lifeguard-derive --all-targets --all-features -- -D warnings`, `cargo test -p lifeguard --test db_integration_suite session_flush_dirty_after_attach`. +- **PRD Phase A closure — infer-schema CLI e2e + compare-schema (DBA confidence) (2026-03-28):** `lifeguard-migrate/tests/infer_schema_cli_subprocess.rs` (`CARGO_BIN_EXE_lifeguard-migrate infer-schema`); `schema_migration_compare` + CLI **`compare-schema`** (live `information_schema` base tables vs merged `*_generated_from_entities.sql` `-- Table:` names); `tests/migration_db_compare_smoke.rs`. Docs: `lifeguard-migrate/README.md`, `DEVELOPMENT.md`, PRD §5.7, `DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md`, `SEAORM_LIFEGUARD_MAPPING.md`. `cargo test -p lifeguard-migrate`. +- **PRD G6 — validators + F() docs (Phase B / D polish) (2026-03-28):** README feature list + competitive matrix — validators (`run_validators`, `ValidationStrategy`, derive `custom`, predicate names, gap vs SeaORM); F() row + partial-summary — PostgreSQL numeric promotion, no implicit casts. [SEAORM_LIFEGUARD_MAPPING.md](docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) parity rows expanded. [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md](docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md) §6.7 G6 bullet, §8.7 PostgreSQL numeric typing paragraph; `ColumnTrait::f_add` rustdoc cross-ref. +- **PRD Phase E — insert-only session flush (pending map keys + promote) (2026-03-28):** `PENDING_INSERT_KEY_PREFIX` / `is_pending_insert_key`; `ModelIdentityMap::register_pending_insert`, `flush_dirty_with_map_key`, `promote_pending_to_loaded`; `Session` + `flush_dirty_in_transaction_*_with_map_key` and pooled `BEGIN`/`COMMIT` map-key path; unit test `register_pending_insert_flush_with_map_key_and_promote` (`#[allow(clippy::expect_used)]` on test). **Docs + integration:** PRD §0.1/§9.7, `DESIGN_SESSION_UOW.md`, README session bullets, `SEAORM_LIFEGUARD_MAPPING.md` parity row; `tests/db_integration/session_identity_flush.rs` — **`identity_map_pending_insert_flush_and_promote_persists_on_postgres`**, **`session_pending_insert_flush_in_transaction_with_map_key_persists_on_postgres`**, **`session_pending_insert_flush_in_transaction_pooled_with_map_key_persists_on_postgres`**. `cargo test -p lifeguard --lib` **430 passed**; `cargo clippy -p lifeguard --all-targets --all-features -- -D warnings`; `db_integration_suite` with `TEST_DATABASE_URL` / compose. +- **PRD Phase C — `#[scope]` attribute macro (2026-03-28):** `lifeguard_derive::scope` + `lifeguard::scope` re-export; `fn foo` → `pub fn scope_foo` on `impl Entity` (no `self`, no `async`); `lifeguard-derive/tests/test_minimal.rs` `scope_attr_tests`. PRD §7.7 / §0.1, `src/query/scope.rs`, README + `SEAORM_LIFEGUARD_MAPPING.md`. Unlocks consistent named scope patterns for `find_related` + scopes follow-on. +- **PRD Phase E U-4 — pooled pin-slot transactional session flush (2026-03-28):** `WorkerPool` per-slot `Mutex` (serialize jobs on same worker index); `dispatch_on_sender` + `dispatch_locked`; [`LifeguardPool::exclusive_primary_write_executor`] → [`ExclusivePrimaryLifeExecutor`] (primary-only reads/writes on one connection); [`Session::flush_dirty_in_transaction_pooled`] (`BEGIN`/`COMMIT`/`ROLLBACK` via raw SQL, default READ COMMITTED). Integration `session_flush_dirty_in_transaction_pooled_persists_via_update` in `tests/db_integration/session_identity_flush.rs`. PRD §9.7, `DESIGN_SESSION_UOW.md`, README session rows. `cargo clippy -p lifeguard --all-targets --all-features -- -D warnings`; `cargo test -p lifeguard --lib` **429 passed**; `db_integration_suite` needs `TEST_DATABASE_URL` locally/CI. +- **G6 README + Phase D F() rustdoc (2026-03-28):** README — validators row/bullets (`lifeguard::predicates`), session/UoW (`Session`, `attach_session`, `flush_dirty_in_transaction`, roadmap: pooled pin-slot, insert-only flush); implementation summary + trade-offs aligned. `ColumnTrait::f_add` — **PostgreSQL numeric typing** limitation (promotion/casts; no auto-cast); `f_sub`/`f_mul`/`f_div` cross-ref. `cargo clippy -p lifeguard --all-targets --all-features -- -D warnings`; `cargo test -p lifeguard` lib **429 passed** (plain `db_integration_suite` SIGSEGV in this env — use nextest serial profile when DB tests required). +- **PRD Phase B — built-in validation predicates (2026-03-31):** `src/active_model/predicates.rs` (`lifeguard::predicates`) — `string_utf8_chars_max`, `string_utf8_chars_in_range`, `blob_or_string_byte_len_max`, `i64_in_range`, `f64_in_range`; PRD §6.7 + SEAORM parity row; `lib.rs` re-export. +- **PRD Phase E — `Session::flush_dirty_in_transaction` (2026-03-31):** `MayPostgresExecutor` + `BEGIN`/`COMMIT`/`ROLLBACK` around `flush_dirty`; docs (PRD §9.7, `DESIGN_SESSION_UOW.md`, `session` module); integration `session_flush_dirty_in_transaction_persists_via_update`; `register_loaded` rustdoc (insert path). +- **PRD Phase E — `Session` + auto-dirty on `LifeRecord` (2026-03-31):** `src/session/uow.rs` (`Session`, `SessionDirtyNotifier`, pending-dirty merge at `flush_dirty`); `LifeRecord` derive: `attach_session` / `detach_session`, `__lg_session_notifier`, hooks on `set_*` / `set` / `take` / `set_col` / `set_*_expr` (PK entities); integration `session_flush_dirty_after_attach_session_and_set_n_on_record`; PRD §0.1/§9.7, `DESIGN_SESSION_UOW.md`, `DEVELOPMENT.md`. `Send`-safe notifier (graph closures). +- **PRD §9 Postgres flush integration (2026-03-30):** `tests/db_integration/session_identity_flush.rs` (`identity_map_flush_dirty_persists_via_update`, `identity_map_flush_dirty_with_mark_dirty_key_and_identity_map_key`); `db_integration_suite` module; PRD §9.7 + `DEVELOPMENT.md`. +- **PRD Phase E bridge + G6 README (2026-03-30):** `ModelIdentityMap::mark_dirty_key`; derived `LifeRecord::identity_map_key`; session/design/PRD §9.7; insert rejects non-empty `__update_exprs`; README competitive table + bullets (validators, scopes, F(), session); `DEVELOPMENT.md` db_integration note; SEAORM session row; tests `mark_dirty_key_matches_fingerprint`, `user_record_identity_map_key_matches_pk_fingerprint`, `insert_rejects_when_set_expr_pending`. +- **PRD Phase D — F-1 ORM `LifeRecord::update` expr RHS (2026-03-30):** `__update_exprs: HashMap`, `set__expr` per non-PK column; SET clause prefers expr over literal/`save_as`; integration test `record_set_n_expr_update_increments_on_postgres` in `column_f_update.rs` (mutex + shared setup); PRD §8.7. +- **PRD Phase D — F() in WHERE/ORDER BY (2026-03-30):** `tests/db_integration/column_f_where.rs` (`f_add_in_where_and_order_by_on_postgres`); `ColumnTrait::f_add` docs (`Expr::expr` + `ExprTrait` / `order_by_expr`); PRD §8.7 Phase D note. +- **PRD V-5 `#[validate(custom = path)]` (2026-03-30):** `lifeguard-derive` parses field `#[validate(custom = path)]`, generates `ActiveModelBehavior::validate_fields` using `ActiveModelTrait::get` + UFCS; `validate` allowed on `LifeModel`/`LifeRecord`; reject on `#[ignore]` fields; tests `validate_attr_tests` in `lifeguard-derive/tests/test_minimal.rs`; PRD §6.7 + SEAORM parity row; `docs/planning/README.md` lists `DESIGN_FIND_RELATED_SCOPES.md`. +- **PRD follow-on batch (2026-03-30):** `ValidateOp::Delete` + `run_validators` on derive `delete`; `SelectQuery::scope_or` / `scope_any`; SI-3 test `infer_schema_table_filter_si3.rs`; `db_integration/column_f_update.rs` (F-add UPDATE); `DESIGN_FIND_RELATED_SCOPES.md`; PRD §5.7/§6.7/§7.7/§8.7, SEAORM mapping, DEVELOPMENT, migrate README. +- **PRD iteration 2 branch (2026-03-30):** Follow-on work for `PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md` uses branch `feat/schema_validators_session_and_scopes_2` (local; set upstream with `git push -u origin feat/schema_validators_session_and_scopes_2` when publishing). PRD header documents the branch. +- **PRD iteration: V-3 aggregate validation + infer-schema smoke (2026-03-30):** `ValidationStrategy` (`FailFast` | `Aggregate`), `ActiveModelBehavior::validation_strategy`, `run_validators_with_strategy`; tests in `active_model/validation.rs`. `lifeguard-migrate/tests/infer_schema_postgres_smoke.rs` (optional `TEST_DATABASE_URL` / `DATABASE_URL`). PRD §5.7/§6.7, `DEVELOPMENT.md`, `lifeguard-migrate/README.md`, `SEAORM_LIFEGUARD_MAPPING.md`. - **find_latest_generated_migration parse skip (2026-03-30):** Non-numeric `*_generated_from_entities.sql` prefix no longer aborts the whole scan (`continue` vs `?`); tests `find_latest_skips_non_numeric_prefix_files`, `list_chronological_skips_non_numeric_prefix_files`. Migration delta behavior unchanged (merged chronological baseline). - **Migration diff merged history (2026-03-30):** `accumulate_table_baselines_from_dir` replays all `*_generated_from_entities.sql` in timestamp order (full CREATE + delta fragments); `build_service_migration_body_from_service_dir` fixes false full regen when latest file is ALTER-only. Removed erroneous `20260330201157_*`; unit test `merged_chronological_full_then_alter_yields_empty_diff`. `migrations/README.md` updated. - **Example entity + ALTER delta verified (2026-03-30):** `Category.sort_order` (`INTEGER NOT NULL DEFAULT 0`) in `examples/entities/src/inventory/category.rs`; `generate-migrations` emitted `migrations/generated/inventory/20260330201148_generated_from_entities.sql` with `ALTER TABLE categories ADD COLUMN IF NOT EXISTS sort_order ...` only (delta header present). Not committed unless user requests. @@ -15,7 +95,7 @@ - **PRD Phase E v0 — session identity map (2026-03-28):** `src/session/` — `ModelIdentityMap`, `fingerprint_pk_values` (`session/pk.rs`); exported from `lib.rs`; unit tests; `docs/planning/DESIGN_SESSION_UOW.md`; PRD §9.7 + milestone §0.1 (v0 only — flush/pool follow-on). `cargo test -p lifeguard --lib`, `cargo clippy -p lifeguard --all-targets -- -D warnings`. - **PRD Phase D v0 — F-style column expr (2026-03-28):** `ColumnTrait::f_add` / `f_sub` / `f_mul` / `f_div` → `SimpleExpr` for `UPDATE SET`; unit tests in `column_trait.rs`. PRD §8.7; `docs/planning/DEV_RUSTDOC_AND_COVERAGE.md` + `DEVELOPMENT.md` link for rustdoc/coverage process. - **PRD Phase C v0 — scopes (2026-03-28):** `src/query/scope.rs` — `SelectQuery::scope`, `IntoScope` (delegates to `filter`); module docs for soft-delete AND semantics; unit tests composition + soft-delete. PRD §7.7; `SEAORM_LIFEGUARD_MAPPING.md` scoped queries row. `cargo test -p lifeguard --lib`, `cargo clippy -p lifeguard --lib -- -D warnings`. -- **PRD Phase B v0 — validators (2026-03-28):** `ValidateOp` (`Insert`/`Update`), `ValidationError`, `ActiveModelError::Validation(Vec<_>)` with `Display`; `ActiveModelBehavior::validate_fields` / `validate_model` (default no-op); `run_validators` (field → model, fail-fast); `lifeguard-derive` `insert`/`update` call `run_validators` after `before_insert`/`before_update`. Unit tests in `src/active_model/validation.rs`. PRD §6.7. `cargo test -p lifeguard`, `cargo test -p lifeguard-derive`, `cargo clippy -p lifeguard -p lifeguard-derive --all-targets -- -D warnings`; `db_integration_suite` OK with `--test-threads=1` (parallel SIGSEGV env quirk). +- **PRD Phase B v0 — validators (2026-03-28):** `ValidateOp` (`Insert` / `Update`; **`Delete`** added in follow-on), `ValidationError`, `ActiveModelError::Validation(Vec<_>)` with `Display`; `ActiveModelBehavior::validate_fields` / `validate_model` (default no-op); `run_validators` (field → model, fail-fast); `lifeguard-derive` `insert`/`update`/`delete` call `run_validators` after `before_*`. Unit tests in `src/active_model/validation.rs`. PRD §6.7. `cargo test -p lifeguard`, `cargo test -p lifeguard-derive`, `cargo clippy -p lifeguard -p lifeguard-derive --all-targets -- -D warnings`; `db_integration_suite` OK with `--test-threads=1` (parallel SIGSEGV env quirk). - **PRD Phase A v0 — schema inference (2026-03-30):** `lifeguard-migrate infer-schema` + `lifeguard_migrate::schema_infer` (`infer_schema_rust`, `InferOptions`); PostgreSQL `information_schema` introspection; conservative type mapping; composite PK TODO; omitted columns commented. PRD §5.7. `cargo test -p lifeguard-migrate schema_infer`, `cargo clippy -p lifeguard-migrate -- -D warnings`. - **Toxiproxy fallback test ignored (2026-03-28):** `tests/db_integration/pool_read_replica.rs` — `pooled_read_falls_back_to_primary_when_replica_lagging` marked `#[ignore]` (flaky: `is_replica_lagging` not true within ~10s after proxy disable in CI). TODO on test + `WalLagMonitor` follow-up. Prior attempt: `src/pool/wal.rs` statement_timeout + reconnect on query error. **Git:** `feat/lifeguard-pool` pushed — main change `ffd0300` (`farm git`; no Cursor co-author). - **perf-idam `Cargo.lock` for `--locked` CI (2026-03-28):** Standalone `examples/perf-idam` had no `Cargo.lock` (or it was missing/stale); `cargo generate-lockfile` in that dir; verified `cargo test --locked --no-fail-fast`. Commit `examples/perf-idam/Cargo.lock` so Tilt/CI `cargo test --locked` succeeds. @@ -124,12 +204,15 @@ - Compile-time warnings for unsupported types - Next core traits from SEAORM_LIFEGUARD_MAPPING.md -## Metrics -- Edge case coverage: 85% (up from 70%) -- Test coverage: 80% (up from 75%) -- Tests passing: 34+ (up from 30) - Added 4 JSON serialization tests +## Metrics — **historical snapshot (not auto-updated)** + +*The bullets below are a **point-in-time** capture from early Memory Bank use. For **current** test counts, CI status, and shipped features, prefer the **Completed ✅** section above, `cargo test` / coverage output, and repository docs—not this block.* + +- Edge case coverage: 85% (up from 70%) — *historical* +- Test coverage: 80% (up from 75%) — *historical* +- Tests passing: 34+ (up from 30) — *historical* (JSON serialization era); **current** lib tests are listed under **Completed ✅** (e.g. hundreds of `cargo test -p lifeguard --lib` passes in later entries) - Memory Bank files: 6/6 initialized -- CRUD operations: Fixed critical bug in get() method for unset field detection -- ActiveModel/Record Operations: 7/12 implemented (58%) - Added from_json() and to_json() -- Query Builder Methods: 19/20 implemented (95%) - Added JOIN operations (join, left_join, right_join, inner_join) -- Relations: RelationTrait implemented with functional query building (belongs_to, has_one, has_many, has_many_through) \ No newline at end of file +- CRUD operations: *historical note:* fixed `get()` unset-field behavior (see **Completed** / git history for current CRUD surface) +- ActiveModel/Record Operations: 7/12 implemented (58%) — *historical*; see **Completed** for `from_json`/`to_json`, validators, session hooks, etc. +- Query Builder Methods: 19/20 implemented (95%) — *historical*; see **Completed** for CTE/window/join additions +- Relations: RelationTrait implemented with functional query building (belongs_to, has_one, has_many, has_many_through) — *still directionally true; see **Completed** for `find_related`/scopes docs* \ No newline at end of file diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 00000000..72e8d8a0 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,239 @@ +# Lifeguard architecture + +This document is the **detailed** architecture reference: numbered call flows (primary vs replicas, Redis, LifeReflector), multi-service deployment, connection pooling, and cache coherence sequences. For repository status see [COMPARISON.md](./COMPARISON.md#repository-status); for quick start and doc index, see the [README](./README.md). + +--- + +## Target architecture + +**Numbered edges** show typical **order of operations** on the data plane. **Solid lines** are the ORM/pool path (writes always hit **primary**; reads go to **primary** or **replica** tier depending on `ReadPreference`, WAL lag, and pool routing). **Dotted lines** are **optional** cache-aside (your app or framework checks Redis before Postgres) and **background** coherence (LifeReflector runs out-of-band after commits—not on your request’s critical path). + +```mermaid +flowchart TB + subgraph ORM["Application & ORM"] + App[Application / LifeModel / LifeRecord] + SQ[SeaQuery → SQL] + end + + subgraph PoolLayer["LifeguardPool + LifeExecutor"] + Pool[LifeguardPool dispatch] + Ex[LifeExecutor + may_postgres] + end + + subgraph PG["PostgreSQL cluster"] + Pri[(Primary
all writes)] + Rep[(Replica tier
reads when healthy)] + end + + subgraph Cache["Redis"] + Redis[(Cache
active set)] + end + + subgraph Refl["LifeReflector — background"] + LR[Leader: LISTEN / NOTIFY] + end + + App -->|1 build query| SQ + SQ -->|2 submit| Pool + Pool -->|3 acquire slot| Ex + + Ex -->|4a writes + strong reads| Pri + Ex -->|4b reads when replica OK| Rep + + App -.->|5 optional GET| Redis + Redis -.->|6 miss → ORM read path| App + + Pri -->|7 NOTIFY after commit| LR + LR -.->|8 if key warm: EXISTS| Redis + LR -->|9 refresh row| Pri + LR -.->|10 SETEX| Redis + + style Pri fill:#c0c0c0,stroke:#333,stroke-width:2px + style Rep fill:#d8d8d8,stroke:#333,stroke-width:2px + style Redis fill:#ffcccb,stroke:#333,stroke-width:2px + style LR fill:#add8e6,stroke:#333,stroke-width:2px + style Pool fill:#90ee90,stroke:#333,stroke-width:2px + style Ex fill:#90ee90,stroke:#333,stroke-width:2px +``` + +**Legend (numbers):** **1–4** — synchronous request path through ORM → pool → **primary** (writes, read-your-writes, or forced primary) or **replicas** (scaled reads when the pool routes there). **5–6** — optional Redis **read-through**: not automatic in every API today; pattern is GET first, on miss run **1–4** then populate Redis. **7–10** — **asynchronous**: after a successful commit, **NOTIFY** wakes LifeReflector; it refreshes Redis only for keys already in the active set (see [The Killer Feature: LifeReflector](./VISION.md#the-killer-feature-lifereflector) in **[VISION.md](./VISION.md)**). + +**Key Components:** +- **LifeguardPool**: Persistent connection pool; routes to primary vs replica **worker pools** using WAL lag and optional [`ReadPreference`](./src/pool/pooled.rs). +- **LifeExecutor**: Database execution abstraction over `may_postgres`. +- **LifeModel/LifeRecord**: Complete ORM layer (replaces SeaORM). +- **SeaQuery**: SQL building (borrowed, compatible with coroutines). +- **may_postgres**: Coroutine-native PostgreSQL client (foundation). +- **Primary vs replicas**: Writes **always** use the primary URL; reads may use the replica tier when configured and healthy. +- **LifeReflector**: Background cache coherence (not on the hot path of a single `SELECT`). +- **Redis**: Optional cache layer; coherence refresh is **7–10**, not **1–4**. + +## Multi-service deployment + +```mermaid +flowchart TB + subgraph Frontend["Frontend / Clients"] + Web[Web App] + Mobile[Mobile App] + API[API Clients] + end + + subgraph BFF["BFF Layer
Built with BRRTRouter"] + BFF_Service[Backend for Frontend
API Gateway / Router] + end + + subgraph Backend["Backend Microservices
Your Business Logic"] + MS1[User Service] + MS2[Product Service] + MS3[Order Service] + MSN[Service N
Your Domain] + end + + subgraph Lifeguard + Pool[LifeguardPool] + Executor[LifeExecutor] + LifeModel[LifeModel / LifeRecord] + SeaQuery[SeaQuery] + end + + subgraph DataLayer["Data layer"] + may_postgres[may_postgres] + PG_P[(PostgreSQL Primary)] + PG_R[(PostgreSQL Replicas)] + Redis[(Redis Cache)] + end + + subgraph LifeReflector + Reflector[LifeReflector Leader] + end + + Web --> BFF_Service + Mobile --> BFF_Service + API --> BFF_Service + + BFF_Service --> MS1 + BFF_Service --> MS2 + BFF_Service --> MS3 + BFF_Service --> MSN + + MS1 -->|1| Pool + MS2 -->|1| Pool + MS3 -->|1| Pool + MSN -->|1| Pool + + Pool -->|2| Executor + Executor --> LifeModel + LifeModel --> SeaQuery + SeaQuery -->|3 SQL| Executor + Executor -->|4| may_postgres + may_postgres -->|5a writes / RYW reads| PG_P + may_postgres -->|5b scaled reads when routed| PG_R + + MS1 -.->|optional cache| Redis + MS2 -.->|optional cache| Redis + MS3 -.->|optional cache| Redis + MSN -.->|optional cache| Redis + + PG_P -->|NOTIFY bg| Reflector + Reflector -.->|refresh warm keys| Redis + Reflector -->|re-read| PG_P + + style Frontend fill:#e1f5ff + style Web fill:#e1f5ff + style Mobile fill:#e1f5ff + style API fill:#e1f5ff + style BFF fill:#add8e6 + style BFF_Service fill:#add8e6 + style Backend fill:#d4edda + style MS1 fill:#d4edda + style MS2 fill:#d4edda + style MS3 fill:#d4edda + style MSN fill:#d4edda + style Pool fill:#90ee90 + style Executor fill:#90ee90 + style LifeModel fill:#90ee90 + style may_postgres fill:#90ee90 + style PG_P fill:#c0c0c0 + style PG_R fill:#d8d8d8 + style Redis fill:#ffcccb + style Reflector fill:#add8e6 +``` + +**Call order on the request path:** **1** service calls into **`LifeguardPool`** → **2** **`LifeExecutor`** → **3–4** SQL via **`may_postgres`** → **5a** **primary** (all writes; reads when forced or RYW) or **5b** **replicas** (reads when the pool’s WAL/routing allows). **Dotted:** optional Redis in front of Postgres; **NOTIFY** + Reflector runs **asynchronously** after commit (not numbered on the hot path). + +## Connection pool architecture + +Each **slot** is a long-lived `may_postgres` connection; the pool maintains separate **primary** and **replica** worker tiers when replicas are configured—**writes** and primary-tier reads use primary slots; **replica** slots serve scaled reads when WAL lag allows (see [pool docs](./docs/POOLING_OPERATIONS.md)). + +```mermaid +graph TD + subgraph LifeguardPool["LifeguardPool
The 300 Spartans"] + S[Semaphore
max_connections tokens
100-500 limit] + subgraph Slots["Connection Slots
Persistent & Reused"] + C1[Slot 1
in_use: false
ready] + C2[Slot 2
in_use: true
executing query] + C3[Slot 3
in_use: false
ready] + CN[Slot N
in_use: false
ready] + end + end + + subgraph Traffic["Incoming Traffic
The Persian Empire"] + R1[Request 1] + R2[Request 2] + R3[Request 3] + RN[Request N
millions/sec] + end + + Traffic -->|acquire| S + S -->|find free| Slots + Slots -->|mark in_use| C2 + C2 -->|execute query| PG[PostgreSQL
The Pass] + PG -->|result| C2 + C2 -->|release| S + C2 -->|mark free| Slots + C2 -->|ready for| Traffic + + style LifeguardPool fill:#fff4e1 + style S fill:#fff4e1 + style Slots fill:#e1ffe1 + style PG fill:#e1ffe1 + style Traffic fill:#ffe1e1 + + Note[100 connections
handle millions of requests
through aggressive reuse] + LifeguardPool --> Note +``` + +## LifeReflector cache coherence + +```mermaid +sequenceDiagram + participant LifeRecord + participant Postgres + participant LifeReflector + participant Redis + participant LifeModel + + LifeRecord->>Postgres: Write (INSERT/UPDATE/DELETE) + Postgres->>Postgres: Commit Transaction + Postgres->>LifeReflector: NOTIFY table_changes, '{"id": 42}' + LifeReflector->>Redis: EXISTS lifeguard:model:table:42? + alt Key Exists (Active Item) + LifeReflector->>Postgres: SELECT * FROM table WHERE id = 42 (from Primary) + Postgres-->>LifeReflector: Fresh Data + LifeReflector->>Redis: SETEX lifeguard:model:table:42 + else Key Not Exists (Inactive) + LifeReflector->>LifeReflector: Ignore (item not cached, TTL expired) + end + LifeModel->>Redis: Read (GET lifeguard:model:table:42) + alt Cache Hit + Redis-->>LifeModel: Cached Data (Fresh) + else Cache Miss + LifeModel->>Postgres: Read (SELECT * FROM table WHERE id = 42) + Postgres-->>LifeModel: Data + LifeModel->>Redis: SETEX lifeguard:model:table:42 + end +``` + +--- + +[← README](./README.md) · [Vision](./VISION.md) · [Comparison](./COMPARISON.md) diff --git a/COMPARISON.md b/COMPARISON.md new file mode 100644 index 00000000..44778474 --- /dev/null +++ b/COMPARISON.md @@ -0,0 +1,181 @@ +# Competitive comparison and ecosystem + +*This document combines **repository truth** (what ships in-tree today) with a **competitive snapshot** versus other Rust ORMs and ecosystem positioning. In the table below, **Implementation Status** labels **shipped** crate behavior (including optional features), **partial** gaps, and **vision** rows (especially transparent cache and explicit read-preference APIs). Authoritative row-by-row coverage and percentages live in [SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) and `cargo doc`. The [repository status](#repository-status) section states what is implemented **today**; the [short summary](#implementation-status-summary-short) below the table complements that with parity-oriented completion notes.* + +## Repository status + +**Ground truth** for what is implemented in-tree versus narrative **target** behavior. For the technical pitch, see the [README](./README.md). + +### Current status (repository truth) + +- **In this crate today:** `LifeExecutor` / `MayPostgresExecutor`, `connect` and connection helpers, `SelectQuery` and the query stack, `#[derive(LifeModel)]` / `#[derive(LifeRecord)]` (`lifeguard-derive`), relations (including loaders and `find_related` / linked paths), migrations (`lifeguard::migration`, `lifeguard-migrate`), transactions, raw SQL helpers, partial models, optional **metrics** (including pool `pool_tier` labels) and **tracing** features, **channel logging** (`lifeguard::logging`), and **`LifeguardPool`** / **`PooledLifeExecutor`** (`lifeguard::pool`, re-exported at the crate root). +- **Pool maturity:** the pool is **production-usable** for the supported design: one OS thread per slot, **bounded** per-worker queues, configurable **acquire timeout**, optional **replica** tier with **WAL lag** routing and monitor give-up, **slot heal** after connectivity-class errors, **idle liveness** probes, and **max connection lifetime** with jitter. Operators should tune from [POOLING_OPERATIONS.md](./docs/POOLING_OPERATIONS.md); the PRD tracks closure and future work in [PRD_CONNECTION_POOLING.md](./docs/planning/PRD_CONNECTION_POOLING.md). **`ReadPreference`** + **`PooledLifeExecutor::with_read_preference`** let callers force primary-tier reads for read-your-writes while writes stay on the primary tier. +- **Migrations / schema tooling (`lifeguard-migrate`):** **`infer-schema`** introspects PostgreSQL and emits Rust entities (including **composite primary keys** via `#[primary_key]` on each PK column). **`compare-schema`** checks live `information_schema` vs merged generated migrations beyond table names—**column-name drift** for tables present in both the database and the migration baseline ([PRD §5](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md), [lifeguard-migrate README](./lifeguard-migrate/README.md)). +- **Scopes vs `find_related`:** parent entity **`scope`** predicates are **not** merged into **`find_related`**; chain filters or scopes on the **`SelectQuery`** returned from `find_related` ([`query::scope`](./src/query/scope.rs), [DESIGN_FIND_RELATED_SCOPES.md](./docs/planning/DESIGN_FIND_RELATED_SCOPES.md)). +- **LifeReflector (`lifeguard-reflector`):** distributed cache coherence is implemented in the workspace crate [`lifeguard-reflector`](./lifeguard-reflector/) (same repository as `lifeguard-derive`, `lifeguard-migrate`, and other `lifeguard-*` packages). Behavior and flow diagrams: [ARCHITECTURE.md](./ARCHITECTURE.md) and [The Killer Feature: LifeReflector](./VISION.md#the-killer-feature-lifereflector) in **[VISION.md](./VISION.md)**; the crate may be published or split out later without renaming it. +- **Docs vs code:** Mermaid diagrams and some marketing sections describe the **target** platform (cache tier, replica routing, pool). Treat [docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md), `cargo doc`, and `examples/` as the ground truth for what compiles. Consumer-facing **`///` docs** are actively maintained (opt-in advanced `SelectQuery` SQL, pool/read preference, session, relations); a strict `RUSTDOCFLAGS='-D warnings'` pass is an ongoing hygiene goal. + +--- + +## Competitive metrics: Lifeguard vs Rust ORMs + +| Feature | Lifeguard Promise | Implementation Status | SeaORM | Diesel | SQLx | +|---------|-------------------|----------------------|--------|--------|------| +| **Concurrency Model** | ✅ Coroutine-native (`may`) | ✅ **Implemented** | ❌ Async/await (Tokio) | ❌ Sync-only | ❌ Async/await (Tokio) | +| **Performance (Hot Paths)** | ✅✅✅ 2-5× faster | 🟡 **Architectural** | ⚠️ Async overhead | ✅ Fast (sync) | ⚠️ Async overhead | +| **Performance (Small Queries)** | ✅✅✅ 10×+ faster | 🟡 **Architectural** | ⚠️ Future allocation | ✅ Fast | ⚠️ Future allocation | +| **Memory Footprint** | ✅✅ Low (stackful coroutines) | 🟡 **Architectural** | ⚠️ Higher (heap futures) | ✅ Low | ⚠️ Higher (heap futures) | +| **Predictable Latency** | ✅✅✅ Deterministic scheduling | 🟡 **Architectural** | ⚠️ Poll-based (variable) | ✅ Predictable | ⚠️ Poll-based (variable) | +| **Type Safety** | ✅✅✅ Compile-time validation | ✅ **Implemented** | ✅✅ Compile-time validation | ✅✅✅ Strong compile-time | ✅✅ Compile-time SQL checks | +| **ORM Features** | ✅✅✅ Complete (SeaORM parity) | 🟡 **High coverage** (core traits, relations, query builder; see mapping doc for %) | ✅✅✅ Complete | ✅✅ Good | ❌ Query builder only | +| **CRUD Operations** | ✅✅✅ Full support | ✅ **Implemented** (insert/update/save/delete via ActiveModelTrait) | ✅✅✅ Full support | ✅✅ Full support | ⚠️ Manual SQL | +| **Relations** | ✅✅✅ All types (has_one, has_many, belongs_to, many_to_many) | ✅ **Implemented** (Complete with eager/lazy loading, composite keys, DeriveLinked) | ✅✅✅ All types | ✅✅ Basic support | ❌ Manual joins | +| **Migrations** | ✅✅✅ Programmatic, data seeding, advanced ops | 🟡 **Partial** (`lifeguard::migration` + `lifeguard-migrate` + **`DeriveMigrationName`** / **`MigrationName`**; codegen paths still evolve) | ✅✅✅ Programmatic | ✅✅ CLI-based | ⚠️ Manual SQL | +| **Schema Inference** | ✅✅✅ From database (Diesel equivalent) | 🟡 **Partial** (`lifeguard-migrate infer-schema` / `schema_infer`, composite PK attributes, **`compare-schema`** column-name drift vs merged migrations; see [PRD §5](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) | ✅✅ From database | ✅✅✅ `table!` macro | ❌ No | +| **Query Builder** | ✅✅✅ Type-safe, chainable | ✅ **Implemented** (19/20 methods, 95% coverage) | ✅✅✅ Type-safe, chainable | ✅✅✅ Compile-time checked | ✅✅ Compile-time SQL | +| **Transactions** | ✅✅✅ Full support | ✅ **Implemented** (Roadmap Epic 01) | ✅✅✅ Full support | ✅✅ Full support | ✅✅ Full support | +| **Batch Operations** | ✅✅✅ insert_many, update_many, delete_many | ✅ **Implemented** | ✅✅✅ Batch support | ✅✅ Batch support | ⚠️ Manual | +| **Upsert** | ✅✅✅ save(), on_conflict() | ✅ **Implemented** (save() method exists) | ✅✅✅ save(), on_conflict() | ✅✅ on_conflict() | ⚠️ Manual SQL | +| **Pagination** | ✅✅✅ paginate(), paginate_and_count() | ✅ **Implemented** | ✅✅✅ Pagination helpers | ⚠️ Manual | ⚠️ Manual | +| **Entity Hooks** | ✅✅✅ before/after lifecycle events | ✅ **Implemented** (ActiveModelBehavior with 8 lifecycle hooks) | ✅✅✅ Hooks support | ❌ No | ❌ No | +| **Validators** | ✅✅✅ Field & model-level | 🟡 **Partial** — `run_validators` / `run_validators_with_strategy`, `ValidationStrategy::{FailFast, Aggregate}`, `ActiveModelBehavior::validate_fields` / `validate_model` / `validation_strategy`, derive `#[validate(custom = …)]`, `ValidateOp::Delete`; [`lifeguard::predicates`](./src/active_model/predicates.rs) for compose-in-`validate_fields`; not SeaORM’s full built-in validator attribute set — [PRD §6](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md) | ⚠️ Limited | ❌ No | ❌ No | +| **Soft Deletes** | ✅✅✅ Built-in support | ✅ **Implemented** (`#[soft_delete]` + `SelectQuery` / loader filtering) | ⚠️ Manual | ❌ No | ❌ No | +| **Auto Timestamps** | ✅✅✅ created_at, updated_at | ✅ **Implemented** (`#[auto_timestamp]` on `LifeRecord` insert/update paths) | ⚠️ Manual | ❌ No | ❌ No | +| **Session/Unit of Work** | ✅✅✅ Identity map, dirty tracking | 🟡 **Partial** (`ModelIdentityMap`, `Session`, `attach_session` / auto-dirty enqueue, `flush_dirty` / `flush_dirty_with_map_key`, pending insert + promote, `flush_dirty_in_transaction` / `flush_dirty_in_transaction_pooled`, `LifeRecord::identity_map_key`; [PRD §9](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) | ❌ No | ❌ No | ❌ No | +| **Scopes** | ✅✅✅ Named query scopes | 🟡 **Partial** (`SelectQuery::scope`, `scope_or` / `scope_any`, `IntoScope`, `lifeguard::scope`; **`find_related`** does not merge parent scopes—chain on returned `SelectQuery` — [PRD §7](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) | ❌ No | ❌ No | ❌ No | +| **Model Managers** | ✅✅✅ Custom query methods | ✅ **Implemented** (ModelManager trait + custom methods pattern) | ❌ No | ❌ No | ❌ No | +| **F() Expressions** | ✅✅✅ Database-level expressions | 🟡 **Partial** — `ColumnTrait::f_add` / `f_sub` / `f_mul` / `f_div`, derived `set_*_expr` + `update()`, `Expr::expr` + `ExprTrait` / `order_by_expr` for `WHERE`/`ORDER BY`; **PostgreSQL:** mixed numeric operand types follow server promotion rules—Lifeguard does not inject casts; use matching types, `SimpleExpr`, or `Expr::cust` for explicit `::bigint` / `::numeric` when required — [PRD §8](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md) | ❌ No | ⚠️ Limited | ❌ No | +| **Subqueries** | ✅✅✅ Full support | 🟡 **Partial** ([`join_subquery`](./src/query/select.rs), [`subquery_column`](./src/query/select.rs); not every SeaQuery subquery surface) | ✅✅✅ Full support | ✅✅ Full support | ✅✅ Manual SQL | +| **CTEs** | ✅✅✅ WITH clauses | 🟡 **Partial** ([`with_cte`](./src/query/select.rs) + lifeguard `all`/`one`; opt-in advanced SQL — [crate `query::select`](./src/query/select.rs)) | ✅✅✅ WITH clauses | ✅✅ WITH clauses | ✅✅ Manual SQL | +| **Window Functions** | ✅✅✅ Full support | 🟡 **Partial** ([`window`](./src/query/select.rs) / [`expr_window*`](./src/query/select.rs) / [`window_function_cust`](./src/query/select.rs)) | ✅✅✅ Full support | ✅✅ Full support | ✅✅ Manual SQL | +| **Eager Loading** | ✅✅✅ Multiple strategies (joinedload, subqueryload, selectinload) | ✅ **Implemented** (selectinload strategy with FK extraction) | ✅✅✅ Eager loading | ⚠️ Manual | ❌ Manual | +| **Raw SQL** | ✅✅✅ find_by_statement(), execute_unprepared() | ✅ **Implemented** (Architecture supports raw SQL) | ✅✅✅ Raw SQL support | ✅✅✅ Raw SQL support | ✅✅✅ Primary feature | +| **Connection Pooling** | ✅✅✅ Persistent, semaphore-based, health monitoring | ✅ **Shipped** ([`LifeguardPool`](./src/pool/pooled.rs): bounded queues, acquire timeout, heal, lifetime, metrics w/ `pool_tier`; see [pooling PRD](./docs/planning/PRD_CONNECTION_POOLING.md) for remaining parity) | ✅✅✅ Built-in pool | ⚠️ External (r2d2) | ✅✅✅ Built-in pool | +| **Replica Read Support** | ✅✅✅ WAL-based health monitoring, automatic routing | ✅ **Shipped** (replica tier + [`WalLagMonitor`](./src/pool/wal.rs); routing is pool-internal, not SeaORM-identical API) | ❌ No | ❌ No | ❌ No | +| **Read Preferences** | ✅✅✅ primary, replica, mixed, strong | 🟡 **Partial** ([`ReadPreference`](./src/pool/pooled.rs) + [`PooledLifeExecutor::with_read_preference`](./src/pool/pooled.rs) for explicit primary-tier reads; default pool routing still WAL/replica-aware; not full SeaORM “mixed/strong” semantics) | ❌ No | ❌ No | ❌ No | +| **Distributed Caching** | ✅✅✅✅ **LifeReflector (UNIQUE)** | 🟡 **Architectural** (Not in SeaORM mapping, may exist) | ❌ No | ❌ No | ❌ No | +| **Cache Coherence** | ✅✅✅✅ **Zero-stale reads (UNIQUE)** | 🟡 **Architectural** (Not in SeaORM mapping, may exist) | ❌ No | ❌ No | ❌ No | +| **TTL-Based Active Set** | ✅✅✅✅ **Adaptive caching (UNIQUE)** | 🟡 **Architectural** (Not in SeaORM mapping, may exist) | ❌ No | ❌ No | ❌ No | +| **PostgreSQL Features** | ✅✅✅ Views, materialized views, JSONB, FTS, PostGIS, partitioning | 🟡 **Partial** (JSONB ✅ core feature, others future) | ✅✅✅ Most features | ✅✅✅ Most features | ✅✅✅ All features (raw SQL) | +| **Observability** | ✅✅✅ Prometheus, OpenTelemetry, comprehensive metrics | ✅ **Implemented** (optional `metrics` / `tracing`; OTel-compatible / OTLP; [OBSERVABILITY.md](./OBSERVABILITY.md); pool series with `pool_tier`) | ✅✅ Basic metrics | ⚠️ Limited | ⚠️ Limited | +| **Developer Experience** | ✅✅✅ Familiar API, no async/await, clear errors | ✅ **Implemented** (SeaORM-like API) | ✅✅✅ Good, async/await required | ⚠️ Complex type system | ✅✅ Good, async/await required | +| **Learning Curve** | ✅✅ Moderate (familiar if you know SeaORM) | ✅ **Implemented** (SeaORM-like API) | ✅✅ Moderate | ⚠️ Steep (complex macros) | ✅✅ Moderate | +| **Production Ready** | ✅✅✅ Complete observability, health checks, metrics | 🟡 **Workload-dependent** (core ORM + pool + metrics/tracing ship; validate migrations, cache, and ops for your deployment) | ✅✅✅ Production ready | ✅✅✅ Production ready | ✅✅✅ Production ready | +| **Multi-Database** | ❌ PostgreSQL only (by design) | ✅ **By Design** | ✅✅ PostgreSQL, MySQL, SQLite | ✅✅ PostgreSQL, MySQL, SQLite | ✅✅✅ PostgreSQL, MySQL, SQLite, MSSQL | +| **Coroutine Runtime** | ✅✅✅✅ **Native support (UNIQUE)** | ✅ **Implemented** | ❌ Incompatible | ❌ Incompatible | ❌ Incompatible | + +### Legend + +**Implementation Status Column:** +- ✅ **Implemented** = Feature is fully implemented and working +- 🟡 **Partial/Future/Architectural** = Partially implemented, planned for future, or architectural feature (not in SeaORM mapping) +- ❌ **Not Implemented** = Feature promised but not yet implemented + +**Feature Comparison Columns:** +- ✅✅✅✅ = **Unique advantage** (no other ORM has this) +- ✅✅✅ = Excellent support +- ✅✅ = Good support +- ✅ = Basic support +- ⚠️ = Limited or manual implementation required +- ❌ = Not supported + +### Implementation status summary (short) + +**Strong in-tree today:** core traits (`LifeModelTrait`, `ModelTrait`, `ActiveModelTrait`, …), CRUD/save paths, `SelectQuery` stack, relations and eager/loader paths (including composite keys and linked traversals), migrations framework (`lifeguard::migration`, `lifeguard-migrate`), JSON column support, derive **`#[soft_delete]`** / **`#[auto_timestamp]`**, partial models, lifecycle hooks, **`LifeguardPool`** / **`PooledLifeExecutor`** with primary+replica tiers, WAL lag routing, slot heal, idle liveness, max connection lifetime, and optional **metrics** (including **`pool_tier`** labels) / **tracing**. + +**Partial (PRD v0 shipped; see [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)):** schema inference CLI/module (**composite PK** `#[primary_key]` on each column; **`compare-schema`** column-name drift vs merged migrations); validators (pipeline + aggregate mode + derive `custom` + **`lifeguard::predicates`** — this document and the mapping doc spell out shipped vs SeaORM gaps); `SelectQuery::scope` + **`scope_or` / `scope_any`** + **`#[scope]`** (parent scopes not merged into **`find_related`**—chain on the returned query); F() on **`UPDATE`** (derived `set_*_expr`) and **`WHERE`/`ORDER BY`** via SeaQuery (**PostgreSQL numeric promotion** documented in PRD §8 / `ColumnTrait::f_add`); **`Session`** / **`ModelIdentityMap`** with **`mark_dirty_key`**, **`attach_session`** (dirty enqueue when PK set), **`flush_dirty_in_transaction`** / **`flush_dirty_in_transaction_pooled`** ( **`LifeguardPool::exclusive_primary_write_executor`** ), **`register_pending_insert`** / **`flush_dirty_with_map_key`** / **`promote_pending_to_loaded`**. + +**Partial or roadmap:** deeper SQL builder coverage (e.g. more `SeaQuery` surface re-exported on [`SelectQuery`](./src/query/select.rs)), further migration tooling parity, and any remaining pooling parity called out in [PRD_CONNECTION_POOLING.md](./docs/planning/PRD_CONNECTION_POOLING.md) and [POOLING_OPERATIONS.md](./docs/POOLING_OPERATIONS.md). **Shipped on `SelectQuery`:** [`with_cte`](./src/query/select.rs) (CTE + `all`/`one`), [`join_subquery`](./src/query/select.rs), [`window`](./src/query/select.rs) / [`expr_window*`](./src/query/select.rs), existing [`subquery_column`](./src/query/select.rs) / [`window_function_cust`](./src/query/select.rs). **Pool reads:** [`ReadPreference`](./src/pool/pooled.rs) + [`PooledLifeExecutor::with_read_preference`](./src/pool/pooled.rs) force primary-tier reads when you need read-your-writes; default routing still follows WAL lag. **Session:** `LifeRecord::attach_session_with_model` auto-syncs literals into the identity-map `Rc` via `to_model()` when mutations notify the session ([PRD §9](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)); F-style `set_*_expr` remains record-only until `update()`. + +**Roadmap / vision:** productized “transparent Redis on every read”; LifeReflector and cache coherence in [`lifeguard-reflector`](./lifeguard-reflector/). + +For percentages and row-by-row status, use the mapping document linked in the section intro rather than this table alone. + +### Key differentiators + +**Lifeguard's Unique Advantages:** +1. **LifeReflector** - Distributed cache coherence (Oracle Coherence–style active set) — **unique**; **🟡** product evolution in [`lifeguard-reflector`](./lifeguard-reflector/) +2. **Coroutine-Native** - No async overhead, deterministic scheduling — **unique** among these ORMs ✅ +3. **WAL-Based Replica Routing** - Pool + [`WalLagMonitor`](./src/pool/wal.rs) — **shipped** for `LifeguardPool` reads ✅ +4. **TTL-Based Active Set** - Adaptive caching — **🟡** vision / reflector path; not automatic on every app read +5. **DeriveLinked Macro** - Multi-hop relationship code generation — **competitive advantage** ✅ (SeaORM has no direct equivalent) +6. **Session/Unit of Work** — **🟡** `Session` + identity map + `flush_dirty` / `flush_dirty_with_map_key` / pending insert + promote / `flush_dirty_in_transaction` / `flush_dirty_in_transaction_pooled` ([PRD §9](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) + +**Where Lifeguard Matches or Exceeds:** +- ✅ Substantial SeaORM-oriented coverage (see mapping doc for %; core ORM paths strong) +- ✅ Relations system with composite keys and eager/lazy loading +- ✅ Query builder with 95% method coverage +- ✅ Better performance potential (2-5× faster on hot paths - architectural) +- ✅ Lower memory footprint (architectural) +- ✅ Predictable latency (architectural) + +**Trade-offs:** +- ❌ PostgreSQL-only (by design - enables advanced features) +- ❌ Requires `may` coroutine runtime (not Tokio) +- ❌ Smaller ecosystem (newer project) +- ⚠️ Some roadmap items remain (further query-builder / migration tooling parity, etc.); see [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md), mapping doc, and pooling docs + +### Performance comparison (estimated) + +| Metric | Lifeguard | SeaORM | Diesel | SQLx | +|--------|-----------|--------|--------|------| +| **Simple Query Latency** | 0.1-0.5ms | 0.5-2ms | 0.2-1ms | 0.5-2ms | +| **Hot Path Throughput** | 2-5× faster | Baseline | 1-2× faster | Baseline | +| **Small Query Overhead** | Minimal | Future allocation | Minimal | Future allocation | +| **Memory per Connection** | ~100 bytes | ~1-2 KB | ~100 bytes | ~1-2 KB | +| **Concurrent Connections** | 800+ (1MB stack) | Limited by Tokio | Limited by threads | Limited by Tokio | +| **p99 Latency** | < 5ms (predictable) | 5-20ms (variable) | < 5ms (predictable) | 5-20ms (variable) | + +*Note: Performance numbers are estimates based on architecture. Actual benchmarks will be published after implementation.* + +*ORM note: the **Diesel** column reflects typical **sync** / blocking **Diesel** usage; **Diesel-async** follows **async**/**Tokio** patterns and is closer in deployment to **SeaORM** / **SQLx** than to sync **Diesel**—see [Ecosystem compatibility](#ecosystem-compatibility).* + +### Target performance claims (product narrative) + +**Target Performance:** +- 2-5× faster than async ORMs on hot paths +- 10×+ faster on small queries (no future allocation overhead) +- Predictable p99 latency (< 5ms for simple queries) +- Lower memory footprint than async alternatives + +**Real-World Use Cases:** +- **BRRTRouter**: High-throughput API routing with sub-millisecond database access (100,000+ requests/second) +- **High-Scale Microservices**: Applications requiring millions of requests/second with limited database connections +- **Low-Latency Systems**: Real-time applications needing predictable p99 latency (< 5ms) for database operations + +### Ecosystem compatibility + +**⚠️ Important:** **BRRTRouter + Lifeguard** and **Tokio + async/await ORMs** are **different supported stacks**. Both are Rust, but they assume **different runtimes**: **`may`** coroutines for **Lifeguard** vs **`async`/`await`** (commonly on **Tokio**) for **SeaORM**, **SQLx**, and **Diesel-async**. That is a **compatibility boundary for how we document and support integrations**—not a claim that Rust forbids linking crates. + +| Stack | Runtime / execution model | ORM or access layer | Notes | +|-------|---------------------------|---------------------|--------| +| **BRRTRouter + Lifeguard** | **`may`** coroutines | **Lifeguard** | Supported path for this repo’s ORM; **not** interchangeable with async-first ORMs as a single documented “drop-in” integration. | +| **SeaORM** | **`async`/`await`** — enable a runtime via crate features (e.g. **`runtime-tokio-native-tls`**, other `runtime-*` flags per upstream docs) | SeaORM | **Tokio** (or async-std) is selected by **feature flags**; not a **`may`** stack. | +| **SQLx** | **`async`/`await`** — requires a runtime feature (**Tokio** or **async-std**) | SQLx | **Unsupported / not recommended** to use without enabling a supported async runtime; misconfiguration may **panic** at runtime rather than failing at compile time—see SQLx feature docs. | +| **Diesel (sync / core)** | Blocking / synchronous API | Diesel | **Runtime-agnostic** for the classic blocking API when used from normal threads. | +| **Diesel-async** | **`async`/`await`**, **Tokio**-centric | Diesel-async | Async API; **not** the same portability story as sync **Diesel**. | + +**Practical status:** Mixing **BRRTRouter + Lifeguard (`may`)** with **SeaORM / SQLx / Diesel-async** in one **supported** application architecture is **not documented here**—each stack expects **`may`** vs **`async`/`await` + **Tokio** (or async-std) respectively. Choose **BRRTRouter + Lifeguard** for the **`may`** path; choose **Tokio + SeaORM**, **Tokio + SQLx**, or **Tokio + Diesel-async** for the mainstream **async/await** path. **Diesel (sync)** stays **runtime-agnostic** for blocking use. + +### When to use each ecosystem + +**Use BRRTRouter + Lifeguard if:** +- ✅ You're building with **BRRTRouter** (the coroutine API framework) +- ✅ You need **distributed cache coherence** (LifeReflector - unique to Lifeguard) +- ✅ You need **extreme scale** (millions of requests/second) +- ✅ You need **predictable latency** (API routers, real-time systems) +- ✅ You're **PostgreSQL-only** (enables advanced features) +- ✅ You want **Oracle Coherence-level functionality** + +**Use Tokio + async ORMs if:** +- ✅ You're using **Tokio** (or another **async** runtime) with **`async`/`await`** +- ✅ You need **multi-database support** (PostgreSQL, MySQL, SQLite, MSSQL) — typical for **SeaORM** / **SQLx** +- ✅ You want **mature, well-documented ORMs** (**SeaORM**, **SQLx**, **Diesel** / **Diesel-async**) +- ✅ You don't need distributed cache coherence +- ✅ You're building traditional **async/await** microservices + +**Ecosystem choice:** **BRRTRouter** implies **Lifeguard** on **`may`**. A **Tokio**-centric service typically picks **SeaORM**, **SQLx**, or **Diesel-async**; **Diesel (sync)** is **runtime-agnostic** when used as blocking I/O from threads. Cross-stack mixing is **unsupported** as a first-class integration story—not “impossible in Rust,” but **out of scope** for the documented paths above. + +--- + +[← README](./README.md) · [Roadmap](./ROADMAP.md) diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index b30593a1..78fa9842 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -37,9 +37,29 @@ For each PRD-driven or user-facing change, follow **`docs/planning/DEV_RUSTDOC_A ### `lifeguard-migrate` and schema inference -- **CLI:** `cargo run -p lifeguard-migrate -- infer-schema --database-url …` (or set `DATABASE_URL` / `LIFEGUARD_DATABASE_URL`). See **`lifeguard-migrate/README.md`** (`infer-schema` section) and **`docs/planning/DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md`**. +**CLI:** `cargo run -p lifeguard-migrate -- infer-schema --database-url …` (or set `DATABASE_URL` / `LIFEGUARD_DATABASE_URL`). **`compare-schema`** compares live base tables to merged `*_generated_from_entities.sql` under `--generated-dir` (DBA / CI table-name reconciliation). See **`lifeguard-migrate/README.md`** (`infer-schema` and `compare-schema`) and **`docs/planning/DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md`**. + - **Emitter goldens:** changing `lifeguard-migrate/src/schema_infer.rs` output may require updating files under **`lifeguard-migrate/tests/golden/`**. Run **`cargo test -p lifeguard-migrate schema_infer`** before merge. +#### Live Postgres tests (optional) + +These integration tests need a reachable Postgres URL; otherwise they skip. + +- **`infer_schema_postgres_smoke.rs`** — `infer_schema_rust` on `public`. +- **`infer_schema_table_filter_si3.rs`** — SI-3 table filter. +- **`infer_schema_cli_subprocess.rs`** — full **`infer-schema`** binary via **`CARGO_BIN_EXE_lifeguard-migrate`** (subprocess; requires the Cargo-built `lifeguard-migrate` test binary). +- **`migration_db_compare_smoke.rs`** — `compare_generated_dir_to_live_db` and **`compare-schema`** CLI. + +**Env (any one):** **`TEST_DATABASE_URL`**, **`DATABASE_URL`**, or **`LIFEGUARD_DATABASE_URL`**. + +#### `db_integration_suite` + +Integration tests under **`tests/db_integration/`** (run as the **`db_integration_suite`** target), including: + +- **`column_f_update`** — F-style `UPDATE SET`, derived **`set_*_expr`** / **`update()`**, insert guard for **`__update_exprs`**. +- **`column_f_where`** — `WHERE` / `ORDER BY` with **`Expr::expr`** and **`ColumnTrait::f_*`**. +- **`session_identity_flush`** — **`ModelIdentityMap`** / **`Session`**, **`attach_session`**, **`flush_dirty`** / **`flush_dirty_in_transaction`**, **`register_pending_insert`**, **`flush_dirty_with_map_key`**, **`flush_dirty_in_transaction_with_map_key`**, **`flush_dirty_in_transaction_pooled_with_map_key`**, **`promote_pending_to_loaded`**, **`LifeRecord::update`** / **`insert`** (PRD §9). + ### Development Workflow **Before writing code:** diff --git a/OBSERVABILITY.md b/OBSERVABILITY.md new file mode 100644 index 00000000..042477f4 --- /dev/null +++ b/OBSERVABILITY.md @@ -0,0 +1,54 @@ +# Observability + +Lifeguard is built for **OpenTelemetry-compatible** observability: optional **`tracing`** integration (spans and events that follow the ecosystem’s tracing model) and optional **Prometheus** metrics (`metrics` feature). You install **one** global `TracerProvider` and **one** `tracing` subscriber in the host process; Lifeguard does not take over globals—see [Host-owned OpenTelemetry](#host-owned-opentelemetry) below. + +**Backends:** the same instrumentation works with **OpenTelemetry-native** stacks (OTLP → Grafana, Jaeger, Tempo, etc.) and with **Datadog** via the [OpenTelemetry Protocol](https://opentelemetry.io/docs/specs/otlp/)—for example Datadog Agent OTLP intake, or `otel-collector` forwarding to Datadog. Use your org’s standard collector/agent; Lifeguard emits **tracing** and **metrics** shapes that fit those pipelines. + +**Deeper reference:** feature flags, Kind/Tilt dashboard refresh, and **metric tables** with labels — [docs/OBSERVABILITY.md](./docs/OBSERVABILITY.md). **App wiring** (Registry + `channel_layer`, no duplicate globals) — [docs/OBSERVABILITY_APP_INTEGRATION.md](./docs/OBSERVABILITY_APP_INTEGRATION.md). + +--- + +## Prometheus metrics + +When the `metrics` feature is enabled, typical series include (non-exhaustive; see [docs/OBSERVABILITY.md](./docs/OBSERVABILITY.md) for the full table): + +- `lifeguard_pool_size` — Current pool size +- `lifeguard_active_connections` — Active connections +- `lifeguard_connection_wait_time` — Time waiting for connection +- `lifeguard_query_duration_seconds` — Query execution time +- `lifeguard_query_errors_total` — Query errors +- `lifeguard_cache_hits_total` — Cache hits +- `lifeguard_cache_misses_total` — Cache misses +- `lifeguard_replica_lag_bytes` — Replica lag (bytes) +- `lifeguard_replica_lag_seconds` — Replica lag (seconds) +- `lifeguard_replicas_healthy` — Number of healthy replicas + +Pool-scoped series use a low-cardinality **`pool_tier`** label (`primary` / `replica`) where applicable. + +## OpenTelemetry tracing + +When the `tracing` feature is enabled: + +- Distributed tracing for database operations +- Spans for: connection acquisition, query execution, cache operations +- Integration with existing OpenTelemetry infrastructure (via your process’s `tracing` + OTLP exporter) + +### Host-owned OpenTelemetry + +Lifeguard does **not** set a global OpenTelemetry `TracerProvider`. Your service (for example **BRRTRouter**) must install **one** provider and **one** `tracing_subscriber::Registry` stack. Optionally add **`lifeguard::channel_layer()`** to that same `.with(...)` chain so events also go through Lifeguard’s may-channel logger. See **[docs/OBSERVABILITY_APP_INTEGRATION.md](./docs/OBSERVABILITY_APP_INTEGRATION.md)** and the **`lifeguard::logging`** rustdoc. + +## LifeReflector metrics + +Metrics for the [`lifeguard-reflector`](./lifeguard-reflector/) service (when enabled in that deployment): + +- `reflector_notifications_total` — Notifications received +- `reflector_refreshes_total` — Cache refreshes +- `reflector_ignored_total` — Ignored notifications (inactive items) +- `reflector_active_keys` — Active cache keys +- `reflector_redis_latency_seconds` — Redis operation latency +- `reflector_pg_latency_seconds` — PostgreSQL operation latency +- `reflector_leader_changes_total` — Leader election events + +--- + +[← README](./README.md) · [Operator guide & Kind/Grafana](./docs/OBSERVABILITY.md) · [App integration](./docs/OBSERVABILITY_APP_INTEGRATION.md) diff --git a/README.md b/README.md index 5c3f6acb..75efea6f 100644 --- a/README.md +++ b/README.md @@ -1,371 +1,52 @@

- Lifeguard logo + Lifeguard CI +   + CI uses Rust nightly (see workflow for pinned toolchain) +

+

+ Lifeguard logo

# 🛟 Lifeguard: Coroutine-Driven Database Runtime for Rust **Lifeguard** is a **coroutine-native PostgreSQL ORM and data access platform** built for Rust's `may` runtime. It aims for SeaORM-like ergonomics without async/`Tokio`: stackful coroutines and `may_postgres` as the database client. -### Current status (repository truth) - -- **In this crate today:** `LifeExecutor` / `MayPostgresExecutor`, `connect` and connection helpers, `SelectQuery` and the query stack, `#[derive(LifeModel)]` / `#[derive(LifeRecord)]` (`lifeguard-derive`), relations (including loaders and `find_related` / linked paths), migrations (`lifeguard::migration`, `lifeguard-migrate`), transactions, raw SQL helpers, partial models, optional **metrics** (including pool `pool_tier` labels) and **tracing** features, **channel logging** (`lifeguard::logging`), and **`LifeguardPool`** / **`PooledLifeExecutor`** (`lifeguard::pool`, re-exported at the crate root). -- **Pool maturity:** the pool is **production-usable** for the supported design: one OS thread per slot, **bounded** per-worker queues, configurable **acquire timeout**, optional **replica** tier with **WAL lag** routing and monitor give-up, **slot heal** after connectivity-class errors, **idle liveness** probes, and **max connection lifetime** with jitter. Operators should tune from [POOLING_OPERATIONS.md](./docs/POOLING_OPERATIONS.md); the PRD tracks closure and future work in [PRD_CONNECTION_POOLING.md](./docs/planning/PRD_CONNECTION_POOLING.md). -- **LifeReflector (`lifeguard-reflector`):** distributed cache coherence is implemented in the workspace crate [`lifeguard-reflector`](./lifeguard-reflector/) (same repository as `lifeguard-derive`, `lifeguard-migrate`, and other `lifeguard-*` packages). Behavior and architecture are described below; the crate may be published or split out later without renaming it. -- **Docs vs code:** Mermaid diagrams and some marketing sections describe the **target** platform (cache tier, replica routing, pool). Treat [docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md), `cargo doc`, and `examples/` as the ground truth for what compiles. - ---- - -## 🔥 Why Lifeguard? +## Why Lifeguard (technical bet) -**The Problem:** Existing Rust ORMs (SeaORM, Diesel, SQLx) are built for async/await and Tokio. The `may` coroutine runtime uses stackful coroutines, not async futures. These are **fundamentally incompatible architectures**—you cannot bridge them without significant performance penalties. +**The problem:** Existing Rust ORMs (SeaORM, Diesel, SQLx) target async/`Tokio`. The `may` coroutine runtime uses stackful coroutines, not async futures—**architectures do not bridge** without significant cost. For the narrative on that mismatch and the pain of forcing async ORMs onto a coroutine stack, see **[LIFEGUARD_BLOG_POST.md](./LIFEGUARD_BLOG_POST.md)**. -**The Solution:** Build a complete ORM from scratch using `may_postgres` (coroutine-native PostgreSQL client). No async runtime. No Tokio. Pure coroutine I/O. +**The approach:** A full ORM on **`may_postgres`** (coroutine-native PostgreSQL). No async runtime on the database path. Pure coroutine I/O. -**Why This Matters:** -- **BRRTRouter** (the coroutine API framework) needs blistering fast database access for high-throughput applications -- High-performance microservices need predictable, low-latency database access without async overhead -- Applications with extreme scale requirements (millions of requests/second) need efficient connection pooling when database connections are limited -- Coroutines offer deterministic scheduling, lower memory overhead, and predictable latency -- But without a proper ORM, developers are forced to choose: async ORM (overhead) or raw SQL (no safety) +**Who it is for:** Teams on **`may`** (for example **BRRTRouter**) who want **SeaORM-like** productivity, **typed models and queries**, a **production connection pool** (primary/replica, WAL-aware routing, optional read preference), **OTel-compatible** metrics/tracing, and a **cache-coherence** story (**LifeReflector** + Redis) aimed at **latency-sensitive** tiers. -**Lifeguard solves this** by providing a complete data platform that matches SeaORM's feature set but is built for coroutines, plus **distributed cache coherence** (LifeReflector) that no other ORM provides. - ---- - -## 🚀 What We're Building - -### Core ORM: LifeModel & LifeRecord - -A complete ORM system with two primary abstractions: - -**LifeModel** (Immutable Database Rows) -- Represents database rows as immutable Rust structs -- Generated via `#[derive(LifeModel)]` procedural macro -- Provides type-safe query builders -- Automatic row-to-struct mapping -- Complete SeaORM API parity - -**LifeRecord** (Mutable Change Sets) -- Separate abstraction for inserts and updates -- Generated via `#[derive(LifeRecord)]` procedural macro -- Type-safe mutation builders -- Automatic SQL generation via SeaQuery -- Change tracking (dirty fields) - -```rust -use lifeguard_derive::{LifeModel, LifeRecord}; - -#[derive(LifeModel, LifeRecord)] -#[table_name = "users"] -struct User { - #[primary_key] - id: i64, - email: String, - is_active: bool, -} - -// Inserts/selects go through LifeExecutor + SelectQuery / ActiveModelTrait; -// see lifeguard-derive tests and examples/ for full patterns (no Tokio required). -``` - -### Connection pool: LifeguardPool - -**In-tree:** [`LifeguardPool`](./src/pool/pooled.rs) (re-exported as `lifeguard::LifeguardPool`) — persistent `may_postgres` connections, one worker per slot, bounded per-worker job queues, configurable acquire timeout ([`LifeError::PoolAcquireTimeout`](./src/executor.rs)), optional read-replica routing with [`WalLagMonitor`](./src/pool/wal.rs), slot heal, idle liveness, max connection lifetime, and Prometheus metrics with a low-cardinality **`pool_tier`** label (`primary` / `replica`) on pool-scoped series. See [POOLING_OPERATIONS.md](./docs/POOLING_OPERATIONS.md), [DESIGN_CONNECTION_POOLING.md](./docs/planning/DESIGN_CONNECTION_POOLING.md), and [OBSERVABILITY.md](./docs/OBSERVABILITY.md). - -**Alternative:** open connections with [`connect`](./src/connection.rs) and run queries through [`MayPostgresExecutor`](./src/executor.rs) / [`LifeExecutor`](./src/executor.rs) when you do not need the pool. See [`examples/query_builder_example.rs`](./examples/query_builder_example.rs) for patterns. - -### The Killer Feature: LifeReflector - -**Distributed cache coherence system**—this is Lifeguard's unique advantage: - -> **Note:** LifeReflector is developed as the **`lifeguard-reflector`** workspace crate in this repository ([`./lifeguard-reflector`](./lifeguard-reflector/)). Enterprise licensing may still apply for some distributions; see that crate’s README. - -A **standalone microservice** that maintains cluster-wide cache coherence: - -- **Leader-elected Raft system:** Only one active reflector at a time (no duplicate work) -- **Postgres LISTEN/NOTIFY integration:** Subscribes to database change events -- **Intelligent cache refresh:** Only **re-writes** keys that already exist in Redis (TTL-based **active set**—no stale copy to fix if the key was never cached) -- **Read path populates Redis:** Cache miss → load from Postgres → `SETEX` (with TTL); new rows enter Redis when something **reads** them (or via warm-up), not from `NOTIFY` alone -- **Horizontal scaling:** All microservices benefit from single reflector - -**How it works:** - -1. **Reads (population):** A service checks **Redis first**. On a **miss**, it reads from **Postgres** and **writes the row into Redis** (e.g. `SETEX` + TTL). First-time and cold rows are cached here—this is how Redis gets populated. -2. **LifeRecord** (or the writer) commits to **Postgres**; the database path emits **`NOTIFY`** (payload identifies the row). -3. **LifeReflector** (leader) receives the notification. -4. Reflector checks whether that entity **key already exists** in Redis (active cached item). -5. **If it exists** → Reflector **re-reads from Postgres** and **updates Redis** so no client keeps a pre-write value. -6. **If it does not exist** → Reflector **ignores** the notify: there is **no cached row to invalidate**—nothing in Redis was wrong. The next read miss still runs step (1) and loads fresh data from Postgres into Redis. -7. **Cross-service reads:** Once a key is in Redis, other services can read it from Redis; steps 2–6 keep **already-cached** keys aligned with Postgres after writes. - -**Result:** Oracle Coherence–style **coherence for the active set** in Redis: lazy (or warmed) population on reads, plus **notify-driven refresh** only where a stale cache entry could otherwise exist. See the **sequence diagram** below (cache miss branch → Postgres → `SETEX`). - -**Enterprise:** commercial or source-available licensing may apply for some LifeReflector deployments. Source and package layout live under [`lifeguard-reflector`](./lifeguard-reflector/); contact enterprise@microscaler.io for licensing questions. - -### Transparent caching system (target) - -**Target behavior** (not fully wired as “magic” on every read path in this crate today): Lifeguard’s design calls for caching that still respects PostgreSQL primaries and replicas: - -- **Check Redis first:** Sub-millisecond reads if cached -- **Read from replicas:** When healthy (WAL lag < threshold) -- **Write to primary:** Always (as PostgreSQL was designed) -- **LifeReflector keeps cache fresh:** Automatic coherence across microservices ([`lifeguard-reflector`](./lifeguard-reflector/)) - -Your application code doesn't need to know about Redis, replicas, or cache coherence. It just calls `User::find_by_id(&pool, 42)?` and Lifeguard handles the rest. - -**Note:** For distributed cache coherence across multiple microservices, [`lifeguard-reflector`](./lifeguard-reflector/) provides automatic cache refresh using PostgreSQL LISTEN/NOTIFY. - -### Replica Read Support - -Advanced read routing with WAL lag awareness: - -- **WAL position tracking:** Monitors `pg_current_wal_lsn()` vs `pg_last_wal_replay_lsn()` -- **Dynamic health checks:** Automatically detects replica lag -- **Intelligent routing:** Routes reads to replicas only when healthy -- **Automatic fallback:** Falls back to primary if replicas are stale -- **Strong consistency mode:** Optional causal read-your-writes consistency - -**Read Preference Modes:** -- `primary` - Always read from primary -- `replica` - Use replicas when healthy -- `mixed` - Automatic selection (Redis → replica → primary) -- `strong` - Causal consistency (wait for replica to catch up) - -### Complete feature set (vision vs crate) - -The lists below mix **shipped**, **partial**, and **planned** capabilities. For a maintained feature matrix, see [SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md). - -**ORM features (SeaORM parity target):** -- ✅ Complete CRUD operations -- ✅ Type-safe query builders -- ✅ Relations (has_one, has_many, belongs_to, many_to_many) -- ✅ Migrations (programmatic, data seeding, advanced operations) -- ✅ Transactions -- ✅ Raw SQL helpers -- ✅ Batch operations -- ✅ Upsert support -- ✅ Pagination helpers -- ✅ Entity hooks & lifecycle events -- 🟡 Validators (trait pipeline + derive; see [PRD §6](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) -- ✅ Soft deletes -- ✅ Auto-managed timestamps - -**Competitive Features:** -- 🟡 Schema inference (`lifeguard-migrate infer-schema`; Diesel-style parity — [PRD §5](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) -- 🟡 Session/Unit of Work (`ModelIdentityMap`, `mark_dirty` / `flush_dirty`; full auto-dirty + pool `Session` — [PRD §9](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) -- 🟡 Scopes (`SelectQuery::scope`; [PRD §7](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) -- ✅ Model Managers (Django) -- 🟡 F() Expressions (`ColumnTrait::f_*` for `UPDATE SET`; [PRD §8](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) -- ✅ Advanced eager loading strategies (SQLAlchemy) - -**Unique Features (No Other ORM Has):** -- ✅ **LifeReflector** - Distributed cache coherence -- ✅ **Coroutine-native** - No async overhead -- ✅ **WAL-based replica routing** - Automatic health monitoring -- ✅ **TTL-based active set** - Adaptive caching +**Shipped vs aspirational:** Treat **[COMPARISON.md](./COMPARISON.md#repository-status)** (repository truth), `cargo doc`, and `examples/` as **source of truth**; some diagrams and marketing copy still describe **target** behavior (for example fully automatic transparent Redis on every read path). Product vision, feature lists, and the LifeReflector narrative: **[VISION.md](./VISION.md)**. Parity tables and competitive framing: **[COMPARISON.md](./COMPARISON.md)** and the [SeaORM mapping](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md). --- ## 🏗️ Architecture overview -Diagrams summarize the **target** platform (including pool and the [`lifeguard-reflector`](./lifeguard-reflector/) service). Components in grey or noted above may not be exposed from this crate yet. - -### Target architecture - -```mermaid -graph TD - App[Application Code] --> Pool[LifeguardPool] - Pool --> Executor[LifeExecutor] - Executor --> may_postgres[may_postgres] - may_postgres --> PostgreSQL[PostgreSQL] - - App --> LifeModel[LifeModel / LifeRecord] - LifeModel --> SeaQuery[SeaQuery SQL Builder] - SeaQuery --> Executor - - App --> Redis[Redis Cache] - Redis --> LifeReflector[LifeReflector Service] - PostgreSQL -- NOTIFY --> LifeReflector - LifeReflector --> Redis - - style App fill:#add8e6,stroke:#333,stroke-width:2px - style Pool fill:#90ee90,stroke:#333,stroke-width:2px - style Executor fill:#90ee90,stroke:#333,stroke-width:2px - style LifeModel fill:#90ee90,stroke:#333,stroke-width:2px - style SeaQuery fill:#90ee90,stroke:#333,stroke-width:2px - style may_postgres fill:#90ee90,stroke:#333,stroke-width:2px - style PostgreSQL fill:#c0c0c0,stroke:#333,stroke-width:2px - style Redis fill:#ffcccb,stroke:#333,stroke-width:2px - style LifeReflector fill:#add8e6,stroke:#333,stroke-width:2px -``` - -**Key Components:** -- **LifeguardPool**: Persistent connection pool with semaphore-based acquisition -- **LifeExecutor**: Database execution abstraction over `may_postgres` -- **LifeModel/LifeRecord**: Complete ORM layer (replaces SeaORM) -- **SeaQuery**: SQL building (borrowed, compatible with coroutines) -- **may_postgres**: Coroutine-native PostgreSQL client (foundation) -- **LifeReflector**: Distributed cache coherence microservice -- **Redis**: Transparent caching layer +High-level data flow: your app and ORM go through **`LifeguardPool`** to **PostgreSQL** (writes and strong reads on the **primary**; scaled reads may use **replicas** when routing allows). **Redis** is optional cache-aside; **[`lifeguard-reflector`](./lifeguard-reflector/)** refreshes warm keys **asynchronously** after commits (not on the `SELECT` hot path). ```mermaid -graph TD - subgraph Frontend["Frontend / Clients"] - Web[Web App] - Mobile[Mobile App] - API[API Clients] - end - - subgraph BFF["BFF Layer
Built with BRRTRouter"] - BFF_Service[Backend for Frontend
API Gateway / Router] - end - - subgraph Backend["Backend Microservices
Your Business Logic"] - MS1[User Service] - MS2[Product Service] - MS3[Order Service] - MSN[Service N
Your Domain] - end - - subgraph Lifeguard - Pool[LifeguardPool] - Executor[LifeExecutor] - LifeModel[LifeModel / LifeRecord] - SeaQuery[SeaQuery] +flowchart LR + subgraph Req["Request path"] + A[App / LifeModel] --> P[LifeguardPool] + P --> PR[(Primary)] + P --> RP[(Replicas)] end - - subgraph Data Layer - may_postgres[may_postgres] - PostgreSQL[(PostgreSQL)] - Redis[(Redis Cache)] + subgraph Opt["Optional + async"] + A -.-> R[(Redis)] + PR -.-> LR[LifeReflector] + LR -.-> R end - - subgraph LifeReflector - Reflector[LifeReflector Leader] - end - - Web --> BFF_Service - Mobile --> BFF_Service - API --> BFF_Service - - BFF_Service --> MS1 - BFF_Service --> MS2 - BFF_Service --> MS3 - BFF_Service --> MSN - - MS1 --> Pool - MS2 --> Pool - MS3 --> Pool - MSN --> Pool - - Pool --> Executor - Executor --> LifeModel - LifeModel --> SeaQuery - SeaQuery --> Executor - Executor --> may_postgres - may_postgres --> PostgreSQL - - MS1 --> Redis - MS2 --> Redis - MS3 --> Redis - MSN --> Redis - PostgreSQL -- NOTIFY --> Reflector - Reflector --> Redis - - style Frontend fill:#e1f5ff - style Web fill:#e1f5ff - style Mobile fill:#e1f5ff - style API fill:#e1f5ff - style BFF fill:#add8e6 - style BFF_Service fill:#add8e6 - style Backend fill:#d4edda - style MS1 fill:#d4edda - style MS2 fill:#d4edda - style MS3 fill:#d4edda - style MSN fill:#d4edda - style Pool fill:#90ee90 - style Executor fill:#90ee90 - style LifeModel fill:#90ee90 - style may_postgres fill:#90ee90 - style PostgreSQL fill:#c0c0c0 - style Redis fill:#ffcccb - style Reflector fill:#add8e6 + style PR fill:#c0c0c0 + style RP fill:#d8d8d8 + style R fill:#ffcccb + style LR fill:#add8e6 + style P fill:#90ee90 ``` -### Connection Pool Architecture - -```mermaid -graph TD - subgraph LifeguardPool["LifeguardPool
The 300 Spartans"] - S[Semaphore
max_connections tokens
100-500 limit] - subgraph Slots["Connection Slots
Persistent & Reused"] - C1[Slot 1
in_use: false
ready] - C2[Slot 2
in_use: true
executing query] - C3[Slot 3
in_use: false
ready] - CN[Slot N
in_use: false
ready] - end - end - - subgraph Traffic["Incoming Traffic
The Persian Empire"] - R1[Request 1] - R2[Request 2] - R3[Request 3] - RN[Request N
millions/sec] - end - - Traffic -->|acquire| S - S -->|find free| Slots - Slots -->|mark in_use| C2 - C2 -->|execute query| PG[PostgreSQL
The Pass] - PG -->|result| C2 - C2 -->|release| S - C2 -->|mark free| Slots - C2 -->|ready for| Traffic - - style LifeguardPool fill:#fff4e1 - style S fill:#fff4e1 - style Slots fill:#e1ffe1 - style PG fill:#e1ffe1 - style Traffic fill:#ffe1e1 - - Note[100 connections
handle millions of requests
through aggressive reuse] - LifeguardPool --> Note -``` - -### LifeReflector Cache Coherence - -```mermaid -sequenceDiagram - participant LifeRecord - participant Postgres - participant LifeReflector - participant Redis - participant LifeModel - - LifeRecord->>Postgres: Write (INSERT/UPDATE/DELETE) - Postgres->>Postgres: Commit Transaction - Postgres->>LifeReflector: NOTIFY table_changes, '{"id": 42}' - LifeReflector->>Redis: EXISTS lifeguard:model:table:42? - alt Key Exists (Active Item) - LifeReflector->>Postgres: SELECT * FROM table WHERE id = 42 (from Primary) - Postgres-->>LifeReflector: Fresh Data - LifeReflector->>Redis: SETEX lifeguard:model:table:42 - else Key Not Exists (Inactive) - LifeReflector->>LifeReflector: Ignore (item not cached, TTL expired) - end - LifeModel->>Redis: Read (GET lifeguard:model:table:42) - alt Cache Hit - Redis-->>LifeModel: Cached Data (Fresh) - else Cache Miss - LifeModel->>Postgres: Read (SELECT * FROM table WHERE id = 42) - Postgres-->>LifeModel: Data - LifeModel->>Redis: SETEX lifeguard:model:table:42 - end -``` - - - - +**Full diagrams** (numbered call order, multi-service deployment, pool slots, LifeReflector sequence): **[ARCHITECTURE.md](./ARCHITECTURE.md)**. --- @@ -395,43 +76,13 @@ Pooling behavior and tunables evolve with [PRD_CONNECTION_POOLING.md](./docs/pla - **[DEVELOPMENT.md](./DEVELOPMENT.md)** — Clippy (CI parity), pre-commit, `just` recipes. - **[docs/TEST_INFRASTRUCTURE.md](./docs/TEST_INFRASTRUCTURE.md)** — Postgres/Redis for integration tests and CI. - --- ## 📊 Observability -Comprehensive instrumentation for production operations: +Lifeguard is **OpenTelemetry-compatible**: optional **`tracing`** spans/events and optional **Prometheus** metrics (`metrics` / `tracing` features) fit standard **OTLP** pipelines—use them with **OpenTelemetry-native** backends (Grafana, Jaeger, Tempo, collectors) or **Datadog** via OTLP intake or the Datadog Agent’s OpenTelemetry support. Lifeguard does not install global OTel or `tracing` subscribers for you; the host app owns **one** provider and subscriber (see **[OBSERVABILITY.md](./OBSERVABILITY.md)** and **[docs/OBSERVABILITY_APP_INTEGRATION.md](./docs/OBSERVABILITY_APP_INTEGRATION.md)**). -### Prometheus Metrics - -- `lifeguard_pool_size` - Current pool size -- `lifeguard_active_connections` - Active connections -- `lifeguard_connection_wait_time` - Time waiting for connection -- `lifeguard_query_duration_seconds` - Query execution time -- `lifeguard_query_errors_total` - Query errors -- `lifeguard_cache_hits_total` - Cache hits -- `lifeguard_cache_misses_total` - Cache misses -- `lifeguard_replica_lag_bytes` - Replica lag (bytes) -- `lifeguard_replica_lag_seconds` - Replica lag (seconds) -- `lifeguard_replicas_healthy` - Number of healthy replicas - -### OpenTelemetry Tracing - -- Distributed tracing for database operations -- Spans for: connection acquisition, query execution, cache operations -- Integration with existing OpenTelemetry infrastructure - -**Host-owned globals:** Lifeguard does **not** set a global OpenTelemetry `TracerProvider`. Your service (for example **BRRTRouter**) must install **one** provider and **one** `tracing_subscriber::Registry` stack. Optionally add **`lifeguard::channel_layer()`** to that same `.with(...)` chain so events also go through Lifeguard’s may-channel logger. See **[docs/OBSERVABILITY_APP_INTEGRATION.md](docs/OBSERVABILITY_APP_INTEGRATION.md)** and the **`lifeguard::logging`** rustdoc. - -### LifeReflector Metrics - -- `reflector_notifications_total` - Notifications received -- `reflector_refreshes_total` - Cache refreshes -- `reflector_ignored_total` - Ignored notifications (inactive items) -- `reflector_active_keys` - Active cache keys -- `reflector_redis_latency_seconds` - Redis operation latency -- `reflector_pg_latency_seconds` - PostgreSQL operation latency -- `reflector_leader_changes_total` - Leader election events +**Details:** Prometheus series (pool, queries, replica lag), tracing scopes, LifeReflector metrics, and **Kind/Tilt** dashboard refresh — **[OBSERVABILITY.md](./OBSERVABILITY.md)** (overview) and **[docs/OBSERVABILITY.md](./docs/OBSERVABILITY.md)** (full metric tables, feature flags, `kubectl apply`). --- @@ -444,196 +95,24 @@ There is **no** `lifeguard::testkit` / `test_pool!` macro in this repository; us --- -## 🗺️ Roadmap - -Epic-style checklists in older docs were overstated relative to this crate. Use these instead: - -| Area | Status | -|------|--------| -| `may_postgres`, `LifeExecutor`, transactions, raw SQL | Shipped | -| `LifeModel` / `LifeRecord`, query builder, relations, loaders | Shipped (ongoing hardening) | -| Migrations (`lifeguard::migration`, `lifeguard-migrate`, example `generate-migrations`) | Shipped (tooling evolves) | -| Optional metrics / tracing / channel logging | Shipped behind features | -| `LifeguardPool` / `PooledLifeExecutor` (primary/replica, WAL, heal, metrics) | Shipped (see [POOLING_OPERATIONS.md](./docs/POOLING_OPERATIONS.md), PRD for remaining parity) | -| Replica **read-preference** API surface, transparent Redis on every query | Planned / partial | -| LifeReflector, enterprise cache coherence | In-tree [`lifeguard-reflector`](./lifeguard-reflector/) (evolving) | - -Story-level detail: [docs/planning/epics-stories/](./docs/planning/epics-stories/) · Feature audit: [docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) · [docs/EPICS/](./docs/EPICS/) (curated notes). - ---- - -## 🎯 Competitive metrics: Lifeguard vs Rust ORMs - -*Snapshot for quick orientation. **Implementation Status** labels **shipped** crate behavior (including optional features), **partial** gaps, and **vision** rows (especially transparent cache and explicit read-preference APIs). Authoritative row-by-row coverage and percentages live in [SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) and `cargo doc`. The [short summary](#implementation-status-summary-short) below tracks README “current status” completion.* - -| Feature | Lifeguard Promise | Implementation Status | SeaORM | Diesel | SQLx | -|---------|-------------------|----------------------|--------|--------|------| -| **Concurrency Model** | ✅ Coroutine-native (`may`) | ✅ **Implemented** | ❌ Async/await (Tokio) | ❌ Sync-only | ❌ Async/await (Tokio) | -| **Performance (Hot Paths)** | ✅✅✅ 2-5× faster | 🟡 **Architectural** | ⚠️ Async overhead | ✅ Fast (sync) | ⚠️ Async overhead | -| **Performance (Small Queries)** | ✅✅✅ 10×+ faster | 🟡 **Architectural** | ⚠️ Future allocation | ✅ Fast | ⚠️ Future allocation | -| **Memory Footprint** | ✅✅ Low (stackful coroutines) | 🟡 **Architectural** | ⚠️ Higher (heap futures) | ✅ Low | ⚠️ Higher (heap futures) | -| **Predictable Latency** | ✅✅✅ Deterministic scheduling | 🟡 **Architectural** | ⚠️ Poll-based (variable) | ✅ Predictable | ⚠️ Poll-based (variable) | -| **Type Safety** | ✅✅✅ Compile-time validation | ✅ **Implemented** | ✅✅ Compile-time validation | ✅✅✅ Strong compile-time | ✅✅ Compile-time SQL checks | -| **ORM Features** | ✅✅✅ Complete (SeaORM parity) | 🟡 **High coverage** (core traits, relations, query builder; see mapping doc for %) | ✅✅✅ Complete | ✅✅ Good | ❌ Query builder only | -| **CRUD Operations** | ✅✅✅ Full support | ✅ **Implemented** (insert/update/save/delete via ActiveModelTrait) | ✅✅✅ Full support | ✅✅ Full support | ⚠️ Manual SQL | -| **Relations** | ✅✅✅ All types (has_one, has_many, belongs_to, many_to_many) | ✅ **Implemented** (Complete with eager/lazy loading, composite keys, DeriveLinked) | ✅✅✅ All types | ✅✅ Basic support | ❌ Manual joins | -| **Migrations** | ✅✅✅ Programmatic, data seeding, advanced ops | 🟡 **Partial** (`lifeguard::migration` + `lifeguard-migrate` shipped; `DeriveMigrationName` etc. still future per mapping) | ✅✅✅ Programmatic | ✅✅ CLI-based | ⚠️ Manual SQL | -| **Schema Inference** | ✅✅✅ From database (Diesel equivalent) | 🟡 **Partial** (`lifeguard-migrate infer-schema` / `schema_infer`; see [PRD §5](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) | ✅✅ From database | ✅✅✅ `table!` macro | ❌ No | -| **Query Builder** | ✅✅✅ Type-safe, chainable | ✅ **Implemented** (19/20 methods, 95% coverage) | ✅✅✅ Type-safe, chainable | ✅✅✅ Compile-time checked | ✅✅ Compile-time SQL | -| **Transactions** | ✅✅✅ Full support | ✅ **Implemented** (Roadmap Epic 01) | ✅✅✅ Full support | ✅✅ Full support | ✅✅ Full support | -| **Batch Operations** | ✅✅✅ insert_many, update_many, delete_many | ✅ **Implemented** | ✅✅✅ Batch support | ✅✅ Batch support | ⚠️ Manual | -| **Upsert** | ✅✅✅ save(), on_conflict() | ✅ **Implemented** (save() method exists) | ✅✅✅ save(), on_conflict() | ✅✅ on_conflict() | ⚠️ Manual SQL | -| **Pagination** | ✅✅✅ paginate(), paginate_and_count() | ✅ **Implemented** | ✅✅✅ Pagination helpers | ⚠️ Manual | ⚠️ Manual | -| **Entity Hooks** | ✅✅✅ before/after lifecycle events | ✅ **Implemented** (ActiveModelBehavior with 8 lifecycle hooks) | ✅✅✅ Hooks support | ❌ No | ❌ No | -| **Validators** | ✅✅✅ Field & model-level | 🟡 **Partial** (`run_validators`, `ActiveModelBehavior::validate_*`, `ActiveModelError::Validation`; [PRD §6](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) | ⚠️ Limited | ❌ No | ❌ No | -| **Soft Deletes** | ✅✅✅ Built-in support | ✅ **Implemented** (`#[soft_delete]` + `SelectQuery` / loader filtering) | ⚠️ Manual | ❌ No | ❌ No | -| **Auto Timestamps** | ✅✅✅ created_at, updated_at | ✅ **Implemented** (`#[auto_timestamp]` on `LifeRecord` insert/update paths) | ⚠️ Manual | ❌ No | ❌ No | -| **Session/Unit of Work** | ✅✅✅ Identity map, dirty tracking | 🟡 **Partial** (`ModelIdentityMap`, `mark_dirty` / `flush_dirty`; **no** auto-dirty on sets; [PRD §9](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) | ❌ No | ❌ No | ❌ No | -| **Scopes** | ✅✅✅ Named query scopes | 🟡 **Partial** (`SelectQuery::scope`, `IntoScope`; [PRD §7](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) | ❌ No | ❌ No | ❌ No | -| **Model Managers** | ✅✅✅ Custom query methods | ✅ **Implemented** (ModelManager trait + custom methods pattern) | ❌ No | ❌ No | ❌ No | -| **F() Expressions** | ✅✅✅ Database-level expressions | 🟡 **Partial** (`ColumnTrait::f_add` / `f_sub` / `f_mul` / `f_div` for `UPDATE SET`; [PRD §8](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) | ❌ No | ⚠️ Limited | ❌ No | -| **Subqueries** | ✅✅✅ Full support | 🟡 **Future** (Not yet implemented) | ✅✅✅ Full support | ✅✅ Full support | ✅✅ Manual SQL | -| **CTEs** | ✅✅✅ WITH clauses | 🟡 **Future** (Not yet implemented) | ✅✅✅ WITH clauses | ✅✅ WITH clauses | ✅✅ Manual SQL | -| **Window Functions** | ✅✅✅ Full support | 🟡 **Future** (Not yet implemented) | ✅✅✅ Full support | ✅✅ Full support | ✅✅ Manual SQL | -| **Eager Loading** | ✅✅✅ Multiple strategies (joinedload, subqueryload, selectinload) | ✅ **Implemented** (selectinload strategy with FK extraction) | ✅✅✅ Eager loading | ⚠️ Manual | ❌ Manual | -| **Raw SQL** | ✅✅✅ find_by_statement(), execute_unprepared() | ✅ **Implemented** (Architecture supports raw SQL) | ✅✅✅ Raw SQL support | ✅✅✅ Raw SQL support | ✅✅✅ Primary feature | -| **Connection Pooling** | ✅✅✅ Persistent, semaphore-based, health monitoring | ✅ **Shipped** ([`LifeguardPool`](./src/pool/pooled.rs): bounded queues, acquire timeout, heal, lifetime, metrics w/ `pool_tier`; see [pooling PRD](./docs/planning/PRD_CONNECTION_POOLING.md) for remaining parity) | ✅✅✅ Built-in pool | ⚠️ External (r2d2) | ✅✅✅ Built-in pool | -| **Replica Read Support** | ✅✅✅ WAL-based health monitoring, automatic routing | ✅ **Shipped** (replica tier + [`WalLagMonitor`](./src/pool/wal.rs); routing is pool-internal, not SeaORM-identical API) | ❌ No | ❌ No | ❌ No | -| **Read Preferences** | ✅✅✅ primary, replica, mixed, strong | 🟡 **Partial** (transparent routing via pool/WAL; no SeaORM-style explicit read-preference enum API) | ❌ No | ❌ No | ❌ No | -| **Distributed Caching** | ✅✅✅✅ **LifeReflector (UNIQUE)** | 🟡 **Architectural** (Not in SeaORM mapping, may exist) | ❌ No | ❌ No | ❌ No | -| **Cache Coherence** | ✅✅✅✅ **Zero-stale reads (UNIQUE)** | 🟡 **Architectural** (Not in SeaORM mapping, may exist) | ❌ No | ❌ No | ❌ No | -| **TTL-Based Active Set** | ✅✅✅✅ **Adaptive caching (UNIQUE)** | 🟡 **Architectural** (Not in SeaORM mapping, may exist) | ❌ No | ❌ No | ❌ No | -| **PostgreSQL Features** | ✅✅✅ Views, materialized views, JSONB, FTS, PostGIS, partitioning | 🟡 **Partial** (JSONB ✅ core feature, others future) | ✅✅✅ Most features | ✅✅✅ Most features | ✅✅✅ All features (raw SQL) | -| **Observability** | ✅✅✅ Prometheus, OpenTelemetry, comprehensive metrics | ✅ **Implemented** (optional `metrics` / `tracing`; Prometheus scrape; pool series with `pool_tier`) | ✅✅ Basic metrics | ⚠️ Limited | ⚠️ Limited | -| **Developer Experience** | ✅✅✅ Familiar API, no async/await, clear errors | ✅ **Implemented** (SeaORM-like API) | ✅✅✅ Good, async/await required | ⚠️ Complex type system | ✅✅ Good, async/await required | -| **Learning Curve** | ✅✅ Moderate (familiar if you know SeaORM) | ✅ **Implemented** (SeaORM-like API) | ✅✅ Moderate | ⚠️ Steep (complex macros) | ✅✅ Moderate | -| **Production Ready** | ✅✅✅ Complete observability, health checks, metrics | 🟡 **Workload-dependent** (core ORM + pool + metrics/tracing ship; validate migrations, cache, and ops for your deployment) | ✅✅✅ Production ready | ✅✅✅ Production ready | ✅✅✅ Production ready | -| **Multi-Database** | ❌ PostgreSQL only (by design) | ✅ **By Design** | ✅✅ PostgreSQL, MySQL, SQLite | ✅✅ PostgreSQL, MySQL, SQLite | ✅✅✅ PostgreSQL, MySQL, SQLite, MSSQL | -| **Coroutine Runtime** | ✅✅✅✅ **Native support (UNIQUE)** | ✅ **Implemented** | ❌ Incompatible | ❌ Incompatible | ❌ Incompatible | - -### Legend - -**Implementation Status Column:** -- ✅ **Implemented** = Feature is fully implemented and working -- 🟡 **Partial/Future/Architectural** = Partially implemented, planned for future, or architectural feature (not in SeaORM mapping) -- ❌ **Not Implemented** = Feature promised but not yet implemented - -**Feature Comparison Columns:** -- ✅✅✅✅ = **Unique advantage** (no other ORM has this) -- ✅✅✅ = Excellent support -- ✅✅ = Good support -- ✅ = Basic support -- ⚠️ = Limited or manual implementation required -- ❌ = Not supported - -### Implementation status summary (short) - -**Strong in-tree today:** core traits (`LifeModelTrait`, `ModelTrait`, `ActiveModelTrait`, …), CRUD/save paths, `SelectQuery` stack, relations and eager/loader paths (including composite keys and linked traversals), migrations framework (`lifeguard::migration`, `lifeguard-migrate`), JSON column support, derive **`#[soft_delete]`** / **`#[auto_timestamp]`**, partial models, lifecycle hooks, **`LifeguardPool`** / **`PooledLifeExecutor`** with primary+replica tiers, WAL lag routing, slot heal, idle liveness, max connection lifetime, and optional **metrics** (including **`pool_tier`** labels) / **tracing**. - -**Partial (PRD v0 shipped; see [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)):** schema inference CLI/module, validator pipeline on save paths, `SelectQuery::scope`, `ColumnTrait::f_*` for `UPDATE SET`, and **`ModelIdentityMap`** with **`mark_dirty` / `flush_dirty`** (explicit dirty flush; no auto-dirty on `LifeRecord::set` yet). - -**Partial or roadmap:** fuller SeaORM parity on those workstreams (flush/UoW, derive sugar, `WHERE`/`ORDER BY` F-exprs), some SQL builder extras (subqueries/CTEs/windows), explicit read-preference API surface (pool routing is already shipped), migration derive niceties (e.g. `DeriveMigrationName` per mapping), and any remaining pooling parity called out in [PRD_CONNECTION_POOLING.md](./docs/planning/PRD_CONNECTION_POOLING.md) and [POOLING_OPERATIONS.md](./docs/POOLING_OPERATIONS.md). - -**Roadmap / vision:** productized “transparent Redis on every read”; LifeReflector and cache coherence in [`lifeguard-reflector`](./lifeguard-reflector/). - -For percentages and row-by-row status, use the mapping document linked in the section intro rather than this README table alone. - -### Key Differentiators - -**Lifeguard's Unique Advantages:** -1. **LifeReflector** - Distributed cache coherence (Oracle Coherence–style active set) — **unique**; **🟡** product evolution in [`lifeguard-reflector`](./lifeguard-reflector/) -2. **Coroutine-Native** - No async overhead, deterministic scheduling — **unique** among these ORMs ✅ -3. **WAL-Based Replica Routing** - Pool + [`WalLagMonitor`](./src/pool/wal.rs) — **shipped** for `LifeguardPool` reads ✅ -4. **TTL-Based Active Set** - Adaptive caching — **🟡** vision / reflector path; not automatic on every app read -5. **DeriveLinked Macro** - Multi-hop relationship code generation — **competitive advantage** ✅ (SeaORM has no direct equivalent) -6. **Session/Unit of Work** — **🟡** identity map + explicit dirty flush (`mark_dirty` / `flush_dirty`); automatic per-field dirty + pool-scoped `Session` still roadmap ([PRD §9](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) - -**Where Lifeguard Matches or Exceeds:** -- ✅ Substantial SeaORM-oriented coverage (see mapping doc for %; core ORM paths strong) -- ✅ Relations system with composite keys and eager/lazy loading -- ✅ Query builder with 95% method coverage -- ✅ Better performance potential (2-5× faster on hot paths - architectural) -- ✅ Lower memory footprint (architectural) -- ✅ Predictable latency (architectural) - -**Trade-offs:** -- ❌ PostgreSQL-only (by design - enables advanced features) -- ❌ Requires `may` coroutine runtime (not Tokio) -- ❌ Smaller ecosystem (newer project) -- ⚠️ Some roadmap items remain (full UoW flush, validator/scope derive sugar, explicit read-preference API, SQL builder extras, migration derives, etc.); see [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md), mapping doc, and pooling docs - -### Performance Comparison (Estimated) - -| Metric | Lifeguard | SeaORM | Diesel | SQLx | -|--------|-----------|--------|--------|------| -| **Simple Query Latency** | 0.1-0.5ms | 0.5-2ms | 0.2-1ms | 0.5-2ms | -| **Hot Path Throughput** | 2-5× faster | Baseline | 1-2× faster | Baseline | -| **Small Query Overhead** | Minimal | Future allocation | Minimal | Future allocation | -| **Memory per Connection** | ~100 bytes | ~1-2 KB | ~100 bytes | ~1-2 KB | -| **Concurrent Connections** | 800+ (1MB stack) | Limited by Tokio | Limited by threads | Limited by Tokio | -| **p99 Latency** | < 5ms (predictable) | 5-20ms (variable) | < 5ms (predictable) | 5-20ms (variable) | - -*Note: Performance numbers are estimates based on architecture. Actual benchmarks will be published after implementation.* - -### Ecosystem Compatibility - -**⚠️ Important: BRRTRouter and Lifeguard are a parallel ecosystem, separate from async/await Rust.** - -These are **two incompatible worlds** with the only commonality being Rust itself: - -| Ecosystem | Runtime | ORM Options | Incompatible With | -|-----------|---------|-------------|-------------------| -| **BRRTRouter + Lifeguard** | `may` coroutines | Lifeguard only | SeaORM, Diesel (async), SQLx, Tokio | -| **Tokio + Async ORMs** | `async/await` | SeaORM, Diesel, SQLx | BRRTRouter, Lifeguard, `may` | - -**You cannot mix and match.** If you're using BRRTRouter, you **must** use Lifeguard. The async/await ORMs (SeaORM, Diesel, SQLx) are fundamentally incompatible with the `may` coroutine runtime. - -### When to Use Each Ecosystem - -**Use BRRTRouter + Lifeguard if:** -- ✅ You're building with **BRRTRouter** (the coroutine API framework) -- ✅ You need **distributed cache coherence** (LifeReflector - unique to Lifeguard) -- ✅ You need **extreme scale** (millions of requests/second) -- ✅ You need **predictable latency** (API routers, real-time systems) -- ✅ You're **PostgreSQL-only** (enables advanced features) -- ✅ You want **Oracle Coherence-level functionality** - -**Use Tokio + Async ORMs if:** -- ✅ You're using **Tokio/async-await** runtime -- ✅ You need **multi-database support** (PostgreSQL, MySQL, SQLite, MSSQL) -- ✅ You want **mature, well-documented ORMs** (SeaORM, Diesel, SQLx) -- ✅ You don't need distributed cache coherence -- ✅ You're building traditional async/await microservices - -**The choice is made at the ecosystem level, not the ORM level.** Once you choose BRRTRouter, Lifeguard is your only ORM option. Once you choose Tokio, you can choose between SeaORM, Diesel, or SQLx—but you cannot use BRRTRouter. - ---- - -## 🚀 Performance - -**Target Performance:** -- 2-5× faster than async ORMs on hot paths -- 10×+ faster on small queries (no future allocation overhead) -- Predictable p99 latency (< 5ms for simple queries) -- Lower memory footprint than async alternatives - -**Real-World Use Cases:** -- **BRRTRouter**: High-throughput API routing with sub-millisecond database access (100,000+ requests/second) -- **High-Scale Microservices**: Applications requiring millions of requests/second with limited database connections -- **Low-Latency Systems**: Real-time applications needing predictable p99 latency (< 5ms) for database operations - ---- - ## 📚 Documentation -- [Developer workflow & Clippy / pre-commit](./DEVELOPMENT.md) -- [Tests & CI Postgres/Redis](./docs/TEST_INFRASTRUCTURE.md) -- [Observability & host-owned OTel/tracing](./docs/OBSERVABILITY_APP_INTEGRATION.md) -- [Metrics, tracing, **Kind/Tilt `kubectl apply`** to refresh Grafana dashboards, and Postgres replication lag (time vs bytes)](./docs/OBSERVABILITY.md#kubernetes-kind-tilt-apply-and-refresh-dashboards) -- [**Connection pool** operations, tuning, non-goals (PgBouncer), migration notes](./docs/POOLING_OPERATIONS.md) · [design doc (queue policy, metrics, decisions)](./docs/planning/DESIGN_CONNECTION_POOLING.md) -- [SeaORM ↔ Lifeguard mapping](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) -- [Epic notes](./docs/EPICS/) · [Story tree](./docs/planning/epics-stories/) -- [Planning index](./docs/planning/README.md) +| Topic | Document | +|--------|----------| +| **Repository truth, competitive matrix, ecosystem, performance** | [COMPARISON.md](./COMPARISON.md) | +| **Roadmap (high-level areas)** | [ROADMAP.md](./ROADMAP.md) | +| **Product vision & long-form “what we’re building”** | [VISION.md](./VISION.md) | +| **Blog: async ORMs vs `may`, and why Lifeguard exists** | [LIFEGUARD_BLOG_POST.md](./LIFEGUARD_BLOG_POST.md) | +| **Architecture (diagrams, flows)** | [ARCHITECTURE.md](./ARCHITECTURE.md) | +| **Observability overview (OTel-compatible, Datadog via OTLP)** | [OBSERVABILITY.md](./OBSERVABILITY.md) | +| **Host-owned OTel/tracing wiring** | [docs/OBSERVABILITY_APP_INTEGRATION.md](./docs/OBSERVABILITY_APP_INTEGRATION.md) | +| **Metrics tables, Kind/Tilt `kubectl apply`** | [docs/OBSERVABILITY.md](./docs/OBSERVABILITY.md#kubernetes-kind-tilt-apply-and-refresh-dashboards) | +| **Connection pool operations & tuning** | [docs/POOLING_OPERATIONS.md](./docs/POOLING_OPERATIONS.md) · [docs/planning/DESIGN_CONNECTION_POOLING.md](./docs/planning/DESIGN_CONNECTION_POOLING.md) | +| **SeaORM ↔ Lifeguard mapping (authoritative parity)** | [docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) | +| **Developer workflow & Clippy / pre-commit** | [DEVELOPMENT.md](./DEVELOPMENT.md) | +| **Tests & CI Postgres/Redis** | [docs/TEST_INFRASTRUCTURE.md](./docs/TEST_INFRASTRUCTURE.md) | +| **Epic notes & story tree** | [docs/EPICS/](./docs/EPICS/) · [docs/planning/epics-stories/](./docs/planning/epics-stories/) | +| **Planning index** | [docs/planning/README.md](./docs/planning/README.md) | --- @@ -647,7 +126,6 @@ Lifeguard is under active development. We welcome: See [EPICS](./docs/EPICS/) for current development priorities. - --- ## 📜 License diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 00000000..95475713 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,19 @@ +# Roadmap + +Epic-style checklists in older docs were overstated relative to this crate. Use this table instead: + +| Area | Status | +|------|--------| +| `may_postgres`, `LifeExecutor`, transactions, raw SQL | Shipped | +| `LifeModel` / `LifeRecord`, query builder, relations, loaders | Shipped (ongoing hardening) | +| Migrations (`lifeguard::migration`, `lifeguard-migrate`, example `generate-migrations`) | Shipped (tooling evolves) | +| Optional metrics / tracing / channel logging | Shipped behind features | +| `LifeguardPool` / `PooledLifeExecutor` (primary/replica, WAL, heal, metrics) | Shipped (see [POOLING_OPERATIONS.md](./docs/POOLING_OPERATIONS.md), PRD for remaining parity) | +| **`ReadPreference`** on `PooledLifeExecutor` (explicit primary-tier reads); transparent Redis on every query | Partial — API shipped; “Redis on every read” remains vision / reflector path | +| LifeReflector, enterprise cache coherence | In-tree [`lifeguard-reflector`](./lifeguard-reflector/) (evolving) | + +Story-level detail: [docs/planning/epics-stories/](./docs/planning/epics-stories/) · Feature audit: [docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) · [docs/EPICS/](./docs/EPICS/) (curated notes). + +--- + +[← README](./README.md) · [Comparison](./COMPARISON.md#repository-status) diff --git a/SECURITY_PROMPT.md b/SECURITY_PROMPT.md new file mode 100644 index 00000000..f78cfb01 --- /dev/null +++ b/SECURITY_PROMPT.md @@ -0,0 +1,109 @@ +# Security assessment prompt (reusable) + +**Purpose:** Copy the block below into an AI assistant, internal review ticket, or external penetration-test brief when you want a **structured security review** of the Lifeguard workspace. The audience for the **output** should be **CTOs, engineering leads, and CSOs**—executive clarity, evidence-backed, and actionable. + +**Scope note:** This repository is the **Lifeguard** PostgreSQL ORM/data-access platform for Rust (`may` / `may_postgres`), with optional **Redis** (cache / LifeReflector), **migrations**, **GraphQL** (optional), and **CI/Compose** test infrastructure. Adjust scope in your paste if you audit only a subdirectory or a release tag. + +--- + +## Prompt to submit (copy from here) + +```text +As an expert in IT security and vulnerability assessment, analyse the Lifeguard codebase (Rust PostgreSQL ORM / data platform: lifeguard crate, lifeguard-derive, lifeguard-migrate, lifeguard-reflector, examples, tests). + +Write the assessment in the prose of an external security audit consultancy. The audience is CTOs, engineering leads, and CSOs. Use clear executive summaries where appropriate. + +For each security concern, provide a table with: +- Location (file path and, where helpful, symbol or feature name) +- Concern (what is risky) +- Potential exploit (realistic attacker scenario or failure mode) +- Possible remediation (controls, design changes, or operational mitigations) + +Where possible, cite hyperlinks to: +- Relevant CWE entries +- OWASP categories or project pages +- Known CVEs or vendor advisories **only when directly applicable** (e.g. a dependency with a tracked CVE); avoid speculative CVE linkage. + +Cover at minimum: +1. **Injection and query construction** — raw SQL APIs, string-built SQL, SeaQuery/ORM paths, migration SQL +2. **Secrets and configuration** — connection strings, env handling, logging of sensitive data +3. **Memory safety and `unsafe`** — any `unsafe` blocks, `unsafe impl`, FFI +4. **Concurrency and TOCTOU** — pool dispatch, session/identity map, replica/WAL routing +5. **Supply chain** — `cargo`/git dependencies, pinned revisions, optional features +6. **Denial of service** — unbounded work, pool exhaustion, channel backpressure +7. **Cache / Redis / LifeReflector** (if present) — cache poisoning, invalidation, NOTIFY abuse +8. **GraphQL surface** (if enabled) — introspection, depth/complexity, authz (consumer responsibility) + +Explicitly state **consumer responsibilities** (e.g. this library does not implement application authentication). + +Do **not** modify source code in your response; recommendations only. + +Deliver: +- Executive summary (1–2 paragraphs) +- Detailed findings in tabular form as specified +- Residual risk and suggested next steps (e.g. SAST, dependency scanning cadence, threat model for deployment) +``` + +--- + +## How to use this file + +| Step | Action | +|------|--------| +| 1 | Pin a **commit SHA** or **release tag** in your audit ticket so results are reproducible. | +| 2 | Paste the prompt into your tool of choice; attach or allow read access to the repo. | +| 3 | For **external** auditors, add: deployment topology (internet-facing or not), data classification, and whether GraphQL/Redis/reflector are in use. | +| 4 | Store the generated report under version control (e.g. `docs/security/`) with **date** and **scope**. | + +--- + +## Appendix A — Representative themes observed in this workspace (non-exhaustive) + +*This appendix is **illustrative** for scoping future runs. It is **not** a substitute for a full assessment on your revision. Locations may shift as the code evolves.* + +### A.1 Themes to review (mapping to code areas) + +| Theme | Example locations / notes | +|-------|---------------------------| +| **Raw / unprepared SQL** | `src/raw_sql.rs` — `execute_unprepared` passes through to `executor.execute(sql, &[])`; consumer misuse enables classic SQL injection if user input is concatenated into `sql`. | +| **Parameterized vs embedded values** | `src/relation/eager.rs` — generated SQL fragments use `value_to_sql_string` / `Expr::cust` in places; code comments note embedding values is not ideal. Review for any path where **untrusted** data influences SQL text. | +| **`unsafe`** | `src/session/identity_model_cell.rs` — `unsafe impl Send`; `lifeguard-reflector` may use `unsafe` for FFI/cache — review soundness and thread contracts. | +| **Migration / lock SQL** | `src/migration/lock.rs` — `format!` with `LOCK_VERSION` constant (not user input); still verify no future refactor introduces interpolation of untrusted input. | +| **Dependency posture** | Root `Cargo.toml` — git-pinned `may_postgres`; direct `protobuf = "3.7.2"` with comment referencing **CVE-2025-53605** — keep `cargo audit` / `cargo deny` in CI. | +| **Cryptography** | `sha2` for migration checksums — appropriate for integrity, not for passwords; ensure no misuse as a KDF. | +| **Test-only mocks** | `src/macros/mock.rs` — `format!` building SQL for tests; must not ship to production paths. | + +### A.2 Reference links (generic classes; not product-specific CVEs unless tied to a dependency) + +| Topic | Reference | +|-------|-----------| +| SQL injection (class) | [CWE-89: Improper Neutralization of Special Elements used in an SQL Command](https://cwe.mitre.org/data/definitions/89.html) | +| Injection (general) | [OWASP Top 10 A03:2021 – Injection](https://owasp.org/Top10/A03_2021-Injection/) | +| Deserialization | [CWE-502](https://cwe.mitre.org/data/definitions/502.html) / secure serde usage | +| Protobuf advisory (if using protobuf) | Track vendor advisories for the **pinned** `protobuf` crate version in `Cargo.toml` | + +### A.3 Sample findings table (illustrative — **re-validate on each audit**) + +| Location | Concern | Potential exploit | Possible remediation | +|----------|---------|-------------------|----------------------| +| `src/raw_sql.rs` (`execute_unprepared`, etc.) | Unprepared execution of caller-supplied SQL strings | Attacker-controlled string concatenated into SQL → **CWE-89** | Prefer `execute_statement` / parameterized APIs; static analysis for call sites; never pass untrusted input into `execute_unprepared` | +| Consumer applications using Lifeguard | No built-in authn/authz | Broken access control at app layer | Enforce authz in application; use least-privilege DB roles; row-level security in PostgreSQL where appropriate | +| `Cargo.toml` / lockfile | Transitive vulnerabilities | Known CVEs in dependencies | `cargo audit`, Dependabot, pin upgrades; review git deps (`may_postgres`) | +| Optional Redis / NOTIFY / reflector | Cache and messaging trust boundaries | Cache poisoning or stale reads if misconfigured | TLS to Redis where required, auth secrets, network segmentation; document trust model | +| `unsafe impl Send` (`SessionIdentityModelCell`) | Protocol contract not enforced by types | Data races / UB if session + record used across threads incorrectly | Document (done); long-term: `Arc>` or API split if multi-threaded session is required | + +--- + +## Appendix B — Document control + +| Field | Value | +|-------|--------| +| Template version | 1.0 | +| Intended use | Lifeguard repository security assessments | +| Code changes | **None** required by this file; it is a prompt + guidance only | + +When you complete an audit, add a row here or in `docs/security/`: + +| Date | Scope (commit/tag) | Report location | Owner | +|------|----------------------|-----------------|-------| +| *—* | *—* | *—* | *—* | diff --git a/VISION.md b/VISION.md new file mode 100644 index 00000000..46dab09e --- /dev/null +++ b/VISION.md @@ -0,0 +1,141 @@ +# Vision: what we’re building + +This document is the **long-form product vision**: core ORM abstractions, pooling, **LifeReflector**, transparent caching targets, replica routing, and parity lists (shipped vs planned). For **what compiles today**, repository truth, and competitive framing, see **[COMPARISON.md](./COMPARISON.md)** (start with [repository status](./COMPARISON.md#repository-status)) and [SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md). + +--- + +## 🚀 What we're building + +### Core ORM: LifeModel & LifeRecord + +A complete ORM system with two primary abstractions: + +**LifeModel** (Immutable Database Rows) +- Represents database rows as immutable Rust structs +- Generated via `#[derive(LifeModel)]` procedural macro +- Provides type-safe query builders +- Automatic row-to-struct mapping +- Complete SeaORM API parity + +**LifeRecord** (Mutable Change Sets) +- Separate abstraction for inserts and updates +- Generated via `#[derive(LifeRecord)]` procedural macro +- Type-safe mutation builders +- Automatic SQL generation via SeaQuery +- Change tracking (dirty fields) + +```rust +use lifeguard_derive::{LifeModel, LifeRecord}; + +#[derive(LifeModel, LifeRecord)] +#[table_name = "users"] +struct User { + #[primary_key] + id: i64, + email: String, + is_active: bool, +} + +// Inserts/selects go through LifeExecutor + SelectQuery / ActiveModelTrait; +// see lifeguard-derive tests and examples/ for full patterns (no Tokio required). +``` + +### Connection pool: LifeguardPool + +**In-tree:** [`LifeguardPool`](./src/pool/pooled.rs) (re-exported as `lifeguard::LifeguardPool`) — persistent `may_postgres` connections, one worker per slot, bounded per-worker job queues, configurable acquire timeout ([`LifeError::PoolAcquireTimeout`](./src/executor.rs)), optional read-replica routing with [`WalLagMonitor`](./src/pool/wal.rs), slot heal, idle liveness, max connection lifetime, and Prometheus metrics with a low-cardinality **`pool_tier`** label (`primary` / `replica`) on pool-scoped series. See [POOLING_OPERATIONS.md](./docs/POOLING_OPERATIONS.md), [DESIGN_CONNECTION_POOLING.md](./docs/planning/DESIGN_CONNECTION_POOLING.md), and [OBSERVABILITY.md](./OBSERVABILITY.md) (summary) / [docs/OBSERVABILITY.md](./docs/OBSERVABILITY.md) (operators, Kind, metric tables). + +**Alternative:** open connections with [`connect`](./src/connection.rs) and run queries through [`MayPostgresExecutor`](./src/executor.rs) / [`LifeExecutor`](./src/executor.rs) when you do not need the pool. See [`examples/query_builder_example.rs`](./examples/query_builder_example.rs) for patterns. + +### The Killer Feature: LifeReflector + +**Distributed cache coherence system**—this is Lifeguard's unique advantage: + +> **Note:** LifeReflector is developed as the **`lifeguard-reflector`** workspace crate in this repository ([`./lifeguard-reflector`](./lifeguard-reflector/)). Enterprise licensing may still apply for some distributions; see that crate’s README. + +A **standalone microservice** that maintains cluster-wide cache coherence: + +- **Leader-elected Raft system:** Only one active reflector at a time (no duplicate work) +- **Postgres LISTEN/NOTIFY integration:** Subscribes to database change events +- **Intelligent cache refresh:** Only **re-writes** keys that already exist in Redis (TTL-based **active set**—no stale copy to fix if the key was never cached) +- **Read path populates Redis:** Cache miss → load from Postgres → `SETEX` (with TTL); new rows enter Redis when something **reads** them (or via warm-up), not from `NOTIFY` alone +- **Horizontal scaling:** All microservices benefit from single reflector + +**How it works:** + +1. **Reads (population):** A service checks **Redis first**. On a **miss**, it reads from **Postgres** and **writes the row into Redis** (e.g. `SETEX` + TTL). First-time and cold rows are cached here—this is how Redis gets populated. +2. **LifeRecord** (or the writer) commits to **Postgres**; the database path emits **`NOTIFY`** (payload identifies the row). +3. **LifeReflector** (leader) receives the notification. +4. Reflector checks whether that entity **key already exists** in Redis (active cached item). +5. **If it exists** → Reflector **re-reads from Postgres** and **updates Redis** so no client keeps a pre-write value. +6. **If it does not exist** → Reflector **ignores** the notify: there is **no cached row to invalidate**—nothing in Redis was wrong. The next read miss still runs step (1) and loads fresh data from Postgres into Redis. +7. **Cross-service reads:** Once a key is in Redis, other services can read it from Redis; steps 2–6 keep **already-cached** keys aligned with Postgres after writes. + +**Result:** Oracle Coherence–style **coherence for the active set** in Redis: lazy (or warmed) population on reads, plus **notify-driven refresh** only where a stale cache entry could otherwise exist. See the **sequence diagram** below (cache miss branch → Postgres → `SETEX`). + +**Enterprise:** commercial or source-available licensing may apply for some LifeReflector deployments. Source and package layout live under [`lifeguard-reflector`](./lifeguard-reflector/); contact enterprise@microscaler.io for licensing questions. + +### Transparent caching system (target) + +**Target behavior** (not fully wired as “magic” on every read path in this crate today): Lifeguard’s design calls for caching that still respects PostgreSQL primaries and replicas: + +- **Check Redis first:** Sub-millisecond reads if cached +- **Read from replicas:** When healthy (WAL lag < threshold) +- **Write to primary:** Always (as PostgreSQL was designed) +- **LifeReflector keeps cache fresh:** Automatic coherence across microservices ([`lifeguard-reflector`](./lifeguard-reflector/)) + +Your application code doesn't need to know about Redis, replicas, or cache coherence. It just calls `User::find_by_id(&pool, 42)?` and Lifeguard handles the rest. + +**Note:** For distributed cache coherence across multiple microservices, [`lifeguard-reflector`](./lifeguard-reflector/) provides automatic cache refresh using PostgreSQL LISTEN/NOTIFY. + +### Replica Read Support + +Advanced read routing with WAL lag awareness: + +- **WAL position tracking:** Monitors `pg_current_wal_lsn()` vs `pg_last_wal_replay_lsn()` +- **Dynamic health checks:** Automatically detects replica lag +- **Intelligent routing:** Routes reads to replicas only when healthy +- **Automatic fallback:** Falls back to primary if replicas are stale +- **Strong consistency mode:** Optional causal read-your-writes consistency + +**Read Preference Modes:** +- `primary` - Always read from primary +- `replica` - Use replicas when healthy +- `mixed` - Automatic selection (Redis → replica → primary) +- `strong` - Causal consistency (wait for replica to catch up) + +### Complete feature set (vision vs crate) + +The lists below mix **shipped**, **partial**, and **planned** capabilities. For a maintained feature matrix, see [SEAORM_LIFEGUARD_MAPPING.md](./docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md). + +**ORM features (SeaORM parity target):** +- ✅ Complete CRUD operations +- ✅ Type-safe query builders +- ✅ Relations (has_one, has_many, belongs_to, many_to_many) +- ✅ Migrations (programmatic, data seeding, advanced operations) +- ✅ Transactions +- ✅ Raw SQL helpers +- ✅ Batch operations +- ✅ Upsert support +- ✅ Pagination helpers +- ✅ Entity hooks & lifecycle events +- 🟡 Validators (`run_validators` / [`ValidationStrategy`](./src/active_model/validate_op.rs), `ActiveModelBehavior::validate_fields` / `validate_model`, `ActiveModelError::Validation`, derive `#[validate(custom = …)]`, `ValidateOp::Delete`; [`lifeguard::predicates`](./src/active_model/predicates.rs) — `string_utf8_chars_max`, `string_utf8_chars_in_range`, `blob_or_string_byte_len_max`, `i64_in_range`, `f64_in_range`; SeaORM-style built-in attribute matrix not replicated — [PRD §6](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) +- ✅ Soft deletes +- ✅ Auto-managed timestamps + +**Competitive Features:** +- 🟡 Schema inference (`lifeguard-migrate infer-schema`, composite PK `#[primary_key]` codegen, `compare-schema` column drift — [PRD §5](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) +- 🟡 Session/Unit of Work (`ModelIdentityMap`, `Session` / `SessionDirtyNotifier`, `attach_session` + record auto-dirty enqueue, `flush_dirty` / `flush_dirty_with_map_key`, `register_pending_insert` / `promote_pending_to_loaded` / `is_pending_insert_key`, `flush_dirty_in_transaction` / `flush_dirty_in_transaction_pooled` + `LifeguardPool::exclusive_primary_write_executor`, `LifeRecord::identity_map_key` — [PRD §9](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) +- 🟡 Scopes (`SelectQuery::scope`, `scope_or` / `scope_any`, `#[scope]` on `impl Entity`; parent scopes are not merged into `find_related`—chain on the returned query — [PRD §7](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) +- ✅ Model Managers (Django) +- 🟡 F() Expressions (`ColumnTrait::f_*`, `LifeRecord::set_*_expr` / `identity_map_key`, `Expr::expr` in `WHERE`/`ORDER BY`; PostgreSQL applies its own numeric promotion for mixed types—match column/RHS types or use explicit casts when you need a specific storage type; [PRD §8](./docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md)) +- ✅ Advanced eager loading strategies (SQLAlchemy) + +**Unique Features (No Other ORM Has):** +- ✅ **LifeReflector** - Distributed cache coherence +- ✅ **Coroutine-native** - No async overhead +- ✅ **WAL-based replica routing** - Automatic health monitoring +- ✅ **TTL-based active set** - Adaptive caching + +--- + +[← README](./README.md) · [Architecture](./ARCHITECTURE.md) diff --git a/docs/OBSERVABILITY.md b/docs/OBSERVABILITY.md index 739c9da9..b008f2b0 100644 --- a/docs/OBSERVABILITY.md +++ b/docs/OBSERVABILITY.md @@ -1,5 +1,7 @@ # Observability in Lifeguard +**Overview (OTel-compatible tracing, Prometheus, Datadog/OTLP):** see the repo root **[OBSERVABILITY.md](../OBSERVABILITY.md)**. + Lifeguard provides comprehensive observability through Prometheus metrics and OpenTelemetry tracing. These features are optional and can be enabled via feature flags. ## Kubernetes (Kind / Tilt): apply and refresh dashboards diff --git a/docs/planning/DESIGN_FIND_RELATED_SCOPES.md b/docs/planning/DESIGN_FIND_RELATED_SCOPES.md new file mode 100644 index 00000000..cb93f7e5 --- /dev/null +++ b/docs/planning/DESIGN_FIND_RELATED_SCOPES.md @@ -0,0 +1,27 @@ +# Design note: `find_related` and named scopes + +**Status:** Default behavior **documented in crate rustdoc** (`query::scope`, `FindRelated`); optional `related_scope` / inherited parent scopes remain future work. +**PRD:** [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md §7](./PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md) (scopes, SC-1–SC-4). + +## Current behavior (v0) + +- [`SelectQuery::scope`](../../src/query/scope.rs) and [`SelectQuery::filter`](../../src/query/select.rs) apply predicates on the **root** entity’s `SELECT`. +- [`SelectQuery::scope_or`](../../src/query/scope.rs) / [`scope_any`](../../src/query/scope.rs) compose **OR** branches for that same root query (PRD SC-2). +- [`FindRelated`](../../src/relation/mod.rs) and loaders build **join** / **subselect** paths; their `WHERE` clauses target related tables via relation metadata, not via the parent entity’s scope helpers. + +## Product question + +When loading related rows (e.g. `post.find_related(Comment)`), should **parent scopes** (e.g. `Post::published()`) automatically constrain which parents participate, while **child scopes** (e.g. `Comment::visible()`) apply only to the related table? + +## Recommended default (**adopted for v0 documentation**) + +1. **Root `SelectQuery` scopes** apply to the root entity only (current behavior). +2. **`find_related`**: filters on the parent entity’s **separate** `SelectQuery` (e.g. `Post::find().scope(...)`) are **not** merged into `find_related` SQL—only join/PK-driven `WHERE` from [`build_where_condition`](../../src/relation/def/condition.rs). Callers chain `.scope` / `.filter` on the `SelectQuery` returned by `find_related` to constrain related rows. A future opt-in API (`related_scope` / `with_scope_on_related`) would be explicit. +3. **Eager loaders** (`Loader*`): same rule—avoid silently ANDing unrelated table scopes onto join SQL without an explicit API, to prevent surprising cartesian restrictions. + +## Next implementation steps (optional) + +- After a write on the primary, **read-your-writes** on pooled executors: use `PooledLifeExecutor::with_read_preference(ReadPreference::Primary)` (see `src/pool/pooled.rs`) so `SELECT` paths do not hit a possibly stale replica (same applies to pooled reads right after `INSERT`/`UPDATE`). +- Add examples under `examples/` showing `find_related` + manual `filter` on the returned query type if the API allows chaining. +- If product wants **inherited scope**, add a dedicated method (name TBD) on the relation builder so call sites opt in. +- Cross-link: [`SEAORM_LIFEGUARD_MAPPING.md`](./lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) scopes row points here for `find_related` + scopes semantics. diff --git a/docs/planning/DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md b/docs/planning/DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md index 4f613247..c0bbe2b4 100644 --- a/docs/planning/DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md +++ b/docs/planning/DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md @@ -13,7 +13,8 @@ | Layer | Crate / binary | Responsibility | |--------|----------------|----------------| | Introspection | `lifeguard_migrate::schema_infer` | Query PostgreSQL `information_schema` (and related catalogs), map types conservatively, emit Rust source **as text** via `infer_schema_rust` → `emit_inferred_rust`. **Golden tests** lock emitter output under `lifeguard-migrate/tests/golden/` (no live DB required). | -| CLI | `lifeguard-migrate` subcommand **`infer-schema`** | Parse `--database-url` / env (`DATABASE_URL`, `LIFEGUARD_DATABASE_URL`), `--schema`, repeatable `--table`; connect via `may_postgres`, call `infer_schema_rust`, print or write output. | +| Reconciliation (tables + columns) | `lifeguard_migrate::schema_migration_compare` | Compare live `information_schema` **base table names** to merged `*_generated_from_entities.sql`; for tables in **both** baselines, compare **column names** (`information_schema.columns` vs `generated_migration_diff::column_map_from_merged_baseline` from `CREATE` + `ADD COLUMN` fragments). Name-level only — not full SQL type equality. | +| CLI | `lifeguard-migrate` subcommands **`infer-schema`**, **`compare-schema`** | **`infer-schema`:** `--database-url` / env, `--schema`, repeatable `--table`; call `infer_schema_rust`, print Rust. **`compare-schema`:** `--generated-dir` (directory of `*_generated_from_entities.sql`), same schema flag; exit non-zero on drift. | | Consumption | Application / examples | Teams **copy, review, and commit** emitted `LifeModel` / `LifeRecord` modules into their crate (e.g. `examples/entities`). No automatic merge into `lifeguard-codegen` today. | **Codegen boundary:** Inference outputs **Rust source strings** that are **compatible** with `#[derive(LifeModel, LifeRecord)]` and existing column attributes. It does **not** invoke `lifeguard-derive` or `lifeguard-codegen` at runtime. The derive macros run later, when the pasted source is compiled. @@ -21,13 +22,13 @@ ## What is intentionally out of scope for v0 - **Bidirectional sync** (DB change → Rust → DB) as a single command. -- **Watch mode** / CI diff gates (PRD stretch; may build on stable sort + golden files). +- **Watch mode** / richer CI golden workflows — **deferred** (PRD §5.7a); may build on stable sort + golden files. - **Emitting migrations** from inferred models — migration SQL continues to flow from entity definitions + `lifeguard-migrate` generators, not from `infer-schema` alone. ## Type mapping policy - **Conservative:** unknown PostgreSQL types → omit column with `// OMITTED:` (see `schema_infer.rs` and PRD SI-2). -- **Composite primary keys:** emitted with `TODO` comments; single-column PKs get `#[primary_key]`. +- **Composite primary keys:** each PK column gets `#[primary_key]` (same as multi-field PK support in `lifeguard-derive`). - **Versioning:** mapping tables live in code; when extending types, update tests and this doc’s PRD cross-reference. ## Safety and configuration @@ -44,4 +45,4 @@ ## References - [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md §5](./PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md) -- Implementation: `lifeguard-migrate/src/schema_infer.rs`, `lifeguard-migrate/src/main.rs` (`infer-schema`). +- Implementation: `lifeguard-migrate/src/schema_infer.rs`, `lifeguard-migrate/src/schema_migration_compare.rs`, `lifeguard-migrate/src/main.rs` (`infer-schema`, `compare-schema`). diff --git a/docs/planning/DESIGN_SESSION_UOW.md b/docs/planning/DESIGN_SESSION_UOW.md index ccb37f1c..87222d81 100644 --- a/docs/planning/DESIGN_SESSION_UOW.md +++ b/docs/planning/DESIGN_SESSION_UOW.md @@ -1,13 +1,13 @@ # Design: Session / Unit of Work (Lifeguard) -**Status:** Companion to [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md §9](./PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md). **`ModelIdentityMap`** in `src/session/mod.rs` implements identity (U-1), **dirty keys** + `flush_dirty` (U-2 partial — closure-based persistence; pool story still U-4). +**Status:** Companion to [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md §9](./PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md). **`ModelIdentityMap`** (`src/session/mod.rs`) implements identity (U-1). **`Session`** + **`SessionDirtyNotifier`** (`src/session/uow.rs`) wrap the map and a **`Send`/`Sync` pending-dirty queue** so derived `LifeRecord` can **`attach_session`** and auto-enqueue dirty keys on `set_*` / `ActiveModelTrait::set` / `set_*_expr` without breaking graph **`Send`** bounds. **`flush_dirty`** remains closure-based persistence (U-2); pool usage is U-4. ## Goals (from PRD) - **U-1:** Identity map — same PK → same in-memory handle (implemented). -- **U-2:** Dirty tracking + **flush** — **`mark_dirty` / `flush_dirty`** ship; callers wire `LifeRecord::update` / `save` inside the flush closure. Automatic derive integration (auto-mark on `set`) is **not** implemented. +- **U-2:** Dirty tracking + **flush** — **`mark_dirty`**, **`mark_dirty_key`**, **`flush_dirty`** on **`ModelIdentityMap`**; **`Session::flush_dirty`** drains a mutex-backed pending set into the map then flushes. Callers wire `LifeRecord::update` / `save` inside the flush closure. **`LifeRecord::attach_session(&session)`** (PK entities only): mutating the record enqueues the PK fingerprint for flush when `identity_map_key()` is `Some`. Keep the registered **`Model`** in sync with record edits before flush if the closure reads from `Rc>` (see integration test in `session_identity_flush.rs`). - **U-3:** Explicit session — no thread-local global; maps are constructed by the app (satisfied). -- **U-4:** **LifeguardPool** — session must document whether it holds one executor, pins a worker, or uses another policy. **Decision (v0):** any future `Session` that performs I/O should hold **`&dyn LifeExecutor`** (or a generic bound) obtained **from the pool per operation** unless we add an explicit “pin slot for this UoW” API on [`LifeguardPool`](./PRD_CONNECTION_POOLING.md). Do **not** assume a session can outlive a single pooled checkout without a design that stores `PooledLifeExecutor` or equivalent. +- **U-4:** **LifeguardPool** — **`Session`** does not hold an executor; pass **`&dyn LifeExecutor`** (e.g. **`PooledLifeExecutor`**) into **`Session::flush_dirty`**. For one DB transaction across a flush on the pool, use **`Session::flush_dirty_in_transaction_pooled`**, which pins one primary worker via **`LifeguardPool::exclusive_primary_write_executor`** (per-slot mutex + all statements on that slot). Plain **`PooledLifeExecutor`** still round-robins workers per call. - **U-5:** **`may` coroutines** — `ModelIdentityMap` uses `Rc`/`RefCell` and is **not** `Send`/`Sync`; treat it like other single-threaded cell state: one map per coroutine/thread, or external `Mutex` if shared. ## Fingerprint keys @@ -16,8 +16,9 @@ ## Flush (current + future) -- **Shipped:** `ModelIdentityMap::flush_dirty` walks dirty entries in **lexicographic PK fingerprint order** and invokes `Fn(&dyn LifeExecutor, Rc>) -> Result<(), ActiveModelError>`. Wrap the executor in `lifeguard::Transaction` if you need a single DB transaction across rows. -- **Future:** auto-dirty on `LifeRecord::set` / derive hooks; optional `Session` struct holding executor + map. +- **Shipped:** `ModelIdentityMap::flush_dirty` walks dirty entries in **lexicographic map-key order** (pending-insert keys under `PENDING_INSERT_KEY_PREFIX` sort before normal PK fingerprints) and invokes `Fn(&dyn LifeExecutor, Rc>) -> Result<(), ActiveModelError>`. **`ModelIdentityMap::register_pending_insert`** / **`flush_dirty_with_map_key`** / **`promote_pending_to_loaded`** plus **`is_pending_insert_key`** support rows **without** a stable PK fingerprint until after `LifeRecord::insert` (callers branch on the map key in the flush closure, then promote). **`Session::flush_dirty`** merges **`SessionDirtyNotifier`** keys into the map first, then calls that logic. **`Session::flush_dirty_in_transaction(&MayPostgresExecutor, …)`** runs the flush inside **`Transaction`** on a **direct** client. **`Session::flush_dirty_in_transaction_pooled(&LifeguardPool, …)`** pins one primary slot (**`ExclusivePrimaryLifeExecutor`**) and uses raw `BEGIN` / `COMMIT` / `ROLLBACK` so the whole flush is one connection; map-key variants exist for insert vs update in one transaction. Per-slot mutexes also serialize unrelated jobs that target the same worker index. +- **LifeRecord → model auto-sync (PRD §9):** derived **`attach_session_with_model(&session, &Rc>)`** — after each mutation that notifies the session, **`to_model()`** runs when it succeeds and writes into the linked `Rc` so flush closures read current literals without `*rc.borrow_mut() = rec.to_model()?`. **`attach_session`** without the `Rc` leaves prior manual sync behavior. F-style **`set_*_expr`** is not stored on the `Model` type; those edits stay on the record until `update()`. +- **Future:** optional executor-holding session type. ## References diff --git a/docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md b/docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md index 7b930fa7..f09ee6f6 100644 --- a/docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md +++ b/docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md @@ -3,7 +3,8 @@ **Slug:** `schema_validators_session_and_scopes` **Status:** **Draft** — Requirements and acceptance criteria; design splits into follow-on `DESIGN_*.md` per workstream as implementation starts. **Audience:** Lifeguard maintainers, `lifeguard-derive` authors, and application teams targeting SeaORM-like ergonomics on `may`. -**References:** [README.md](../../README.md) competitive matrix (“Not Implemented” rows); [SEAORM_LIFEGUARD_MAPPING.md](./lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md); `src/query/`, `lifeguard-derive/`, `lifeguard::LifeRecord` / `LifeModel` patterns. +**Iteration 2 (PRD follow-on):** default git branch for the next tranche of work — `feat/schema_validators_session_and_scopes_2` (v0 landed via PR #56 on `main`; this branch continues §5–§9 “still to do” items). +**References:** [COMPARISON.md](../../COMPARISON.md) competitive matrix (“Not Implemented” rows); [SEAORM_LIFEGUARD_MAPPING.md](./lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md); `src/query/`, `lifeguard-derive/`, `lifeguard::LifeRecord` / `LifeModel` patterns. --- @@ -13,11 +14,11 @@ - [x] PRD published (`PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md`) - [x] Design note(s): schema inference CLI / codegen boundary — [DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md](./DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md) -- [x] **Phase A — Schema inference** ([§5](#5-schema-inference-from-db--diesel-style)) — *can ship independently* — **v0 landed:** `lifeguard-migrate infer-schema` + `lifeguard_migrate::schema_infer` (see §5.7) +- [x] **Phase A — Schema inference** ([§5](#5-schema-inference-from-db--diesel-style)) — *can ship independently* — **v0 landed:** `lifeguard-migrate infer-schema` + `compare-schema` + `lifeguard_migrate::schema_infer` / `schema_migration_compare` (see §5.7) - [x] **Phase B — Validators** ([§6](#6-validators-field--model-level)) — **v0 landed:** trait hooks + `run_validators` + `ActiveModelError::Validation`; see [§6.7](#67-implementation-status-v0) -- [x] **Phase C — Scopes** ([§7](#7-scopes-named-query-scopes)) — **v0 landed:** `SelectQuery::scope`, `IntoScope`; see [§7.7](#77-implementation-status-v0) +- [x] **Phase C — Scopes** ([§7](#7-scopes-named-query-scopes)) — **v0 landed:** `SelectQuery::scope`, `IntoScope`, **`#[scope]`** attribute (`lifeguard::scope`); see [§7.7](#77-implementation-status-v0) - [x] **Phase D — F() expressions** ([§8](#8-f-expressions-database-level-expressions)) — **v0 landed:** `ColumnTrait::f_add` / `f_sub` / `f_mul` / `f_div`; see [§8.7](#87-implementation-status-v0) -- [x] **Phase E — Session / Unit of Work (v0 — identity map)** ([§9](#9-session--unit-of-work-identity-map-dirty-tracking)) — **v0:** `ModelIdentityMap`, `fingerprint_pk_values`; **deferred:** dirty flush (U-2), pool-bound session (U-4); see [§9.7](#97-implementation-status-v0) +- [x] **Phase E — Session / Unit of Work (v0 — identity map + session handle)** ([§9](#9-session--unit-of-work-identity-map-dirty-tracking)) — **v0:** `ModelIdentityMap`, `Session`, `SessionDirtyNotifier`, `attach_session` / record auto-dirty enqueue, **`LifeguardPool::exclusive_primary_write_executor`** / **`Session::flush_dirty_in_transaction_pooled`** (U-4 pin-slot); **insert-only flush:** `register_pending_insert`, `flush_dirty_with_map_key`, `promote_pending_to_loaded`, `is_pending_insert_key`; see [§9.7](#97-implementation-status-v0--u-2-partial) - [x] [§10 Success criteria](#10-success-criteria) satisfied for **PRD v0** (partial parity per phase; follow-on work remains in §5–§9 “still to do” bullets) ### 0.2 Workstream rollup @@ -126,20 +127,27 @@ Success means developers can (where applicable) **generate or refresh** models f **Still product-open:** - Emit `LifeModel` only vs also `LifeRecord` stubs vs `PartialModel` hints (v0 emits both derives where applicable). -- Watch mode / CI golden snapshots (PRD stretch). ### 5.7 Implementation status (v0) **Shipped in-tree:** - **CLI:** `cargo run -p lifeguard-migrate -- infer-schema --database-url ` (or `DATABASE_URL` / `LIFEGUARD_DATABASE_URL`). Flags: `--schema` (default `public`), `--table TABLE` (repeatable) to restrict tables. -- **Library:** `lifeguard_migrate::schema_infer::{infer_schema_rust, InferOptions}` — introspects `information_schema`, maps common PostgreSQL types to Rust types conservatively, emits `#[derive(LifeModel, LifeRecord)]` structs with `#[primary_key]` when the PK is a **single** column; **composite** PKs get a `TODO` comment; unsupported types are **omitted** with `// OMITTED:` lines (SI-2). +- **Library:** `lifeguard_migrate::schema_infer::{infer_schema_rust, InferOptions}` — introspects `information_schema`, maps common PostgreSQL types to Rust types conservatively, emits `#[derive(LifeModel, LifeRecord)]` structs with `#[primary_key]` on **each** primary-key column (including **composite** PKs — multiple `#[primary_key]` attributes, matching `lifeguard-derive`); unsupported types are **omitted** with `// OMITTED:` lines (SI-2). -**SI-1 / golden coverage:** deterministic output is covered by unit tests on `emit_inferred_rust` in `lifeguard-migrate/src/schema_infer.rs` against `lifeguard-migrate/tests/golden/*.expected.rs` (single table, omitted column, composite PK TODO, table filter, SQL keyword field). +**SI-1 / golden coverage:** deterministic output is covered by unit tests on `emit_inferred_rust` in `lifeguard-migrate/src/schema_infer.rs` against `lifeguard-migrate/tests/golden/*.expected.rs` (single table, omitted column, composite PK, table filter, SQL keyword field). -**Still to do for Phase A closure:** optional live-Postgres smoke for the **`infer-schema` CLI** end-to-end; CI doc hook. **Docs:** `lifeguard-migrate/README.md` (`infer-schema`), `DEVELOPMENT.md` (migrate / goldens). +**Phase A closure (documentation + tests):** **`infer-schema` CLI subprocess e2e** — `lifeguard-migrate/tests/infer_schema_cli_subprocess.rs` (spawns `CARGO_BIN_EXE_lifeguard-migrate infer-schema`, asserts banner; skips without DB URL). **Library / CI:** `infer_schema_postgres_smoke.rs`, `infer_schema_table_filter_si3.rs` (unchanged). **DBA confidence — live DB vs on-disk generated migrations:** `lifeguard_migrate::schema_migration_compare` + CLI **`compare-schema`** — reconciles **`information_schema` base table names** and, for tables present in both baselines, **column names** (`information_schema.columns` vs columns parsed from merged `CREATE TABLE` + `ADD COLUMN` fragments via `column_map_from_merged_baseline`); **does not** compare SQL type text, constraints, or **indexes** in depth (`pg_indexes` / `CREATE INDEX` reconciliation is a **stretch** — see §5.7a). `tests/migration_db_compare_smoke.rs`. **Docs:** `lifeguard-migrate/README.md` (`infer-schema`, `compare-schema`), `DEVELOPMENT.md` (migrate section). -**Design:** [DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md](./DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md) (CLI vs codegen boundary). +**Design:** [DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md](./DESIGN_SCHEMA_INFERENCE_CLI_CODEGEN.md) (CLI vs codegen boundary; `compare-schema` column reconciliation is name-level only). + +### 5.7a Deferred (Phase A stretch — end of backlog) + +Tackle after core PRD follow-through items: + +- **Watch mode** for `infer-schema` +- **Richer CI golden workflows** (snapshot automation beyond current unit goldens) +- **Index reconciliation (stretch):** extend `lifeguard_migrate::schema_migration_compare` and the **`compare-schema`** CLI to fetch index definitions (e.g. `pg_indexes` / `information_schema.statistics`), parse each index’s column list, and verify indexed columns exist in both `column_map_from_merged_baseline` and the live DB; surface mismatches as failures or warnings; align generated migrations so **`CREATE INDEX`** is represented and validated. Follow-up: optional **lifeguard-derive** / migration-time checks that struct fields map to indexed columns where the schema expects them. (See §5.7 shipped scope: table + column **names** only today.) --- @@ -159,14 +167,14 @@ Success means developers can (where applicable) **generate or refresh** models f ### 6.3 What (scope) -- **Field validators:** Run on values present or changed on `LifeRecord` for insert/update/save paths (exact operations TBD in design). +- **Field validators:** Run on values present or changed on `LifeRecord` for insert/update/delete/save paths (exact operations TBD in design). - **Model validators:** Access multiple fields; run after field validators. - **API surface:** Minimum = **traits** + manual registration or inherent impls; **stretch** = derive attributes (`#[validate(...)]`) where macro hygiene allows. - **Errors:** Typed error type or `LifeError` variant that carries **field paths** and **messages**; optional aggregation mode. ### 6.4 How (approach) -- **Integration point:** Call validator pipeline from **`ActiveModelTrait` save/insert/update** (or a single internal choke point) **before** building SQL. +- **Integration point:** Call validator pipeline from **`ActiveModelTrait` save/insert/update/delete** (or a single internal choke point) **before** building SQL. - **Sync only:** Validators are synchronous closures or trait methods; **no** async/`await` (matches `may` stack). - **Composition:** Small building blocks (`validate_len`, custom `Fn`) composed into a **validator list** per model; optional derive generates lists. - **Testing:** Pure unit tests on validator functions without Postgres; integration tests optional for end-to-end rejection. @@ -175,7 +183,7 @@ Success means developers can (where applicable) **generate or refresh** models f | Req ID | Requirement | Acceptance criteria | |--------|-------------|---------------------| -| V-1 | **Field validators** run for present/changed fields on `LifeRecord` save paths (insert/update as applicable). | Unit tests: failing field validator blocks persistence and returns typed error. | +| V-1 | **Field validators** run for present/changed fields on `LifeRecord` save paths (insert/update/delete as applicable). | Unit tests: failing field validator blocks persistence and returns typed error. | | V-2 | **Model validators** run after field validators and may inspect multiple fields. | Unit test: cross-field rule works. | | V-3 | Errors are **aggregated** or **fail-fast** per explicit policy (default documented). | Tests cover both modes if both are exposed. | | V-4 | Opt-out / skip for specific operations if needed (e.g. `save` vs `insert`) — **if** we expose hooks; otherwise document use of hooks. | Documented in rustdoc. | @@ -190,13 +198,17 @@ Success means developers can (where applicable) **generate or refresh** models f **Shipped in-tree:** -- **Types:** `lifeguard::ValidateOp` (`Insert` | `Update`), `lifeguard::ValidationError` (`field: Option`, `message: String`, with `field` / `model` constructors). +- **Types:** `lifeguard::ValidateOp` (`Insert` | `Update` | `Delete`), `lifeguard::ValidationError` (`field: Option`, `message: String`, with `field` / `model` constructors). - **Errors:** `ActiveModelError::Validation(Vec)` with `Display` listing field-scoped and model-scoped messages (fail-fast; no multi-error aggregation yet). -- **Traits:** `ActiveModelBehavior::validate_fields` / `validate_model` (default no-op), invoked via `lifeguard::run_validators` in order **field → model**. -- **Integration:** `lifeguard-derive` generated `insert` / `update` call `run_validators` **after** `before_insert` / `before_update` (so hook defaults are visible to validation) and **before** SQL build. -- **Tests:** Unit tests on `run_validators` ordering and short-circuit; `cargo clippy` / `lifeguard-derive` tests pass. +- **Traits:** `ActiveModelBehavior::validate_fields` / `validate_model` (default no-op), `validation_strategy` (default [`ValidationStrategy::FailFast`]), invoked via `lifeguard::run_validators` in order **field → model**. +- **V-3:** `ValidationStrategy::Aggregate` collects all `Validation` errors from `validate_fields` then `validate_model`; override `validation_strategy` on the record or call `run_validators_with_strategy` directly. +- **Delete:** `ValidateOp::Delete` after `before_delete`, before SQL; same validator hooks as insert/update. +- **Integration:** `lifeguard-derive` generated `insert` / `update` / `delete` call `run_validators` **after** the corresponding `before_*` hook and **before** SQL build. +- **Tests:** Unit tests on `run_validators` ordering, fail-fast, aggregate collection, and `Delete` op; `cargo clippy` / `lifeguard-derive` tests pass. +- **V-5 (derive sugar):** `#[validate(custom = path)]` on model fields — `path` is `fn(&sea_query::Value) -> Result<(), String>`; `LifeRecord` implements `validate_fields` to run each custom validator when `ActiveModelTrait::get` is `Some` for that column. Unsupported on `#[ignore]`/`#[skip]` fields. Tests: `lifeguard-derive/tests/test_minimal.rs` (`validate_attr_tests`). +- **Built-in predicates:** `lifeguard::predicates` (`src/active_model/predicates.rs`) — `string_utf8_chars_max`, `string_utf8_chars_in_range`, `blob_or_string_byte_len_max`, `i64_in_range`, `f64_in_range` on `sea_query::Value`; unit tests in-module. -**Still to do for fuller Phase B:** optional `#[validate(...)]` derive (V-5), explicit aggregate-errors mode if required (V-3), `DELETE` path validation if product wants it, README / mapping matrix updates (G6). +**G6 (documentation):** [COMPARISON.md](../../COMPARISON.md) competitive/feature bullets and [SEAORM_LIFEGUARD_MAPPING.md](./lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) parity row list shipped validator APIs, predicate names, and the intentional gap vs SeaORM’s broader built-in attribute set. --- @@ -242,9 +254,9 @@ Success means developers can (where applicable) **generate or refresh** models f - **API:** `lifeguard::SelectQuery::scope` and `lifeguard::IntoScope` in `src/query/scope.rs`. Any `sea_query::IntoCondition` (column expressions, `Condition`, etc.) applies as a scope; implementation delegates to `SelectQuery::filter` so predicates **AND** together. - **Pattern:** Entity-associated functions (e.g. `UserEntity::scope_active() -> impl IntoCondition`) are composed with `User::find().scope(UserEntity::scope_active())`. - **Soft delete:** `query::scope` module documents that `LifeModelTrait::soft_delete_column` is applied at execution time and **AND**ed with scoped predicates unless `with_trashed` is set; unit test `scope_and_soft_delete_both_anded_at_execution`. -- **Tests:** `src/query/scope.rs` — composition + soft-delete interaction. +- **Tests:** `src/query/scope.rs` — composition + soft-delete interaction + `scope_or` / `scope_any`. -**Still to do for fuller Phase C:** derive sugar (`#[scope]` / codegen), OR-composition helpers, `find_related`/loader interaction notes in a design doc, README matrix row (G6). +**Still to do for fuller Phase C:** optional codegen beyond `#[scope]` (e.g. scope lists on the struct). **Done in-tree:** `SelectQuery::scope_or` / `scope_any` (PRD SC-2); **`#[scope]`** attribute macro (`lifeguard::scope` / `lifeguard_derive::scope`) on `impl Entity` renames `fn foo` → `scope_foo`. **`find_related` vs scopes:** default behavior documented in crate rustdoc (`query::scope`, `FindRelated`) and [DESIGN_FIND_RELATED_SCOPES.md](./DESIGN_FIND_RELATED_SCOPES.md) (parent scopes are not merged into `find_related` SQL; chain on the returned query). Opt-in `related_scope` / inherited parent scopes remain future work. README matrix (G6) updated for scopes. --- @@ -291,11 +303,15 @@ Success means developers can (where applicable) **generate or refresh** models f - **Tests:** `src/query/column/column_trait.rs` — `test_f_add_update_sql_contains_arithmetic`, basic compile tests for `f_*`. - **Process:** `docs/planning/DEV_RUSTDOC_AND_COVERAGE.md` and `DEVELOPMENT.md` (rustdoc + coverage checklist for feature work). -**Still to do for fuller Phase D:** wire **`LifeRecord::update`** / derive to accept expression RHS without hand-built `Query::update` (F-1 end-to-end on ORM path), `WHERE`/`ORDER BY` examples, README matrix row (G6), integration test on Postgres. +**F-3 (limitations vs raw SQL):** `ColumnTrait::f_add` rustdoc (aggregates/subqueries → `Expr::cust`); [COMPARISON.md](../../COMPARISON.md) competitive section + feature bullets (§10 / G6). + +**PostgreSQL numeric typing (F-style ops):** SeaQuery emits `SimpleExpr` arithmetic; PostgreSQL applies **binary promotion** (e.g. `integer` + `numeric` → `numeric`). Lifeguard does **not** inject implicit casts—align operand types in the query builder, or use `Expr::cust` / explicit SQL for `::bigint`, `::numeric`, etc. [COMPARISON.md](../../COMPARISON.md) + [SEAORM_LIFEGUARD_MAPPING.md](./lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) F() row; rustdoc on `ColumnTrait::f_add` (`src/query/column/column_trait.rs`). + +**Done in-tree:** `LifeRecord` `set__expr` / `__update_exprs` / derived `update()`; `identity_map_key` for session bridge; `insert()` rejects non-empty `__update_exprs`; Postgres `column_f_update.rs` + `column_f_where.rs`; [COMPARISON.md](../../COMPARISON.md) + mapping G6 for F(). ### 8.8 Dependency note -Coordinate with **`SelectQuery`** and **`ActiveModelTrait`** update paths; likely **SeaQuery extensions** or thin lifeguard wrappers (see §8.4). **LifeRecord** integration remains the main follow-on. +**LifeRecord** `update()` / `set_*_expr` path coordinates with SeaQuery `UpdateStatement::value`. **`SelectQuery`** F-style filters use `Expr::expr` + SeaQuery’s `ExprTrait` at the call site. --- @@ -347,13 +363,13 @@ Coordinate with **`SelectQuery`** and **`ActiveModelTrait`** update paths; likel **Shipped in-tree:** - **API:** `lifeguard::ModelIdentityMap` and `lifeguard::fingerprint_pk_values` in `src/session/` — identity map keyed by stable PK fingerprints (`src/session/pk.rs`); same primary key → same `Rc>` (first registration wins; duplicate model dropped). -- **U-2 (partial):** `mark_dirty`, `unmark_dirty`, `is_marked_dirty`, `dirty_len`, `clear_dirty`, `flush_dirty` — dirty keys flushed in **lexicographic order of PK fingerprint** via a closure `Fn(&dyn LifeExecutor, Rc>) -> Result<(), ActiveModelError>` (callers wire `LifeRecord::update` / `save`). **Not** shipped: auto-mark dirty on `LifeRecord::set`, insert-only flush, transactional batching inside the map. +- **U-2 (partial):** `mark_dirty`, **`mark_dirty_key`** (fingerprint string), `unmark_dirty`, `is_marked_dirty`, `dirty_len`, `clear_dirty`, `flush_dirty` / **`flush_dirty_with_map_key`** on **`ModelIdentityMap`** — dirty keys flushed in **lexicographic order of internal map key** (pending-insert keys first, then PK fingerprints) via a closure; callers wire `LifeRecord::update` / `save` / **`insert`**. **`register_pending_insert`**, **`promote_pending_to_loaded`**, **`is_pending_insert_key`** / **`PENDING_INSERT_KEY_PREFIX`** support **insert-only** rows until a real PK exists after `insert`. Derived **`LifeRecord::identity_map_key()`** returns `Some(fingerprint)` when all PK columns are set. **`Session`** (`src/session/uow.rs`) shares an identity map and merges a **`Send`/`Sync` pending-dirty queue** at **`Session::flush_dirty`**; **`Session::flush_dirty_in_transaction`** (`MayPostgresExecutor` + **`Transaction`**). **`Session::flush_dirty_in_transaction_pooled`** + **`LifeguardPool::exclusive_primary_write_executor`** (U-4: per-slot mutex, one primary connection for `BEGIN`/`COMMIT`/`ROLLBACK` around flush); **`flush_dirty_*_with_map_key`** variants for transactional insert vs update. **`LifeRecord::attach_session` / `detach_session`** (PK entities): `set_*`, **`ActiveModelTrait::set` / `take` / `set_col`**, and **`set_*_expr`** enqueue dirty via **`SessionDirtyNotifier`** when the PK is set on the record. - **Design:** `docs/planning/DESIGN_SESSION_UOW.md` — pool pinning, flush, and `may`/threading notes (U-4, U-5). - **Rustdoc:** `session` module documents identity, dirty flush, threading (`Send`/`Sync`). -- **Tests:** `src/session/mod.rs`, `src/session/pk.rs` — identity map, fingerprint, dirty order, flush error retention. +- **Tests:** `src/session/mod.rs`, `src/session/pk.rs`, `src/session/uow.rs` — identity map, fingerprint, dirty order, flush error retention, pending insert flush + promote (unit), `Session` pending merge, `SessionDirtyNotifier` `Send`. **`db_integration_suite`:** `tests/db_integration/session_identity_flush.rs` — raw map flush, `mark_dirty_key` + `identity_map_key`, **`Session` + `attach_session` + record `set_*`**, **`Session::flush_dirty_in_transaction`** / **`flush_dirty_in_transaction_pooled`** → `LifeRecord::update`, **`register_pending_insert`** + **`flush_dirty_with_map_key`** + **`promote_pending_to_loaded`** → `LifeRecord::insert`, same path inside **`flush_dirty_in_transaction_with_map_key`** / **`flush_dirty_in_transaction_pooled_with_map_key`** on Postgres. - **Process:** `docs/planning/DEV_RUSTDOC_AND_COVERAGE.md` and `DEVELOPMENT.md` (rustdoc + coverage checklist for feature work). -**Still to do for fuller Phase E:** auto-dirty / derive integration, optional `Session` type holding executor/pool policy (U-4 integration), README / mapping matrix row if needed beyond §10 snapshot, Postgres integration tests as the API stabilizes. +**Still to do for fuller Phase E:** mapping matrix row tweaks as APIs grow. **Done in-tree:** **`LifeRecord::attach_session_with_model`** — linked `Rc>` updated via **`to_model()`** on each session-notifying mutation when conversion succeeds (PRD §9 / `DESIGN_SESSION_UOW.md`). --- @@ -361,7 +377,7 @@ Coordinate with **`SelectQuery`** and **`ActiveModelTrait`** update paths; likel - [x] Each **phase** (A–E) has **passing tests** (unit and, where needed, integration with `TEST_DATABASE_URL`) — **v0:** phases ship unit tests; integration coverage varies by workstream (see §5.7–§9.7). - [x] **Public rustdoc** describes the supported API surface and sharp edges — **v0:** each phase documents limitations in-module (ongoing: expand examples as APIs stabilize). -- [x] [SEAORM_LIFEGUARD_MAPPING.md](./lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) and [README.md](../../README.md) competitive table updated: **Partial** / **Implemented** labels for schema inference, validators, scopes, F(), session/UoW; mapping doc **PRD parity snapshot** table. +- [x] [SEAORM_LIFEGUARD_MAPPING.md](./lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md) and [COMPARISON.md](../../COMPARISON.md) competitive table updated: **Partial** / **Implemented** labels for schema inference, validators, scopes, F(), session/UoW; mapping doc **PRD parity snapshot** table. - [x] No new **unwrap** in library paths without JSF policy compliance; clippy `-D warnings` on touched crates — **policy:** `#![deny(clippy::unwrap_used)]` / `expect_used` on `lifeguard` crate; run clippy on touched crates before merge. --- @@ -389,6 +405,6 @@ Coordinate with **`SelectQuery`** and **`ActiveModelTrait`** update paths; likel ## 13. References -- [README.md](../../README.md) — “Competitive metrics” table (Not Implemented rows). +- [COMPARISON.md](../../COMPARISON.md) — competitive metrics table (Not Implemented rows). - [PRD_CONNECTION_POOLING.md](./PRD_CONNECTION_POOLING.md) — pool semantics that Session must align with. - PostgreSQL information schema — introspection source of truth for Phase A. diff --git a/docs/planning/README.md b/docs/planning/README.md index 5cf66c22..aeff6f2f 100644 --- a/docs/planning/README.md +++ b/docs/planning/README.md @@ -37,6 +37,7 @@ Generated: 2026-01-22 ## Active design docs (pooling) - [`DESIGN_CONNECTION_POOLING.md`](./DESIGN_CONNECTION_POOLING.md) — in-process pool behavior, metrics, PRD §9 decisions (companion to [`PRD_CONNECTION_POOLING.md`](./PRD_CONNECTION_POOLING.md)). +- [`DESIGN_FIND_RELATED_SCOPES.md`](./DESIGN_FIND_RELATED_SCOPES.md) — how named scopes interact with `find_related` / loaders (PRD Phase C follow-on). ## Active PRDs (ORM parity) diff --git a/docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md b/docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md index 5670fc65..8270cd9e 100644 --- a/docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md +++ b/docs/planning/lifeguard-derive/SEAORM_LIFEGUARD_MAPPING.md @@ -6,15 +6,15 @@ This document maps SeaORM (v2.0.0-rc.28) and SeaQuery (v0.32.7) components to th ### PRD parity snapshot (schema, validators, scopes, F(), session) -Cross-reference: [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md](../PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md). These rows summarize **v0** shipped behavior vs SeaORM-style **vision**; the README competitive matrix tracks the same features. +Cross-reference: [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md](../PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md). These rows summarize **v0** shipped behavior vs SeaORM-style **vision**; [COMPARISON.md](../../../COMPARISON.md) tracks the same features in table form. | Capability | Primary API / location | Status | Notes | |------------|------------------------|--------|-------| -| **Schema inference (DB → Rust)** | `lifeguard-migrate infer-schema`, `schema_infer::emit_inferred_rust`, `tests/golden/*.expected.rs` | 🟡 **Partial** | PRD §5.7; deterministic emitter golden tests; conservative type mapping; composite PK gaps possible | -| **Validators** | `run_validators`, `ActiveModelBehavior::validate_fields` / `validate_model`, `ActiveModelError::Validation` | 🟡 **Partial** | PRD §6.7; derive optional sugar TBD | -| **Scopes** | `SelectQuery::scope`, `IntoScope`, `src/query/scope.rs` | 🟡 **Partial** | PRD §7.7; AND composition; soft-delete interaction documented | -| **F() expressions** | `ColumnTrait::f_add` / `f_sub` / `f_mul` / `f_div` | 🟡 **Partial** | PRD §8.7; `UPDATE SET` RHS; `WHERE`/`ORDER BY` / LifeRecord path TBD | -| **Session / UoW** | `ModelIdentityMap`, `fingerprint_pk_values`, `mark_dirty` / `flush_dirty`, `src/session/` | 🟡 **Partial** | PRD §9.7; identity + closure-based dirty flush — **no** auto-dirty on `LifeRecord::set`, **no** pool-bound `Session` type yet | +| **Schema inference (DB → Rust)** | `lifeguard-migrate infer-schema`, `compare-schema` (tables + column names for shared tables vs merged SQL), `schema_infer::emit_inferred_rust`, `schema_migration_compare`, `generated_migration_diff::column_map_from_merged_baseline`, `tests/golden/*.expected.rs` | 🟡 **Partial** | PRD §5.7; deterministic emitter golden tests; conservative type mapping; composite PKs emit multiple `#[primary_key]`; `compare-schema` reconciles column **names** (not SQL type/column-def equality); **Deferred:** watch mode / richer CI golden workflows (PRD §5.7a) | +| **Validators** | `run_validators`, `run_validators_with_strategy`, `ValidationStrategy` (FailFast / Aggregate), `ActiveModelBehavior::validate_fields` / `validate_model` / `validation_strategy`, `ActiveModelError::Validation`, `ValidateOp` (Insert / Update / Delete), `#[validate(custom = path)]` on fields, `lifeguard::predicates` | 🟡 **Partial** | PRD §6.7. **Shipped:** field → model order; fail-fast default; aggregate collects field+model errors; delete path runs validators; derive `custom` (`fn(&Value) -> Result<(), String>`). **Predicates:** `string_utf8_chars_max`, `string_utf8_chars_in_range`, `blob_or_string_byte_len_max`, `i64_in_range`, `f64_in_range` on `Value`. **Gap vs SeaORM:** no full built-in attribute matrix (length/range as derive attrs on every type); compose via `validate_fields` + predicates or custom fns. | +| **Scopes** | `SelectQuery::scope`, `scope_or`, `scope_any`, `IntoScope`, `#[scope]` on `impl Entity` (`lifeguard::scope`), `src/query/scope.rs` | 🟡 **Partial** | PRD §7.7; AND + OR composition; soft-delete interaction; **`find_related` does not inherit parent `scope` predicates** — chain `.scope`/`.filter` on the query `find_related` returns; see [DESIGN_FIND_RELATED_SCOPES.md](../DESIGN_FIND_RELATED_SCOPES.md) and rustdoc on `query::scope` / `FindRelated` | +| **F() expressions** | `ColumnTrait::f_add` / `f_sub` / `f_mul` / `f_div`; `LifeRecord::set_*_expr` + `__update_exprs` on derived `update()`; `Expr::expr` + `ExprTrait` / `order_by_expr` for `WHERE` / `ORDER BY` | 🟡 **Partial** | PRD §8.7; Postgres integration tests in `column_f_update.rs` / `column_f_where.rs`. **PostgreSQL numeric typing:** expressions use SeaQuery `SimpleExpr` arithmetic; the server applies **binary promotion** (e.g. `integer` ± `numeric` → `numeric`). Lifeguard does **not** auto-cast operands—if you need a specific result or storage type (e.g. force `bigint`), align column and RHS types in the query builder or use `Expr::cust` / raw SQL for explicit `::type` casts. See `ColumnTrait::f_add` rustdoc and PRD §8.7. | +| **Session / UoW** | `ModelIdentityMap`, `fingerprint_pk_values`, `mark_dirty` / `mark_dirty_key` / `flush_dirty` / `flush_dirty_with_map_key`, `register_pending_insert` / `promote_pending_to_loaded` / `is_pending_insert_key`, `Session`, `LifeRecord::identity_map_key`, `src/session/` | 🟡 **Partial** | PRD §9.7; identity + dirty flush + insert-only pending keys; `attach_session` auto-dirty when PK set — not full SeaORM session semantics | ## Core Features @@ -69,7 +69,7 @@ Cross-reference: [PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md](../PRD_SCHEMA_VAL | `DeriveIntoActiveModel` | ❌ Missing | 🔴 **Future** | Conversion from Model to ActiveModel - **Not needed for migrations** | | `DeriveActiveModelBehavior` | ✅ Implemented | ✅ Complete | ActiveModelBehavior trait implementation (default impl generated for all Records) | | `DeriveActiveEnum` | ❌ Missing | 🟡 **Future** | Enum support for ActiveModel - **Not needed for migrations** | -| `DeriveMigrationName` | ❌ Missing | 🟡 **Future** | Migration name generation - **Nice-to-have, not a blocker for migrations** | +| `DeriveMigrationName` | ✅ `lifeguard::migration::DeriveMigrationName` + `MigrationName` | ✅ **Implemented** | Unit struct → snake_case `MIGRATION_NAME` + `MigrationName`; pair with manual `Migration` | | `FromJsonQueryResult` | ❌ Missing | 🟡 **Future** | JSON query result deserialization (JSON column support is ✅ core feature) | | `DeriveValueType` | ❌ Missing | 🟡 **Future** | ValueType trait for wrapper types - **Not needed for migrations** | | `DeriveDisplay` | ❌ Missing | 🟡 **Future** | Display trait for ActiveEnum - **Not needed for migrations** | @@ -392,7 +392,7 @@ This design simplifies the API while maintaining the same functionality. **Note:** The missing derive macros listed above are **NOT prerequisites** for migrations. See `MIGRATION_PREREQUISITES_DISCOVERY.md` for detailed analysis. **Future State:** -- `DeriveMigrationName` - Generate migration names (nice-to-have, not a blocker) +- `DeriveMigrationName` — **shipped:** `lifeguard::migration::DeriveMigrationName` + `MigrationName` trait (`MIGRATION_NAME` constant) - Migration CLI tool - Integration with migration tools #### JSON Support @@ -892,7 +892,7 @@ SQL Views are **virtual tables** based on the result of a SQL query. They: | **Materialized View** | **Cached Query Model** | 🟡 **Future** | Model backed by materialized view table, refresh support | | **View with JOINs** | **Query-based Model** | ✅ **Partial** | Use query builder with joins, map to struct | | **View with Aggregations** | **Projection/Partial Model** | ✅ **Implemented** | `DerivePartialModel` for selected columns | -| **View as Security Layer** | **Scoped Queries** | 🟡 **Partial** | `SelectQuery::scope` / `IntoScope` + entity helpers returning `IntoCondition` (`src/query/scope.rs`); derive sugar TBD | +| **View as Security Layer** | **Scoped Queries** | 🟡 **Partial** | `SelectQuery::scope` / `IntoScope` + `#[scope]` (`lifeguard::scope`) + entity helpers returning `IntoCondition` (`src/query/scope.rs`) | #### Implementation Patterns diff --git a/lifeguard-derive/src/attributes.rs b/lifeguard-derive/src/attributes.rs index 73f4356e..860f2a9b 100644 --- a/lifeguard-derive/src/attributes.rs +++ b/lifeguard-derive/src/attributes.rs @@ -118,6 +118,32 @@ pub fn has_attribute(field: &Field, attr_name: &str) -> bool { .any(|attr| attr.path().is_ident(attr_name)) } +/// Parse `#[validate(custom = path)]` on a model field (PRD V-5). +/// +/// Multiple attributes or `custom = a, custom = b` in one `#[validate(...)]` are supported. +/// The generated code calls each path as `path(&sea_query::Value) -> Result<(), String>`. +pub fn parse_field_validate_custom_paths(field: &Field) -> syn::Result> { + let mut paths = Vec::new(); + for attr in &field.attrs { + if !attr.path().is_ident("validate") { + continue; + } + attr.parse_nested_meta(|meta| { + if meta.path.is_ident("custom") { + let value = meta.value()?; + let path: syn::Path = value.parse()?; + paths.push(path); + Ok(()) + } else { + Err(meta.error( + "unknown `validate` item; expected `custom = path` (function `fn(&sea_query::Value) -> Result<(), String>`)", + )) + } + })?; + } + Ok(paths) +} + /// Holds the configuration extracted from `#[has_many]`, `#[belongs_to]`, etc. #[derive(Debug, Clone, Default)] pub struct RelationAttribute { @@ -373,6 +399,16 @@ pub fn parse_column_attributes(field: &Field) -> Result, } /// Parse table-level attributes from struct attributes @@ -585,6 +624,25 @@ pub fn parse_table_attributes( } else if let Ok(meta) = attr.meta.require_list() { table_attrs.after_delete = Some(meta.tokens.to_string()); } + } else if attr.path().is_ident("validation_strategy") { + if let Ok(meta) = attr.meta.require_name_value() { + if let syn::Expr::Lit(ExprLit { + lit: Lit::Str(s), .. + }) = &meta.value + { + let v = s.value().to_ascii_lowercase(); + table_attrs.validation_strategy = Some(match v.as_str() { + "aggregate" => TableValidationStrategy::Aggregate, + "fail_fast" | "failfast" => TableValidationStrategy::FailFast, + _ => { + return Err(syn::Error::new_spanned( + attr, + "validation_strategy must be \"aggregate\" or \"fail_fast\"", + )); + } + }); + } + } } } diff --git a/lifeguard-derive/src/lib.rs b/lifeguard-derive/src/lib.rs index 1c115430..42d10f0e 100644 --- a/lifeguard-derive/src/lib.rs +++ b/lifeguard-derive/src/lib.rs @@ -82,7 +82,9 @@ pub fn derive_from_row(input: TokenStream) -> TokenStream { has_many, belongs_to, has_one, - cursor_tiebreak + cursor_tiebreak, + validate, + validation_strategy ) )] pub fn derive_life_model(input: TokenStream) -> TokenStream { @@ -98,6 +100,9 @@ pub fn derive_life_model(input: TokenStream) -> TokenStream { /// - `dirty_fields()` method (returns list of changed fields) /// - `is_dirty()` method (checks if any fields changed) /// - Setter methods for each field +/// - Optional `#[validate(custom = path)]` on fields: `path` is `fn(&sea_query::Value) -> Result<(), String>`; runs when the field is set (`get` is `Some`) during `validate_fields`. +/// - Optional `#[validation_strategy = "aggregate"]` or `"fail_fast"` on the struct: controls how multiple field validators combine (default: fail fast). +/// - F-style **`UPDATE`**: `set__expr(sea_query::SimpleExpr)` schedules `SET col = ` (e.g. `Column::n.f_add(1)`); stored in `__update_exprs` until `reset` / `from_model`. Literal `set_*` clears the expression for that column. #[proc_macro_derive( LifeRecord, attributes( @@ -120,7 +125,9 @@ pub fn derive_life_model(input: TokenStream) -> TokenStream { has_many, belongs_to, has_one, - cursor_tiebreak + cursor_tiebreak, + validate, + validation_strategy ) )] pub fn derive_life_record(input: TokenStream) -> TokenStream { @@ -252,3 +259,47 @@ pub fn derive_partial_model(input: TokenStream) -> TokenStream { pub fn derive_try_into_model(input: TokenStream) -> TokenStream { macros::derive_try_into_model(input) } + +/// Derive macro for a **unit struct** migration name (`snake_case` of the type name). +/// +/// Implements [`lifeguard::migration::MigrationName`] and defines an associated constant +/// `MIGRATION_NAME`. Pair with manual [`lifeguard::migration::Migration`] for `up` / `down`. +/// +/// ```ignore +/// #[derive(DeriveMigrationName)] +/// pub struct CreateUsersTable; +/// +/// impl lifeguard::migration::Migration for CreateUsersTable { +/// fn name(&self) -> &str { +/// lifeguard::migration::MigrationName::migration_name(self) +/// } +/// // ... +/// } +/// ``` +#[proc_macro_derive(DeriveMigrationName)] +pub fn derive_migration_name(input: TokenStream) -> TokenStream { + macros::derive_migration_name(input) +} + +/// Attribute for named query scopes on `impl Entity` (PRD Phase C). +/// +/// Transforms `fn active() -> …` into `pub fn scope_active() -> …` so call sites use +/// `Entity::scope_active()` with `SelectQuery::scope` on the query builder. +/// +/// - Must be on an **associated function** with **no** `self` receiver. +/// - If the function is already named `scope_*`, the name is left unchanged. +/// - Unannotated visibility becomes `pub` (inherited → `pub`); `pub(crate)` and `pub` are kept. +/// +/// ```ignore +/// impl Entity { +/// #[lifeguard_derive::scope] +/// fn active() -> impl sea_query::IntoCondition { +/// Column::Status.eq("active") +/// } +/// } +/// // User::find().scope(Entity::scope_active()) +/// ``` +#[proc_macro_attribute] +pub fn scope(attr: TokenStream, item: TokenStream) -> TokenStream { + macros::scope_attr::scope_attr(attr, item) +} diff --git a/lifeguard-derive/src/macros/life_record.rs b/lifeguard-derive/src/macros/life_record.rs index cae3af00..85133c7c 100644 --- a/lifeguard-derive/src/macros/life_record.rs +++ b/lifeguard-derive/src/macros/life_record.rs @@ -79,6 +79,16 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { } } + let has_primary_keys = fields.iter().any(|field| { + if attributes::has_attribute(field, "skip") || attributes::has_attribute(field, "ignore") { + return false; + } + match attributes::parse_column_attributes(field) { + Ok(attrs) => attrs.is_primary_key, + Err(_) => false, + } + }); + // Parse table-level attributes to get hook metadata let table_attrs = match attributes::parse_table_attributes(&input.attrs, &valid_columns) { Ok(attrs) => attrs, @@ -120,6 +130,9 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { let mut delete_where_clauses = Vec::new(); // WHERE clauses for DELETE let mut returning_extractors: Vec = Vec::new(); // Code to extract returned PK values let mut to_json_field_conversions = Vec::new(); // Code to convert each field to JSON + let mut field_validate_fail_fast_fragments: Vec = Vec::new(); // #[validate(custom = ...)] — FailFast (`?`) + let mut field_validate_aggregate_fragments: Vec = Vec::new(); // same — Aggregate (collect into `errs`) + let mut update_expr_setters: Vec = Vec::new(); // set__expr for UPDATE SET expr RHS (F-style) for field in fields.iter() { let field_name = match utils::field_ident(field) { @@ -171,6 +184,14 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { // Skip ignored fields - they're not included in database operations // But we still need to add them to the Record struct and conversion methods if is_ignored { + if attributes::has_attribute(field, "validate") { + return syn::Error::new_spanned( + field, + "`#[validate]` is not supported on `#[ignore]` / `#[skip]` fields", + ) + .to_compile_error() + .into(); + } // Still include in Record struct with original type (not Option) record_fields.push(quote! { pub #field_name: #field_type, @@ -205,6 +226,44 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { let column_variant_name = utils::pascal_case(&field_name.to_string()); let column_variant = Ident::new(&column_variant_name, field_name.span()); + let validate_custom_paths = match attributes::parse_field_validate_custom_paths(field) { + Ok(p) => p, + Err(e) => return e.to_compile_error().into(), + }; + if !validate_custom_paths.is_empty() { + let col_name_lit = LitStr::new(&db_column_name, field_name.span()); + let validator_calls_fail_fast = validate_custom_paths.iter().map(|path| { + quote! { + #path(&val).map_err(|msg| lifeguard::ActiveModelError::Validation( + vec![lifeguard::active_model::validate_op::ValidationError::field(#col_name_lit, msg)], + ))?; + } + }); + let validator_calls_aggregate = validate_custom_paths.iter().map(|path| { + quote! { + if let Err(msg) = #path(&val) { + errs.push(lifeguard::active_model::validate_op::ValidationError::field(#col_name_lit, msg)); + } + } + }); + field_validate_fail_fast_fragments.push(quote! { + if let Some(val) = lifeguard::ActiveModelTrait::get( + self, + <#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant, + ) { + #(#validator_calls_fail_fast)* + } + }); + field_validate_aggregate_fragments.push(quote! { + if let Some(val) = lifeguard::ActiveModelTrait::get( + self, + <#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant, + ) { + #(#validator_calls_aggregate)* + } + }); + } + // Track primary key information if is_primary_key { primary_key_field_names.push(field_name.clone()); @@ -264,23 +323,39 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { to_model_struct_fields.push(quote! { #field_name, }); } - // Generate dirty field check - // For Option fields (both cases), check if Some - dirty_fields_check.push(quote! { - if self.#field_name.is_some() { - dirty.push(stringify!(#field_name).to_string()); - } - }); + // Generate dirty field check: literal set, or F-style expression scheduled for UPDATE + if is_primary_key { + dirty_fields_check.push(quote! { + if self.#field_name.is_some() { + dirty.push(stringify!(#field_name).to_string()); + } + }); + } else { + dirty_fields_check.push(quote! { + if self.#field_name.is_some() + || self.__update_exprs.contains_key(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant) + { + dirty.push(stringify!(#field_name).to_string()); + } + }); + } // Generate setter method // If field is already Option, setter accepts Option directly // Otherwise, setter accepts T and wraps in Some() let setter_name = Ident::new(&format!("set_{field_name}"), field_name.span()); + let session_notify = if has_primary_keys { + quote! { self.__lg_session_notify_dirty(); } + } else { + quote! {} + }; if is_already_option { setter_methods.push(quote! { /// Set the #field_name field pub fn #setter_name(&mut self, value: #field_type) -> &mut Self { + self.__update_exprs.remove(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant); self.#field_name = value; + #session_notify self } }); @@ -288,7 +363,29 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { setter_methods.push(quote! { /// Set the #field_name field pub fn #setter_name(&mut self, value: #field_type) -> &mut Self { + self.__update_exprs.remove(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant); self.#field_name = Some(value); + #session_notify + self + } + }); + } + + if !is_primary_key { + let expr_setter_name = + Ident::new(&format!("{setter_name}_expr"), field_name.span()); + let expr_session_notify = if has_primary_keys { + quote! { self.__lg_session_notify_dirty(); } + } else { + quote! {} + }; + update_expr_setters.push(quote! { + /// Schedule a database expression for this column on [`ActiveModelTrait::update`](lifeguard::ActiveModelTrait::update) (e.g. [`ColumnTrait::f_add`](lifeguard::ColumnTrait::f_add)). + /// Clears any literal value previously set for this field. + pub fn #expr_setter_name(&mut self, expr: sea_query::SimpleExpr) -> &mut Self { + self.#field_name = None; + self.__update_exprs.insert(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant, expr); + #expr_session_notify self } }); @@ -320,13 +417,21 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { ); active_model_set_match_arms.push(quote! { <#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant => { - #value_to_field_conversion + let __lg_set_column_result = #value_to_field_conversion; + if __lg_set_column_result.is_ok() { + self.__update_exprs.remove(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant); + } + __lg_set_column_result } }); active_model_set_col_match_arms.push(quote! { #db_column_name => { - #value_to_field_conversion + let __lg_set_column_result = #value_to_field_conversion; + if __lg_set_column_result.is_ok() { + self.__update_exprs.remove(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant); + } + __lg_set_column_result } }); @@ -334,13 +439,30 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { // Use inner_type for type conversion (e.g., String from Option) let field_to_value_conversion = type_conversion::generate_option_field_to_value(field_name, inner_type); - active_model_take_match_arms.push(quote! { - <#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant => { - let value = #field_to_value_conversion; - self.#field_name = None; - value - } - }); + if has_primary_keys { + active_model_take_match_arms.push(quote! { + <#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant => { + let __lg_take_notify = self.#field_name.is_some() + || self.__update_exprs.contains_key(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant); + self.__update_exprs.remove(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant); + let value = #field_to_value_conversion; + self.#field_name = None; + if __lg_take_notify { + self.__lg_session_notify_dirty(); + } + value + } + }); + } else { + active_model_take_match_arms.push(quote! { + <#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant => { + self.__update_exprs.remove(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant); + let value = #field_to_value_conversion; + self.#field_name = None; + value + } + }); + } active_model_reset_fields.push(quote! { self.#field_name = None; @@ -426,7 +548,9 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { let has_save_as = col_attrs.save_as.is_some(); if has_save_as { update_set_clauses.push(quote! { - if self.get(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant).is_some() { + if let Some(expr_entry) = self.__update_exprs.get(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant) { + query.value(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant, expr_entry.clone()); + } else if self.get(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant).is_some() { use lifeguard::query::column::definition::get_static_expr; if let Some(save_expr) = <#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant.column_save_as() { let static_str = get_static_expr(&save_expr); @@ -436,7 +560,9 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { }); } else { update_set_clauses.push(quote! { - if let Some(value) = self.get(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant) { + if let Some(expr_entry) = self.__update_exprs.get(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant) { + query.value(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant, expr_entry.clone()); + } else if let Some(value) = self.get(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant) { query.value(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant, sea_query::Expr::val(value)); } }); @@ -446,7 +572,9 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { // This is the one actually used in update() method if has_save_as { update_set_clauses_from_hooks.push(quote! { - if record_for_hooks.get(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant).is_some() { + if let Some(expr_entry) = record_for_hooks.__update_exprs.get(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant) { + query.value(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant, expr_entry.clone()); + } else if record_for_hooks.get(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant).is_some() { use lifeguard::query::column::definition::get_static_expr; if let Some(save_expr) = <#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant.column_save_as() { let static_str = get_static_expr(&save_expr); @@ -456,7 +584,9 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { }); } else { update_set_clauses_from_hooks.push(quote! { - if let Some(value) = record_for_hooks.get(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant) { + if let Some(expr_entry) = record_for_hooks.__update_exprs.get(&<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant) { + query.value(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant, expr_entry.clone()); + } else if let Some(value) = record_for_hooks.get(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant) { query.value(<#entity_name as lifeguard::LifeModelTrait>::Column::#column_variant, sea_query::Expr::val(value)); } }); @@ -554,9 +684,26 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { }); } - // Generate primary key check code for save() - // If there are no primary keys, save() should always do insert - let has_primary_keys = !primary_key_field_names.is_empty(); + let identity_map_key_method = if has_primary_keys { + let pk_cap = primary_key_column_variants.len(); + quote! { + /// Stable fingerprint for [`lifeguard::ModelIdentityMap::mark_dirty_key`](lifeguard::ModelIdentityMap::mark_dirty_key) when **all** PK columns are set on this record. + #[must_use] + pub fn identity_map_key(&self) -> Option { + let mut __pk_vals = Vec::with_capacity(#pk_cap); + #( + __pk_vals.push(lifeguard::ActiveModelTrait::get( + self, + <#entity_name as lifeguard::LifeModelTrait>::Column::#primary_key_column_variants, + )?); + )* + Some(lifeguard::session::fingerprint_pk_values(&__pk_vals)) + } + } + } else { + quote! {} + }; + let mut save_pk_checks = Vec::new(); for field_name in primary_key_field_names.iter() { save_pk_checks.push(quote! { @@ -697,6 +844,53 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { quote! { Ok(()) } }; + let validate_fields_impl = if field_validate_fail_fast_fragments.is_empty() { + quote! {} + } else { + quote! { + fn validate_fields( + &self, + op: lifeguard::active_model::validate_op::ValidateOp, + ) -> Result<(), lifeguard::ActiveModelError> { + match self.validation_strategy(op) { + lifeguard::active_model::validate_op::ValidationStrategy::FailFast => { + #(#field_validate_fail_fast_fragments)* + Ok(()) + } + lifeguard::active_model::validate_op::ValidationStrategy::Aggregate => { + let mut errs: Vec = Vec::new(); + #(#field_validate_aggregate_fragments)* + if errs.is_empty() { + Ok(()) + } else { + Err(lifeguard::ActiveModelError::Validation(errs)) + } + } + } + } + } + }; + + let validation_strategy_impl = match table_attrs.validation_strategy { + None => quote! {}, + Some(attributes::TableValidationStrategy::FailFast) => quote! { + fn validation_strategy( + &self, + _op: lifeguard::active_model::validate_op::ValidateOp, + ) -> lifeguard::active_model::validate_op::ValidationStrategy { + lifeguard::active_model::validate_op::ValidationStrategy::FailFast + } + }, + Some(attributes::TableValidationStrategy::Aggregate) => quote! { + fn validation_strategy( + &self, + _op: lifeguard::active_model::validate_op::ValidateOp, + ) -> lifeguard::active_model::validate_op::ValidationStrategy { + lifeguard::active_model::validate_op::ValidationStrategy::Aggregate + } + }, + }; + let build_delete_query_ts = if table_attrs.soft_delete { let set_updated_at = if table_attrs.auto_timestamp { quote! { @@ -727,12 +921,148 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { } }; + let session_link_struct_field = if has_primary_keys { + quote! { + #[doc(hidden)] + pub __lg_session_notifier: Option, + /// When [`Self::attach_session_with_model`] is used, mutations sync into this handle via [`Self::to_model`]. + #[doc(hidden)] + pub __lg_session_model: Option>, + } + } else { + quote! {} + }; + + let session_new_init = if has_primary_keys { + quote! { + __lg_session_notifier: None, + __lg_session_model: None, + } + } else { + quote! {} + }; + + // TODO(lifeguard-session): Defer/batch `to_model()` + identity-map `replace_with` until flush + // (e.g. dirty bit + single sync before `Session::flush_dirty`) to avoid O(n fields) per + // mutation when `attach_session_with_model` is used. Non-trivial: keep parity with F-style + // `set_*_expr` (not on `Model`) and `to_model()` / `FieldRequired` semantics. + let session_helpers = if has_primary_keys { + quote! { + /// Wire this record to `session` so `set_*`, [`ActiveModelTrait::set`](lifeguard::ActiveModelTrait::set), and F-style `set_*_expr` enqueue dirty keys (merged at [`Session::flush_dirty`](lifeguard::session::Session::flush_dirty)) when the primary key is set (PRD §9). + /// + /// Does **not** link the identity map [`Rc`]; use [`Self::attach_session_with_model`] to keep the registered model in sync when you mutate this record (see PRD §9). + pub fn attach_session(&mut self, session: &lifeguard::session::Session<#entity_name>) { + self.__lg_session_notifier = Some(session.dirty_notifier()); + self.__lg_session_model = None; + } + + /// Like [`Self::attach_session`], but also keeps `model_rc` (typically from [`Session::register_loaded`](lifeguard::session::Session::register_loaded)) updated on each notifying mutation by calling [`Self::to_model`] when it succeeds—so [`Session::flush_dirty`](lifeguard::session::Session::flush_dirty) closures see current literals without a manual `*rc.borrow_mut() = rec.to_model()?`. + /// + /// If [`Self::to_model`] returns `Err` (e.g. required field unset), the model is left unchanged for that mutation. F-style `set_*_expr` values are not represented on the [`Model`](lifeguard::ModelTrait) type; they remain on the record only. + /// + /// # Thread safety + /// + /// This links the record to the same [`Rc`](std::rc::Rc) as the identity map. The session is single-threaded; **do not** use this record from another OS thread while [`Session`](lifeguard::session::Session) on the original thread can still access that `Rc`. Call [`Self::detach_session`] before moving the record across threads, or keep session and record on one thread. See [`SessionIdentityModelCell`](lifeguard::session::SessionIdentityModelCell). + /// + /// # Performance + /// + /// On each notifying mutation (`set_*`, [`ActiveModelTrait::set`](lifeguard::ActiveModelTrait::set), etc.), the derive calls [`Self::to_model`] and writes the result into the linked `Rc`. That **rebuilds the full [`Model`](lifeguard::ModelTrait) from the record** (typically **O(n fields)** in clones and allocations) so [`Session::flush_dirty`](lifeguard::session::Session::flush_dirty) closures always see current literals without a separate sync step. This is intentional (see `DESIGN_SESSION_UOW.md`); for **many** field updates in a row on wide entities, consider [`Self::attach_session`] only and sync the map once before flush, or batch work and accept the trade-off. + pub fn attach_session_with_model( + &mut self, + session: &lifeguard::session::Session<#entity_name>, + model_rc: &std::rc::Rc>, + ) { + self.__lg_session_notifier = Some(session.dirty_notifier()); + self.__lg_session_model = Some(lifeguard::session::SessionIdentityModelCell::new(model_rc)); + } + + /// Stop forwarding mutations to the session dirty queue and clear any linked identity-map handle. + pub fn detach_session(&mut self) { + self.__lg_session_notifier = None; + self.__lg_session_model = None; + } + + #[doc(hidden)] + #[inline] + fn __lg_session_notify_dirty(&self) { + if let Some(ref n) = self.__lg_session_notifier { + n.notify_identity_map_dirty(self.identity_map_key()); + } + if let Some(ref cell) = self.__lg_session_model { + // `Self::to_model()` clones every field into `Model` (see `attach_session_with_model`); hot-path cost if notification behavior changes. + if let Ok(m) = self.to_model() { + let _ = cell.replace_with(m); + } + } + } + } + } else { + quote! {} + }; + + let active_model_set_impl = if has_primary_keys { + quote! { + fn set(&mut self, column: <#entity_name as lifeguard::LifeModelTrait>::Column, value: sea_query::Value) -> Result<(), lifeguard::ActiveModelError> { + let __lg_set_out = match column { + #(#active_model_set_match_arms)* + }; + if __lg_set_out.is_ok() { + self.__lg_session_notify_dirty(); + } + __lg_set_out + } + } + } else { + quote! { + fn set(&mut self, column: <#entity_name as lifeguard::LifeModelTrait>::Column, value: sea_query::Value) -> Result<(), lifeguard::ActiveModelError> { + match column { + #(#active_model_set_match_arms)* + } + } + } + }; + + let active_model_take_impl = quote! { + fn take(&mut self, column: <#entity_name as lifeguard::LifeModelTrait>::Column) -> Option { + match column { + #(#active_model_take_match_arms)* + } + } + }; + + let active_model_set_col_impl = if has_primary_keys { + quote! { + fn set_col(&mut self, col_name: &str, value: sea_query::Value) -> Result<(), lifeguard::ActiveModelError> { + let __lg_set_col_out = match col_name { + #(#active_model_set_col_match_arms)* + _ => Err(lifeguard::ActiveModelError::Other(format!("Column string not found on record: {}", col_name))) + }; + if __lg_set_col_out.is_ok() { + self.__lg_session_notify_dirty(); + } + __lg_set_col_out + } + } + } else { + quote! { + fn set_col(&mut self, col_name: &str, value: sea_query::Value) -> Result<(), lifeguard::ActiveModelError> { + match col_name { + #(#active_model_set_col_match_arms)* + _ => Err(lifeguard::ActiveModelError::Other(format!("Column string not found on record: {}", col_name))) + } + } + } + }; + // Generate the expanded code let expanded = quote! { // Record struct (mutable change-set) #[derive(Debug, Clone)] pub struct #record_name { #(#record_fields)* + #session_link_struct_field + /// F-style `UPDATE SET col = ` assignments (see `set_*_expr` methods). Cleared on `reset` / `from_model`. + pub __update_exprs: std::collections::HashMap<<#entity_name as lifeguard::LifeModelTrait>::Column, sea_query::SimpleExpr>, pub __graph: lifeguard::active_model::graph::GraphContainer, } @@ -756,6 +1086,8 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { #( #ignored_field_names: #ignored_field_defaults, )* + #session_new_init + __update_exprs: std::collections::HashMap::new(), __graph: lifeguard::active_model::graph::GraphContainer::default(), } } @@ -765,6 +1097,8 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { pub fn from_model(model: &#model_name) -> Self { Self { #(#from_model_fields)* + #session_new_init + __update_exprs: std::collections::HashMap::new(), __graph: lifeguard::active_model::graph::GraphContainer::default(), } } @@ -795,7 +1129,12 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { !self.dirty_fields().is_empty() } + #identity_map_key_method + + #session_helpers + #(#setter_methods)* + #(#update_expr_setters)* } impl Default for #record_name { @@ -815,17 +1154,9 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { } } - fn set(&mut self, column: <#entity_name as lifeguard::LifeModelTrait>::Column, value: sea_query::Value) -> Result<(), lifeguard::ActiveModelError> { - match column { - #(#active_model_set_match_arms)* - } - } + #active_model_set_impl - fn take(&mut self, column: <#entity_name as lifeguard::LifeModelTrait>::Column) -> Option { - match column { - #(#active_model_take_match_arms)* - } - } + #active_model_take_impl fn get_col(&self, col_name: &str) -> Option { match col_name { @@ -834,14 +1165,10 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { } } - fn set_col(&mut self, col_name: &str, value: sea_query::Value) -> Result<(), lifeguard::ActiveModelError> { - match col_name { - #(#active_model_set_col_match_arms)* - _ => Err(lifeguard::ActiveModelError::Other(format!("Column string not found on record: {}", col_name))) - } - } + #active_model_set_col_impl fn reset(&mut self) { + self.__update_exprs.clear(); #(#active_model_reset_fields)* } @@ -865,6 +1192,13 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { lifeguard::active_model::validate_op::ValidateOp::Insert, )?; + if !record_for_hooks.__update_exprs.is_empty() { + return Err(lifeguard::ActiveModelError::Other( + "`set_*_expr` / `__update_exprs` apply only to `update()`; clear them with `reset()` or use `update()`" + .to_string(), + )); + } + // Build INSERT statement let mut query = Query::insert(); let entity = #entity_name::default(); @@ -1149,6 +1483,10 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { // Call before_delete hook let mut record_for_hooks = self.clone(); record_for_hooks.before_delete()?; + lifeguard::active_model::validation::run_validators( + &record_for_hooks, + lifeguard::active_model::validate_op::ValidateOp::Delete, + )?; // Build DELETE or UPDATE (soft-delete) statement // If soft_delete is enabled, this issues an UPDATE instead of DELETE @@ -1211,6 +1549,8 @@ pub fn derive_life_record(input: TokenStream) -> TokenStream { // Implement ActiveModelBehavior with optionally customized hooks impl lifeguard::ActiveModelBehavior for #record_name { + #validate_fields_impl + #validation_strategy_impl fn before_insert(&mut self) -> Result<(), lifeguard::ActiveModelError> { #before_insert_impl } diff --git a/lifeguard-derive/src/macros/migration_name_derive.rs b/lifeguard-derive/src/macros/migration_name_derive.rs new file mode 100644 index 00000000..67f54219 --- /dev/null +++ b/lifeguard-derive/src/macros/migration_name_derive.rs @@ -0,0 +1,65 @@ +//! `#[derive(DeriveMigrationName)]` for unit structs that implement [`lifeguard::migration::Migration`]. + +use proc_macro::TokenStream; +use quote::quote; +use syn::{parse_macro_input, Data, DeriveInput, Fields, Generics}; + +pub fn derive_migration_name(input: TokenStream) -> TokenStream { + let input = parse_macro_input!(input as DeriveInput); + if let Err(e) = validate_no_generics(&input.generics) { + return e.to_compile_error().into(); + } + let ident = &input.ident; + let struct_err = match &input.data { + Data::Struct(s) => match &s.fields { + Fields::Unit => None, + Fields::Named(f) => Some(syn::Error::new_spanned( + f, + "DeriveMigrationName requires a unit struct (e.g. `struct MyMigration;`)", + )), + Fields::Unnamed(f) => Some(syn::Error::new_spanned( + f, + "DeriveMigrationName requires a unit struct (e.g. `struct MyMigration;`)", + )), + }, + Data::Enum(e) => Some(syn::Error::new_spanned( + e.enum_token, + "DeriveMigrationName only supports unit structs", + )), + Data::Union(u) => Some(syn::Error::new_spanned( + u.union_token, + "DeriveMigrationName only supports unit structs", + )), + }; + if let Some(e) = struct_err { + return e.to_compile_error().into(); + } + + let snake = crate::utils::snake_case(&ident.to_string()); + let lit = syn::LitStr::new(&snake, ident.span()); + + let expanded = quote! { + impl #ident { + /// Stable migration identifier (snake_case of the struct name). + pub const MIGRATION_NAME: &'static str = #lit; + } + + impl ::lifeguard::migration::MigrationName for #ident { + fn migration_name(&self) -> &'static str { + Self::MIGRATION_NAME + } + } + }; + TokenStream::from(expanded) +} + +fn validate_no_generics(generics: &Generics) -> Result<(), syn::Error> { + if generics.params.is_empty() { + Ok(()) + } else { + Err(syn::Error::new_spanned( + generics, + "DeriveMigrationName does not support generic parameters", + )) + } +} diff --git a/lifeguard-derive/src/macros/mod.rs b/lifeguard-derive/src/macros/mod.rs index 537bc4a5..7de05436 100644 --- a/lifeguard-derive/src/macros/mod.rs +++ b/lifeguard-derive/src/macros/mod.rs @@ -1,5 +1,6 @@ //! Macro implementations +pub mod migration_name_derive; pub mod entity; pub mod from_row; pub mod life_model; @@ -7,8 +8,10 @@ pub mod life_record; pub mod linked; pub mod partial_model; pub mod relation; +pub mod scope_attr; pub mod try_into_model; +pub use migration_name_derive::derive_migration_name; pub use entity::derive_entity; pub use from_row::derive_from_row; pub use life_model::derive_life_model; diff --git a/lifeguard-derive/src/macros/scope_attr.rs b/lifeguard-derive/src/macros/scope_attr.rs new file mode 100644 index 00000000..c5abb1af --- /dev/null +++ b/lifeguard-derive/src/macros/scope_attr.rs @@ -0,0 +1,43 @@ +//! `#[scope]` attribute macro — PRD Phase C derive sugar (named scopes on `Entity`). + +use proc_macro::TokenStream; +use quote::quote; +use syn::spanned::Spanned; +use syn::{parse_macro_input, Error, ItemFn}; + +/// Renames `fn foo` → `fn scope_foo` (or keeps `fn scope_foo` as-is) and defaults to `pub` so +/// scopes read as `Entity::scope_active()` in rustdoc (PRD SC-1). +pub fn scope_attr(_attr: TokenStream, item: TokenStream) -> TokenStream { + let mut func = parse_macro_input!(item as ItemFn); + + if func.sig.asyncness.is_some() { + return Error::new( + func.sig.asyncness.span(), + "`#[scope]` does not support async functions", + ) + .to_compile_error() + .into(); + } + if func.sig.receiver().is_some() { + return Error::new( + func.sig.fn_token.span(), + "`#[scope]` must be on an associated function without `self` (e.g. `fn active() -> impl IntoCondition`)", + ) + .to_compile_error() + .into(); + } + + let orig = &func.sig.ident; + let new_ident = if orig.to_string().starts_with("scope_") { + orig.clone() + } else { + syn::Ident::new(&format!("scope_{orig}"), orig.span()) + }; + func.sig.ident = new_ident; + + if matches!(func.vis, syn::Visibility::Inherited) { + func.vis = syn::parse_quote!(pub); + } + + quote!(#func).into() +} diff --git a/lifeguard-derive/tests/test_derive_migration_name.rs b/lifeguard-derive/tests/test_derive_migration_name.rs new file mode 100644 index 00000000..5592ca35 --- /dev/null +++ b/lifeguard-derive/tests/test_derive_migration_name.rs @@ -0,0 +1,24 @@ +//! Tests for `DeriveMigrationName`. + +use lifeguard::migration::MigrationName; +use lifeguard_derive::DeriveMigrationName; + +#[derive(DeriveMigrationName)] +pub struct CreateUsersTable; + +#[derive(DeriveMigrationName)] +pub struct AddSortOrderColumn; + +#[test] +fn migration_name_snake_case_from_struct() { + assert_eq!(CreateUsersTable::MIGRATION_NAME, "create_users_table"); + assert_eq!( + MigrationName::migration_name(&CreateUsersTable), + "create_users_table" + ); + assert_eq!(AddSortOrderColumn::MIGRATION_NAME, "add_sort_order_column"); + assert_eq!( + MigrationName::migration_name(&AddSortOrderColumn), + "add_sort_order_column" + ); +} diff --git a/lifeguard-derive/tests/test_minimal.rs b/lifeguard-derive/tests/test_minimal.rs index 3493a65e..502d5564 100644 --- a/lifeguard-derive/tests/test_minimal.rs +++ b/lifeguard-derive/tests/test_minimal.rs @@ -53,6 +53,181 @@ pub mod column_name_tests { } } +/// `#[scope]` attribute on `impl Entity` (PRD Phase C derive sugar). +pub mod scope_attr_tests { + use lifeguard::ColumnTrait; + use lifeguard::LifeModelTrait; + use lifeguard::scope; + use lifeguard_derive::{LifeModel, LifeRecord}; + use sea_query::IntoCondition; + + #[derive(LifeModel, LifeRecord)] + #[table_name = "scope_macro_users"] + pub struct ScopeMacroUser { + #[primary_key] + pub id: i32, + pub active: bool, + } + + impl Entity { + #[scope] + fn active() -> impl IntoCondition { + Column::Active.eq(true) + } + } + + #[test] + fn scope_attr_renames_to_scope_active_and_chains() { + let _q = Entity::find().scope(Entity::scope_active()); + } +} + +/// `#[validate(custom = ...)]` on `LifeRecord` fields (PRD V-5). +pub mod validate_attr_tests { + use lifeguard::{run_validators, ValidateOp}; + use lifeguard_derive::{LifeModel, LifeRecord}; + + pub fn email_non_empty(v: &sea_query::Value) -> Result<(), String> { + match v { + sea_query::Value::String(Some(s)) if !s.is_empty() => Ok(()), + sea_query::Value::String(Some(_)) => Err("email must be non-empty".to_string()), + _ => Err("email must be a non-empty string".to_string()), + } + } + + pub fn name_non_empty(v: &sea_query::Value) -> Result<(), String> { + match v { + sea_query::Value::String(Some(s)) if !s.is_empty() => Ok(()), + _ => Err("name must be non-empty".to_string()), + } + } + + #[derive(LifeModel, LifeRecord)] + #[table_name = "validated_users"] + pub struct ValidatedUser { + #[primary_key] + pub id: i32, + #[validate(custom = email_non_empty)] + pub email: String, + } + + #[test] + fn validate_custom_rejects_empty_email() { + let mut r = ValidatedUserRecord::new(); + r.set_email(String::new()); + let err = run_validators(&r, ValidateOp::Insert).expect_err("validation should fail"); + match err { + lifeguard::ActiveModelError::Validation(v) => { + assert_eq!(v.len(), 1); + assert_eq!(v[0].field.as_deref(), Some("email")); + assert!(v[0].message.contains("non-empty")); + } + e => panic!("expected Validation error, got {:?}", e), + } + } + + #[test] + fn validate_custom_accepts_non_empty_email() { + let mut r = ValidatedUserRecord::new(); + r.set_email("a@b.c".to_string()); + run_validators(&r, ValidateOp::Insert).expect("validation should pass"); + } + + #[test] + fn validate_custom_skips_when_field_unset() { + let r = ValidatedUserRecord::new(); + run_validators(&r, ValidateOp::Insert).expect("no field set => custom validator not run"); + } +} + +/// Two `#[validate(custom)]` fields — FailFast (default): first failing field only. +mod validate_multi_fail_fast { + use super::validate_attr_tests::email_non_empty; + use lifeguard::{run_validators, ValidateOp}; + use lifeguard_derive::{LifeModel, LifeRecord}; + + #[derive(LifeModel, LifeRecord)] + #[table_name = "validated_users_multi_ff"] + pub struct ValidatedUserMulti { + #[primary_key] + pub id: i32, + #[validate(custom = email_non_empty)] + pub email: String, + #[validate(custom = super::validate_attr_tests::name_non_empty)] + pub name: String, + } + + #[test] + fn validate_fail_fast_stops_after_first_invalid_field() { + let mut r = ValidatedUserMultiRecord::new(); + r.set_id(1); + r.set_email(String::new()); + r.set_name(String::new()); + let err = run_validators(&r, ValidateOp::Insert).expect_err("first field fails"); + match err { + lifeguard::ActiveModelError::Validation(v) => { + assert_eq!(v.len(), 1, "FailFast should stop after first failing field validator"); + assert_eq!(v[0].field.as_deref(), Some("email")); + } + e => panic!("expected Validation error, got {:?}", e), + } + } +} + +/// Two `#[validate(custom)]` fields — `ValidationStrategy::Aggregate`: all field errors. +mod validate_multi_aggregate { + use super::validate_attr_tests::{email_non_empty, name_non_empty}; + use lifeguard::{run_validators, ValidateOp}; + use lifeguard_derive::{LifeModel, LifeRecord}; + + #[derive(LifeModel, LifeRecord)] + #[table_name = "validated_users_multi_agg"] + #[validation_strategy = "aggregate"] + pub struct ValidatedUserMultiAgg { + #[primary_key] + pub id: i32, + #[validate(custom = email_non_empty)] + pub email: String, + #[validate(custom = name_non_empty)] + pub name: String, + } + + #[test] + fn validate_aggregate_collects_all_field_errors() { + let mut r = ValidatedUserMultiAggRecord::new(); + r.set_id(1); + r.set_email(String::new()); + r.set_name(String::new()); + let err = run_validators(&r, ValidateOp::Insert).expect_err("both fields invalid"); + match err { + lifeguard::ActiveModelError::Validation(v) => { + assert_eq!(v.len(), 2, "Aggregate should run every field validator"); + let fields: Vec> = v.iter().map(|e| e.field.as_deref()).collect(); + assert!(fields.contains(&Some("email"))); + assert!(fields.contains(&Some("name"))); + } + e => panic!("expected Validation error, got {:?}", e), + } + } +} + +#[test] +fn user_record_identity_map_key_matches_pk_fingerprint() { + use lifeguard::session::fingerprint_pk_values; + use sea_query::Value; + + let m = UserModel { + id: 42, + name: "n".into(), + email: "e@x".into(), + }; + let r = UserRecord::from_model(&m); + assert_eq!( + r.identity_map_key(), + Some(fingerprint_pk_values(&[Value::Int(Some(42))])) + ); +} + // Entity with numeric fields for testing all numeric types // Using a module to avoid name conflicts // NOTE: may_postgres doesn't support u8, u16, u64 in FromSql, so we manually implement ModelTrait @@ -2712,6 +2887,202 @@ mod active_model_trait_tests { ); } + #[test] + fn test_set_invalid_type_preserves_update_expr() { + use lifeguard::ColumnTrait; + use sea_query::Expr; + + let mut record = UserRecord::new(); + record.set_id(1); + record.set_name("a".to_string()); + record.set_email("e@example.com".to_string()); + // Non-PK column: F-style expr is only generated for non-primary-key fields. + record.set_name_expr(Expr::cust("1")); + assert!(record + .__update_exprs + .contains_key(&::Column::Name)); + + let err = record + .set( + ::Column::Name, + sea_query::Value::Int(Some(42)), + ) + .expect_err("wrong type for name column"); + assert!(matches!( + err, + lifeguard::ActiveModelError::InvalidValueType { .. } + )); + + assert!( + record + .__update_exprs + .contains_key(&::Column::Name), + "failed set() must not clear scheduled F-style expression" + ); + } + + #[test] + fn test_set_col_invalid_type_preserves_update_expr() { + use lifeguard::ColumnTrait; + use sea_query::Expr; + + let mut record = UserRecord::new(); + record.set_id(1); + record.set_name("a".to_string()); + record.set_email("e@example.com".to_string()); + record.set_name_expr(Expr::cust("1")); + assert!(record + .__update_exprs + .contains_key(&::Column::Name)); + + let err = record + .set_col("name", sea_query::Value::Int(Some(42))) + .expect_err("wrong type for name column"); + assert!(matches!( + err, + lifeguard::ActiveModelError::InvalidValueType { .. } + )); + + assert!( + record + .__update_exprs + .contains_key(&::Column::Name), + "failed set_col() must not clear scheduled F-style expression" + ); + } + + // Session: flush closure runs once per dirty row — `take()` on an already-unset column must not + // enqueue dirty (parity with `set()` only notifying on success). + #[test] + fn test_take_noop_when_unset_skips_session_dirty() { + use lifeguard::executor::{LifeError, LifeExecutor}; + use lifeguard::session::Session; + use may_postgres::Row; + + struct NopExecutor; + + impl LifeExecutor for NopExecutor { + fn execute( + &self, + _query: &str, + _params: &[&dyn may_postgres::types::ToSql], + ) -> Result { + Ok(0) + } + + fn query_one( + &self, + _query: &str, + _params: &[&dyn may_postgres::types::ToSql], + ) -> Result { + Err(LifeError::QueryError("nop".into())) + } + + fn query_all( + &self, + _query: &str, + _params: &[&dyn may_postgres::types::ToSql], + ) -> Result, LifeError> { + Ok(vec![]) + } + } + + let session = Session::::new(); + session.register_loaded(UserModel { + id: 1, + name: "n".to_string(), + email: "e@e.com".to_string(), + }); + session.clear_dirty(); + + let mut record = UserRecord::new(); + record.set_id(1); + record.attach_session(&session); + + let _ = record.take(::Column::Name); + + let mut flush_count = 0; + let ex = NopExecutor; + session + .flush_dirty(&ex as &dyn LifeExecutor, |_, _| { + flush_count += 1; + Ok(()) + }) + .expect("flush"); + + assert_eq!( + flush_count, 0, + "take on unset column should not enqueue session dirty" + ); + } + + #[test] + fn test_take_with_value_or_expr_notifies_session_dirty() { + use lifeguard::executor::{LifeError, LifeExecutor}; + use lifeguard::session::Session; + use lifeguard::ColumnTrait; + use may_postgres::Row; + use sea_query::Expr; + + struct NopExecutor; + + impl LifeExecutor for NopExecutor { + fn execute( + &self, + _query: &str, + _params: &[&dyn may_postgres::types::ToSql], + ) -> Result { + Ok(0) + } + + fn query_one( + &self, + _query: &str, + _params: &[&dyn may_postgres::types::ToSql], + ) -> Result { + Err(LifeError::QueryError("nop".into())) + } + + fn query_all( + &self, + _query: &str, + _params: &[&dyn may_postgres::types::ToSql], + ) -> Result, LifeError> { + Ok(vec![]) + } + } + + let session = Session::::new(); + session.register_loaded(UserModel { + id: 1, + name: "n".to_string(), + email: "e@e.com".to_string(), + }); + session.clear_dirty(); + + let mut record = UserRecord::new(); + record.set_id(1); + record.set_email("e@e.com".to_string()); + record.set_name_expr(Expr::cust("1")); + record.attach_session(&session); + + let _ = record.take(::Column::Name); + + let mut flush_count = 0; + let ex = NopExecutor; + session + .flush_dirty(&ex as &dyn LifeExecutor, |_, _| { + flush_count += 1; + Ok(()) + }) + .expect("flush"); + + assert_eq!( + flush_count, 1, + "take that clears a literal or F-style expr should enqueue session dirty" + ); + } + // ============================================================================ // EDGE CASES FOR get() // ============================================================================ diff --git a/lifeguard-migrate/README.md b/lifeguard-migrate/README.md index 1e5bd211..3b8485de 100644 --- a/lifeguard-migrate/README.md +++ b/lifeguard-migrate/README.md @@ -15,6 +15,7 @@ Migration CLI tool for Lifeguard ORM - manage database schema changes with versi - ✅ **Status Tracking** - View applied vs pending migrations - ✅ **Entity-Driven Generation** - Generate SQL migrations from Lifeguard entity definitions - ✅ **Schema inference (`infer-schema`)** - Introspect PostgreSQL and print `LifeModel` / `LifeRecord` Rust sketches (stdout); see below +- ✅ **DB vs generated migration baseline (`compare-schema`)** - Reconcile live `information_schema` tables and (for tables in both baselines) **column names** vs merged `*_generated_from_entities.sql` (`CREATE` + `ADD COLUMN`); see below - ✅ **CI/CD Integration** - Designed for automated deployment pipelines - ✅ **Dry Run Mode** - Preview migrations without executing them @@ -163,7 +164,7 @@ This scans entity definitions and generates SQL migration files for tables, colu ### `infer-schema` -Introspect a live **PostgreSQL** database (`information_schema`) and print conservative Rust entity sketches to **stdout**. Output is **review-first**: paste into your crate, adjust types, and fix composite PKs (`TODO` comments) as needed. +Introspect a live **PostgreSQL** database (`information_schema`) and print conservative Rust entity sketches to **stdout**. Output is **review-first**: paste into your crate and adjust types; **composite** primary keys emit `#[primary_key]` on each PK column (same as single-column PKs). **Requirements:** `--database-url` or `DATABASE_URL` / `LIFEGUARD_DATABASE_URL` (same as other DB commands). @@ -194,6 +195,32 @@ lifeguard-migrate infer-schema \ cargo test -p lifeguard-migrate schema_infer ``` +**Optional live DB tests** (skip when no URL): `tests/infer_schema_postgres_smoke.rs` (introspect `public`); `tests/infer_schema_table_filter_si3.rs` (SI-3 — table filter excludes other tables). Set `TEST_DATABASE_URL`, `DATABASE_URL`, or `LIFEGUARD_DATABASE_URL`. See **`DEVELOPMENT.md`** (`lifeguard-migrate` section). + +**CLI subprocess e2e** (optional): `tests/infer_schema_cli_subprocess.rs` runs the **`lifeguard-migrate infer-schema`** binary via `CARGO_BIN_EXE_lifeguard-migrate` and asserts the stdout banner (same env URL as above; skips when unset). + +### `compare-schema` + +Compare **live PostgreSQL** to merged **`*_generated_from_entities.sql`** under a directory: + +1. **Table names:** `information_schema` base tables (`table_type = 'BASE TABLE'`) vs `-- Table: name` sections (after chronological merge). +2. **Column names:** for each table present in **both** baselines, `information_schema.columns` vs columns parsed from the merged `CREATE TABLE` body plus `ADD COLUMN` / `ADD COLUMN IF NOT EXISTS` lines (`column_map_from_merged_baseline`). + +Column reconciliation is **name-level** (presence of columns), not equality of SQL types or full `CREATE` definitions. Use **`--schema`** for a service or scratch namespace when you must not compare against every table in `public` (shared dev/CI databases often contain many unrelated tables). + +**Exit code:** `0` when there is no drift; non-zero when extra/missing tables or extra/missing column names on shared tables (CI-friendly). + +```bash +lifeguard-migrate compare-schema \ + --database-url "$DATABASE_URL" \ + --schema public \ + --generated-dir migrations/generated/inventory +``` + +**Library:** `lifeguard_migrate::schema_migration_compare::{compare_generated_dir_to_live_db, MigrationDbCompareReport}`. + +**Optional live DB tests:** `tests/migration_db_compare_smoke.rs` (library + CLI; skips without URL / binary env). + ### `info` Show detailed migration information: diff --git a/lifeguard-migrate/src/generated_migration_diff.rs b/lifeguard-migrate/src/generated_migration_diff.rs index 39fca8f5..ef24af27 100644 --- a/lifeguard-migrate/src/generated_migration_diff.rs +++ b/lifeguard-migrate/src/generated_migration_diff.rs @@ -65,6 +65,25 @@ pub struct TableBaselineParts { pub delta_section_fragments: Vec, } +/// Column names → definition tail (everything after the column name) from a merged table baseline: +/// `CREATE TABLE` column lines plus `ADD COLUMN` / `ADD COLUMN IF NOT EXISTS` from merged deltas. +/// +/// Used by [`crate::schema_migration_compare`] for column-level reconciliation with `information_schema`. +#[must_use] +pub fn column_map_from_merged_baseline(parts: &TableBaselineParts) -> BTreeMap { + let combined = combined_old_section(parts); + let mut cols = BTreeMap::new(); + if let Some(sec) = parts.last_create_section.as_ref() { + if let Some((_, body, _)) = extract_create_and_tail(sec) { + cols = parse_column_defs_from_create_body(&body); + } + } + for (name, def) in parse_add_columns_from_alter_blob(&combined) { + cols.insert(name, def); + } + cols +} + /// Replay every `*_generated_from_entities.sql` under `dir` (oldest first) and merge table sections. #[must_use] pub fn accumulate_table_baselines_from_dir(dir: &Path) -> BTreeMap { @@ -660,4 +679,24 @@ CREATE TABLE IF NOT EXISTS widgets ( &[("widgets".into(), new_sql.into())] )); } + + #[test] + fn column_map_from_merged_baseline_merges_create_and_alter() { + let mut parts = TableBaselineParts::default(); + parts.last_create_section = Some( + r"CREATE TABLE IF NOT EXISTS widgets ( + id INTEGER PRIMARY KEY, + name VARCHAR(255) NOT NULL +);" + .to_string(), + ); + parts.delta_section_fragments.push( + "ALTER TABLE widgets ADD COLUMN IF NOT EXISTS sku VARCHAR(50) NOT NULL DEFAULT '';\n" + .to_string(), + ); + let m = column_map_from_merged_baseline(&parts); + assert!(m.contains_key("id")); + assert!(m.contains_key("name")); + assert!(m.contains_key("sku")); + } } diff --git a/lifeguard-migrate/src/lib.rs b/lifeguard-migrate/src/lib.rs index d8bd4485..733edf99 100644 --- a/lifeguard-migrate/src/lib.rs +++ b/lifeguard-migrate/src/lib.rs @@ -13,6 +13,7 @@ pub mod entity_loader; pub mod generated_migration_diff; pub mod registry_loader; pub mod schema_infer; +pub mod schema_migration_compare; pub mod sql_dependency_order; pub mod sql_generator; diff --git a/lifeguard-migrate/src/main.rs b/lifeguard-migrate/src/main.rs index 5d9c4205..1f7069dd 100644 --- a/lifeguard-migrate/src/main.rs +++ b/lifeguard-migrate/src/main.rs @@ -102,6 +102,21 @@ enum Commands { tables: Vec, }, + /// Compare live database to merged **`*_generated_from_entities.sql`** baselines (`-- Table:` sections). + /// + /// Reconciles **table names** (base tables in `information_schema`) and, for tables present in both + /// baselines, **column names** from `information_schema.columns` vs parsed `CREATE TABLE` + `ADD COLUMN` + /// lines. Does not compare SQL type text or constraints in depth (name-level column diff only). + CompareSchema { + /// PostgreSQL schema (namespace) for `information_schema.tables` + #[arg(long, default_value = "public")] + schema: String, + + /// Directory containing `*_generated_from_entities.sql` (e.g. `migrations/generated/inventory`) + #[arg(long)] + generated_dir: PathBuf, + }, + /// Show detailed migration information Info { /// Show information for a specific migration version @@ -160,6 +175,15 @@ fn main() { .expect("database URL validated when command requires DB"); handle_infer_schema(db_url, schema, tables) } + Commands::CompareSchema { + schema, + generated_dir, + } => { + let db_url = database_url + .as_ref() + .expect("database URL validated when command requires DB"); + handle_compare_schema(db_url, schema, &generated_dir) + } _ => { // All other commands need database connection let db_url = database_url.expect("Database URL should be validated above"); @@ -203,6 +227,37 @@ fn main() { } } +fn handle_compare_schema( + database_url: &str, + schema: String, + generated_dir: &std::path::Path, +) -> Result<(), MigrationError> { + use lifeguard_migrate::schema_migration_compare::compare_generated_dir_to_live_db; + + if !generated_dir.is_dir() { + return Err(MigrationError::InvalidFormat(format!( + "compare-schema: not a directory: {path}", + path = generated_dir.display() + ))); + } + + let client = connect(database_url).map_err(|e| { + MigrationError::InvalidFormat(format!("compare-schema: database connect failed: {e}")) + })?; + let executor = MayPostgresExecutor::new(client); + let report = compare_generated_dir_to_live_db(&executor, &schema, generated_dir).map_err( + |e| MigrationError::InvalidFormat(format!("compare-schema: {e}")), + )?; + print!("{report}"); + if report.has_drift() { + return Err(MigrationError::InvalidFormat( + "compare-schema: live database does not match merged generated migration baseline (tables and/or column names — see above)." + .to_string(), + )); + } + Ok(()) +} + fn handle_infer_schema( database_url: &str, schema: String, diff --git a/lifeguard-migrate/src/schema_infer.rs b/lifeguard-migrate/src/schema_infer.rs index bd48cfc3..4594ec32 100644 --- a/lifeguard-migrate/src/schema_infer.rs +++ b/lifeguard-migrate/src/schema_infer.rs @@ -147,15 +147,6 @@ pub(crate) fn emit_inferred_rust( writeln!(out, "#[table_name = \"{}\"]", table).unwrap(); writeln!(out, "pub struct {} {{", struct_name).unwrap(); - if pk_cols.len() > 1 { - writeln!( - out, - " // TODO: composite PRIMARY KEY ({}) — confirm Lifeguard PK tuple layout.", - pk_cols.join(", ") - ) - .unwrap(); - } - for col in &cols { let rust_type = map_pg_to_rust(&col.data_type, &col.udt_name, col.is_nullable); if let Some((ty, attrs)) = rust_type { @@ -163,7 +154,7 @@ pub(crate) fn emit_inferred_rust( for line in attrs { writeln!(out, " {}", line).unwrap(); } - if pk_set.len() == 1 && pk_set.contains(&col.column_name) { + if pk_set.contains(&col.column_name) { writeln!(out, " #[primary_key]").unwrap(); } writeln!(out, " pub {}: {},", field, ty).unwrap(); @@ -399,7 +390,7 @@ mod tests { } #[test] - fn golden_emit_composite_primary_key_todo() { + fn golden_emit_composite_primary_key() { let columns = vec![ ColumnMeta { table_name: "pair_keys".into(), diff --git a/lifeguard-migrate/src/schema_migration_compare.rs b/lifeguard-migrate/src/schema_migration_compare.rs new file mode 100644 index 00000000..0ddf79f5 --- /dev/null +++ b/lifeguard-migrate/src/schema_migration_compare.rs @@ -0,0 +1,270 @@ +//! Compare **live PostgreSQL** to merged **`*_generated_from_entities.sql`** baselines. +//! +//! - **Table names:** [`accumulate_table_baselines_from_dir`] (`-- Table:` sections) vs +//! `information_schema.tables`. +//! - **Column names:** for tables present in **both** baselines, compare `information_schema.columns` +//! to column lines from merged `CREATE TABLE` + `ADD COLUMN` fragments (see +//! [`crate::generated_migration_diff::column_map_from_merged_baseline`]). +//! +//! Does **not** compare SQL type text literally (PG `data_type` vs migration `INTEGER` spelling); +//! name-level reconciliation is the Phase A column diff scope. + +use lifeguard::LifeExecutor; +use lifeguard::LifeError; +use std::collections::BTreeSet; +use std::fmt; +use std::path::Path; + +use crate::generated_migration_diff::{ + accumulate_table_baselines_from_dir, column_map_from_merged_baseline, +}; + +/// Table names from merged `*_generated_from_entities.sql` in `dir` (from `-- Table:` headers). +#[must_use] +pub fn table_names_from_generated_migrations_dir(dir: &Path) -> BTreeSet { + accumulate_table_baselines_from_dir(dir) + .into_keys() + .collect() +} + +/// `BASE TABLE` names in `information_schema.tables` for `schema` (e.g. `public`). +pub fn fetch_live_base_table_names( + executor: &dyn LifeExecutor, + schema: &str, +) -> Result, LifeError> { + let sql = r" + SELECT table_name::text + FROM information_schema.tables + WHERE table_schema = $1 AND table_type = 'BASE TABLE' + ORDER BY table_name + "; + let rows = executor.query_all(sql, &[&schema])?; + let mut set = BTreeSet::new(); + for row in rows { + let name: String = row + .try_get(0) + .map_err(|e| LifeError::Other(format!("compare-schema table_name: {e}")))?; + set.insert(name); + } + Ok(set) +} + +/// Column names for one table in `information_schema.columns` (ordered for stable diffs). +pub fn fetch_live_table_column_names( + executor: &dyn LifeExecutor, + schema: &str, + table: &str, +) -> Result, LifeError> { + let sql = r" + SELECT column_name::text + FROM information_schema.columns + WHERE table_schema = $1 AND table_name = $2 + ORDER BY ordinal_position + "; + let rows = executor.query_all(sql, &[&schema, &table])?; + let mut set = BTreeSet::new(); + for row in rows { + let name: String = row + .try_get(0) + .map_err(|e| LifeError::Other(format!("compare-schema column_name: {e}")))?; + set.insert(name); + } + Ok(set) +} + +/// Column-level drift for a single table that exists in both the live DB and merged migrations. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TableColumnDrift { + pub table: String, + /// Columns present in `information_schema` but not parsed from the merged baseline. + pub only_in_database: Vec, + /// Columns parsed from the merged baseline but not in `information_schema`. + pub only_in_migrations: Vec, +} + +/// Result of [`compare_generated_dir_to_live_db`]. +#[derive(Debug, Clone)] +pub struct MigrationDbCompareReport { + /// PostgreSQL schema (namespace) used for the live query. + pub schema: String, + /// Directory scanned for `*_generated_from_entities.sql`. + pub generated_dir: std::path::PathBuf, + /// Present in the database but not in any merged `-- Table:` baseline. + pub only_in_database: Vec, + /// Present in merged migration baselines but not as a base table in the database. + pub only_in_migrations: Vec, + /// Tables in both baselines where **column name** sets differ. + pub column_drifts: Vec, +} + +impl MigrationDbCompareReport { + /// `true` when table sets differ or any shared table has column name drift. + #[must_use] + pub fn has_drift(&self) -> bool { + !self.only_in_database.is_empty() + || !self.only_in_migrations.is_empty() + || !self.column_drifts.is_empty() + } +} + +/// Compare merged generated migration baselines to live `information_schema` (tables + column names). +pub fn compare_generated_dir_to_live_db( + executor: &dyn LifeExecutor, + schema: &str, + generated_dir: &Path, +) -> Result { + let acc = accumulate_table_baselines_from_dir(generated_dir); + let on_disk: BTreeSet = acc.keys().cloned().collect(); + let live = fetch_live_base_table_names(executor, schema)?; + let mut only_in_db: Vec = live.difference(&on_disk).cloned().collect(); + let mut only_mig: Vec = on_disk.difference(&live).cloned().collect(); + only_in_db.sort(); + only_mig.sort(); + + let mut column_drifts = Vec::new(); + for table in on_disk.intersection(&live) { + let Some(parts) = acc.get(table.as_str()) else { + continue; + }; + let mig_map = column_map_from_merged_baseline(parts); + let mig_names: BTreeSet = mig_map.keys().cloned().collect(); + let live_names = fetch_live_table_column_names(executor, schema, table)?; + let mut only_col_db: Vec = live_names.difference(&mig_names).cloned().collect(); + let mut only_col_mig: Vec = mig_names.difference(&live_names).cloned().collect(); + only_col_db.sort(); + only_col_mig.sort(); + if !only_col_db.is_empty() || !only_col_mig.is_empty() { + column_drifts.push(TableColumnDrift { + table: table.clone(), + only_in_database: only_col_db, + only_in_migrations: only_col_mig, + }); + } + } + column_drifts.sort_by(|a, b| a.table.cmp(&b.table)); + + Ok(MigrationDbCompareReport { + schema: schema.to_string(), + generated_dir: generated_dir.to_path_buf(), + only_in_database: only_in_db, + only_in_migrations: only_mig, + column_drifts, + }) +} + +impl fmt::Display for MigrationDbCompareReport { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + writeln!( + f, + "Schema/migration reconciliation (PostgreSQL schema `{}`)", + self.schema + )?; + writeln!( + f, + " Generated migrations dir: {}", + self.generated_dir.display() + )?; + writeln!(f)?; + if !self.has_drift() { + writeln!( + f, + " No drift: table names align, and column names match for tables present in both the database and merged migration baselines." + )?; + return Ok(()); + } + if !self.only_in_database.is_empty() { + writeln!( + f, + " Tables only in database (not in merged migration baseline):" + )?; + for t in &self.only_in_database { + writeln!(f, " - {t}")?; + } + writeln!(f)?; + } + if !self.only_in_migrations.is_empty() { + writeln!( + f, + " Tables only in merged migration files (not in live DB):" + )?; + for t in &self.only_in_migrations { + writeln!(f, " - {t}")?; + } + writeln!(f)?; + } + if !self.column_drifts.is_empty() { + writeln!( + f, + " Column name differences (tables in both live DB and merged migrations):" + )?; + for d in &self.column_drifts { + writeln!(f, " Table `{}`:", d.table)?; + if !d.only_in_database.is_empty() { + writeln!( + f, + " Columns only in database (not in merged baseline): {}", + d.only_in_database.join(", ") + )?; + } + if !d.only_in_migrations.is_empty() { + writeln!( + f, + " Columns only in merged baseline (not in database): {}", + d.only_in_migrations.join(", ") + )?; + } + } + } + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn report_display_ok_when_no_drift() { + let r = MigrationDbCompareReport { + schema: "public".into(), + generated_dir: Path::new("/tmp/x").to_path_buf(), + only_in_database: vec![], + only_in_migrations: vec![], + column_drifts: vec![], + }; + assert!(!r.has_drift()); + let s = r.to_string(); + assert!(s.contains("No drift")); + } + + #[test] + fn has_drift_when_only_in_db() { + let r = MigrationDbCompareReport { + schema: "public".into(), + generated_dir: Path::new("/x").to_path_buf(), + only_in_database: vec!["orphan".into()], + only_in_migrations: vec![], + column_drifts: vec![], + }; + assert!(r.has_drift()); + } + + #[test] + fn has_drift_when_column_drift_only() { + let r = MigrationDbCompareReport { + schema: "public".into(), + generated_dir: Path::new("/x").to_path_buf(), + only_in_database: vec![], + only_in_migrations: vec![], + column_drifts: vec![TableColumnDrift { + table: "t".into(), + only_in_database: vec!["extra".into()], + only_in_migrations: vec![], + }], + }; + assert!(r.has_drift()); + let s = r.to_string(); + assert!(s.contains("Column name differences")); + assert!(s.contains("extra")); + } +} diff --git a/lifeguard-migrate/tests/golden/infer_composite_pk.expected.rs b/lifeguard-migrate/tests/golden/infer_composite_pk.expected.rs index a361ef83..46328262 100644 --- a/lifeguard-migrate/tests/golden/infer_composite_pk.expected.rs +++ b/lifeguard-migrate/tests/golden/infer_composite_pk.expected.rs @@ -7,7 +7,8 @@ use lifeguard_derive::{LifeModel, LifeRecord}; #[derive(Debug, Clone, PartialEq, LifeModel, LifeRecord)] #[table_name = "pair_keys"] pub struct PairKeys { - // TODO: composite PRIMARY KEY (site_id, item_id) — confirm Lifeguard PK tuple layout. + #[primary_key] pub site_id: i32, + #[primary_key] pub item_id: i32, } diff --git a/lifeguard-migrate/tests/infer_schema_cli_subprocess.rs b/lifeguard-migrate/tests/infer_schema_cli_subprocess.rs new file mode 100644 index 00000000..cbd8f440 --- /dev/null +++ b/lifeguard-migrate/tests/infer_schema_cli_subprocess.rs @@ -0,0 +1,46 @@ +//! End-to-end subprocess test: `lifeguard-migrate infer-schema` (PRD Phase A — CLI e2e). +//! +//! Requires `CARGO_BIN_EXE_lifeguard-migrate` (set by Cargo when building integration tests) and a +//! Postgres URL; skips when either is missing so `cargo test -p lifeguard-migrate` stays usable offline. + +use std::process::Command; + +fn postgres_url() -> Option { + std::env::var("TEST_DATABASE_URL") + .or_else(|_| std::env::var("DATABASE_URL")) + .or_else(|_| std::env::var("LIFEGUARD_DATABASE_URL")) + .ok() + .filter(|s| !s.trim().is_empty()) +} + +#[test] +fn infer_schema_cli_prints_generated_banner() { + let Some(url) = postgres_url() else { + eprintln!( + "infer_schema_cli_prints_generated_banner: skipped (set TEST_DATABASE_URL, DATABASE_URL, or LIFEGUARD_DATABASE_URL)" + ); + return; + }; + + let Some(bin) = std::env::var_os("CARGO_BIN_EXE_lifeguard-migrate") else { + eprintln!("infer_schema_cli_prints_generated_banner: skipped (CARGO_BIN_EXE_lifeguard-migrate not set — run cargo test -p lifeguard-migrate)"); + return; + }; + + let out = Command::new(&bin) + .args(["infer-schema", "--database-url", &url]) + .output() + .unwrap_or_else(|e| panic!("spawn lifeguard-migrate infer-schema: {e}")); + + assert!( + out.status.success(), + "infer-schema failed: stderr={stderr}", + stderr = String::from_utf8_lossy(&out.stderr) + ); + let stdout = String::from_utf8_lossy(&out.stdout); + assert!( + stdout.contains("Generated by lifeguard-migrate infer-schema"), + "expected banner in stdout, got {} bytes", + stdout.len() + ); +} diff --git a/lifeguard-migrate/tests/infer_schema_postgres_smoke.rs b/lifeguard-migrate/tests/infer_schema_postgres_smoke.rs new file mode 100644 index 00000000..e06eb0eb --- /dev/null +++ b/lifeguard-migrate/tests/infer_schema_postgres_smoke.rs @@ -0,0 +1,43 @@ +//! Optional live-PostgreSQL smoke for `infer_schema_rust` (PRD §5.7). +//! +//! When `TEST_DATABASE_URL`, `DATABASE_URL`, or `LIFEGUARD_DATABASE_URL` is set, connects and +//! runs introspection on `public`. Without a URL the test returns immediately (passes) so default +//! `cargo test -p lifeguard-migrate` stays offline. CI can set the same env vars used for other DB tests. + +use lifeguard::{connect, MayPostgresExecutor}; +use lifeguard_migrate::schema_infer::{infer_schema_rust, InferOptions}; + +fn postgres_url() -> Option { + std::env::var("TEST_DATABASE_URL") + .or_else(|_| std::env::var("DATABASE_URL")) + .or_else(|_| std::env::var("LIFEGUARD_DATABASE_URL")) + .ok() + .filter(|s| !s.trim().is_empty()) +} + +#[test] +fn infer_schema_rust_live_public_smoke() { + let Some(url) = postgres_url() else { + eprintln!( + "infer_schema_rust_live_public_smoke: skipped (set TEST_DATABASE_URL, DATABASE_URL, or LIFEGUARD_DATABASE_URL)" + ); + return; + }; + + let client = connect(&url).expect("connect to Postgres for infer-schema smoke"); + let executor = MayPostgresExecutor::new(client); + let out = infer_schema_rust( + &executor, + &InferOptions { + schema: "public".to_string(), + tables: vec![], + }, + ) + .expect("infer_schema_rust"); + + assert!( + out.contains("Generated by lifeguard-migrate infer-schema"), + "expected banner in output, got {} bytes", + out.len() + ); +} diff --git a/lifeguard-migrate/tests/infer_schema_table_filter_si3.rs b/lifeguard-migrate/tests/infer_schema_table_filter_si3.rs new file mode 100644 index 00000000..1860747d --- /dev/null +++ b/lifeguard-migrate/tests/infer_schema_table_filter_si3.rs @@ -0,0 +1,58 @@ +//! SI-3: `--table` / `InferOptions.tables` restricts emitted structs (PRD §5.5). +//! +//! Requires `TEST_DATABASE_URL`, `DATABASE_URL`, or `LIFEGUARD_DATABASE_URL`; otherwise skips. + +use lifeguard::{connect, LifeExecutor, MayPostgresExecutor}; +use lifeguard_migrate::schema_infer::{infer_schema_rust, InferOptions}; + +const T_ALPHA: &str = "lg_infer_si3_alpha"; +const T_BETA: &str = "lg_infer_si3_beta"; + +fn postgres_url() -> Option { + std::env::var("TEST_DATABASE_URL") + .or_else(|_| std::env::var("DATABASE_URL")) + .or_else(|_| std::env::var("LIFEGUARD_DATABASE_URL")) + .ok() + .filter(|s| !s.trim().is_empty()) +} + +#[test] +fn infer_schema_table_filter_excludes_other_tables() { + let Some(url) = postgres_url() else { + eprintln!("infer_schema_table_filter_excludes_other_tables: skipped (no DB URL)"); + return; + }; + + let client = connect(&url).expect("connect"); + let ex = MayPostgresExecutor::new(client); + + let create_a = format!( + "CREATE TABLE IF NOT EXISTS public.{T_ALPHA} (id SERIAL PRIMARY KEY, note TEXT NOT NULL DEFAULT '')" + ); + let create_b = format!( + "CREATE TABLE IF NOT EXISTS public.{T_BETA} (id SERIAL PRIMARY KEY, note TEXT NOT NULL DEFAULT '')" + ); + ex.execute(&create_a, &[]).expect("create alpha"); + ex.execute(&create_b, &[]).expect("create beta"); + + let out = infer_schema_rust( + &ex, + &InferOptions { + schema: "public".to_string(), + tables: vec![T_ALPHA.to_string()], + }, + ) + .expect("infer_schema_rust"); + + assert!( + out.contains(&format!("`{T_ALPHA}`")) || out.contains(T_ALPHA), + "expected alpha table in output: {out}" + ); + assert!( + !out.contains(T_BETA), + "beta table should be excluded when filtering to {T_ALPHA}: {out}" + ); + + let _ = ex.execute(&format!("DROP TABLE IF EXISTS public.{T_ALPHA}"), &[]); + let _ = ex.execute(&format!("DROP TABLE IF EXISTS public.{T_BETA}"), &[]); +} diff --git a/lifeguard-migrate/tests/migration_db_compare_smoke.rs b/lifeguard-migrate/tests/migration_db_compare_smoke.rs new file mode 100644 index 00000000..53999879 --- /dev/null +++ b/lifeguard-migrate/tests/migration_db_compare_smoke.rs @@ -0,0 +1,150 @@ +//! Optional live-PostgreSQL smoke for [`lifeguard_migrate::schema_migration_compare`] + `compare-schema` CLI. +//! +//! Uses a **dedicated schema** per run so we only compare the scratch table against merged migration +//! baselines — not the entire shared `public` schema (which holds many integration-test tables in CI). + +use std::fs; + +use lifeguard::{connect, LifeExecutor, MayPostgresExecutor}; +use lifeguard_migrate::schema_migration_compare::compare_generated_dir_to_live_db; +use uuid::Uuid; + +const FILE: &str = "20990101000000_generated_from_entities.sql"; +const TABLE: &str = "smoke_t"; + +fn postgres_url() -> Option { + std::env::var("TEST_DATABASE_URL") + .or_else(|_| std::env::var("DATABASE_URL")) + .or_else(|_| std::env::var("LIFEGUARD_DATABASE_URL")) + .ok() + .filter(|s| !s.trim().is_empty()) +} + +/// Unique schema so parallel tests and shared DBs do not see unrelated `public` tables. +fn scratch_schema_name() -> String { + format!("lg_cmp_{id}", id = Uuid::new_v4().simple()) +} + +#[test] +fn compare_generated_dir_matches_live_table_set() { + let Some(url) = postgres_url() else { + eprintln!( + "compare_generated_dir_matches_live_table_set: skipped (set TEST_DATABASE_URL, DATABASE_URL, or LIFEGUARD_DATABASE_URL)" + ); + return; + }; + + let schema = scratch_schema_name(); + let dir = tempfile::tempdir().expect("tempdir"); + let sql = format!( + "-- Table: {TABLE}\n\ + CREATE TABLE IF NOT EXISTS {TABLE} (id INTEGER PRIMARY KEY);\n" + ); + fs::write(dir.path().join(FILE), sql).expect("write generated sql"); + + let client = connect(&url).expect("connect"); + let executor = MayPostgresExecutor::new(client); + + executor + .execute( + &format!("DROP SCHEMA IF EXISTS {schema} CASCADE"), + &[], + ) + .ok(); + executor + .execute(&format!("CREATE SCHEMA {schema}"), &[]) + .expect("create schema"); + executor + .execute( + &format!("CREATE TABLE {schema}.{TABLE} (id INTEGER PRIMARY KEY)"), + &[], + ) + .expect("create scratch table"); + + let report = compare_generated_dir_to_live_db(&executor, &schema, dir.path()) + .expect("compare_generated_dir_to_live_db"); + + executor + .execute( + &format!("DROP SCHEMA IF EXISTS {schema} CASCADE"), + &[], + ) + .ok(); + + assert!( + !report.has_drift(), + "expected no drift; only_in_db={:?} only_mig={:?}", + report.only_in_database, + report.only_in_migrations + ); +} + +#[test] +fn compare_schema_cli_succeeds_when_no_drift() { + let Some(url) = postgres_url() else { + eprintln!("compare_schema_cli_succeeds_when_no_drift: skipped (no DB URL)"); + return; + }; + + let Some(bin) = std::env::var_os("CARGO_BIN_EXE_lifeguard-migrate") else { + eprintln!("compare_schema_cli_succeeds_when_no_drift: skipped (no CARGO_BIN_EXE_lifeguard-migrate)"); + return; + }; + + let schema = scratch_schema_name(); + let dir = tempfile::tempdir().expect("tempdir"); + let sql = format!( + "-- Table: {TABLE}\n\ + CREATE TABLE IF NOT EXISTS {TABLE} (id INTEGER PRIMARY KEY);\n" + ); + fs::write(dir.path().join(FILE), sql).expect("write generated sql"); + + let client = connect(&url).expect("connect"); + let executor = MayPostgresExecutor::new(client); + executor + .execute( + &format!("DROP SCHEMA IF EXISTS {schema} CASCADE"), + &[], + ) + .ok(); + executor + .execute(&format!("CREATE SCHEMA {schema}"), &[]) + .expect("create schema"); + executor + .execute( + &format!("CREATE TABLE {schema}.{TABLE} (id INTEGER PRIMARY KEY)"), + &[], + ) + .expect("create scratch table"); + + let out = std::process::Command::new(&bin) + .args([ + "compare-schema", + "--database-url", + &url, + "--schema", + &schema, + "--generated-dir", + dir.path().to_str().expect("utf8 temp path"), + ]) + .output() + .expect("spawn compare-schema"); + + executor + .execute( + &format!("DROP SCHEMA IF EXISTS {schema} CASCADE"), + &[], + ) + .ok(); + + assert!( + out.status.success(), + "compare-schema failed: stderr={stderr}", + stderr = String::from_utf8_lossy(&out.stderr) + ); + let stdout = String::from_utf8_lossy(&out.stdout); + assert!( + stdout.contains("No drift"), + "unexpected output: {stdout}" + ); +} diff --git a/src/active_model/mod.rs b/src/active_model/mod.rs index 81f0b6bf..ee67fd01 100644 --- a/src/active_model/mod.rs +++ b/src/active_model/mod.rs @@ -10,7 +10,7 @@ //! - **Traits**: Core `ActiveModel` traits (`ActiveModelTrait`, `ActiveModelBehavior`) //! - **Value**: `ActiveValue` enum for field value metadata //! - **Error**: `ActiveModelError` for operation errors -//! - **Conversion**: `SeaQuery` → `ToSql` via shared [`crate::query::converted_params`] (`ActiveModelError`) +//! - **Conversion**: `SeaQuery` → `ToSql` via [`crate::active_model::with_converted_params`] (`ActiveModelError`) //! //! # Examples //! @@ -38,7 +38,7 @@ // Validation types (PRD Phase B; no dependency on traits) pub mod validate_op; #[doc(inline)] -pub use validate_op::{ValidateOp, ValidationError}; +pub use validate_op::{ValidateOp, ValidationError, ValidationStrategy}; // Core traits pub mod traits; @@ -58,7 +58,10 @@ pub use error::ActiveModelError; // Validation orchestration (`run_validators` after lifecycle hooks) pub mod validation; #[doc(inline)] -pub use validation::run_validators; +pub use validation::{run_validators, run_validators_with_strategy}; + +/// Built-in `len` / `range`-style validators on [`sea_query::Value`] (PRD Phase B follow-on). +pub mod predicates; // Graph sorting and nesting mechanics pub mod graph; diff --git a/src/active_model/predicates.rs b/src/active_model/predicates.rs new file mode 100644 index 00000000..961fd497 --- /dev/null +++ b/src/active_model/predicates.rs @@ -0,0 +1,161 @@ +//! Built-in validation helpers for use in [`ActiveModelBehavior::validate_fields`](crate::active_model::ActiveModelBehavior::validate_fields) +//! and `#[validate(custom = path)]` (PRD Phase B — `range`, `len`-style predicates). +//! +//! Each function takes a [`sea_query::Value`] and returns `Result<(), String>` — the same shape as +//! `#[validate(custom = path)]` — so you can map to [`ValidationError`](crate::active_model::ValidationError) in +//! [`ActiveModelBehavior::validate_fields`](crate::active_model::ActiveModelBehavior::validate_fields). +//! +//! **Unset fields:** `None` / null scalar variants are treated as “nothing to validate” and return `Ok(())`. + +use sea_query::Value; + +/// Maximum UTF-8 **character** length (`.chars().count()`, not `.len()` bytes). +/// +/// Unset `String(None)` or `Null` → `Ok(())`. Non-string values → `Ok(())` (use only on string columns). +pub fn string_utf8_chars_max(value: &Value, max: usize) -> Result<(), String> { + if let Value::String(Some(s)) = value { + let count = s.chars().count(); + if count > max { + return Err(format!("must be at most {max} characters (got {count})")); + } + } + Ok(()) +} + +/// Minimum and maximum UTF-8 character length (inclusive). +pub fn string_utf8_chars_in_range(value: &Value, min: usize, max: usize) -> Result<(), String> { + match value { + Value::String(Some(s)) => { + let n = s.chars().count(); + if n < min || n > max { + Err(format!("must be between {min} and {max} characters (got {n})")) + } else { + Ok(()) + } + } + Value::String(None) => Ok(()), + _ => Ok(()), + } +} + +/// Maximum **byte** length for `String` or `Bytes` payloads. +pub fn blob_or_string_byte_len_max(value: &Value, max: usize) -> Result<(), String> { + match value { + Value::String(Some(s)) if s.len() > max => Err(format!( + "must be at most {max} bytes (got {})", + s.len() + )), + Value::Bytes(Some(b)) if b.len() > max => Err(format!( + "must be at most {max} bytes (got {})", + b.len() + )), + Value::String(Some(_)) + | Value::String(None) + | Value::Bytes(Some(_)) + | Value::Bytes(None) => Ok(()), + _ => Ok(()), + } +} + +fn value_as_i64(value: &Value) -> Option { + match value { + Value::TinyInt(Some(v)) => Some(i64::from(*v)), + Value::SmallInt(Some(v)) => Some(i64::from(*v)), + Value::Int(Some(v)) => Some(i64::from(*v)), + Value::BigInt(Some(v)) => Some(*v), + Value::TinyUnsigned(Some(v)) => Some(i64::from(*v)), + Value::SmallUnsigned(Some(v)) => Some(i64::from(*v)), + Value::Unsigned(Some(v)) => Some(i64::from(*v)), + Value::BigUnsigned(Some(v)) => i64::try_from(*v).ok(), + _ => None, + } +} + +/// Inclusive `i64` range for integer-like [`Value`] variants. +/// +/// Unset / null integer → `Ok(())`. Values that do not map to `i64` (e.g. float, string) → `Ok(())`; +/// use only on columns you know are integral, or combine with type checks. +pub fn i64_in_range(value: &Value, min: i64, max: i64) -> Result<(), String> { + let Some(n) = value_as_i64(value) else { + return Ok(()); + }; + if n < min || n > max { + return Err(format!("must be between {min} and {max} (got {n})")); + } + Ok(()) +} + +fn value_as_f64(value: &Value) -> Option { + match value { + Value::Float(Some(v)) => Some(f64::from(*v)), + Value::Double(Some(v)) => Some(*v), + _ => None, + } +} + +/// Inclusive `f64` range for `Float` / `Double` values. +/// +/// Unset / null → `Ok(())`. Non-float values → `Ok(())`. +pub fn f64_in_range(value: &Value, min: f64, max: f64) -> Result<(), String> { + let Some(x) = value_as_f64(value) else { + return Ok(()); + }; + if !x.is_finite() { + return Err("must be a finite number".to_string()); + } + if x < min || x > max { + return Err(format!("must be between {min} and {max} (got {x})")); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn string_utf8_chars_max_ok_empty() { + assert!(string_utf8_chars_max(&Value::String(Some(String::new())), 5).is_ok()); + } + + #[test] + fn string_utf8_chars_max_err() { + assert_eq!( + string_utf8_chars_max(&Value::String(Some("abcdef".to_string())), 5), + Err("must be at most 5 characters (got 6)".to_string()) + ); + } + + #[test] + fn string_utf8_chars_max_unicode_counts_chars() { + assert!(string_utf8_chars_max(&Value::String(Some("é".to_string())), 1).is_ok()); + assert!(string_utf8_chars_max(&Value::String(Some("éé".to_string())), 1).is_err()); + } + + #[test] + fn string_utf8_chars_in_range_unset_ok() { + assert!(string_utf8_chars_in_range(&Value::String(None), 1, 10).is_ok()); + } + + #[test] + fn i64_in_range_unset_ok() { + assert!(i64_in_range(&Value::Int(None), 0, 10).is_ok()); + } + + #[test] + fn i64_in_range_bounds() { + assert!(i64_in_range(&Value::Int(Some(5)), 0, 10).is_ok()); + assert_eq!( + i64_in_range(&Value::Int(Some(11)), 0, 10), + Err("must be between 0 and 10 (got 11)".to_string()) + ); + } + + #[test] + fn f64_in_range_rejects_nan() { + assert_eq!( + f64_in_range(&Value::Double(Some(f64::NAN)), 0.0, 1.0), + Err("must be a finite number".to_string()) + ); + } +} diff --git a/src/active_model/traits.rs b/src/active_model/traits.rs index 177dbd29..f223ce0f 100644 --- a/src/active_model/traits.rs +++ b/src/active_model/traits.rs @@ -4,7 +4,7 @@ //! model operations including field access, `CRUD` operations, and lifecycle hooks. use super::error::ActiveModelError; -use super::validate_op::ValidateOp; +use super::validate_op::{ValidateOp, ValidationStrategy}; use super::value::ActiveValue; use crate::model::ModelTrait; use crate::query::LifeModelTrait; @@ -696,10 +696,17 @@ pub trait ActiveModelBehavior: ActiveModelTrait { Ok(()) } + /// How to combine validation errors from [`validate_fields`](Self::validate_fields) and + /// [`validate_model`](Self::validate_model). Default: [`ValidationStrategy::FailFast`]. + fn validation_strategy(&self, _op: ValidateOp) -> ValidationStrategy { + ValidationStrategy::FailFast + } + /// Field-level validation (PRD Phase B). Default: no-op. /// /// Runs after [`before_insert`](Self::before_insert) / [`before_update`](Self::before_update) - /// so hook-applied defaults are visible; before SQL is built. + /// so hook-applied defaults are visible; before SQL is built. For deletes, runs after + /// [`before_delete`](Self::before_delete) with [`ValidateOp::Delete`]. fn validate_fields(&self, _op: ValidateOp) -> Result<(), ActiveModelError> { Ok(()) } diff --git a/src/active_model/validate_op.rs b/src/active_model/validate_op.rs index 8ca19ee1..a51a14bd 100644 --- a/src/active_model/validate_op.rs +++ b/src/active_model/validate_op.rs @@ -1,6 +1,19 @@ //! Operation discriminator for [`super::traits::ActiveModelBehavior::validate_fields`] / //! [`super::traits::ActiveModelBehavior::validate_model`] (PRD Phase B). +/// How [`super::validation::run_validators`] combines errors from [`super::traits::ActiveModelBehavior::validate_fields`] +/// and [`super::traits::ActiveModelBehavior::validate_model`] (PRD V-3). +#[derive(Copy, Clone, Debug, PartialEq, Eq, Default)] +pub enum ValidationStrategy { + /// Stop on the first failing validator hook (default). + #[default] + FailFast, + /// Run `validate_fields`, then `validate_model`, and return all `Validation` errors in one `Vec`. + /// + /// Non-validation errors (e.g. `DatabaseError`) still abort immediately on first occurrence. + Aggregate, +} + /// Which persistence path is running validation. #[derive(Copy, Clone, Debug, PartialEq, Eq, Hash)] pub enum ValidateOp { @@ -8,6 +21,8 @@ pub enum ValidateOp { Insert, /// `ActiveModelTrait::update` / `save` update branch Update, + /// `ActiveModelTrait::delete` (after `before_delete`, before SQL) + Delete, } /// One failed validation rule (field-level or model-level). diff --git a/src/active_model/validation.rs b/src/active_model/validation.rs index 4d62d21e..1bed40d8 100644 --- a/src/active_model/validation.rs +++ b/src/active_model/validation.rs @@ -1,24 +1,56 @@ //! Orchestrates [`super::traits::ActiveModelBehavior::validate_fields`] then -//! [`super::traits::ActiveModelBehavior::validate_model`] (fail-fast). +//! [`super::traits::ActiveModelBehavior::validate_model`] (PRD ordering: field → model). use super::error::ActiveModelError; use super::traits::ActiveModelBehavior; -use super::validate_op::ValidateOp; +use super::validate_op::{ValidateOp, ValidationStrategy}; -/// Run field-level then model-level validation (PRD ordering: field → model). +/// Run field-level then model-level validation using [`ActiveModelBehavior::validation_strategy`]. #[inline] pub fn run_validators(record: &R, op: ValidateOp) -> Result<(), ActiveModelError> { - record.validate_fields(op)?; - record.validate_model(op)?; - Ok(()) + run_validators_with_strategy(record, op, record.validation_strategy(op)) +} + +/// Run validators with an explicit [`ValidationStrategy`] (ignores [`ActiveModelBehavior::validation_strategy`]). +#[inline] +pub fn run_validators_with_strategy( + record: &R, + op: ValidateOp, + strategy: ValidationStrategy, +) -> Result<(), ActiveModelError> { + match strategy { + ValidationStrategy::FailFast => { + record.validate_fields(op)?; + record.validate_model(op)?; + Ok(()) + } + ValidationStrategy::Aggregate => { + let mut errs = Vec::new(); + match record.validate_fields(op) { + Ok(()) => {} + Err(ActiveModelError::Validation(v)) => errs.extend(v), + Err(e) => return Err(e), + } + match record.validate_model(op) { + Ok(()) => {} + Err(ActiveModelError::Validation(v)) => errs.extend(v), + Err(e) => return Err(e), + } + if errs.is_empty() { + Ok(()) + } else { + Err(ActiveModelError::Validation(errs)) + } + } + } } #[cfg(test)] mod tests { use super::super::error::ActiveModelError; use super::super::traits::{ActiveModelBehavior, ActiveModelTrait}; - use super::super::validate_op::{ValidateOp, ValidationError}; - use super::run_validators; + use super::super::validate_op::{ValidateOp, ValidationError, ValidationStrategy}; + use super::{run_validators, run_validators_with_strategy}; use crate::LifeModelTrait; use sea_query::{Iden, IdenStatic}; @@ -89,6 +121,7 @@ mod tests { order: std::cell::RefCell>, fail_field: bool, fail_model: bool, + strategy: ValidationStrategy, } impl ActiveModelTrait for StubRecord { @@ -151,6 +184,10 @@ mod tests { } impl ActiveModelBehavior for StubRecord { + fn validation_strategy(&self, _op: ValidateOp) -> ValidationStrategy { + self.strategy + } + fn validate_fields(&self, _op: ValidateOp) -> Result<(), ActiveModelError> { self.order.borrow_mut().push("validate_fields"); if self.fail_field { @@ -181,6 +218,7 @@ mod tests { order: std::cell::RefCell::new(Vec::new()), fail_field: false, fail_model: false, + strategy: ValidationStrategy::FailFast, }; assert!(matches!( run_validators(&r, ValidateOp::Insert), @@ -195,6 +233,7 @@ mod tests { order: std::cell::RefCell::new(Vec::new()), fail_field: true, fail_model: false, + strategy: ValidationStrategy::FailFast, }; assert!(matches!( run_validators(&r, ValidateOp::Insert), @@ -202,4 +241,60 @@ mod tests { )); assert_eq!(&*r.order.borrow(), &["validate_fields"]); } + + #[test] + fn run_validators_aggregate_runs_model_after_field_errors() { + let r = StubRecord { + order: std::cell::RefCell::new(Vec::new()), + fail_field: true, + fail_model: true, + strategy: ValidationStrategy::Aggregate, + }; + let res = run_validators(&r, ValidateOp::Insert); + assert!( + matches!( + res, + Err(ActiveModelError::Validation(ref v)) + if v.len() == 2 + && v[0].field.as_deref() == Some("x") + && v[1].field.is_none() + ), + "unexpected result: {res:?}" + ); + assert_eq!( + &*r.order.borrow(), + &["validate_fields", "validate_model"] + ); + } + + #[test] + fn run_validators_with_strategy_overrides_record_strategy() { + let r = StubRecord { + order: std::cell::RefCell::new(Vec::new()), + fail_field: true, + fail_model: true, + strategy: ValidationStrategy::FailFast, + }; + let err = run_validators_with_strategy(&r, ValidateOp::Insert, ValidationStrategy::Aggregate); + assert!( + matches!(err, Err(ActiveModelError::Validation(ref v)) if v.len() == 2), + "expected Validation with 2 errors, got {err:?}" + ); + assert_eq!( + &*r.order.borrow(), + &["validate_fields", "validate_model"] + ); + } + + #[test] + fn run_validators_delete_runs_field_then_model() { + let r = StubRecord { + order: std::cell::RefCell::new(Vec::new()), + fail_field: false, + fail_model: false, + strategy: ValidationStrategy::FailFast, + }; + assert!(run_validators(&r, ValidateOp::Delete).is_ok()); + assert_eq!(&*r.order.borrow(), &["validate_fields", "validate_model"]); + } } diff --git a/src/lib.rs b/src/lib.rs index b85dc957..3d5b84ab 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -38,6 +38,13 @@ //! [operator tuning / non-goals](https://github.com/microscaler/lifeguard/blob/main/docs/POOLING_OPERATIONS.md), //! and [TCP keepalive / idle tuning](https://github.com/microscaler/lifeguard/blob/main/docs/POOL_TCP_KEEPALIVE.md) //! (PRD and ops links work on **docs.rs** via GitHub URLs; clone has the same paths under `docs/`). +//! +//! ## Explicit opt-in APIs +//! +//! Advanced `SELECT` features (CTEs, subquery joins, windows, raw `WITH` escape hatches) are **not** +//! implied by normal [`SelectQuery`] usage — you chain the methods documented in +//! [`crate::query::select`]. For connection pools, [`ReadPreference`] overrides where **reads** go +//! ([`PooledLifeExecutor::with_read_preference`]); writes stay on the primary tier. pub mod config; @@ -77,10 +84,13 @@ pub mod test_helpers; // mod tests_cfg; pub use pool::{ - DatabaseConfig, LifeguardPool, LifeguardPoolSettings, OwnedParam, PooledLifeExecutor, - WalLagPolicy, + DatabaseConfig, ExclusivePrimaryLifeExecutor, LifeguardPool, LifeguardPoolSettings, OwnedParam, + PooledLifeExecutor, ReadPreference, WalLagPolicy, }; +#[doc(inline)] +pub use lifeguard_derive::scope; + // Optional GraphQL: `LifeModel` nests `async_graphql::SimpleObject` on the generated `Model`. // Crates that enable `lifeguard`/`graphql` should depend on the same `async-graphql` version // and enable the scalar features they use (e.g. `chrono`, `uuid`, `decimal`); the workspace @@ -110,8 +120,9 @@ pub use query::{ // ActiveModel operations - Epic 02 Story 07 pub mod active_model; pub use active_model::{ - run_validators, with_converted_params, ActiveModelBehavior, ActiveModelError, ActiveModelTrait, - ActiveValue, ValidateOp, ValidationError, + predicates, run_validators, run_validators_with_strategy, with_converted_params, + ActiveModelBehavior, ActiveModelError, ActiveModelTrait, ActiveValue, ValidateOp, + ValidationError, ValidationStrategy, }; // Model trait - Core Traits & Types @@ -120,7 +131,10 @@ pub use model::{ModelError, ModelTrait, TryIntoModel}; // Session / identity map (PRD Phase E v0) pub mod session; -pub use session::{fingerprint_pk_values, ModelIdentityMap}; +pub use session::{ + fingerprint_pk_values, is_pending_insert_key, ModelIdentityMap, PENDING_INSERT_KEY_PREFIX, + Session, SessionDirtyNotifier, SessionIdentityModelCell, +}; // Relation trait - Epic 02 Story 08 pub mod relation; diff --git a/src/logging/mod.rs b/src/logging/mod.rs index 22449e75..a6c45dad 100644 --- a/src/logging/mod.rs +++ b/src/logging/mod.rs @@ -1,8 +1,8 @@ //! Global logging through a single [`may::sync::mpsc`] queue. //! -//! Producers call [`enqueue`] (or the [`lifeguard_log!`] macro). One coroutine drains the +//! Producers call [`enqueue`] (or the [`lifeguard_log!`](macro@crate::logging::lifeguard_log) macro). One coroutine drains the //! channel and writes lines to stderr so formatting stays sequential without locking on the -//! send path. [`flush_log_channel`] and [`ChannelLogger`](log_bridge::ChannelLogger)'s +//! send path. [`flush_log_channel`] and [`ChannelLogger`]'s //! [`log::Log::flush`] block until prior enqueued records have been written (see reentrancy note //! there). //! diff --git a/src/migration/migration_name.rs b/src/migration/migration_name.rs new file mode 100644 index 00000000..ae34943e --- /dev/null +++ b/src/migration/migration_name.rs @@ -0,0 +1,23 @@ +//! Stable names for [`super::Migration`] unit structs. + +/// Stable **snake_case** migration identifier, usually implemented via **`DeriveMigrationName`** +/// (`lifeguard_derive` / `lifeguard::DeriveMigrationName`). +/// +/// Use [`Migration::name`](super::Migration::name) in [`super::Migration`] implementations: +/// +/// ```ignore +/// use lifeguard::migration::{DeriveMigrationName, Migration, MigrationName, SchemaManager}; +/// +/// #[derive(DeriveMigrationName)] +/// pub struct CreateUsersTable; +/// +/// impl Migration for CreateUsersTable { +/// fn name(&self) -> &str { +/// MigrationName::migration_name(self) +/// } +/// // ... +/// } +/// ``` +pub trait MigrationName: Send + Sync { + fn migration_name(&self) -> &'static str; +} diff --git a/src/migration/mod.rs b/src/migration/mod.rs index 2160adaf..d4f7a554 100644 --- a/src/migration/mod.rs +++ b/src/migration/mod.rs @@ -43,6 +43,7 @@ pub mod checksum; pub mod error; pub mod file; pub mod lock; +pub mod migration_name; #[allow(clippy::module_inception)] pub mod migration; pub mod migrator; @@ -60,6 +61,10 @@ pub use lock::{ acquire_migration_lock, is_migration_lock_held, release_migration_lock, MigrationLockGuard, }; pub use migration::Migration; +pub use migration_name::MigrationName; + +#[doc(inline)] +pub use lifeguard_derive::DeriveMigrationName; pub use migrator::Migrator; pub use record::MigrationRecord; pub use registry::{ diff --git a/src/pool/mod.rs b/src/pool/mod.rs index 5fd93800..0d9e73c9 100644 --- a/src/pool/mod.rs +++ b/src/pool/mod.rs @@ -7,6 +7,8 @@ //! [`LifeguardPoolSettings`] and [`crate::LifeError::PoolAcquireTimeout`]). //! - **[`PooledLifeExecutor`]**: [`crate::executor::LifeExecutor`] over the pool via //! [`crate::executor::LifeExecutor::execute_values`] / `query_*_values` (and ORM paths). +//! Use [`ReadPreference`] with [`PooledLifeExecutor::with_read_preference`] to force primary +//! reads when you need read-your-writes consistency. //! - **[`OwnedParam`]**: owned bind parameters for jobs crossing channels (cannot send `&dyn ToSql` //! across threads/coroutine boundaries). //! - **[`wal::WalLagMonitor`]** + **[`WalLagPolicy`]**: optional background polling used when routing @@ -39,4 +41,4 @@ pub mod wal; pub use config::{DatabaseConfig, LifeguardPoolSettings}; pub use wal::WalLagPolicy; pub use owned_param::OwnedParam; -pub use pooled::{LifeguardPool, PooledLifeExecutor}; +pub use pooled::{ExclusivePrimaryLifeExecutor, LifeguardPool, PooledLifeExecutor, ReadPreference}; diff --git a/src/pool/pooled.rs b/src/pool/pooled.rs index bd86a200..6c7fb965 100644 --- a/src/pool/pooled.rs +++ b/src/pool/pooled.rs @@ -14,6 +14,9 @@ //! **replica** pool when replica URLs and a non-zero replica pool size are configured **and** //! [`crate::pool::wal::WalLagMonitor`] reports the replica is not lagging; otherwise reads use //! the primary pool. +//! - Callers can override routing per [`PooledLifeExecutor`] with [`ReadPreference`]: +//! [`ReadPreference::Primary`] forces reads onto the primary tier (read-your-writes); the default +//! is [`ReadPreference::Default`] (WAL-based routing above). //! //! With the **`metrics`** feature, pool-scoped series use the OpenTelemetry attribute **`pool_tier`** //! (`primary` \| `replica`); see [`crate::metrics::METRICS`]. @@ -27,8 +30,9 @@ use crate::pool::wal::{WalLagMonitor, WalLagPolicy}; use crossbeam_channel::{RecvTimeoutError, SendTimeoutError}; use may_postgres::types::ToSql; use may_postgres::{Client, Row}; +use std::fmt; use std::sync::atomic::{AtomicUsize, Ordering}; -use std::sync::Arc; +use std::sync::{Arc, Mutex, MutexGuard}; use std::thread; use std::time::{Duration, Instant}; @@ -37,9 +41,45 @@ use crate::metrics::tracing_helpers; #[cfg(feature = "metrics")] use crate::metrics::METRICS; +/// Where [`PooledLifeExecutor`] should send **read** queries (`query_one_values` / `query_all_values`). +/// +/// Writes always use the primary tier regardless of this value. +/// +/// # When to use [`ReadPreference::Primary`] +/// +/// After your code **writes** and then **reads** through the same logical pool, the replica may not +/// have applied WAL yet. Use [`PooledLifeExecutor::with_read_preference`] with [`ReadPreference::Primary`] +/// for those reads so they hit the primary and see your own commit. Leave [`ReadPreference::Default`] +/// for read-mostly paths where slightly stale reads are acceptable and you want load on standbys. +/// +/// # Example +/// +/// ```no_run +/// use lifeguard::{LifeguardPool, PooledLifeExecutor, ReadPreference}; +/// use std::sync::Arc; +/// +/// # fn take_pool() -> Arc { todo!() } +/// let pool = take_pool(); +/// let exec = PooledLifeExecutor::new(pool).with_read_preference(ReadPreference::Primary); +/// let _ = exec.read_preference(); +/// ``` +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] +pub enum ReadPreference { + /// Route reads using [`LifeguardPool`]'s built-in policy: replica tier when configured and WAL + /// lag allows, otherwise primary. + #[default] + Default, + /// Always read from the **primary** tier (strong consistency, read-your-writes after a write + /// on the same pool). + Primary, +} + /// One tier of workers (primary or replica) with round-robin dispatch. struct WorkerPool { worker_txs: Arc<[crossbeam_channel::Sender]>, + /// One mutex per slot: held for the duration of each dispatched job, or for the whole lifetime + /// of [`ExclusivePrimaryLifeExecutor`] (U-4 pin-slot) so other dispatchers block on that slot. + slot_locks: Arc<[Mutex<()>]>, next_worker: AtomicUsize, pool_size: usize, acquire_timeout: Duration, @@ -98,8 +138,12 @@ impl WorkerPool { let worker_txs: Arc<[crossbeam_channel::Sender]> = txs.into(); + let slot_locks: Vec> = (0..pool_size).map(|_| Mutex::new(())).collect(); + let slot_locks: Arc<[Mutex<()>]> = slot_locks.into_boxed_slice().into(); + Ok(Self { worker_txs, + slot_locks, next_worker: AtomicUsize::new(0), pool_size, acquire_timeout: settings.acquire_timeout, @@ -107,14 +151,55 @@ impl WorkerPool { }) } - fn pick_worker(&self) -> &crossbeam_channel::Sender { - let i = self.next_worker.fetch_add(1, Ordering::Relaxed) % self.pool_size; - &self.worker_txs[i] + fn pick_worker_index(&self) -> usize { + self.next_worker.fetch_add(1, Ordering::Relaxed) % self.pool_size } + /// Enqueue work on a worker after taking that slot’s [`Mutex`]. + /// + /// **Latency:** The slot index is chosen **before** the lock ([`Self::pick_worker_index`], + /// round-robin). If another caller holds the same slot’s mutex via + /// [`LifeguardPool::exclusive_primary_write_executor`], this path **blocks** until that + /// [`ExclusivePrimaryLifeExecutor`] is dropped—so a long `BEGIN`…`COMMIT` on one slot can delay + /// unrelated traffic that round-robins onto that slot. Larger [`Self::pool_size`] spreads load; + /// keeping exclusive transactions short reduces head-of-line blocking. + /// + /// **Scheduling:** [`std::sync::Mutex`] is an OS-thread lock. Blocking here blocks the **calling + /// thread** (including a `may` worker thread if your coroutine runs on one), not the pool’s + /// dedicated DB worker threads. That matches this pool’s synchronous “submit job → wait for + /// reply” API; cooperative runtimes should avoid holding the pool across very long critical + /// sections on shared executor threads. fn dispatch( &self, build: impl FnOnce(std::sync::mpsc::SyncSender>) -> WorkerJob, + ) -> Result { + let slot = self.pick_worker_index(); + let _slot_guard = self.slot_locks[slot] + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + self.dispatch_locked(slot, build) + } + + /// Dispatch to `slot` without acquiring [`Self::slot_locks`]. Caller must already hold the + /// mutex for `slot` (see [`ExclusivePrimaryLifeExecutor`]). + fn dispatch_locked( + &self, + slot: usize, + build: impl FnOnce(std::sync::mpsc::SyncSender>) -> WorkerJob, + ) -> Result { + if slot >= self.pool_size { + return Err(LifeError::Pool(format!( + "internal pool error: slot {slot} out of range (pool_size {})", + self.pool_size + ))); + } + self.dispatch_on_sender(&self.worker_txs[slot], build) + } + + fn dispatch_on_sender( + &self, + tx: &crossbeam_channel::Sender, + build: impl FnOnce(std::sync::mpsc::SyncSender>) -> WorkerJob, ) -> Result { #[cfg(feature = "tracing")] let _span = tracing_helpers::acquire_connection_span().entered(); @@ -123,7 +208,6 @@ impl WorkerPool { let deadline = wait_start + self.acquire_timeout; let (reply_tx, reply_rx) = std::sync::mpsc::sync_channel(1); let job = build(reply_tx); - let tx = self.pick_worker(); let mut current_job = job; loop { @@ -383,6 +467,13 @@ impl LifeguardPool { } } + fn read_pool_for(&self, preference: ReadPreference) -> &WorkerPool { + match preference { + ReadPreference::Primary => &self.primary, + ReadPreference::Default => self.read_tier(), + } + } + fn dispatch_write( &self, build: impl FnOnce(std::sync::mpsc::SyncSender>) -> WorkerJob, @@ -390,11 +481,121 @@ impl LifeguardPool { self.primary.dispatch(build) } - fn dispatch_read( + fn dispatch_read_with_preference( &self, + preference: ReadPreference, build: impl FnOnce(std::sync::mpsc::SyncSender>) -> WorkerJob, ) -> Result { - self.read_tier().dispatch(build) + self.read_pool_for(preference).dispatch(build) + } + + /// Pin one primary worker slot for a multi-statement unit of work (for example `BEGIN` → ORM + /// work → `COMMIT`). While the returned [`ExclusivePrimaryLifeExecutor`] is alive, other + /// dispatchers that target the same slot block on the per-slot mutex, so work on this handle is + /// not interleaved with unrelated jobs on that connection. + pub fn exclusive_primary_write_executor( + &self, + ) -> Result, LifeError> { + let slot = self.primary.pick_worker_index(); + let _guard = self.primary.slot_locks[slot] + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + Ok(ExclusivePrimaryLifeExecutor { + pool: self, + slot, + _guard, + }) + } +} + +/// Pins one primary [`LifeguardPool`] worker slot: every [`LifeExecutor`] call uses the same +/// underlying client until this value is dropped (PRD U-4). +/// +/// Both reads and writes go through the **primary** tier (no replica routing), which matches +/// PostgreSQL transaction semantics for `BEGIN`/`COMMIT` on a single connection. +pub struct ExclusivePrimaryLifeExecutor<'a> { + pool: &'a LifeguardPool, + slot: usize, + _guard: MutexGuard<'a, ()>, +} + +impl fmt::Debug for ExclusivePrimaryLifeExecutor<'_> { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("ExclusivePrimaryLifeExecutor") + .field("slot", &self.slot) + .finish_non_exhaustive() + } +} + +impl LifeExecutor for ExclusivePrimaryLifeExecutor<'_> { + fn execute(&self, query: &str, params: &[&dyn ToSql]) -> Result { + if params.is_empty() { + return self.execute_values(query, &sea_query::Values(Vec::new())); + } + Err(LifeError::Pool( + "ExclusivePrimaryLifeExecutor: use execute_values(query, &sea_query::Values) or ORM APIs; dynamic &dyn ToSql cannot cross the pool channel".to_string(), + )) + } + + fn query_one(&self, query: &str, params: &[&dyn ToSql]) -> Result { + if params.is_empty() { + return self.query_one_values(query, &sea_query::Values(Vec::new())); + } + Err(LifeError::Pool( + "ExclusivePrimaryLifeExecutor: use query_one_values(query, &sea_query::Values) or ORM APIs; dynamic &dyn ToSql cannot cross the pool channel".to_string(), + )) + } + + fn query_all(&self, query: &str, params: &[&dyn ToSql]) -> Result, LifeError> { + if params.is_empty() { + return self.query_all_values(query, &sea_query::Values(Vec::new())); + } + Err(LifeError::Pool( + "ExclusivePrimaryLifeExecutor: use query_all_values(query, &sea_query::Values) or ORM APIs; dynamic &dyn ToSql cannot cross the pool channel".to_string(), + )) + } + + fn execute_values(&self, query: &str, values: &sea_query::Values) -> Result { + let params = values_to_owned(values)?; + let query = query.to_string(); + self.pool + .primary + .dispatch_locked(self.slot, |reply| WorkerJob::Execute { + enqueued_at: Instant::now(), + query, + params, + reply, + }) + } + + fn query_one_values(&self, query: &str, values: &sea_query::Values) -> Result { + let params = values_to_owned(values)?; + let query = query.to_string(); + self.pool + .primary + .dispatch_locked(self.slot, |reply| WorkerJob::QueryOne { + enqueued_at: Instant::now(), + query, + params, + reply, + }) + } + + fn query_all_values( + &self, + query: &str, + values: &sea_query::Values, + ) -> Result, LifeError> { + let params = values_to_owned(values)?; + let query = query.to_string(); + self.pool + .primary + .dispatch_locked(self.slot, |reply| WorkerJob::QueryAll { + enqueued_at: Instant::now(), + query, + params, + reply, + }) } } @@ -735,21 +936,66 @@ fn values_to_owned(values: &sea_query::Values) -> Result, LifeEr /// [`LifeExecutor::query_all_values`] (or ORM methods built on them). Raw /// `execute` / `query_*` with non-empty `&[&dyn ToSql]` are rejected because those /// references cannot cross the pool channel. +/// +/// **Read routing:** by default, reads may go to a configured replica when WAL lag allows; see +/// [`ReadPreference`] and [`Self::with_read_preference`]. Writes always use the primary tier. #[derive(Clone)] pub struct PooledLifeExecutor { pool: Arc, + read_preference: ReadPreference, } impl PooledLifeExecutor { #[must_use] pub fn new(pool: Arc) -> Self { - Self { pool } + Self { + pool, + read_preference: ReadPreference::default(), + } + } + + fn dispatch_read( + &self, + build: impl FnOnce(std::sync::mpsc::SyncSender>) -> WorkerJob, + ) -> Result { + self.pool + .dispatch_read_with_preference(self.read_preference, build) } #[must_use] pub fn pool(&self) -> &Arc { &self.pool } + + /// Returns a copy of this executor with the given [`ReadPreference`]. + /// + /// Use [`ReadPreference::Primary`] when you need **read-your-writes** (e.g. insert then select + /// on the same request). [`ReadPreference::Default`] restores pool policy (replica when allowed). + /// + /// # Example + /// + /// ```no_run + /// use lifeguard::{LifeguardPool, LifeExecutor, PooledLifeExecutor, ReadPreference}; + /// use sea_query::Values; + /// use std::sync::Arc; + /// + /// # fn take_pool() -> Arc { todo!() } + /// let base = PooledLifeExecutor::new(take_pool()); + /// let primary_reads = base.with_read_preference(ReadPreference::Primary); + /// let _ = primary_reads.query_one_values("SELECT 1", &Values(Vec::new())); + /// ``` + #[must_use] + pub fn with_read_preference(self, read_preference: ReadPreference) -> Self { + Self { + read_preference, + ..self + } + } + + #[must_use] + pub fn read_preference(&self) -> ReadPreference { + self.read_preference + } } impl LifeExecutor for PooledLifeExecutor { @@ -794,7 +1040,7 @@ impl LifeExecutor for PooledLifeExecutor { fn query_one_values(&self, query: &str, values: &sea_query::Values) -> Result { let params = values_to_owned(values)?; let query = query.to_string(); - self.pool.dispatch_read(|reply| WorkerJob::QueryOne { + self.dispatch_read(|reply| WorkerJob::QueryOne { enqueued_at: Instant::now(), query, params, @@ -809,7 +1055,7 @@ impl LifeExecutor for PooledLifeExecutor { ) -> Result, LifeError> { let params = values_to_owned(values)?; let query = query.to_string(); - self.pool.dispatch_read(|reply| WorkerJob::QueryAll { + self.dispatch_read(|reply| WorkerJob::QueryAll { enqueued_at: Instant::now(), query, params, @@ -821,8 +1067,14 @@ impl LifeExecutor for PooledLifeExecutor { #[cfg(test)] mod lifetime_effective_limit_tests { use super::connection_lifetime_effective_limit; + use super::ReadPreference; use std::time::Duration; + #[test] + fn read_preference_default_matches_variant() { + assert_eq!(ReadPreference::default(), ReadPreference::Default); + } + #[test] fn same_slot_same_limit_across_calls() { let base = Duration::from_secs(60); diff --git a/src/query/column/column_trait.rs b/src/query/column/column_trait.rs index d3127c2d..9ba10a23 100644 --- a/src/query/column/column_trait.rs +++ b/src/query/column/column_trait.rs @@ -110,25 +110,42 @@ pub trait ColumnTrait: IntoColumnRef { /// /// This is the primary **F-style** helper for column-on-both-sides updates (see project PRD: F expressions / database-level expressions). /// + /// For **`WHERE`** / **`ORDER BY`**, wrap the [`sea_query::SimpleExpr`] in [`sea_query::Expr::expr`] + /// and use [`sea_query::ExprTrait`] (e.g. `.gt(…)`), or pass to [`sea_query::SelectStatement::order_by_expr`]. + /// + /// **F-3:** Shipped coverage vs raw-SQL fallbacks are summarized in the repository README (F() / competitive table) and `docs/planning/PRD_SCHEMA_VALIDATORS_SESSION_AND_SCOPES.md` §8. + /// /// # Limitations /// /// For nested aggregates, subqueries, or vendor-only functions, use [`Expr::cust`](sea_query::Expr::cust) /// or SeaQuery’s expression API directly. + /// + /// **PostgreSQL numeric typing:** mixed operand types (e.g. `integer` + `numeric`) follow the server’s + /// **binary promotion** rules (see PostgreSQL docs: *value expressions*, *type conversion*). If you need a + /// specific result type (e.g. `bigint`), match column and RHS types or add an explicit cast in SQL via + /// [`Expr::cust`](sea_query::Expr::cust) / SeaQuery—Lifeguard does not inject casts. PRD §8.7 and the + /// README competitive matrix (F() row) summarize this alongside raw-SQL fallbacks. fn f_add>(self, rhs: R) -> sea_query::SimpleExpr { Expr::col(self).add(rhs) } /// Database-side **subtract**: `column - rhs`. + /// + /// Same PostgreSQL numeric promotion considerations as [`ColumnTrait::f_add`]. fn f_sub>(self, rhs: R) -> sea_query::SimpleExpr { Expr::col(self).sub(rhs) } /// Database-side **multiply**: `column * rhs`. + /// + /// Same PostgreSQL numeric promotion considerations as [`ColumnTrait::f_add`]. fn f_mul>(self, rhs: R) -> sea_query::SimpleExpr { Expr::col(self).mul(rhs) } /// Database-side **divide**: `column / rhs`. + /// + /// Same PostgreSQL numeric promotion considerations as [`ColumnTrait::f_add`]. fn f_div>(self, rhs: R) -> sea_query::SimpleExpr { Expr::col(self).div(rhs) } diff --git a/src/query/mod.rs b/src/query/mod.rs index fed3f67d..e54bf887 100644 --- a/src/query/mod.rs +++ b/src/query/mod.rs @@ -4,16 +4,24 @@ //! against database entities. It includes traits for entity definitions, query builders //! for SELECT/INSERT/UPDATE/DELETE operations, and execution methods. //! +//! # Default path vs advanced SQL +//! +//! The usual flow is [`LifeModelTrait::find`] → filters / order / limit → [`SelectQuery::all`] / +//! [`SelectQuery::one`] (see [`crate::query::execution`]). CTEs, joins to subqueries, window clauses, +//! and similar features are **opt-in** on [`SelectQuery`]; see [crate::query::select] for the full +//! list and `sea_query` types to import. +//! //! # Architecture //! //! The query module follows `Sea-ORM`'s organizational patterns: //! - **Traits**: Core entity and model traits (`LifeModelTrait`, `LifeEntityName`) //! - **Select**: SELECT query builder (`SelectQuery`) //! - **Scopes**: Named composable predicates (`scope` module, `SelectQuery::scope`, `IntoScope`) +//! - **SQL extras on `SelectQuery`**: [`SelectQuery::with_cte`](select::SelectQuery::with_cte) (CTE + lifeguard `all`/`one`), [`join_subquery`](select::SelectQuery::join_subquery), typed [`window`](select::SelectQuery::window) / [`expr_window_as`](select::SelectQuery::expr_window_as) (see also [`subquery_column`](select::SelectQuery::subquery_column), [`window_function_cust`](select::SelectQuery::window_function_cust)) //! - **Execution**: Query execution methods (`all`, `one`, `first`) //! - **Value Conversion**: `SeaQuery` `Value` to `ToSql` (`converted_params` + `value_conversion`) //! - **Error Handling**: Error detection and classification utilities -//! - **Column**: Type-safe column operations (including **F-style** `f_add` / `f_sub` / `f_mul` / `f_div` on [`ColumnTrait`](column::column_trait::ColumnTrait) for `UPDATE SET`) +//! - **Column**: Type-safe column operations (including **F-style** `f_add` / `f_sub` / `f_mul` / `f_div` on [`ColumnTrait`] for `UPDATE SET`) //! - **Primary Key**: Primary key operations and traits //! //! # Examples diff --git a/src/query/scope.rs b/src/query/scope.rs index ef6f955f..39af2948 100644 --- a/src/query/scope.rs +++ b/src/query/scope.rs @@ -4,7 +4,9 @@ //! //! Define associated functions (or constants) that return anything implementing //! [`sea_query::IntoCondition`] (e.g. [`sea_query::Expr`] from [`crate::ColumnTrait`]), -//! then chain with [`SelectQuery::scope`] (or [`SelectQuery::filter`]): +//! then chain with [`SelectQuery::scope`] (or [`SelectQuery::filter`]). Prefer the +//! [`crate::scope`] attribute on `impl Entity` so `fn active()` becomes `scope_active()` +//! (see `lifeguard-derive` / PRD Phase C). //! //! ```ignore //! User::find() @@ -15,7 +17,8 @@ //! # Composition //! //! Each [`SelectQuery::scope`] call **AND**s its condition with the rest of the `WHERE` -//! clause (same as [`SelectQuery::filter`]). +//! clause (same as [`SelectQuery::filter`]). Use [`SelectQuery::scope_or`] or +//! [`SelectQuery::scope_any`] when you need **OR** between predicates (PRD SC-2). //! //! # Soft delete //! @@ -24,9 +27,19 @@ //! `deleted_at IS NULL` (or the configured column) **unless** [`SelectQuery::with_trashed`] //! was used. That predicate is **AND**ed with every `scope` / `filter` you added. Scopes do //! not replace the global soft-delete filter. +//! +//! # Relations and `find_related` +//! +//! Scopes apply to the **root** entity of the [`SelectQuery`] you are building. They are **not** +//! automatically merged into [`crate::FindRelated::find_related`] SQL, which selects from the **related** +//! table and applies only the relation `WHERE` from [`crate::build_where_condition`]. Chain +//! [`.scope`](crate::SelectQuery::scope) / [`.filter`](crate::SelectQuery::filter) on the query returned +//! by `find_related` to filter related rows. Design note (repository): +//! `docs/planning/DESIGN_FIND_RELATED_SCOPES.md`. use crate::query::select::SelectQuery; use crate::query::traits::LifeModelTrait; +use sea_query::Condition; /// Something that can be applied to a [`SelectQuery`] as a named scope. /// @@ -56,6 +69,37 @@ where pub fn scope>(self, s: S) -> Self { s.apply_scope(self) } + + /// OR two scope conditions: `(a) OR (b)` (PRD SC-2). + #[must_use] + pub fn scope_or(self, a: A, b: B) -> Self + where + A: sea_query::IntoCondition, + B: sea_query::IntoCondition, + { + let c = Condition::any() + .add(a.into_condition()) + .add(b.into_condition()); + self.filter(c) + } + + /// OR an iterator of conditions. Empty iterator returns `self` unchanged. + #[must_use] + pub fn scope_any(self, iter: I) -> Self + where + I: IntoIterator, + C: sea_query::IntoCondition, + { + let mut it = iter.into_iter(); + let Some(first) = it.next() else { + return self; + }; + let mut c = Condition::any().add(first.into_condition()); + for cond in it { + c = c.add(cond.into_condition()); + } + self.filter(c) + } } #[cfg(test)] @@ -242,4 +286,36 @@ mod tests { "user scope id: {sql}" ); } + + #[test] + fn scope_or_produces_or_in_sql() { + let q = SelectQuery::::new().scope_or( + ScopeTestColumn::Status.eq(1i32), + ScopeTestColumn::Status.eq(2i32), + ); + let (sql, _) = q.query.build(PostgresQueryBuilder); + let upper = sql.to_uppercase(); + assert!(upper.contains(" OR "), "expected OR in WHERE: {sql}"); + } + + #[test] + fn scope_any_empty_is_noop() { + let q = SelectQuery::::new(); + let q2 = q.clone().scope_any(std::iter::empty::()); + let (s1, _) = q.query.build(PostgresQueryBuilder); + let (s2, _) = q2.query.build(PostgresQueryBuilder); + assert_eq!(s1, s2); + } + + #[test] + fn scope_any_three_branches() { + let q = SelectQuery::::new().scope_any([ + ScopeTestColumn::Status.eq(1i32), + ScopeTestColumn::Status.eq(2i32), + ScopeTestColumn::Id.eq(99i32), + ]); + let (sql, _) = q.query.build(PostgresQueryBuilder); + let upper = sql.to_uppercase(); + assert!(upper.matches(" OR ").count() >= 2, "expected multiple OR: {sql}"); + } } diff --git a/src/query/select.rs b/src/query/select.rs index ea9728dd..d21074a7 100644 --- a/src/query/select.rs +++ b/src/query/select.rs @@ -1,8 +1,30 @@ //! Select query builder for `LifeModel`. //! -//! This module provides `SelectQuery` and `SelectModel` for building and executing -//! type-safe database queries. Query building methods (`filter`, `order_by`, `limit`, etc.) -//! are defined here, while execution methods are in the execution module. +//! This module provides [`SelectQuery`] and [`SelectModel`] for building and executing +//! type-safe database queries. Builder methods (`filter`, `order_by`, `limit`, …) live here; +//! execution (`all`, `one`, …) is in [`crate::query::execution`]. +//! +//! # Default path vs advanced SQL (nothing is “magic”) +//! +//! Most handlers only need **`Entity::find()` → `filter` → `order_by` → `limit` → +//! [`all`](crate::query::select::SelectQuery::all)**. That path is the default; the ORM does **not** +//! inject CTEs, subquery joins, or window functions unless **you** chain the APIs below. +//! +//! **When you need a richer `SELECT`**, use these **explicit** methods (all keep [`SelectQuery`] so +//! loaders, soft-delete, and [`all`](crate::query::select::SelectQuery::all) / +//! [`one`](crate::query::select::SelectQuery::one) still work), or compose [`sea_query::Expr`] for `WHERE`: +//! +//! - **`WITH` (CTE)** — [`SelectQuery::with_cte`] (preferred). Avoid [`SelectQuery::with`] unless you +//! intentionally want a raw [`sea_query::WithQuery`] and will hand-build SQL. +//! - **`JOIN (SELECT …)`** — [`SelectQuery::join_subquery`]. +//! - **Subquery as a SELECT column** — [`SelectQuery::subquery_column`]. +//! - **Window functions (`OVER`, `WINDOW`)** — [`SelectQuery::window`], [`SelectQuery::expr_window_as`], +//! [`SelectQuery::expr_window_name_as`], or raw SQL via [`SelectQuery::window_function_cust`]. +//! +//! Bring in [`sea_query`] types as needed: [`CommonTableExpression`](sea_query::CommonTableExpression), +//! [`WithClause`](sea_query::WithClause), [`WindowStatement`](sea_query::WindowStatement), +//! [`JoinType`](sea_query::JoinType), and [`ExprTrait`](sea_query::ExprTrait) for `.equals` / `.eq` on +//! expressions. use crate::query::column::column_trait::ColumnDefHelper; use crate::query::column::definition::get_static_expr; @@ -14,10 +36,11 @@ use std::rc::Rc; /// Query builder for selecting records /// -/// This is returned by `LifeModelTrait::find()` and can be chained with filters, -/// ordering, pagination, and grouping. +/// Returned by [`LifeModelTrait::find`]. Chain filters, ordering, +/// pagination, scopes, and (optionally) advanced SQL helpers documented in the [module +/// prelude](crate::query::select#default-path-vs-advanced-sql-nothing-is-magic). /// -/// # Example +/// # Example (typical) /// /// ```no_run /// use lifeguard::{SelectQuery, LifeModelTrait, LifeExecutor}; @@ -40,9 +63,10 @@ use std::rc::Rc; /// .all(executor)?; /// ``` /// -/// Following `SeaORM`'s pattern: `SelectQuery` where `E: LifeModelTrait`. -/// The Entity (not Model) is the type parameter, and Model is accessed via -/// the associated type `E::Model`. +/// CTEs with the same executor path: [`SelectQuery::with_cte`]. +/// +/// Following SeaORM-style naming: `SelectQuery` where `E: LifeModelTrait`. The **entity** is the +/// type parameter; the row type is `E::Model`. pub struct SelectQuery where E: LifeModelTrait, @@ -509,11 +533,168 @@ where self } + /// Attach a **`WITH`** clause while keeping this [`SelectQuery`] so [`all`](crate::query::select::SelectQuery::all), + /// [`one`](crate::query::select::SelectQuery::one), loaders, and soft-delete still apply. + /// + /// Wraps [`SelectStatement::with_cte`](sea_query::SelectStatement::with_cte). Prefer this over + /// [`Self::with`], which returns a raw [`sea_query::WithQuery`] outside the lifeguard execution + /// API. + /// + /// # Example + /// + /// Build a [`CommonTableExpression`](sea_query::CommonTableExpression) with the `new` / `table_name` / `query` + /// builder, wrap it in [`WithClause`](sea_query::WithClause), then pass it here. + /// + /// ```no_run + /// use sea_query::{CommonTableExpression, SelectStatement, WithClause}; + /// + /// let mut inner = SelectStatement::default(); + /// inner.column(sea_query::Asterisk).from("other_table"); + /// let mut cte = CommonTableExpression::new(); + /// cte.table_name("picked").query(inner); + /// let wc = WithClause::new().cte(cte.to_owned()).to_owned(); + /// // Then: `MyEntity::find().with_cte(wc)` — still a `SelectQuery`; chain `.all(executor)` etc. + /// ``` + #[must_use] + pub fn with_cte(mut self, clause: C) -> Self + where + C: Into, + { + self.query.with_cte(clause); + self + } + + /// Join the main query to a **subquery** (`JOIN (SELECT …) AS alias ON …`). + /// + /// Wraps [`SelectStatement::join_subquery`](sea_query::SelectStatement::join_subquery). Use + /// [`sea_query::ExprTrait`] (e.g. `.equals`) for the join condition. + /// + /// # Example + /// + /// ```no_run + /// use sea_query::SelectStatement; + /// + /// let mut sq = SelectStatement::default(); + /// sq.column("id").from("inner_t"); + /// // Then: `MyEntity::find().join_subquery( + /// // JoinType::LeftJoin, + /// // sq, + /// // "sub_a", + /// // Expr::col("my_table.id").equals(("sub_a", "id")), + /// // )` + /// ``` + #[must_use] + pub fn join_subquery( + mut self, + join: sea_query::JoinType, + subquery: SelectStatement, + alias: T, + on: C, + ) -> Self + where + T: sea_query::IntoIden, + C: sea_query::IntoCondition, + { + self.query.join_subquery(join, subquery, alias, on); + self + } + + /// Define a named **`WINDOW`** clause (`WINDOW name AS (PARTITION BY …)`). + /// + /// Pair with [`Self::expr_window_name`] / [`Self::expr_window_name_as`], or use [`Self::expr_window`] + /// / [`Self::expr_window_as`] for inline `OVER (…)`. + /// + /// # Example + /// + /// ```no_run + /// use sea_query::WindowStatement; + /// + /// let w = WindowStatement::partition_by("grp"); + /// // Then: `MyEntity::find().window("w", w).expr_window_name_as(Expr::col("my_table.id"), "w", "rn")` + /// ``` + #[must_use] + pub fn window(mut self, name: W, def: sea_query::WindowStatement) -> Self + where + W: sea_query::IntoIden, + { + self.query.window(name, def); + self + } + + /// `SELECT … OVER (window)` with an inline [`WindowStatement`](sea_query::WindowStatement) (no + /// named `WINDOW` clause). Prefer [`Self::window`] + [`Self::expr_window_name_as`] when the same + /// window definition is reused across several columns. + /// + /// # Example + /// + /// ```no_run + /// use sea_query::{Order, WindowStatement}; + /// + /// let w = WindowStatement::partition_by("grp").order_by("ts", Order::Asc); + /// // Then: `MyEntity::find().expr_window(Expr::col("my_table.id"), w)` + /// ``` + #[must_use] + pub fn expr_window(mut self, expr: T, window: sea_query::WindowStatement) -> Self + where + T: Into, + { + self.query.expr_window(expr, window); + self + } + + /// `SELECT … OVER (window) AS alias` with an inline [`WindowStatement`](sea_query::WindowStatement). + /// + /// # Example + /// + /// ```no_run + /// use sea_query::WindowStatement; + /// + /// let w = WindowStatement::partition_by("grp"); + /// // Then: `MyEntity::find().expr_window_as(Expr::col("my_table.id"), w, "rn")` + /// ``` + #[must_use] + pub fn expr_window_as( + mut self, + expr: T, + window: sea_query::WindowStatement, + alias: A, + ) -> Self + where + T: Into, + A: sea_query::IntoIden, + { + self.query.expr_window_as(expr, window, alias); + self + } + + /// `SELECT … OVER window_name` (use after [`Self::window`]). + #[must_use] + pub fn expr_window_name(mut self, expr: T, window_name: W) -> Self + where + T: Into, + W: sea_query::IntoIden, + { + self.query.expr_window_name(expr, window_name); + self + } + + /// `SELECT … OVER window_name AS alias` (use after [`Self::window`]). + #[must_use] + pub fn expr_window_name_as(mut self, expr: T, window_name: W, alias: A) -> Self + where + T: Into, + W: sea_query::IntoIden, + A: sea_query::IntoIden, + { + self.query.expr_window_name_as(expr, window_name, alias); + self + } + /// Add a Common Table Expression (CTE) using WITH clause /// /// CTEs allow you to define temporary named result sets that exist only for the duration of a query. - /// **Note:** This method returns a `WithQuery` which has a different API than `SelectQuery`. - /// You can use `with_query.select()` to continue building the query. + /// **Note:** This method returns a [`sea_query::WithQuery`], not [`SelectQuery`]. For lifeguard + /// execution (`all` / `one`), use [`Self::with_cte`] instead. /// /// # Arguments /// @@ -627,35 +808,18 @@ where /// # Example /// /// ```no_run - /// use lifeguard::SelectQuery; /// use sea_query::Iden; /// - /// # struct UserModel { id: i32, name: String, row_num: i64 }; - /// # impl lifeguard::FromRow for UserModel { - /// # fn from_row(_row: &may_postgres::Row) -> Result { todo!() } - /// # } /// struct RowNumber; /// impl sea_query::Iden for RowNumber { /// fn unquoted(&self) -> &'static str { "row_number" } /// } /// - /// // Add window function to query using custom SQL - /// # let query = UserModel::find(); - /// let query_with_window = query.window_function_cust( - /// "ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary DESC)", - /// Some(RowNumber) - /// ); + /// // Then: `MyEntity::find().window_function_cust( + /// // "ROW_NUMBER() OVER (PARTITION BY department_id ORDER BY salary DESC)", + /// // Some(RowNumber), + /// // )` /// ``` - /// Add a custom window function expression to the SELECT clause - /// - /// # Arguments - /// - /// * `window_expr` - The window function expression as a static string - /// * `alias` - Optional alias for the window function - /// - /// # Returns - /// - /// Returns `Self` for method chaining. #[must_use] pub fn window_function_cust( mut self, @@ -938,4 +1102,58 @@ mod tests { "Should not have CONCAT when no select_as is present. SQL: {sql}" ); } + + #[test] + fn with_cte_prepends_with_clause() { + use sea_query::{ + CommonTableExpression, PostgresQueryBuilder, SelectStatement, WithClause, + }; + + let mut inner = SelectStatement::default(); + inner.column(sea_query::Asterisk).from("cte_source"); + let mut cte = CommonTableExpression::new(); + cte.table_name("my_cte").query(inner); + let wc = WithClause::new().cte(cte.to_owned()).to_owned(); + + let q = SelectQuery::::new().with_cte(wc); + let (sql, _) = q.query.build(PostgresQueryBuilder); + let upper = sql.to_uppercase(); + assert!( + upper.starts_with("WITH"), + "expected WITH prefix, got: {sql}" + ); + } + + #[test] + fn join_subquery_emits_subselect_join() { + use sea_query::{ExprTrait, JoinType, PostgresQueryBuilder, SelectStatement}; + + let mut sq = SelectStatement::default(); + sq.column("id").from("inner_t"); + + let q = SelectQuery::::new().join_subquery( + JoinType::LeftJoin, + sq, + "sub_a", + sea_query::Expr::col("test_table.id").equals(("sub_a", "id")), + ); + let (sql, _) = q.query.build(PostgresQueryBuilder); + let upper = sql.to_uppercase(); + assert!(upper.contains("LEFT JOIN"), "SQL: {sql}"); + assert!(upper.contains("SUB_A"), "SQL: {sql}"); + } + + #[test] + fn window_clause_and_expr_window_name_as() { + use sea_query::{Expr, PostgresQueryBuilder, WindowStatement}; + + let w = WindowStatement::partition_by("grp"); + let q = SelectQuery::::new() + .window("w", w) + .expr_window_name_as(Expr::col("test_table.id"), "w", "rn"); + let (sql, _) = q.query.build(PostgresQueryBuilder); + let upper = sql.to_uppercase(); + assert!(upper.contains("WINDOW"), "SQL: {sql}"); + assert!(upper.contains("OVER"), "SQL: {sql}"); + } } diff --git a/src/query/traits.rs b/src/query/traits.rs index 4e1030ec..7c0db329 100644 --- a/src/query/traits.rs +++ b/src/query/traits.rs @@ -6,7 +6,7 @@ //! //! Generic methods take `executor: &E` where `E: LifeExecutor`. If you hold a trait object as //! `let ex: &dyn LifeExecutor`, pass **`&ex`** (so `E` is `&dyn LifeExecutor`, which implements -//! [`LifeExecutor`](crate::executor::LifeExecutor) via a blanket impl), not `ex` alone. +//! [`LifeExecutor`] via a blanket impl), not `ex` alone. //! //! # Examples //! diff --git a/src/relation/traits.rs b/src/relation/traits.rs index afce9159..288c01b7 100644 --- a/src/relation/traits.rs +++ b/src/relation/traits.rs @@ -355,13 +355,13 @@ where /// Trait for finding related entities from a model instance /// -/// Implementations return static [`RelationDef`](crate::RelationDef) metadata for the edge from +/// Implementations return static [`RelationDef`] metadata for the edge from /// **`Self` (entity)** to **`R` (related entity)**. See that type’s **“Orientation for `Related`”** /// section for how `from_tbl`, `to_tbl`, `from_col`, and `to_col` must be filled for /// `BelongsTo` vs `HasMany`. /// -/// [`FindRelated::find_related`](FindRelated::find_related) and [`LazyLoader::load`](crate::LazyLoader::load) -/// both use [`build_where_condition`](crate::build_where_condition) so the lazy and eager paths stay consistent. +/// [`FindRelated::find_related`] and [`LazyLoader`](crate::relation::lazy::LazyLoader) both use +/// [`build_where_condition`] so the lazy and eager paths stay consistent. pub trait Related where Self: LifeModelTrait, @@ -399,7 +399,16 @@ where /// allowing you to find related entities based on the current model's /// primary key value. /// -/// # Example +/// ## Named scopes +/// +/// [`crate::SelectQuery::scope`] applies only to queries you build on a given root entity. +/// `find_related` returns a [`SelectQuery`] for the **related** table with a `WHERE` from the +/// relation metadata only—it does **not** inherit scopes from a separate `Parent::find().scope(…)` +/// query. Chain [`.scope`](crate::SelectQuery::scope) or [`.filter`](crate::SelectQuery::filter) on +/// the returned query to narrow related rows. See `docs/planning/DESIGN_FIND_RELATED_SCOPES.md` in +/// the repository. +/// +/// ## Example /// /// ```no_run /// use lifeguard::{FindRelated, Related, LifeModelTrait, ModelTrait, LifeExecutor}; diff --git a/src/session/identity_model_cell.rs b/src/session/identity_model_cell.rs new file mode 100644 index 00000000..fe43b6ca --- /dev/null +++ b/src/session/identity_model_cell.rs @@ -0,0 +1,79 @@ +//! Link from a derived [`LifeRecord`] to an identity-map [`Rc`]`<`[`RefCell`]``>` (PRD §9). +//! +//! # Threading and soundness +//! +//! [`SessionIdentityModelCell`] holds a clone of the same [`Rc`] as [`super::ModelIdentityMap`]. That +//! [`Rc`] **must not** be accessed concurrently from multiple OS threads: [`RefCell`] is not +//! [`Sync`], and [`Rc`] reference counts are not atomic in the sense required for safe cross-thread +//! sharing. +//! +//! Lifeguard’s session model is **single-threaded per unit of work** ([`super::Session`] is not +//! [`Send`] because it wraps `Rc>`). [`crate::active_model::ActiveModelTrait`] +//! nonetheless requires [`Send`] on records, so this type uses an **`unsafe impl` [`Send`]** to keep +//! derived [`LifeRecord`] types [`Send`] when `M: Send`. +//! +//! **Contract:** Treat a derived record with an attached session model link like the session itself: +//! only one thread may use the linked [`Rc`] at a time. Before moving a record to another thread, +//! call `detach_session()` on that record (or otherwise end the link) so the [`Rc`] is not used from +//! two threads. If the original thread still holds [`Session`] / the map while another thread uses an +//! attached record, that is **undefined behavior** (data race on [`RefCell`] / [`Rc`]). +//! +//! A type-system–sound alternative is shared storage such as [`Arc`](std::sync::Arc)`<`[`Mutex`](std::sync::Mutex)`>` +//! (or [`RwLock`](std::sync::RwLock)) instead of [`Rc`]/[`RefCell`] for identity-map entries; that would be a +//! larger API and performance change. Note: [`Arc`](std::sync::Arc)`<`[`RefCell`](std::cell::RefCell)`>` is **not** a drop-in +//! `Send` fix — [`RefCell`] is [`!Sync`](Sync), so `Arc>` does not regain `Send` the way `Arc>` does. +//! +//! See **`SECURITY_PROMPT.md`** (§A.3, `SessionIdentityModelCell`) for audit-tracked remediation options. + +use crate::model::ModelTrait; +use std::cell::{BorrowMutError, RefCell}; +use std::fmt; +use std::rc::Rc; + +/// Opaque handle to the identity-map cell used by derived `LifeRecord::attach_session_with_model` (PRD §9). +/// +/// See the **Threading and soundness** section in the module documentation for why this type +/// implements [`Send`] via `unsafe impl` and the **protocol** you must follow to avoid undefined behavior. +#[derive(Clone)] +pub struct SessionIdentityModelCell { + rc: Rc>, +} + +// SAFETY: `Rc` uses non-atomic reference counts; using clones from two OS threads is undefined +// behavior. `RefCell` is `!Sync`, so the shared cell must not be accessed concurrently from multiple +// threads either. +// +// We use `unsafe impl Send` only to satisfy `ActiveModelTrait: Send` on derived records. +// `LifeRecord` stores `Option>`, and `Option: Send` requires `T: Send`, +// so without this impl `LifeRecord` could not be `Send` even when no session link is active. +// +// Runtime invariant (not enforced by the type system): every clone of the inner `Rc` (including the +// `ModelIdentityMap` entry and this cell) must be used from at most one OS thread at a time, matching +// the single-threaded `Session` / map model (PRD §9). Before moving a record that used +// `attach_session_with_model`, call `detach_session` on that record (or otherwise end the link) so no +// other thread can touch the same `Rc`. +// +// Violating this after moving the record is UB (races on `Rc` refcount and/or `RefCell`). A sound +// alternative is `Arc>` (or `RwLock`) for identity-map storage; see module docs. +unsafe impl Send for SessionIdentityModelCell {} + +impl SessionIdentityModelCell { + #[must_use] + pub fn new(rc: &Rc>) -> Self { + Self { + rc: Rc::clone(rc), + } + } + + pub fn replace_with(&self, model: M) -> Result<(), BorrowMutError> { + let mut guard = self.rc.try_borrow_mut()?; + *guard = model; + Ok(()) + } +} + +impl fmt::Debug for SessionIdentityModelCell { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str("SessionIdentityModelCell(..)") + } +} diff --git a/src/session/mod.rs b/src/session/mod.rs index 208a4ee6..fa530ef9 100644 --- a/src/session/mod.rs +++ b/src/session/mod.rs @@ -7,10 +7,15 @@ //! # Dirty keys and flush (U-2) //! //! After you mutate a model through [`Rc`]`<`[`RefCell`]`<…>`>``, call [`ModelIdentityMap::mark_dirty`] -//! with that model’s primary key. [`ModelIdentityMap::flush_dirty`] visits **dirty** entries in +//! with that model’s primary key. If you edit a **`#[derive(LifeRecord)]`** value instead, call +//! [`ModelIdentityMap::mark_dirty_key`] with `record.identity_map_key()?` (all PK columns must be set). +//! [`ModelIdentityMap::flush_dirty`] visits **dirty** entries in //! **lexicographic order of PK fingerprint** (stable, deterministic) and invokes your closure. //! The closure typically builds a `LifeRecord` and calls [`crate::active_model::ActiveModelTrait::update`] //! or `save` — the map does not generate SQL itself. +//! [`ModelIdentityMap::flush_dirty_with_map_key`] walks dirty rows in **lexicographic map-key order** +//! (pending-insert keys first; see [`ModelIdentityMap::register_pending_insert`]) and passes the key so the closure can +//! call `insert` vs `update`. //! //! Flush is **not** implicitly transactional; wrap the executor in [`crate::Transaction`] if you //! need all writes in one database transaction. @@ -22,16 +27,32 @@ //! //! # Pooling (U-4) //! -//! See `docs/planning/DESIGN_SESSION_UOW.md` for how a future pool-bound session relates to -//! [`LifeguardPool`](crate::LifeguardPool). +//! See `docs/planning/DESIGN_SESSION_UOW.md` for pooling (U-4). [`Session`] bundles the map and a +//! sendable [`SessionDirtyNotifier`] for `LifeRecord::attach_session`, `attach_session_with_model`, and `detach_session` (entities with a PK). +//! +//! Flush with [`LifeguardPool`](crate::LifeguardPool) via [`Session::flush_dirty`] and a [`PooledLifeExecutor`](crate::pool::PooledLifeExecutor) (or any [`LifeExecutor`]). For one DB transaction around the flush on a **direct** client, use [`Session::flush_dirty_in_transaction`](crate::session::Session::flush_dirty_in_transaction). For the same on a pool, use [`Session::flush_dirty_in_transaction_pooled`](crate::session::Session::flush_dirty_in_transaction_pooled). //! //! # See also //! //! - Project PRD §9 (session / UoW). +mod identity_model_cell; mod pk; +mod uow; +pub use identity_model_cell::SessionIdentityModelCell; pub use pk::fingerprint_pk_values; +pub use uow::{Session, SessionDirtyNotifier}; + +/// Prefix for [`ModelIdentityMap::register_pending_insert`] keys (not a primary-key fingerprint). +pub const PENDING_INSERT_KEY_PREFIX: &str = "__lg_insert__\x1f"; + +/// `true` when `key` was returned from [`ModelIdentityMap::register_pending_insert`]. +#[inline] +#[must_use] +pub fn is_pending_insert_key(key: &str) -> bool { + key.starts_with(PENDING_INSERT_KEY_PREFIX) +} use crate::active_model::ActiveModelError; use crate::executor::LifeExecutor; @@ -50,8 +71,10 @@ use std::rc::Rc; /// are application-defined). /// /// [`mark_dirty`](Self::mark_dirty) only records keys that already exist in the map (registered -/// rows). Inserts pending only in memory are out of scope — use your normal `insert` path, then -/// [`register_loaded`] after the database assigns keys if you want them in the map. +/// rows). For **insert-only** rows (no stable PK fingerprint yet), use +/// [`register_pending_insert`](Self::register_pending_insert) and flush with +/// [`flush_dirty_with_map_key`](Self::flush_dirty_with_map_key) so the closure can branch on +/// [`is_pending_insert_key`]. pub struct ModelIdentityMap where E: LifeModelTrait, @@ -59,6 +82,8 @@ where { map: HashMap>>, dirty: HashSet, + /// Monotonic id for [`Self::register_pending_insert`] keys (`PENDING_INSERT_KEY_PREFIX` + id). + next_pending_id: u64, } impl ModelIdentityMap @@ -72,9 +97,55 @@ where Self { map: HashMap::new(), dirty: HashSet::new(), + next_pending_id: 0, } } + /// Register a new row that will be **inserted** (no stable PK fingerprint in the map yet). + /// + /// Returns `(map_key, rc)` — keep `map_key` for [`Self::promote_pending_to_loaded`] after a + /// successful insert, and use [`Self::flush_dirty_with_map_key`] in the flush closure to call + /// `insert` when [`is_pending_insert_key`](`map_key`). + /// + /// Synthetic keys are [`PENDING_INSERT_KEY_PREFIX`] plus a decimal id from the map’s internal + /// `next_pending_id` field, advanced with `wrapping_add(1)` each time this method runs. After + /// \(2^{64}\) such calls, ids could theoretically wrap and collide; in practice unreachable. + /// Promote or remove pending entries so `next_pending_id` does not grow without bound in an + /// extremely long-lived, high-throughput process. + pub fn register_pending_insert(&mut self, model: E::Model) -> (String, Rc>) { + let id = self.next_pending_id; + self.next_pending_id = self.next_pending_id.wrapping_add(1); + let key = format!("{PENDING_INSERT_KEY_PREFIX}{id}"); + let rc = Rc::new(RefCell::new(model)); + self.map.insert(key.clone(), rc.clone()); + self.dirty.insert(key.clone()); + (key, rc) + } + + /// After a successful insert, replace the pending entry with the loaded row (real PK). + /// + /// Removes the synthetic pending key, then [`register_loaded`](Self::register_loaded) with + /// `model` (typically with generated PK from the database). + pub fn promote_pending_to_loaded( + &mut self, + pending_key: &str, + model: E::Model, + ) -> Result>, ActiveModelError> { + if !is_pending_insert_key(pending_key) { + return Err(ActiveModelError::Other( + "promote_pending_to_loaded: key is not a pending insert key".to_string(), + )); + } + if !self.map.contains_key(pending_key) { + return Err(ActiveModelError::Other( + "promote_pending_to_loaded: key not in identity map".to_string(), + )); + } + self.map.remove(pending_key); + self.dirty.remove(pending_key); + Ok(self.register_loaded(model)) + } + /// Register a model instance. Same PK → same [`Rc`]; duplicate model is dropped. pub fn register_loaded(&mut self, model: E::Model) -> Rc> { let key = fingerprint_pk_values(&model.get_primary_key_values()); @@ -114,6 +185,15 @@ where } } + /// Mark dirty using a fingerprint string (e.g. from [`crate::session::fingerprint_pk_values`] + /// or a derived [`LifeRecord`](crate::active_model::ActiveModelTrait)’s `identity_map_key()`). + /// No-op if the key is not registered. + pub fn mark_dirty_key(&mut self, key: &str) { + if self.map.contains_key(key) { + self.dirty.insert(key.to_string()); + } + } + /// Remove the dirty flag without persisting. pub fn unmark_dirty(&mut self, model: &E::Model) { let key = fingerprint_pk_values(&model.get_primary_key_values()); @@ -138,12 +218,17 @@ where self.dirty.clear(); } - /// Flush every dirty row in **lexicographic order of PK fingerprint** by calling `f` with the - /// executor and the shared [`Rc`]. On success for a row, its dirty flag is cleared. On the - /// first error, remaining dirty keys are left unchanged (including the failing key). - pub fn flush_dirty(&mut self, executor: &dyn LifeExecutor, mut f: F) -> Result<(), ActiveModelError> + /// Flush every dirty row in **lexicographic order of map key** (pending-insert keys sort under + /// [`PENDING_INSERT_KEY_PREFIX`], then normal PK fingerprints) by calling `f` with the executor, + /// the shared [`Rc`], and the internal map key string (use [`is_pending_insert_key`] to branch + /// insert vs update). + pub fn flush_dirty_with_map_key( + &mut self, + executor: &dyn LifeExecutor, + mut f: F, + ) -> Result<(), ActiveModelError> where - F: FnMut(&dyn LifeExecutor, Rc>) -> Result<(), ActiveModelError>, + F: FnMut(&dyn LifeExecutor, Rc>, &str) -> Result<(), ActiveModelError>, { let mut keys: Vec = self.dirty.iter().cloned().collect(); keys.sort(); @@ -155,7 +240,7 @@ where self.dirty.remove(&key); continue; }; - match f(executor, rc) { + match f(executor, rc, key.as_str()) { Ok(()) => { self.dirty.remove(&key); } @@ -164,6 +249,16 @@ where } Ok(()) } + + /// Flush every dirty row in **lexicographic order of map key** by calling `f` with the executor + /// and the shared [`Rc`]. On success for a row, its dirty flag is cleared. On the first error, + /// remaining dirty keys are left unchanged (including the failing key). + pub fn flush_dirty(&mut self, executor: &dyn LifeExecutor, mut f: F) -> Result<(), ActiveModelError> + where + F: FnMut(&dyn LifeExecutor, Rc>) -> Result<(), ActiveModelError>, + { + self.flush_dirty_with_map_key(executor, |ex, rc, _key| f(ex, rc)) + } } impl Default for ModelIdentityMap @@ -187,6 +282,7 @@ mod tests { use may_postgres::Row; use sea_query::{Iden, IdenStatic, Value}; use std::cell::RefCell; + use std::rc::Rc; struct NopExecutor; @@ -345,6 +441,22 @@ mod tests { assert_eq!(map.dirty_len(), 0); } + #[test] + fn mark_dirty_key_matches_fingerprint() { + let mut map = ModelIdentityMap::::new(); + let _ = map.register_loaded(SessModel { + id: 5, + label: "a", + }); + let key = fingerprint_pk_values(&[Value::Int(Some(5))]); + map.mark_dirty_key(&key); + assert_eq!(map.dirty_len(), 1); + assert!(map.is_marked_dirty(&SessModel { + id: 5, + label: "x", + })); + } + #[test] fn flush_dirty_lexicographic_order() { let mut map = ModelIdentityMap::::new(); @@ -418,4 +530,58 @@ mod tests { label: "x", })); } + + #[test] + #[allow(clippy::expect_used)] // Test code - expect is acceptable + fn register_pending_insert_flush_with_map_key_and_promote() { + let mut map = ModelIdentityMap::::new(); + let (k, _rc) = map.register_pending_insert(SessModel { + id: 0, + label: "new", + }); + assert!(is_pending_insert_key(&k)); + assert_eq!(map.dirty_len(), 1); + let ex = NopExecutor; + let ex_ref: &dyn LifeExecutor = &ex; + map.flush_dirty_with_map_key(ex_ref, |_, _, key| { + assert!(is_pending_insert_key(key)); + Ok(()) + }) + .expect("flush"); + assert_eq!(map.dirty_len(), 0); + assert_eq!(map.len(), 1); + let r2 = map + .promote_pending_to_loaded(&k, SessModel { + id: 42, + label: "saved", + }) + .expect("promote"); + assert_eq!(r2.borrow().id, 42); + assert_eq!(map.len(), 1); + assert!(map + .get_existing(&SessModel { + id: 42, + label: "probe", + }) + .is_some_and(|r| Rc::ptr_eq(&r, &r2))); + } + + #[test] + fn session_identity_model_cell_replace_with_updates_rc() { + let mut map = ModelIdentityMap::::new(); + let rc = map.register_loaded(SessModel { + id: 1, + label: "a", + }); + let cell = SessionIdentityModelCell::new(&rc); + assert!( + cell + .replace_with(SessModel { + id: 1, + label: "b", + }) + .is_ok() + ); + assert_eq!(rc.borrow().label, "b"); + } } diff --git a/src/session/uow.rs b/src/session/uow.rs new file mode 100644 index 00000000..b60b806a --- /dev/null +++ b/src/session/uow.rs @@ -0,0 +1,501 @@ +//! Explicit [`Session`] handle for PRD Phase E (U-3, U-4). +//! +//! [`Session`] wraps a shared [`ModelIdentityMap`](super::ModelIdentityMap) and a **sendable** +//! [`SessionDirtyNotifier`] so derived `LifeRecord` can call `attach_session(&session)` without +//! breaking `Send` (required by [`ActiveModel`](crate::active_model) graph closures). +//! +//! Pending dirty keys are merged into the identity map at [`Session::flush_dirty`] time. +//! +//! # Pooling (U-4) +//! +//! Flush with any [`LifeExecutor`](crate::executor::LifeExecutor), including [`PooledLifeExecutor`](crate::pool::PooledLifeExecutor). +//! For a single transaction across dirty rows on a **direct** [`MayPostgresExecutor`](crate::MayPostgresExecutor), +//! use [`Session::flush_dirty_in_transaction`]. For a [`crate::pool::LifeguardPool`], use +//! [`Session::flush_dirty_in_transaction_pooled`] (pins one primary worker slot and runs `BEGIN` / +//! flush / `COMMIT` on that connection). See `docs/planning/DESIGN_SESSION_UOW.md`. + +use std::cell::RefCell; +use std::collections::HashSet; +use std::fmt; +use std::rc::Rc; +use std::sync::{Arc, Mutex}; + +use crate::active_model::ActiveModelError; +use crate::executor::{LifeExecutor, MayPostgresExecutor}; +use crate::model::ModelTrait; +use crate::pool::LifeguardPool; +use crate::query::LifeModelTrait; + +use super::ModelIdentityMap; + +/// Notifies a [`Session`] that a primary-key fingerprint should be treated as dirty. +/// +/// Cloning shares the same backing queue. Safe to store on a derived `LifeRecord` (`Send` + `Sync`). +/// +/// # Threading / `may` +/// +/// The pending set uses [`std::sync::Mutex`] so this handle can be [`Send`] + [`Sync`] while +/// [`Session`]'s identity map remains `Rc`/`RefCell` on **one** thread/coroutine (PRD U-3). A +/// contended [`Mutex::lock`] blocks the **calling OS thread**; cooperative `may` coroutines on that +/// thread do not run until the lock is released. In the **intended** model (session + attached +/// records on the same thread), contention is rare and the critical section is a small +/// [`HashSet::insert`](std::collections::HashSet::insert). Replacing this with a lock-free structure +/// would still need a defined dedup/ordering story and does not remove blocking from synchronous +/// `flush_dirty` paths that merge into the map. +#[derive(Clone)] +pub struct SessionDirtyNotifier { + pending: Arc>>, +} + +impl fmt::Debug for SessionDirtyNotifier { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("SessionDirtyNotifier").finish_non_exhaustive() + } +} + +impl SessionDirtyNotifier { + pub(crate) fn new(pending: Arc>>) -> Self { + Self { pending } + } + + /// Queue `key` for the next [`Session::flush_dirty`]. No-op if `key` is `None`. + pub fn notify_identity_map_dirty(&self, key: Option) { + let Some(k) = key else { + return; + }; + let mut g = match self.pending.lock() { + Ok(guard) => guard, + Err(poisoned) => poisoned.into_inner(), + }; + g.insert(k); + } +} + +/// Unit-of-work boundary: shared identity map and dirty tracking (explicit; no thread-local — U-3). +/// +/// [`Clone`]: all clones share the same map and pending-dirty queue. The inner map is **not** +/// [`Send`]/[`Sync`] (same as [`ModelIdentityMap`](super::ModelIdentityMap)); use one session per +/// thread / coroutine, or keep records on the same thread as the session. +#[derive(Clone)] +pub struct Session +where + E: LifeModelTrait, + E::Model: ModelTrait + Clone, +{ + inner: Rc>>, + pending_dirty: Arc>>, +} + +impl Default for Session +where + E: LifeModelTrait, + E::Model: ModelTrait + Clone, +{ + fn default() -> Self { + Self::new() + } +} + +impl fmt::Debug for Session +where + E: LifeModelTrait, + E::Model: ModelTrait + Clone, +{ + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self.inner.try_borrow() { + Ok(m) => f + .debug_struct("Session") + .field("len", &m.len()) + .field("dirty_len", &m.dirty_len()) + .finish(), + Err(_) => f.write_str("Session(..)"), + } + } +} + +impl Session +where + E: LifeModelTrait, + E::Model: ModelTrait + Clone, +{ + #[must_use] + pub fn new() -> Self { + Self { + inner: Rc::new(RefCell::new(ModelIdentityMap::new())), + pending_dirty: Arc::new(Mutex::new(HashSet::new())), + } + } + + /// [`SessionDirtyNotifier`] for `LifeRecord::attach_session`. + #[must_use] + pub fn dirty_notifier(&self) -> SessionDirtyNotifier { + SessionDirtyNotifier::new(Arc::clone(&self.pending_dirty)) + } + + fn drain_pending_into_map(&self) { + let keys: Vec = { + let mut p = match self.pending_dirty.lock() { + Ok(guard) => guard, + Err(poisoned) => poisoned.into_inner(), + }; + p.drain().collect() + }; + let mut map = self.inner.borrow_mut(); + for k in keys { + map.mark_dirty_key(&k); + } + } + + /// Register a row already materialized in memory (e.g. from a loader) or returned from + /// [`ActiveModelTrait::insert`](crate::active_model::ActiveModelTrait::insert) / `save` — same + /// as [`ModelIdentityMap::register_loaded`](super::ModelIdentityMap::register_loaded). + pub fn register_loaded(&self, model: E::Model) -> Rc> { + self.inner.borrow_mut().register_loaded(model) + } + + /// Register a new row for **insert-only** flush — same as + /// [`ModelIdentityMap::register_pending_insert`](super::ModelIdentityMap::register_pending_insert). + pub fn register_pending_insert(&self, model: E::Model) -> (String, Rc>) { + self.inner.borrow_mut().register_pending_insert(model) + } + + /// Same as [`ModelIdentityMap::promote_pending_to_loaded`](super::ModelIdentityMap::promote_pending_to_loaded). + pub fn promote_pending_to_loaded( + &self, + pending_key: &str, + model: E::Model, + ) -> Result>, ActiveModelError> { + self.inner.borrow_mut().promote_pending_to_loaded(pending_key, model) + } + + #[must_use] + pub fn get_existing(&self, model: &E::Model) -> Option>> { + self.inner.borrow().get_existing(model) + } + + #[must_use] + pub fn len(&self) -> usize { + self.inner.borrow().len() + } + + #[must_use] + pub fn is_empty(&self) -> bool { + self.inner.borrow().is_empty() + } + + #[must_use] + pub fn dirty_len(&self) -> usize { + self.inner.borrow().dirty_len() + } + + /// Marks dirty on the underlying map (immediate). Prefer [`SessionDirtyNotifier`] when mutating an attached `LifeRecord`. + pub fn mark_dirty(&self, model: &E::Model) { + self.inner.borrow_mut().mark_dirty(model); + } + + pub fn mark_dirty_key(&self, key: &str) { + self.inner.borrow_mut().mark_dirty_key(key); + } + + pub fn clear_dirty(&self) { + let mut p = match self.pending_dirty.lock() { + Ok(guard) => guard, + Err(poisoned) => poisoned.into_inner(), + }; + p.clear(); + self.inner.borrow_mut().clear_dirty(); + } + + /// Merges any [`SessionDirtyNotifier`] keys into the map, then flushes (same semantics as [`ModelIdentityMap::flush_dirty`](super::ModelIdentityMap::flush_dirty)). + pub fn flush_dirty(&self, executor: &dyn LifeExecutor, f: F) -> Result<(), ActiveModelError> + where + F: FnMut(&dyn LifeExecutor, Rc>) -> Result<(), ActiveModelError>, + { + self.drain_pending_into_map(); + self.inner.borrow_mut().flush_dirty(executor, f) + } + + /// Like [`Self::flush_dirty`], but passes each row’s internal map key (see [`super::ModelIdentityMap::flush_dirty_with_map_key`](super::ModelIdentityMap::flush_dirty_with_map_key)). + pub fn flush_dirty_with_map_key(&self, executor: &dyn LifeExecutor, f: F) -> Result<(), ActiveModelError> + where + F: FnMut(&dyn LifeExecutor, Rc>, &str) -> Result<(), ActiveModelError>, + { + self.drain_pending_into_map(); + self.inner.borrow_mut().flush_dirty_with_map_key(executor, f) + } + + /// Like [`Self::flush_dirty`], but runs all persistence callbacks inside **one** PostgreSQL + /// transaction (`BEGIN` → flush → `COMMIT`, or `ROLLBACK` on first error). + /// + /// Uses [`MayPostgresExecutor::begin`](MayPostgresExecutor::begin) (isolation level and + /// `BEGIN` semantics match [`crate::transaction::Transaction`]). For [`LifeguardPool`], prefer + /// [`Self::flush_dirty_in_transaction_pooled`]. + pub fn flush_dirty_in_transaction( + &self, + executor: &MayPostgresExecutor, + mut f: F, + ) -> Result<(), ActiveModelError> + where + F: FnMut(&dyn LifeExecutor, Rc>) -> Result<(), ActiveModelError>, + { + let tx = executor.begin().map_err(|e| { + ActiveModelError::DatabaseError(format!("begin transaction: {e}")) + })?; + let ex: &dyn LifeExecutor = &tx; + match self.flush_dirty(ex, |e, m| f(e, m)) { + Ok(()) => tx.commit().map_err(|e| { + ActiveModelError::DatabaseError(format!("commit: {e}")) + }), + Err(e) => { + if let Err(rb_err) = tx.rollback() { + return Err(ActiveModelError::DatabaseError(format!( + "flush failed: {e}; rollback failed: {rb_err}" + ))); + } + Err(e) + } + } + } + + /// Like [`Self::flush_dirty_in_transaction`], but the closure receives the map key (third argument). + pub fn flush_dirty_in_transaction_with_map_key( + &self, + executor: &MayPostgresExecutor, + mut f: F, + ) -> Result<(), ActiveModelError> + where + F: FnMut(&dyn LifeExecutor, Rc>, &str) -> Result<(), ActiveModelError>, + { + let tx = executor.begin().map_err(|e| { + ActiveModelError::DatabaseError(format!("begin transaction: {e}")) + })?; + let ex: &dyn LifeExecutor = &tx; + match self.flush_dirty_with_map_key(ex, |e, m, k| f(e, m, k)) { + Ok(()) => tx.commit().map_err(|e| { + ActiveModelError::DatabaseError(format!("commit: {e}")) + }), + Err(e) => { + if let Err(rb_err) = tx.rollback() { + return Err(ActiveModelError::DatabaseError(format!( + "flush failed: {e}; rollback failed: {rb_err}" + ))); + } + Err(e) + } + } + } + + /// Like [`Self::flush_dirty_in_transaction`], but uses [`LifeguardPool::exclusive_primary_write_executor`] + /// so every ORM statement runs on **one** pinned primary connection (`BEGIN` / flush / `COMMIT` + /// with default `READ COMMITTED` via raw `BEGIN`/`COMMIT`/`ROLLBACK` SQL). + /// + /// Does **not** use [`crate::pool::PooledLifeExecutor`] round-robin dispatch (which can split + /// statements across workers). + pub fn flush_dirty_in_transaction_pooled( + &self, + pool: &LifeguardPool, + mut f: F, + ) -> Result<(), ActiveModelError> + where + F: FnMut(&dyn LifeExecutor, Rc>) -> Result<(), ActiveModelError>, + { + let exec = pool.exclusive_primary_write_executor().map_err(|e| { + ActiveModelError::DatabaseError(format!("exclusive primary executor: {e}")) + })?; + Self::flush_dirty_with_begin_commit_sql(self, &exec, &mut f) + } + + /// Like [`Self::flush_dirty_in_transaction_pooled`], but the closure receives the map key. + pub fn flush_dirty_in_transaction_pooled_with_map_key( + &self, + pool: &LifeguardPool, + mut f: F, + ) -> Result<(), ActiveModelError> + where + F: FnMut(&dyn LifeExecutor, Rc>, &str) -> Result<(), ActiveModelError>, + { + let exec = pool.exclusive_primary_write_executor().map_err(|e| { + ActiveModelError::DatabaseError(format!("exclusive primary executor: {e}")) + })?; + Self::flush_dirty_with_begin_commit_sql_map_key(self, &exec, &mut f) + } + + fn flush_dirty_with_begin_commit_sql( + session: &Self, + executor: &dyn LifeExecutor, + f: &mut F, + ) -> Result<(), ActiveModelError> + where + F: FnMut(&dyn LifeExecutor, Rc>) -> Result<(), ActiveModelError>, + { + let mut g = + |e: &dyn LifeExecutor, m: Rc>, _k: &str| -> Result<(), ActiveModelError> { + f(e, m) + }; + Self::flush_dirty_with_begin_commit_sql_map_key(session, executor, &mut g) + } + + fn flush_dirty_with_begin_commit_sql_map_key( + session: &Self, + executor: &dyn LifeExecutor, + f: &mut F, + ) -> Result<(), ActiveModelError> + where + F: FnMut(&dyn LifeExecutor, Rc>, &str) -> Result<(), ActiveModelError>, + { + executor.execute("BEGIN", &[]).map_err(|e| { + ActiveModelError::DatabaseError(format!("begin transaction: {e}")) + })?; + match session.flush_dirty_with_map_key(executor, |e, m, k| f(e, m, k)) { + Ok(()) => { + executor.execute("COMMIT", &[]).map_err(|e| { + ActiveModelError::DatabaseError(format!("commit: {e}")) + })?; + Ok(()) + } + Err(e) => { + if let Err(rb_err) = executor.execute("ROLLBACK", &[]) { + return Err(ActiveModelError::DatabaseError(format!( + "flush failed: {e}; rollback failed: {rb_err}" + ))); + } + Err(e) + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::executor::{LifeError, LifeExecutor}; + use crate::model::ModelTrait; + use crate::relation::identity::Identity; + use crate::{LifeEntityName, LifeModelTrait}; + use may_postgres::Row; + use sea_query::{Iden, IdenStatic, Value}; + + struct NopExecutor; + + impl LifeExecutor for NopExecutor { + fn execute(&self, _query: &str, _params: &[&dyn may_postgres::types::ToSql]) -> Result { + Ok(0) + } + + fn query_one( + &self, + _query: &str, + _params: &[&dyn may_postgres::types::ToSql], + ) -> Result { + Err(LifeError::QueryError("nop".into())) + } + + fn query_all( + &self, + _query: &str, + _params: &[&dyn may_postgres::types::ToSql], + ) -> Result, LifeError> { + Ok(vec![]) + } + } + + #[derive(Copy, Clone, Debug)] + enum UCol { + Id, + } + + impl Iden for UCol { + fn unquoted(&self) -> &'static str { + match self { + UCol::Id => "id", + } + } + } + + impl IdenStatic for UCol { + fn as_str(&self) -> &'static str { + match self { + UCol::Id => "id", + } + } + } + + crate::impl_column_def_helper_for_test!(UCol); + + #[derive(Copy, Clone, Debug, Default)] + struct UEnt; + + impl LifeEntityName for UEnt { + fn table_name(&self) -> &'static str { + "u" + } + } + + impl LifeModelTrait for UEnt { + type Model = UMod; + type Column = UCol; + } + + #[derive(Clone, Debug)] + struct UMod { + id: i32, + } + + impl ModelTrait for UMod { + type Entity = UEnt; + + fn get(&self, column: UCol) -> Value { + match column { + UCol::Id => Value::Int(Some(self.id)), + } + } + + fn set(&mut self, column: UCol, value: Value) -> Result<(), crate::model::ModelError> { + match column { + UCol::Id => { + if let Value::Int(Some(v)) = value { + self.id = v; + Ok(()) + } else { + Err(crate::model::ModelError::Other("bad id".into())) + } + } + } + } + + fn get_primary_key_value(&self) -> Value { + Value::Int(Some(self.id)) + } + + fn get_primary_key_identity(&self) -> Identity { + Identity::Unary(sea_query::DynIden::from(UCol::Id.as_str())) + } + + fn get_primary_key_values(&self) -> Vec { + vec![Value::Int(Some(self.id))] + } + } + + #[test] + fn session_register_and_flush_merges_pending() { + let s = Session::::new(); + let n = s.dirty_notifier(); + let _ = s.register_loaded(UMod { id: 1 }); + let key = crate::session::fingerprint_pk_values(&[Value::Int(Some(1))]); + n.notify_identity_map_dirty(Some(key)); + let ex = NopExecutor; + let ex_ref: &dyn LifeExecutor = &ex; + assert_eq!( + s.flush_dirty(ex_ref, |_, _| Ok(())), + Ok(()), + "flush_dirty should succeed" + ); + assert_eq!(s.dirty_len(), 0); + } + + #[test] + fn session_dirty_notifier_is_send() { + fn assert_send() {} + assert_send::(); + } +} diff --git a/tests/db_integration/column_f_update.rs b/tests/db_integration/column_f_update.rs new file mode 100644 index 00000000..709c0387 --- /dev/null +++ b/tests/db_integration/column_f_update.rs @@ -0,0 +1,125 @@ +//! Postgres integration: `ColumnTrait::f_add` on `UPDATE SET` (PRD §8, F-1/F-2). + +use std::sync::Mutex; + +use crate::context::get_test_context; +use lifeguard::executor::LifeError; +use lifeguard::test_helpers::TestDatabase; +use lifeguard::{ + ActiveModelError, ActiveModelTrait, ColumnTrait, LifeExecutor, LifeModelTrait, +}; +use lifeguard_derive::{LifeModel, LifeRecord}; +use sea_query::{PostgresQueryBuilder, Query}; + +static LOCK: Mutex<()> = Mutex::new(()); + +#[derive(LifeModel, LifeRecord, Debug, Clone)] +#[table_name = "lg_f_update_counter"] +pub struct Counter { + #[primary_key] + #[auto_increment] + pub id: i32, + pub n: i32, +} + +fn setup(executor: &dyn lifeguard::LifeExecutor) -> Result<(), LifeError> { + executor.execute("DROP TABLE IF EXISTS lg_f_update_counter CASCADE", &[])?; + executor.execute( + "CREATE TABLE lg_f_update_counter (id SERIAL PRIMARY KEY, n INTEGER NOT NULL DEFAULT 0)", + &[], + )?; + executor.execute("INSERT INTO lg_f_update_counter (n) VALUES (0)", &[])?; + Ok(()) +} + +#[test] +fn f_add_update_increments_column_on_postgres() { + let _guard = LOCK.lock().expect("column_f_update lock"); + + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + let mut q = Query::update(); + q.table(Entity); + q.value( + ::Column::N, + ::Column::N.f_add(1), + ); + q.and_where(::Column::Id.eq(1i32)); + + let (sql, values) = q.build(PostgresQueryBuilder); + let n = executor + .execute_values(&sql, &values) + .expect("execute update"); + + assert_eq!(n, 1, "one row updated"); + + let row = executor + .query_one("SELECT n FROM lg_f_update_counter WHERE id = 1", &[]) + .expect("select"); + let n_after: i32 = row.get(0); + assert_eq!(n_after, 1, "column incremented in database"); +} + +/// `LifeRecord::set_*_expr` + `ActiveModelTrait::update` (F-1 on ORM path). +#[test] +fn record_set_n_expr_update_increments_on_postgres() { + let _guard = LOCK.lock().expect("column_f_update lock"); + + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + let row = executor + .query_one("SELECT id, n FROM lg_f_update_counter WHERE id = 1", &[]) + .expect("select seed row"); + let id: i32 = row.get(0); + let n0: i32 = row.get(1); + assert_eq!((id, n0), (1, 0)); + + let model = CounterModel { id: 1, n: 0 }; + let mut rec = CounterRecord::from_model(&model); + rec.set_n_expr(::Column::N.f_add(1i32)); + + let updated = rec + .update(&executor) + .expect("record update with f_add expr"); + + assert_eq!(updated.n, 1); + + let row = executor + .query_one("SELECT n FROM lg_f_update_counter WHERE id = 1", &[]) + .expect("select after"); + let n_after: i32 = row.get(0); + assert_eq!(n_after, 1); +} + +#[test] +fn insert_rejects_when_set_expr_pending() { + let _guard = LOCK.lock().expect("column_f_update lock"); + + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + let mut rec = CounterRecord::new(); + rec.set_n_expr(::Column::N.f_add(1i32)); + + let err = rec.insert(&executor).expect_err("insert must reject pending __update_exprs"); + match err { + ActiveModelError::Other(msg) => { + assert!( + msg.contains("set_*_expr") || msg.contains("__update_exprs"), + "unexpected message: {msg}" + ); + } + e => panic!("expected Other, got {e:?}"), + } +} diff --git a/tests/db_integration/column_f_where.rs b/tests/db_integration/column_f_where.rs new file mode 100644 index 00000000..0703adbc --- /dev/null +++ b/tests/db_integration/column_f_where.rs @@ -0,0 +1,73 @@ +//! Postgres integration: `ColumnTrait::f_*` in `WHERE` and `ORDER BY` via `Expr::expr` (PRD §8, F-1/F-2). + +use crate::context::get_test_context; +use lifeguard::executor::LifeError; +use lifeguard::test_helpers::TestDatabase; +use lifeguard::{ColumnTrait, LifeExecutor, LifeModelTrait}; +use lifeguard_derive::{LifeModel, LifeRecord}; +use sea_query::{Expr, ExprTrait, Order, PostgresQueryBuilder, Query}; + +#[derive(LifeModel, LifeRecord, Debug, Clone)] +#[table_name = "lg_f_where_counter"] +pub struct Counter { + #[primary_key] + #[auto_increment] + pub id: i32, + pub n: i32, +} + +fn setup(executor: &dyn lifeguard::LifeExecutor) -> Result<(), LifeError> { + executor.execute("DROP TABLE IF EXISTS lg_f_where_counter CASCADE", &[])?; + executor.execute( + "CREATE TABLE lg_f_where_counter (id SERIAL PRIMARY KEY, n INTEGER NOT NULL)", + &[], + )?; + executor.execute( + "INSERT INTO lg_f_where_counter (n) VALUES (4), (6)", + &[], + )?; + Ok(()) +} + +#[test] +fn f_add_in_where_and_order_by_on_postgres() { + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + // Rows: id=1 n=4, id=2 n=6. Condition (n + 1) > 5 → only id=2 (7 > 5). + let mut q_where = Query::select(); + q_where + .column(::Column::Id) + .from(Entity) + .and_where( + Expr::expr(::Column::N.f_add(1i32)).gt(5i32), + ); + + let (sql, values) = q_where.build(PostgresQueryBuilder); + let row = executor + .query_one_values(&sql, &values) + .expect("where query"); + let id: i32 = row.get(0); + assert_eq!(id, 2); + + // Order by (n + 1) DESC: (4+1)=5, (6+1)=7 → first row id=2. + let mut q_order = Query::select(); + q_order + .column(::Column::Id) + .from(Entity) + .order_by_expr( + Expr::expr(::Column::N.f_add(1i32)), + Order::Desc, + ) + .limit(1); + + let (sql, values) = q_order.build(PostgresQueryBuilder); + let row = executor + .query_one_values(&sql, &values) + .expect("order by query"); + let id: i32 = row.get(0); + assert_eq!(id, 2); +} diff --git a/tests/db_integration/pool_read_replica.rs b/tests/db_integration/pool_read_replica.rs index 684d5e32..1e2e96c2 100644 --- a/tests/db_integration/pool_read_replica.rs +++ b/tests/db_integration/pool_read_replica.rs @@ -24,12 +24,26 @@ //! latency, and CI runner load. Cross‑AZ or WAN replicas are **outside** this crate’s control. use lifeguard::test_helpers::TestDatabase; -use lifeguard::{LifeExecutor, LifeguardPool, LifeguardPoolSettings, PooledLifeExecutor}; +use lifeguard::{ + LifeExecutor, LifeguardPool, LifeguardPoolSettings, PooledLifeExecutor, ReadPreference, +}; use sea_query::{Value, Values}; +use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::Arc; use std::thread; use std::time::{Duration, Instant}; +/// Unique schema + table per test so parallel `db_integration` runs do not `DROP` each other's objects. +static POOL_REPLICA_SCHEMA_SEQ: AtomicU64 = AtomicU64::new(0); + +fn unique_pool_replica_schema_names() -> (String, String) { + let n = POOL_REPLICA_SCHEMA_SEQ.fetch_add(1, Ordering::Relaxed); + ( + format!("pool_replica_test_{n}"), + format!("t_pool_replica_smoke_{n}"), + ) +} + const BATCH_LOAD_ROWS: i32 = 64; const BATCH_LOAD_BASE_ID: i32 = 10_000; @@ -111,24 +125,33 @@ fn assert_pooled_pg_is_in_recovery(exec: &PooledLifeExecutor, expect_standby: bo ); } -fn setup_schema_on_primary(executor: &lifeguard::MayPostgresExecutor) { +fn setup_schema_on_primary( + executor: &lifeguard::MayPostgresExecutor, + schema: &str, + table: &str, +) { executor - .execute("CREATE SCHEMA IF NOT EXISTS pool_replica_test", &[]) + .execute( + &format!("CREATE SCHEMA IF NOT EXISTS {schema}"), + &[], + ) .expect("create schema"); executor .execute( - "DROP TABLE IF EXISTS pool_replica_test.t_pool_replica_smoke CASCADE", + &format!("DROP TABLE IF EXISTS {schema}.{table} CASCADE"), &[], ) .expect("drop table"); executor .execute( - r#" - CREATE TABLE pool_replica_test.t_pool_replica_smoke ( + &format!( + r#" + CREATE TABLE {schema}.{table} ( id INTEGER PRIMARY KEY, note TEXT NOT NULL ) - "#, + "# + ), &[], ) .expect("create table"); @@ -143,6 +166,7 @@ fn pooled_pool_construct_write_read_with_replica() { let t_total = Instant::now(); let ctx = crate::context::get_test_context(); let primary_url = ctx.pg_url.clone(); + let (schema, table) = unique_pool_replica_schema_names(); let ep_p = pg_tcp_endpoint_key(&primary_url).expect("parse primary URL host:port"); let ep_r = pg_tcp_endpoint_key(&replica_url).expect("parse replica URL host:port"); @@ -155,7 +179,7 @@ fn pooled_pool_construct_write_read_with_replica() { let t0 = Instant::now(); let mut db = TestDatabase::with_url(&primary_url); let setup_exec = db.executor().expect("primary executor"); - setup_schema_on_primary(&setup_exec); + setup_schema_on_primary(&setup_exec, &schema, &table); log_timing("setup_schema (primary)", t0.elapsed()); let pool_settings = integration_pool_settings(); @@ -180,7 +204,7 @@ fn pooled_pool_construct_write_read_with_replica() { Value::String(Some("via-pool".into())), ]); exec.execute_values( - "INSERT INTO pool_replica_test.t_pool_replica_smoke (id, note) VALUES ($1, $2)", + &format!("INSERT INTO {schema}.{table} (id, note) VALUES ($1, $2)"), &insert_vals, ) .expect("pooled insert"); @@ -228,7 +252,7 @@ fn pooled_pool_construct_write_read_with_replica() { let read_vals = Values(vec![Value::Int(Some(7))]); let row = exec .query_one_values( - "SELECT note FROM pool_replica_test.t_pool_replica_smoke WHERE id = $1", + &format!("SELECT note FROM {schema}.{table} WHERE id = $1"), &read_vals, ) .expect("pooled read"); @@ -245,11 +269,9 @@ fn pooled_pool_construct_write_read_with_replica() { let t6 = Instant::now(); let rep = may_postgres::connect(&replica_url).expect("replica direct connect"); + let direct_note_sql = format!("SELECT note FROM {schema}.{table} WHERE id = 7"); let r2 = rep - .query_one( - "SELECT note FROM pool_replica_test.t_pool_replica_smoke WHERE id = 7", - &[], - ) + .query_one(direct_note_sql.as_str(), &[]) .expect("direct replica read"); let note2: String = r2.get(0); assert_eq!(note2, "via-pool"); @@ -259,7 +281,7 @@ fn pooled_pool_construct_write_read_with_replica() { let t7 = Instant::now(); let batch_hi = BATCH_LOAD_BASE_ID + BATCH_LOAD_ROWS - 1; let batch_sql = format!( - "INSERT INTO pool_replica_test.t_pool_replica_smoke (id, note) \ + "INSERT INTO {schema}.{table} (id, note) \ SELECT g, 'batch-load' FROM generate_series({BATCH_LOAD_BASE_ID}, {batch_hi}) AS g" ); exec.execute_values(&batch_sql, &Values(vec![])) @@ -290,7 +312,7 @@ fn pooled_pool_construct_write_read_with_replica() { let t9 = Instant::now(); let cnt_row = exec .query_one_values( - "SELECT COUNT(*)::bigint AS c FROM pool_replica_test.t_pool_replica_smoke WHERE id >= $1", + &format!("SELECT COUNT(*)::bigint AS c FROM {schema}.{table} WHERE id >= $1"), &Values(vec![Value::Int(Some(BATCH_LOAD_BASE_ID))]), ) .expect("count on replica tier"); @@ -301,6 +323,94 @@ fn pooled_pool_construct_write_read_with_replica() { log_timing("TOTAL (smoke + batch)", t_total.elapsed()); } +/// [`ReadPreference::Primary`] must hit the primary tier even when default routing uses the replica. +#[test] +fn pooled_read_preference_primary_forces_primary_tier() { + let Some(replica_url) = replica_url_or_skip() else { + return; + }; + + let ctx = crate::context::get_test_context(); + let primary_url = ctx.pg_url.clone(); + let (schema, table) = unique_pool_replica_schema_names(); + + let ep_p = pg_tcp_endpoint_key(&primary_url).expect("parse primary URL host:port"); + let ep_r = pg_tcp_endpoint_key(&replica_url).expect("parse replica URL host:port"); + assert_ne!( + ep_p, ep_r, + "TEST_REPLICA_URL must not target the same host:port as TEST_DATABASE_URL" + ); + + let mut db = TestDatabase::with_url(&primary_url); + let setup_exec = db.executor().expect("primary executor"); + setup_schema_on_primary(&setup_exec, &schema, &table); + + let pool = Arc::new( + LifeguardPool::new_with_settings( + &primary_url, + 1, + vec![replica_url.clone()], + 1, + &integration_pool_settings(), + ) + .expect("LifeguardPool::new_with_settings with replica"), + ); + let exec = PooledLifeExecutor::new(pool.clone()); + + let lsn = crate::replication_sync::primary_current_wal_lsn(&primary_url).expect("primary lsn"); + crate::replication_sync::wait_replica_replayed_at_least( + &replica_url, + &lsn, + Duration::from_secs(45), + Duration::from_millis(5), + ) + .expect("replica replay wait"); + + assert!( + crate::replication_sync::postgres_is_in_recovery(&replica_url) + .expect("is_in_recovery query"), + "TEST_REPLICA_URL must be a standby (pg_is_in_recovery)" + ); + + let mut lag_ok = false; + for _ in 0..400 { + if !pool.is_replica_lagging() { + lag_ok = true; + break; + } + thread::sleep(Duration::from_millis(5)); + } + assert!( + lag_ok, + "expected replica not lagging for pool read routing after warmup" + ); + + assert_pooled_pg_is_in_recovery( + &exec, + true, + "default ReadPreference routes to replica tier when healthy", + ); + + let exec_primary = exec + .clone() + .with_read_preference(ReadPreference::Primary); + assert_eq!( + exec_primary.read_preference(), + ReadPreference::Primary + ); + assert_pooled_pg_is_in_recovery( + &exec_primary, + false, + "ReadPreference::Primary must use primary tier", + ); + + assert_pooled_pg_is_in_recovery( + &exec, + true, + "original executor should still use default (replica) routing", + ); +} + #[cfg(test)] mod pg_endpoint_key_tests { use super::pg_tcp_endpoint_key; @@ -351,6 +461,7 @@ fn pooled_read_falls_back_to_primary_when_replica_lagging() { let ctx = crate::context::get_test_context(); let primary_url = ctx.pg_url.clone(); + let (schema, table) = unique_pool_replica_schema_names(); let ep_p = pg_tcp_endpoint_key(&primary_url).expect("parse primary URL host:port"); let ep_r = pg_tcp_endpoint_key(&replica_url).expect("parse replica URL host:port"); @@ -362,7 +473,7 @@ fn pooled_read_falls_back_to_primary_when_replica_lagging() { let mut db = TestDatabase::with_url(&primary_url); let setup_exec = db.executor().expect("primary executor"); - setup_schema_on_primary(&setup_exec); + setup_schema_on_primary(&setup_exec, &schema, &table); const FALLBACK_ID: i32 = 42; let pool_settings = integration_pool_settings(); @@ -383,7 +494,7 @@ fn pooled_read_falls_back_to_primary_when_replica_lagging() { Value::String(Some("primary-fallback".into())), ]); exec.execute_values( - "INSERT INTO pool_replica_test.t_pool_replica_smoke (id, note) VALUES ($1, $2)", + &format!("INSERT INTO {schema}.{table} (id, note) VALUES ($1, $2)"), &insert_vals, ) .expect("pooled insert"); @@ -439,7 +550,7 @@ fn pooled_read_falls_back_to_primary_when_replica_lagging() { let read_vals = Values(vec![Value::Int(Some(FALLBACK_ID))]); let row = exec .query_one_values( - "SELECT note FROM pool_replica_test.t_pool_replica_smoke WHERE id = $1", + &format!("SELECT note FROM {schema}.{table} WHERE id = $1"), &read_vals, ) .expect("pooled read should succeed via primary when replica tier is unavailable"); diff --git a/tests/db_integration/session_identity_flush.rs b/tests/db_integration/session_identity_flush.rs new file mode 100644 index 00000000..7b0e4658 --- /dev/null +++ b/tests/db_integration/session_identity_flush.rs @@ -0,0 +1,323 @@ +//! PRD §9: `ModelIdentityMap` / `Session` + `flush_dirty` / `flush_dirty_in_transaction` with derived `LifeRecord::update`. + +use std::sync::Arc; +use std::sync::Mutex; + +use crate::context::get_test_context; +use lifeguard::executor::LifeError; +use lifeguard::session::{ModelIdentityMap, Session}; +use lifeguard::test_helpers::TestDatabase; +use lifeguard::{ActiveModelTrait, LifeExecutor, LifeguardPool, LifeModelTrait, PooledLifeExecutor}; +use lifeguard_derive::{LifeModel, LifeRecord}; + +static LOCK: Mutex<()> = Mutex::new(()); + +#[derive(LifeModel, LifeRecord, Clone, Debug)] +#[table_name = "lg_sess_flush_counter"] +pub struct Counter { + #[primary_key] + pub id: i32, + pub n: i32, +} + +fn setup(executor: &dyn LifeExecutor) -> Result<(), LifeError> { + executor.execute("DROP TABLE IF EXISTS lg_sess_flush_counter CASCADE", &[])?; + executor.execute( + "CREATE TABLE lg_sess_flush_counter (id INTEGER PRIMARY KEY, n INTEGER NOT NULL)", + &[], + )?; + executor.execute( + "INSERT INTO lg_sess_flush_counter (id, n) VALUES (1, 0)", + &[], + )?; + Ok(()) +} + +#[test] +fn identity_map_flush_dirty_persists_via_update() { + let _guard = LOCK.lock().expect("session_identity_flush lock"); + + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + let mut map = ModelIdentityMap::::new(); + let rc = map.register_loaded(CounterModel { id: 1, n: 0 }); + rc.borrow_mut().n = 8; + + map.mark_dirty(&CounterModel { id: 1, n: 0 }); + + map.flush_dirty(&executor, |ex, mrc| { + let model = mrc.borrow().clone(); + let rec = CounterRecord::from_model(&model); + let _ = rec.update(ex)?; + Ok(()) + }) + .expect("flush_dirty"); + + let row = executor + .query_one("SELECT n FROM lg_sess_flush_counter WHERE id = 1", &[]) + .expect("select"); + let n: i32 = row.get(0); + assert_eq!(n, 8); +} + +#[test] +fn identity_map_flush_dirty_with_mark_dirty_key_and_identity_map_key() { + let _guard = LOCK.lock().expect("session_identity_flush lock"); + + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + let mut map = ModelIdentityMap::::new(); + let rc = map.register_loaded(CounterModel { id: 1, n: 0 }); + rc.borrow_mut().n = 15; + + let key = CounterRecord::from_model(&rc.borrow()) + .identity_map_key() + .expect("identity_map_key"); + map.mark_dirty_key(&key); + + map.flush_dirty(&executor, |ex, mrc| { + let model = mrc.borrow().clone(); + let rec = CounterRecord::from_model(&model); + let _ = rec.update(ex)?; + Ok(()) + }) + .expect("flush_dirty"); + + let row = executor + .query_one("SELECT n FROM lg_sess_flush_counter WHERE id = 1", &[]) + .expect("select"); + let n: i32 = row.get(0); + assert_eq!(n, 15); +} + +#[test] +fn session_flush_dirty_after_attach_session_and_set_n_on_record() { + let _guard = LOCK.lock().expect("session_identity_flush lock"); + + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + let session = Session::::new(); + let rc = session.register_loaded(CounterModel { id: 1, n: 0 }); + + let mut rec = CounterRecord::from_model(&rc.borrow()); + rec.attach_session_with_model(&session, &rc); + rec.set_n(42); + + session + .flush_dirty(&executor, |ex, mrc| { + let model = mrc.borrow().clone(); + let r = CounterRecord::from_model(&model); + let _ = r.update(ex)?; + Ok(()) + }) + .expect("flush_dirty"); + + let row = executor + .query_one("SELECT n FROM lg_sess_flush_counter WHERE id = 1", &[]) + .expect("select"); + let n: i32 = row.get(0); + assert_eq!(n, 42); +} + +#[test] +fn session_flush_dirty_in_transaction_persists_via_update() { + let _guard = LOCK.lock().expect("session_identity_flush lock"); + + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + let session = Session::::new(); + let rc = session.register_loaded(CounterModel { id: 1, n: 0 }); + rc.borrow_mut().n = 99; + session.mark_dirty(&CounterModel { id: 1, n: 0 }); + + session + .flush_dirty_in_transaction(&executor, |ex, mrc| { + let model = mrc.borrow().clone(); + let rec = CounterRecord::from_model(&model); + let _ = rec.update(ex)?; + Ok(()) + }) + .expect("flush_dirty_in_transaction"); + + let row = executor + .query_one("SELECT n FROM lg_sess_flush_counter WHERE id = 1", &[]) + .expect("select"); + let n: i32 = row.get(0); + assert_eq!(n, 99); +} + +#[test] +fn session_flush_dirty_in_transaction_pooled_persists_via_update() { + let _guard = LOCK.lock().expect("session_identity_flush lock"); + + let ctx = get_test_context(); + let pool = Arc::new( + LifeguardPool::new(&ctx.pg_url, 1, vec![], 0).expect("LifeguardPool primary-only"), + ); + let setup_ex = PooledLifeExecutor::new(pool.clone()); + setup(&setup_ex).expect("setup"); + + let session = Session::::new(); + let rc = session.register_loaded(CounterModel { id: 1, n: 0 }); + rc.borrow_mut().n = 77; + session.mark_dirty(&CounterModel { id: 1, n: 0 }); + + session + .flush_dirty_in_transaction_pooled(&pool, |ex, mrc| { + let model = mrc.borrow().clone(); + let rec = CounterRecord::from_model(&model); + let _ = rec.update(ex)?; + Ok(()) + }) + .expect("flush_dirty_in_transaction_pooled"); + + let verify = PooledLifeExecutor::new(pool); + let row = verify + .query_one("SELECT n FROM lg_sess_flush_counter WHERE id = 1", &[]) + .expect("select"); + let n: i32 = row.get(0); + assert_eq!(n, 77); +} + +/// Pending-insert keys (`register_pending_insert`) + `flush_dirty_with_map_key` + `promote_pending_to_loaded` +/// (PRD Phase E insert-only flush path). +#[test] +fn identity_map_pending_insert_flush_and_promote_persists_on_postgres() { + let _guard = LOCK.lock().expect("session_identity_flush lock"); + + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + let mut map = ModelIdentityMap::::new(); + let (pending_key, rc) = map.register_pending_insert(CounterModel { + id: 0, + n: 33, + }); + + map.flush_dirty_with_map_key(&executor, |ex, mrc, key| { + assert!(lifeguard::is_pending_insert_key(key)); + let mut m = mrc.borrow().clone(); + m.id = 2; + *mrc.borrow_mut() = m.clone(); + let rec = CounterRecord::from_model(&m); + let inserted = rec.insert(ex)?; + *mrc.borrow_mut() = inserted; + Ok(()) + }) + .expect("flush_dirty_with_map_key"); + + map.promote_pending_to_loaded(&pending_key, rc.borrow().clone()) + .expect("promote_pending_to_loaded"); + + let row = executor + .query_one("SELECT n FROM lg_sess_flush_counter WHERE id = 2", &[]) + .expect("select"); + let n: i32 = row.get(0); + assert_eq!(n, 33); + + assert_eq!(map.dirty_len(), 0); + assert!(map + .get_existing(&CounterModel { id: 2, n: 0 }) + .is_some()); +} + +#[test] +fn session_pending_insert_flush_in_transaction_with_map_key_persists_on_postgres() { + let _guard = LOCK.lock().expect("session_identity_flush lock"); + + let ctx = get_test_context(); + let mut db = TestDatabase::with_url(&ctx.pg_url); + let executor = db.executor().expect("executor"); + + setup(&executor).expect("setup"); + + let session = Session::::new(); + let (pending_key, rc) = session.register_pending_insert(CounterModel { + id: 0, + n: 44, + }); + + session + .flush_dirty_in_transaction_with_map_key(&executor, |ex, mrc, key| { + assert!(lifeguard::is_pending_insert_key(key)); + let mut m = mrc.borrow().clone(); + m.id = 2; + *mrc.borrow_mut() = m.clone(); + let rec = CounterRecord::from_model(&m); + let inserted = rec.insert(ex)?; + *mrc.borrow_mut() = inserted; + Ok(()) + }) + .expect("flush_dirty_in_transaction_with_map_key"); + + session + .promote_pending_to_loaded(&pending_key, rc.borrow().clone()) + .expect("promote_pending_to_loaded"); + + let row = executor + .query_one("SELECT n FROM lg_sess_flush_counter WHERE id = 2", &[]) + .expect("select"); + let n: i32 = row.get(0); + assert_eq!(n, 44); +} + +#[test] +fn session_pending_insert_flush_in_transaction_pooled_with_map_key_persists_on_postgres() { + let _guard = LOCK.lock().expect("session_identity_flush lock"); + + let ctx = get_test_context(); + let pool = Arc::new( + LifeguardPool::new(&ctx.pg_url, 1, vec![], 0).expect("LifeguardPool primary-only"), + ); + let setup_ex = PooledLifeExecutor::new(pool.clone()); + setup(&setup_ex).expect("setup"); + + let session = Session::::new(); + let (pending_key, rc) = session.register_pending_insert(CounterModel { + id: 0, + n: 55, + }); + + session + .flush_dirty_in_transaction_pooled_with_map_key(&pool, |ex, mrc, key| { + assert!(lifeguard::is_pending_insert_key(key)); + let mut m = mrc.borrow().clone(); + m.id = 2; + *mrc.borrow_mut() = m.clone(); + let rec = CounterRecord::from_model(&m); + let inserted = rec.insert(ex)?; + *mrc.borrow_mut() = inserted; + Ok(()) + }) + .expect("flush_dirty_in_transaction_pooled_with_map_key"); + + session + .promote_pending_to_loaded(&pending_key, rc.borrow().clone()) + .expect("promote_pending_to_loaded"); + + let verify = PooledLifeExecutor::new(pool); + let row = verify + .query_one("SELECT n FROM lg_sess_flush_counter WHERE id = 2", &[]) + .expect("select"); + let n: i32 = row.get(0); + assert_eq!(n, 55); +} diff --git a/tests/db_integration_suite.rs b/tests/db_integration_suite.rs index 141ba66b..7e45ceae 100644 --- a/tests/db_integration_suite.rs +++ b/tests/db_integration_suite.rs @@ -27,6 +27,15 @@ mod replication_sync; #[path = "db_integration/active_model_crud.rs"] mod active_model_crud; +#[path = "db_integration/column_f_update.rs"] +mod column_f_update; + +#[path = "db_integration/column_f_where.rs"] +mod column_f_where; + +#[path = "db_integration/session_identity_flush.rs"] +mod session_identity_flush; + #[path = "db_integration/active_model_graph.rs"] mod active_model_graph;