Skip to content

fix(recorder): path-idempotent storage seeding + name->path lookups - #585

Merged
badbread merged 3 commits into
mainfrom
fix/storage-path-idempotent-seed
Aug 8, 2026
Merged

fix(recorder): path-idempotent storage seeding + name->path lookups#585
badbread merged 3 commits into
mainfrom
fix/storage-path-idempotent-seed

Conversation

@badbread

@badbread badbread commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Fixes #584.

Problem

db::upsert_storage is idempotent by storage name only
(ON CONFLICT (name) DO UPDATE SET path). The recorder seeds the two configured
storages on every boot (main.rs::seed_storages), and the seed binary does the same
at container start, keyed on LIVE_STORAGE_NAME / ARCHIVE_STORAGE_NAME. After an
operator renames a storage in the console (PUT /config/storages/{id}), no row carries
the configured name, so the next boot INSERTs a second, empty row for the same directory:
the duplicate Live / Archive rows sitting next to the real renamed rows.

PR #557 added a warn-only detector (duplicate_path_under_other_name) but deliberately
did not prevent the duplicate. Its revisit trigger (the duplicates actually materialize
and confuse the operator) has now fired. The ghost rows are also not inert: the recorder
resolves runtime defaults by config name (the free-space-floor fallback in
archive.rs, live/archive stage labelling in reconcile.rs), so once the real rows are
renamed those lookups miss and silently degrade.

Change

  • Promote the detector from warn to a gate. Both seed paths now share one decision
    function, db::seed_storage_path_idempotent: retarget a same-named row's path; else if
    a row already covers the path under another name, skip the insert (never create a
    duplicate) and return the covering row; else insert (fresh install). It never renames
    or deletes the operator's row, and folds the resolved row into the in-pass set so a
    shared live == archive directory resolves to one row.
  • Add db::get_storage_by_path (deterministic: oldest created_at wins on a shared path)
    and db::get_storage_by_name_or_path (name first, then path), and use the name-or-path
    variant at the two runtime sites so a renamed install keeps full behavior and, after the
    empty rows are cleaned up, the paths still resolve to the real rows.
  • Path comparison normalizes trailing slashes (normalize_storage_path).
  • No migration, no UNIQUE(path): existing DBs already carry duplicate paths and a shared
    live == archive layout is legal.
  • docs/DECISIONS.md entry added, superseding fix: correct release-prep copy defects and code/compose config drift #557's detect-never-fix stance.

Why this cannot orphan footage

Existing segments keep their storage_id, the default policy keeps its live_storage_id,
ensure_default_policy wires the default policy by path, and the fix only ever skips
creating a new empty row
— it never mutates or removes any storage row. Referenced
storage rows are further protected by the ON DELETE RESTRICT FK on segments.storage_id.

Tests

DB-backed tests (gated on TEST_DATABASE_URL; confirmed they actually run on a throwaway
Postgres, not skipped):

  • seed_is_path_idempotent_against_renamed_rows (the exact fix: correct release-prep copy defects and code/compose config drift #557 scenario run twice: still
    two rows, operator names untouched, no growth);
  • seed_fresh_db_creates_both_defaults;
  • seed_retargets_existing_name_in_place;
  • seed_shared_layout_yields_one_row;
  • get_storage_by_path_prefers_oldest (determinism + trailing-slash);
  • get_storage_by_name_or_path_falls_back_to_path (the free-floor / labelling fallback).

Plus pure-function tests for normalize_storage_path and the trailing-slash-aware
detector.

Gate on the build box: cargo fmt --all -- --check, cargo clippy --all-targets -D warnings, and cargo test --workspace against a throwaway Postgres all green.

Follow-up (not in this PR)

The one-time cleanup of the two already-present empty duplicate rows on an affected
deployment is a maintainer DB action, sequenced after this deploys. A referenced row
cannot be deleted (FK RESTRICT), and the empty ghosts are unreferenced, so the cleanup
is a small, verified transactional DELETE.

Boot seeding (recorder main + the seed binary) upserted the two configured
storages by NAME only. After an operator renames a storage in the console
(PUT /config/storages/{id}), no row carries the configured *_STORAGE_NAME, so
the next boot INSERTs a second, empty row for the same directory: the duplicate
Live/Archive rows that showed up next to the real 2TB NVMe / 16TB Spinner rows.

PR #557 added a warn-only detector (duplicate_path_under_other_name) and
deliberately did nothing else; its revisit trigger (the duplicates actually
materialize and confuse the operator) has now fired. The ghost rows are also not
inert: the recorder resolves runtime defaults by config NAME (archive.rs
free-space-floor fallback, reconcile.rs live/archive stage labelling), so once
the real rows are renamed those lookups miss and silently degrade.

- Promote the detector from warn to a GATE. Both seed paths now share one
  decision function, db::seed_storage_path_idempotent: retarget a same-named
  row's path; else if a row already covers the path under another name, SKIP the
  insert (never create a duplicate); else insert (fresh install). It never
  renames or deletes the operator's row, and folds the resolved row into the
  in-pass set so a shared live==archive directory resolves to one row.
- Add db::get_storage_by_path (deterministic: oldest created_at wins on a shared
  path) + db::get_storage_by_name_or_path (name first, then path), and use the
  name-or-path variant at the two runtime sites so a renamed install keeps full
  behavior and, after the empty rows are cleaned up, the paths still resolve to
  the real rows.
- Path comparison normalizes trailing slashes (normalize_storage_path).
- No migration, no UNIQUE(path): existing DBs already carry duplicate paths and
  a shared live==archive layout is legal.

Tests (DB-backed, gated on TEST_DATABASE_URL): the exact #557 scenario run twice
(still 2 rows, operator names untouched, no growth), fresh-install two rows,
same-name retarget, shared-layout one row, get_storage_by_path oldest-wins +
trailing-slash, and the name->path free-floor/labelling fallback; plus pure
tests for normalize_storage_path and the trailing-slash-aware detector.

Cannot orphan footage: existing segments keep their storage_id, the default
policy keeps live_storage_id, ensure_default_policy wires by PATH, and the fix
only ever skips creating a NEW empty row. DECISIONS.md entry added (supersedes
#557's detect-never-fix). The one-time cleanup of the two already-present empty
rows remains a maintainer DB action, sequenced after this deploys.

Signed-off-by: badbread <badbread@users.noreply.github.com>
Signed-off-by: badbread <badbread@users.noreply.github.com>
@badbread
badbread merged commit 7969224 into main Aug 8, 2026
8 checks passed
@badbread
badbread deleted the fix/storage-path-idempotent-seed branch August 8, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Renamed storage causes duplicate empty storage rows on next boot (name-only seed idempotency)

1 participant