Skip to content

ps-dbgen-04-cli: implement tetra3-gen-db CLI and reproducibility fixes#81

Merged
bryantharpe merged 2 commits into
mainfrom
polecat/furiosa/ps-dbgen-04-cli@mrtv1snc
Jul 21, 2026
Merged

ps-dbgen-04-cli: implement tetra3-gen-db CLI and reproducibility fixes#81
bryantharpe merged 2 commits into
mainfrom
polecat/furiosa/ps-dbgen-04-cli@mrtv1snc

Conversation

@bryantharpe

Copy link
Copy Markdown
Owner

Implements the tetra3-gen-db command-line binary for the database-generation crate and fixes the non-determinism that prevented byte-identical output across runs.

Changes

  • Add clap-based tetra3-gen-db binary mirroring the reference CLI:
    • Positional STAR_CATALOG and SAVE_AS
    • --max-fov, --min-fov, --linear-probe, --epoch-proper-motion, --star-max-magnitude
    • Tuning knobs: --verification-stars-per-fov, --lattice-field-oversampling, --patterns-per-lattice-field, --pattern-max-error, --multiscale-step
  • Re-export propagate and HIP_TYC_PM_ORIGIN from database_generation.
  • Fix BSC5 parser to match upstream tetra3 layout (STARN at offset 8, 32-byte entries with f32 id, f64 RA/Dec, i16 spectral type, i16 mag, f32 proper motions in radians/year).
  • Pin ZIP last_modified_time to 2000-01-01 for reproducible .npz archives.
  • Sort deduplicated patterns before hash-table insertion to remove HashSet iteration-order non-determinism.
  • Update catalog_parsing tests for the corrected BSC5 layout.

Verification

  • cargo test -p database-generation --locked passes.
  • cargo clippy -p database-generation --all-targets --locked is clean.
  • cargo deny check passes.
  • Two consecutive CLI runs on reference-solutions/cedar-solve/tests/data/bsc5 produce byte-identical .npz files.

🤖 Generated with Claude Code

- Add clap-based tetra3-gen-db binary mirroring the reference CLI:
  positional STAR_CATALOG and SAVE_AS, --max-fov, --min-fov,
  --linear-probe, --epoch-proper-motion, --star-max-magnitude,
  and tuning knobs for lattice-field oversampling, patterns-per-field,
  pattern-max-error, and multiscale-step.
- Re-export propagate and HIP_TYC_PM_ORIGIN from database_generation lib.
- Fix BSC5 parser to match upstream tetra3 layout: STARN at offset 8,
  32-byte entries with f32 id, f64 RA/Dec, i16 spectral type, i16 mag,
  and f32 proper motions in radians/year.
- Pin zip last_modified_time to 2000-01-01 for byte-identical .npz output.
- Sort deduplicated patterns before hash-table insertion to remove
  HashSet iteration-order non-determinism.
- Update catalog_parsing tests for the corrected BSC5 layout.

Two consecutive CLI runs on the reference bsc5 fixture now produce
byte-identical .npz files.

Co-Authored-By: Claude <noreply@anthropic.com>
@bryantharpe

Copy link
Copy Markdown
Owner Author

review/judge — APPROVE on b50ecd37ea2b

VERDICT: APPROVE
BLOCKING: NONE
NITS: crates/database-generation/src/main.rs:172 — when --epoch-proper-motion none is passed, epoch_proper_motion becomes None and the DatabaseProperties.epoch_proper_motion falls back to pm_origin (e.g. 2000.0), which may misrepresent the database metadata since no propagation was actually performed; consider using a sentinel like 0.0 to signal "no propagation."
NITS: crates/database-generation/src/main.rs:248 — name.starts_with("tyc") and name.starts_with("hip") are redundant with the preceding name.contains("tyc") / name.contains("hip") checks since starts-with implies contains.
NITS: crates/database-generation/src/main.rs:155 — config is constructed with epoch_proper_motion but that field is never read from config afterward; only config.star_max_magnitude is used, so the epoch_proper_motion field in the local GenerationConfig is dead in this function.


Independent U3 review by the judge role (glm-5.2, Zhipu — different lineage
than every author seat). Verdict is on blocking findings, not a score. To
supersede with human sign-off (U4): judge-pr.sh override 81 "<reason>".

@bryantharpe
bryantharpe enabled auto-merge (squash) July 21, 2026 01:30
@bryantharpe bryantharpe added the hold:human operator hold: automation must not review, merge, undraft or escalate this PR label Jul 21, 2026
@bryantharpe
bryantharpe disabled auto-merge July 21, 2026 01:31
@bryantharpe
bryantharpe enabled auto-merge (squash) July 21, 2026 04:26
@bryantharpe
bryantharpe disabled auto-merge July 21, 2026 04:27
…ty test

Judge nits from `review/judge` on b50ecd3:

- main.rs:248 — dropped `starts_with("tyc")`/`starts_with("hip")`; they are
  subsumed by the preceding `contains` checks.
- main.rs:155 — `GenerationConfig::epoch_proper_motion` was written and never
  read. The properties block now reads it from `config` instead of the local,
  so the field is live rather than decorative.
- main.rs:172 — NOT applied, deliberately. The nit proposed a 0.0 sentinel when
  `--epoch-proper-motion none` disables propagation. Falling back to `pm_origin`
  is correct: with propagation disabled the positions ARE at the catalog's origin
  epoch, so that is what the properties must report. It is also what the reference
  does (tetra3.py: "If pm propagation was disabled, set end date to origin"), and
  the spec requires reference parity. A sentinel would describe the data less
  accurately, not more. Comment added so this is not re-raised.

Also adds the acceptance test for `Deterministic, offline generation`, which the
implementation claimed but nothing verified:

- `reproducible_build` — runs the real binary twice over a deterministic synthetic
  BSC5 catalog and byte-compares, with a size floor so two empty files cannot pass
  vacuously.
- `archive_timestamps_are_pinned_not_wall_clock` — the two-run compare has a blind
  spot: runs 10 ms apart share a clock tick, so a clock-dependent archive still
  compares equal. Measured — re-deriving the ZIP mtime from `SystemTime::now()` at
  one-second granularity left `reproducible_build` passing. This asserts the pinned
  2000-01-01 stamp in every local file header, where run duration cannot hide it.
- `byte_comparison_can_detect_a_difference` — calibration; `--linear-probe` must
  change the bytes, or the equality assertion is not capable of failing.

All three calibrated against known-positives rather than trusting exit codes: a
nanosecond-varying mtime makes `reproducible_build` fail, and a wall-clock mtime
makes the timestamp test fail while the byte compare passes blind.

cargo fmt / clippy -D warnings / 140 workspace tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qz1AHuSpjFqRYJWVVVq9mp
@bryantharpe bryantharpe added integrity:cleared OPERATOR ONLY: a human ruled this authoring-integrity finding acceptable and removed hold:human operator hold: automation must not review, merge, undraft or escalate this PR labels Jul 21, 2026
@bryantharpe

Copy link
Copy Markdown
Owner Author

review/judge — APPROVE on 53f5892d247c

VERDICT: APPROVE
BLOCKING: NONE
NITS: crates/database-generation/src/main.rs:173 — config is constructed with epoch_proper_motion that duplicates the local epoch_proper_motion variable; consider using the local directly for properties to avoid two sources of truth for the same value.
NITS: crates/database-generation/src/catalog.rs:83 — f32::from_le_bytes(...) as u32 silently truncates/rounds if the f32 catalog number exceeds 2²⁴; a comment noting the safe range would aid future readers, though BSC5 IDs are well within it.
NITS: crates/database-generation/src/main.rs:248 — catalog_name lowercases stem twice in the match arms (once per arm via to_ascii_lowercase()); a single let stem_lower = stem.to_ascii_lowercase() before the match would be cleaner.


Independent U3 review by the judge role (glm-5.2, Zhipu — different lineage
than every author seat). Verdict is on blocking findings, not a score. To
supersede with human sign-off (U4): judge-pr.sh override 81 "<reason>".

@bryantharpe
bryantharpe merged commit b1fcaa1 into main Jul 21, 2026
23 of 37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integrity:cleared OPERATOR ONLY: a human ruled this authoring-integrity finding acceptable

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant