feat(cli): add export/compact/check subcommands (#18)#29
Merged
Conversation
Adds three new datawal-cli subcommands that round out the 0.1.x CLI
surface without touching the source store on disk.
- export STORE OUTFILE Live KV projection -> JSONL via
DataWal::export_jsonl. Refuses to
clobber an existing outfile (exit 1).
- compact STORE TARGET Snapshot-style rebuild into TARGET
via DataWal::compact_to. Refuses
non-empty target (exit 1).
- check STORE DataWal-level health: open + get every
live key (per-record CRC revalidation)
+ RecoveryReport. Exit 3 on hard get
failure; exit 2 on tail truncation
/ mid-stream error.
Source store on disk is never modified by any subcommand. cli.rs
top-doc and commands.rs top-doc reframed accordingly: subcommands
are now in two groups (read-only inspection vs source-untouched
mutations). Public CLI binary scope is unchanged otherwise.
JSON schema 'datawal.cli.v1' additively extended with three new
kinds: 'export', 'compact', 'check' (additive, non-breaking).
Tests: +6 integration tests in tests/integration.rs (33 total,
0 failures). Local validation: cargo fmt --check, cargo clippy -D
warnings, cargo test --workspace, cargo doc -D warnings, cargo
bench --no-run, cargo publish --dry-run -p datawal — all green.
Closes #18 (CLI-2 remainder; CLI-1 closed by PR #24).
Refs #7.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three new
datawal-clisubcommands that round out the 0.1.x CLI surface without touching the source store on disk.export STORE OUTFILEDataWal::export_jsonl. Refuses to clobber an existingOUTFILE(exit 1).compact STORE TARGETTARGETviaDataWal::compact_to. Refuses non-empty target (exit 1).check STOREDataWal::open+getevery live key (per-record CRC revalidation through the fd pool) +RecoveryReport. Exit 3 on hardgetfailure; exit 2 on tail truncation / mid-stream error.cli.rstop-doc andcommands.rstop-doc are reframed accordingly: subcommands are now in two groups — read-only inspection (scan,get,report,verify,dump,check) and source-untouched mutations (export,compact). Noput/delete/rotatein 0.1.x.What stays the same
datawal; positional store path;--jsonglobal flag.datawal.cli.v1is unchanged. Three newkinds (export,compact,check) are additive — no field renames, no encoding changes.DataWal::open/RecordLog::open.1now also coversexport/compactclobber refusals;3now also covers per-recordgetfailures duringcheck.query,select,where,index,server) remain unused.Tests
+6 integration tests in
crates/datawal-cli/tests/integration.rs(33 total, 0 failures):export_writes_one_jsonl_line_per_live_keyexport_refuses_to_overwrite_existing_outfilecompact_writes_to_empty_target_and_emits_stats(also reopens compacted target and confirms both live keys resolve)compact_refuses_non_empty_targetcheck_clean_store_succeedscheck_detects_tail_truncation(appends 8 junk bytes to the active segment, asserts exit 2 + non-zerotail_truncated)Local validation
All six gates green on
feat/cli-export:cargo fmt --all -- --checkcargo clippy --workspace --all-targets -- -D warningscargo test --workspace --all-targets(full workspace; CLI suite: 33 passed)RUSTDOCFLAGS='-D warnings' cargo doc --workspace --no-depscargo bench --workspace --no-runcargo publish --dry-run -p datawal --allow-dirtyCloses / Refs
Closes #18 (CLI-2 remainder; CLI-1 was closed by PR #24).
Refs #7.