Skip to content

feat(cli): add export/compact/check subcommands (#18)#29

Merged
robertoberto merged 1 commit into
mainfrom
feat/cli-export
May 21, 2026
Merged

feat(cli): add export/compact/check subcommands (#18)#29
robertoberto merged 1 commit into
mainfrom
feat/cli-export

Conversation

@robertoberto
Copy link
Copy Markdown
Contributor

Summary

Adds three new datawal-cli subcommands that round out the 0.1.x CLI surface without touching the source store on disk.

Subcommand Synopsis Source on disk
export STORE OUTFILE Live KV projection → JSONL via DataWal::export_jsonl. Refuses to clobber an existing OUTFILE (exit 1). not modified
compact STORE TARGET Snapshot-style rebuild into TARGET via DataWal::compact_to. Refuses non-empty target (exit 1). not modified
check STORE DataWal::open + get every live key (per-record CRC revalidation through the fd pool) + RecoveryReport. Exit 3 on hard get failure; exit 2 on tail truncation / mid-stream error. not modified

cli.rs top-doc and commands.rs top-doc are reframed accordingly: subcommands are now in two groups — read-only inspection (scan, get, report, verify, dump, check) and source-untouched mutations (export, compact). No put / delete / rotate in 0.1.x.

What stays the same

  • Single binary datawal; positional store path; --json global flag.
  • JSON schema string datawal.cli.v1 is unchanged. Three new kinds (export, compact, check) are additive — no field renames, no encoding changes.
  • Cooperative single-writer lock: every subcommand still goes through DataWal::open / RecordLog::open.
  • Exit-code grammar (0/1/2/3) extended additively: 1 now also covers export/compact clobber refusals; 3 now also covers per-record get failures during check.
  • Reserved CLI names (query, select, where, index, server) remain unused.

Tests

+6 integration tests in crates/datawal-cli/tests/integration.rs (33 total, 0 failures):

  • export_writes_one_jsonl_line_per_live_key
  • export_refuses_to_overwrite_existing_outfile
  • compact_writes_to_empty_target_and_emits_stats (also reopens compacted target and confirms both live keys resolve)
  • compact_refuses_non_empty_target
  • check_clean_store_succeeds
  • check_detects_tail_truncation (appends 8 junk bytes to the active segment, asserts exit 2 + non-zero tail_truncated)

Local validation

All six gates green on feat/cli-export:

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace --all-targets (full workspace; CLI suite: 33 passed)
  • RUSTDOCFLAGS='-D warnings' cargo doc --workspace --no-deps
  • cargo bench --workspace --no-run
  • cargo publish --dry-run -p datawal --allow-dirty

Closes / Refs

Closes #18 (CLI-2 remainder; CLI-1 was closed by PR #24).
Refs #7.

Adds three new datawal-cli subcommands that round out the 0.1.x CLI
surface without touching the source store on disk.

- export STORE OUTFILE        Live KV projection -> JSONL via
                              DataWal::export_jsonl. Refuses to
                              clobber an existing outfile (exit 1).
- compact STORE TARGET        Snapshot-style rebuild into TARGET
                              via DataWal::compact_to. Refuses
                              non-empty target (exit 1).
- check STORE                 DataWal-level health: open + get every
                              live key (per-record CRC revalidation)
                              + RecoveryReport. Exit 3 on hard get
                              failure; exit 2 on tail truncation
                              / mid-stream error.

Source store on disk is never modified by any subcommand. cli.rs
top-doc and commands.rs top-doc reframed accordingly: subcommands
are now in two groups (read-only inspection vs source-untouched
mutations). Public CLI binary scope is unchanged otherwise.

JSON schema 'datawal.cli.v1' additively extended with three new
kinds: 'export', 'compact', 'check' (additive, non-breaking).

Tests: +6 integration tests in tests/integration.rs (33 total,
0 failures). Local validation: cargo fmt --check, cargo clippy -D
warnings, cargo test --workspace, cargo doc -D warnings, cargo
bench --no-run, cargo publish --dry-run -p datawal — all green.

Closes #18 (CLI-2 remainder; CLI-1 closed by PR #24).
Refs #7.
@robertoberto robertoberto merged commit ae50f8f into main May 21, 2026
7 checks passed
@robertoberto robertoberto deleted the feat/cli-export branch May 21, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Roadmap: datawal-cli (inspect + export + maintenance)

1 participant