Skip to content

feat(go): add table write bindings - #658

Merged
JingsongLi merged 12 commits into
apache:mainfrom
XiaoHongbo-Hope:codex/go-write-binding
Aug 16, 2026
Merged

feat(go): add table write bindings#658
JingsongLi merged 12 commits into
apache:mainfrom
XiaoHongbo-Hope:codex/go-write-binding

Conversation

@XiaoHongbo-Hope

@XiaoHongbo-Hope XiaoHongbo-Hope commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Purpose

Expose standard table writes to Go.

Changes

  • Add write, prepare, merge, commit, retry, overwrite, truncate, and abort APIs.
  • Carry append/overwrite mode from the builder into the committer.
  • Preserve Arrow caller ownership by copying input into C-owned memory.
  • Add Go integration tests and usage documentation.

Scope

Commit messages can be merged only within one Go process. Cross-process
serialization is out of scope. With multiple primary-key fixed-bucket
writers, callers must assign each (partition, bucket) to one writer.

This PR adds Go APIs and does not change the storage format.

@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as draft August 3, 2026 07:25
@XiaoHongbo-Hope
XiaoHongbo-Hope force-pushed the codex/go-write-binding branch 3 times, most recently from 270de6a to d10b1b8 Compare August 15, 2026 09:52
@XiaoHongbo-Hope XiaoHongbo-Hope changed the title feat(go): add write bindings with fixed postpone buckets feat(go): add table write and postpone fixed-bucket bindings Aug 15, 2026
@XiaoHongbo-Hope
XiaoHongbo-Hope force-pushed the codex/go-write-binding branch 2 times, most recently from ff70c19 to 26cfe86 Compare August 15, 2026 14:00
@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as ready for review August 15, 2026 14:52
@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as draft August 15, 2026 15:17
@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as ready for review August 15, 2026 15:56
@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as draft August 16, 2026 01:18
Narrow this PR to the standard Go table write binding. Postpone
fixed-bucket builder, writer, commit messages, committer, tests,
test table, and documentation move to a follow-up PR.
@XiaoHongbo-Hope XiaoHongbo-Hope changed the title feat(go): add table write and postpone fixed-bucket bindings feat(go): add table write bindings Aug 16, 2026
@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as ready for review August 16, 2026 11:42
The C API has no filtered overwrite, so FilterAndCommitWithIdentifier
silently degraded to a non-filtered overwrite; a retry after a
successful commit would re-add the same files. Return an explicit
error instead, cover it with a test, and tighten doc comments.
Rust overwrite_with_identifier already runs with filter_committed=true,
so routing filtered retries to it is the correct idempotent behavior
and matches Java StreamTableCommit.filterAndCommit. Replace the wrong
rejection with a retry test asserting no new snapshot and preserved
intermediate commits, and document one-shot Commit identifier usage.

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit be5456e into apache:main Aug 16, 2026
13 checks passed
jerry-024 added a commit to jerry-024/paimon-rust that referenced this pull request Aug 17, 2026
* main:
  perf(vindex): size native batches by active indexes (apache#709)
  fix(data-evolution): require row tracking and row IDs (apache#718)
  feat(go): add table write bindings (apache#658)
  fix(scan): preserve Data Evolution file order in row-id groups (apache#717)
  fix(file_index): align file index format with Java V1 (apache#719)
  fix: configure OpenDAL writer chunk size (apache#713)
  fix(python): release GIL during catalog I/O (apache#716)
  feat(write): add fixed-bucket write primitives for postpone tables (apache#659)
  feat: rust examples for creating and querying Paimon tables (apache#648)
  fix(table): reject row ranges for format tables at read construction (apache#700)
  perf(arrow): prune IN predicates with row-group stats (apache#705)
  feat(io): support in-memory local cache (apache#710)
  fix(dlf): refresh expiring credentials (apache#714)
  fix(datafusion): normalize index_type in global index procedures (apache#715)
  fix(auth): fail closed on query-auth tables outside the read boundary (apache#691)

# Conflicts:
#	crates/paimon/src/table/vector_search_builder.rs
#	crates/paimon/src/vindex/reader.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants