Skip to content

fix(datafusion): normalize index_type in global index procedures - #715

Merged
JingsongLi merged 1 commit into
apache:mainfrom
jackylee-ch:fix/case-insensitive-global-index-type
Aug 15, 2026
Merged

fix(datafusion): normalize index_type in global index procedures#715
JingsongLi merged 1 commit into
apache:mainfrom
jackylee-ch:fix/case-insensitive-global-index-type

Conversation

@jackylee-ch

Copy link
Copy Markdown
Contributor

sys.create_global_index matched index_type case-insensitively for btree and
bitmap but exactly for the vindex types, so index_type => 'BTREE' worked while
index_type => 'IVF-FLAT' was rejected as unsupported. sys.drop_global_index
already canonicalizes every type. None of the three procedures trimmed the
argument, so ' btree ' was reported as unsupported too.

The three procedures now canonicalize index_type at their boundary through a
shared helper (trim, then lowercase), mirroring
indexType.toLowerCase(Locale.ROOT).trim() in Java's Flink and Spark
Create/DropGlobalIndexProcedure. The core builders keep exact matching: they
are the analog of Java's GlobalIndexer and persist index_type into index
metadata, which the read path matches exactly. An unsupported type is echoed
back as written so a typo stays recognizable.

create_lumina_index shares the helper but has no end-to-end test here because
it needs LUMINA_LIB_PATH, which CI does not provide.

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit 9672107 into apache:main Aug 15, 2026
13 checks passed
jerry-024 added a commit to jerry-024/paimon-rust that referenced this pull request Aug 17, 2026
* main:
  perf(vindex): size native batches by active indexes (apache#709)
  fix(data-evolution): require row tracking and row IDs (apache#718)
  feat(go): add table write bindings (apache#658)
  fix(scan): preserve Data Evolution file order in row-id groups (apache#717)
  fix(file_index): align file index format with Java V1 (apache#719)
  fix: configure OpenDAL writer chunk size (apache#713)
  fix(python): release GIL during catalog I/O (apache#716)
  feat(write): add fixed-bucket write primitives for postpone tables (apache#659)
  feat: rust examples for creating and querying Paimon tables (apache#648)
  fix(table): reject row ranges for format tables at read construction (apache#700)
  perf(arrow): prune IN predicates with row-group stats (apache#705)
  feat(io): support in-memory local cache (apache#710)
  fix(dlf): refresh expiring credentials (apache#714)
  fix(datafusion): normalize index_type in global index procedures (apache#715)
  fix(auth): fail closed on query-auth tables outside the read boundary (apache#691)

# Conflicts:
#	crates/paimon/src/table/vector_search_builder.rs
#	crates/paimon/src/vindex/reader.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants