Skip to content

Upgrade bundled arrow to ≥59.0.0 to fix panic on decimal128 from FFI #15

Description

@ianmcook

databow panics when a query returns a decimal128 column whose buffer is
8-byte but not 16-byte aligned:

arrow-buffer/src/buffer/scalar.rs: Memory pointer from external source (e.g, FFI)
is not aligned with the specified scalar type.

Repro

  1. Start Spark in Docker as described here.
  2. Run:
databow --driver spark \
  --uri "spark://admin@localhost:15002?auth_type=none&api=connect" \
  --query "SELECT CAST(19.99 AS DECIMAL(5,2)) AS p"

The ADBC Spark driver emits such buffers (spec-legal). The C++ ADBC manager
reads them fine; databow's Rust stack panics.

Cause / fix

databow bundles arrow-buffer 58.1.0. arrow-rs 59.0.0 fixes this by calling
align_buffers() in from_ffi (apache/arrow-rs#10030, closes #10028).
Upgrade to arrow ≥59.0.0.

Version

databow 0.1.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions