Skip to content

Support non-arrow writes without reading parquet metadata #713

@rockwotj

Description

@rockwotj

Feature Request / Improvement

As in the title, right now AddFiles will read the files I just wrote. I'd like to be able to skip the arrow cost (I write from my internal format directly to parquet, while also bookkeeping schema, etc), but if I use the AddFiles API then I immediately am forced to read the files I just wrote, but I'd rather supply the metadata correctly (I'm also setting the field IDs correct which is part of the requirements for reading)

iceberg slack ref: https://apache-iceberg.slack.com/archives/C05J3MJ42BD/p1770135889569909

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions