Skip to content

[EPIC] Implement Missing Write Actions #2269

@Shekharrajak

Description

@Shekharrajak

iceberg-rust currently supports FastAppendAction for simple appends but is missing several transaction actions available in the Java Iceberg API that are needed for complete write support.

Java API (api/src/main/java/org/apache/iceberg/) iceberg-rust (crates/iceberg/src/transaction/) Status
AppendFiles.java append.rs (FastAppendAction) Available
OverwriteFiles.java - Missing
ReplacePartitions.java - Missing
DeleteFiles.java - Missing
RowDelta.java - Missing
RewriteFiles.java - Missing

TODOs:

  • Implement OverwriteAction - needed for INSERT OVERWRITE (static partition overwrite)
  • Implement ReplacePartitionsAction - needed for dynamic partition overwrite
  • Implement DeleteFilesAction - needed for DELETE FROM table operations
  • Implement RowDeltaAction - needed for row-level DELETE, UPDATE, MERGE INTO (CoW/MoR)
  • Implement RewriteFilesAction - needed for compaction and file rewriting

Related open issues:

#1104 - Support RowDeltaAction
#2202 - Implement RowDeltaAction for row-level modifications (CoW)
#2201 - Add MERGE INTO support for DataFusion integration
#2205 - SQL UPDATE support for DataFusion integration
#2186 - Copy-on-Write and Merge-on-Read support
#1382 - [EPIC] Support for appending data to iceberg table
#2218 - Support for a DeltaWriter

Motivation:

These transaction actions are required for apache/datafusion-comet to implement a fully native Iceberg write path (datafusion-comet#3595, datafusion-comet#2967). Comet currently writes Parquet data files natively via Rust/DataFusion but still relies on the Java Iceberg layer for table commits. Completing these actions in iceberg-rust would enable end-to-end native Iceberg writes - from Arrow batches through DataFusion execution to Iceberg table commits - eliminating JVM dependency for the entire write path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions