Skip to content

Commit c82c42e

Browse files
authored
feat(datafusion): show pushed-down limit in IcebergTableScan EXPLAIN output (#2360)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - Closes #2359. ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> Emit ` limit:[N]` in `IcebergTableScan`'s `DisplayAs` output when a `LIMIT` is pushed down to the scan. When no limit is pushed down, the output is unchanged. - Before (unchanged): `IcebergTableScan projection:[id,name] predicate:[...]` - After (new, only when a limit reaches the scan): `IcebergTableScan projection:[...] predicate:[] limit:[3]` ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> Yes: new `EXPLAIN ... LIMIT 3` assertion in `crates/sqllogictest/testdata/slts/df_test/basic_queries.slt`. Existing snapshots are unchanged, which confirms the additive-only behavior.
1 parent 4b4ffd0 commit c82c42e

2 files changed

Lines changed: 17 additions & 1 deletion

File tree

crates/integrations/datafusion/src/physical_plan/scan.rs

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -196,7 +196,11 @@ impl DisplayAs for IcebergTableScan {
196196
self.predicates
197197
.clone()
198198
.map_or(String::from(""), |p| format!("{p}"))
199-
)
199+
)?;
200+
if let Some(limit) = self.limit {
201+
write!(f, " limit:[{limit}]")?;
202+
}
203+
Ok(())
200204
}
201205
}
202206

crates/sqllogictest/testdata/slts/df_test/basic_queries.slt

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,18 @@ INSERT INTO default.default.query_test_table VALUES
4343
----
4444
10
4545

46+
# Verify EXPLAIN shows limit is pushed down to IcebergTableScan
47+
query TT
48+
EXPLAIN SELECT * FROM default.default.query_test_table LIMIT 3
49+
----
50+
logical_plan
51+
01)Limit: skip=0, fetch=3
52+
02)--TableScan: default.default.query_test_table projection=[id, name, score, category], fetch=3
53+
physical_plan
54+
01)GlobalLimitExec: skip=0, fetch=3
55+
02)--CooperativeExec
56+
03)----IcebergTableScan projection:[id,name,score,category] predicate:[] limit:[3]
57+
4658
# Test SELECT * with ORDER BY and LIMIT
4759
query ITRT
4860
SELECT * FROM default.default.query_test_table ORDER BY id LIMIT 3

0 commit comments

Comments
 (0)