Auto-create dictionary when secondary indexes need it (inverted, FST, range)#17269

Merged

xiangfu0 merged 40 commits intoapache:masterfrom

xiangfu0:dictionary-auto-creation-when-corresponding-indexes-enabled

May 7, 2026

Commits on May 7, 2026

Auto-create dictionary when required by secondary indexes (inverted, FST, range)

xiangfu0
and
claude
committed
Address remaining MAJOR review findings on shared-dict feature

xiangfu0
and
claude
committed
Add v3-format legacy fixture + white-box on-disk migration assertions

xiangfu0
and
claude
committed
Drop FilterOperatorUtilsTest cases for reverted RawValueInvertedIndex path

xiangfu0
and
claude
committed
Range index always requires dictionary; inline ENABLE_DICTIONARY helper

xiangfu0
and
claude
committed
Simplify overlapping tests and align range validation with new contract

xiangfu0
and
claude
committed
Fix shared-dict + RAW forward path in segment creator and update tests

xiangfu0
and
claude
committed
Reject range index on BYTES upfront in RangeIndexType.validate

xiangfu0
and
claude
committed
Move legacy RawInvertedIndex cleanup to its own file; revert range-requires-dict

xiangfu0
and
claude
committed
Inline single-use methods to reduce unnecessary code structure

xiangfu0
and
claude
committed
Add mixed-predicate tests for RAW + dict + inverted index column

xiangfu0
and
claude
committed
DataFetcher cleanup; route shared-dict + RAW columns to raw-value path

xiangfu0
and
claude
committed
Restore readDictIdsFromRawValues as opt-in DataFetcher API

xiangfu0
and
claude
committed
Update DataFetcherTest for the new dict-id contract on RAW forward index

xiangfu0
and
claude
committed
Revert "Update DataFetcherTest for the new dict-id contract on RAW forward index"
xiangfu0
committed
Revert "Restore readDictIdsFromRawValues as opt-in DataFetcher API"
xiangfu0
committed
Revert "DataFetcher cleanup; route shared-dict + RAW columns to raw-value path"
xiangfu0
committed
DataFetcher: split readDictIds and readDictIdsFromRawValues read paths

xiangfu0
and
claude
committed
DataFetcher.fetchDictIds: drop auto-fallback to readDictIdsFromRawValues

xiangfu0
and
claude
committed
DefaultGroupByExecutor: route shared-dict + RAW columns to no-dict GROUP BY

xiangfu0
and
claude
committed
Simplify ForwardIndexHandler.computeOperations; auto-toggle dict by index need

xiangfu0
and
claude
committed
ForwardIndexHandler: track encoding (DICT/RAW/null) and split ENABLE_FORWARD_INDEX

xiangfu0
and
claude
committed
ForwardIndexHandler: support DICT⇄RAW encoding flip with dict kept

xiangfu0
and
claude
committed
DataFetcher: drop the (shared) dictionary on RAW forward in addDataSource

xiangfu0
and
claude
committed
ColumnContext: drop the dictionary on RAW forward at the source

xiangfu0
and
claude
committed
Revert ColumnContext dict-drop; gate dict-id consumers per call site

xiangfu0
and
claude
committed
Test: filterMv predicate evaluation on shared-dict + RAW MV column

xiangfu0
and
claude
committed
Test: filterMv on dict + RAW + inverted, plus DataSource-aware evaluator overload

xiangfu0
and
claude
committed
PredicateEvaluatorProvider: collapse to a single public entry point

xiangfu0
and
claude
committed
Address Jackie-Jiang's open review comments

xiangfu0
and
claude
committed
Drop redundant null guards: trust segment-loader invariants

xiangfu0
and
claude
committed
Auto-enable dictionary at config time for FST/IFST/INVERTED, dedup factory dispatch

xiangfu0
and
claude
committed
Revert TableIndexingTest; remove redundant getStatsCollector boolean parameter

xiangfu0
and
claude
committed
Unified PredicateEvaluator entry; PredicateEvaluator handles dict-vs-raw internally

xiangfu0
and
claude
committed
Revert TableIndexingTest.java to master — keep this file out of the PR

xiangfu0
and
claude
committed
Fix CI unit test regressions from rebase: ExpressionFilterOperator dict, getStatsCollector flag

xiangfu0
and
claude
committed
Fix predicate evaluator provider
Jackie-Jiang
committed
Fix RAW-forward and non-exact range correctness gaps

xiangfu0
and
claude
committed
Fix TransformFunction
Jackie-Jiang
committed
Reject legacy range index v1 on RAW + dict; rebuild range index on version change

xiangfu0
and
claude
committed