Skip to content

[FLINK-39034][runtime] Preserve timestamps in SortPartitionOperator#28644

Open
Jackeyzhe wants to merge 1 commit into
apache:masterfrom
Jackeyzhe:FLINK-39034
Open

[FLINK-39034][runtime] Preserve timestamps in SortPartitionOperator#28644
Jackeyzhe wants to merge 1 commit into
apache:masterfrom
Jackeyzhe:FLINK-39034

Conversation

@Jackeyzhe

Copy link
Copy Markdown
Contributor

What is the purpose of the change

SortPartitionOperator currently writes only record values into the external sorter for non-key-selector sorting. When records are emitted after sorting, this recreates output records without preserving the original StreamRecord timestamp state.

This change stores StreamRecord instances in the sorter and compares their wrapped values through a StreamRecordComparator, so sorted output preserves whether the input record had a timestamp and, when present, the original timestamp.

Brief change log

  • Store StreamRecord<INPUT> in the non-key-selector sorter path.
  • Add StreamRecordComparator to compare records by their wrapped values while preserving stream record metadata.
  • Extend SortPartitionOperatorTest to verify sorted output keeps timestamps and no-timestamp records remain timestamp-less.

Verifying this change

This change is covered by existing and extended tests.

Verified locally with:

./mvnw -pl flink-runtime -am -Dtest=SortPartitionOperatorTest -Dsurefire.failIfNoSpecifiedTests=false test

The target test passed with 2 tests, 0 failures, 0 errors, and 0 skipped.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: yes
  • The runtime per-record code paths (performance sensitive): yes
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable
Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: OpenAI GPT-5.5, reviewd by human
I'm responsible for the quality and correctness of every change in this PR regardless of the tooling used.

@flinkbot

flinkbot commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants