[BP-2.2][FLINK-39387][FLINK-39914][FLINK-39922][FLINK-39921][FLINK-39929][tests] Fix flaky scheduler tests under async TDD creation

[MartijnVisser]

What is the purpose of the change

Backport of the async-TDD ComponentMainThreadExecutor test-stability fix family from master to release-2.2. This PR intentionally bundles five JIRAs because they form one coherent cluster with an internal dependency (the FLINK-39387 pick introduces NoMainThreadCheckComponentMainThreadExecutor, which the later picks need to compile); each is a separate clean -x cherry-pick, so individual reverts remain possible.

Observed on release-2.2 nightlies: ExecutionVertexCancelTest.testSendCancelAndReceiveFail and ExecutionTimeBasedSlowTaskDetectorTest.testFinishedTaskNotExceedRatio fail with IllegalStateException: BUG: trying to schedule a region which is not in CREATED state at startScheduling (build 76677, test_cron_jdk21 core and test_cron_azure core legs).

Root cause (same as on master): since FLINK-38114, TaskDeploymentDescriptor creation is offloaded to the I/O executor and deploy continuations complete on background threads. The ComponentMainThreadExecutorServiceAdapter.forMainThread() test executor asserts the caller is the test main thread, so these completions trip the assertion. FLINK-38114 is on release-2.2, so the assertion relaxation applied by the master fixes is equally justified here. Test-only change; the production scheduler is not affected. The same fixes are already on release-2.3 (partly inherited, partly via the parallel backport PR).

Brief change log

Clean git cherry-pick -x of the merged master fixes, in dependency order (byte-identical, original authorship preserved):

• FLINK-39387 (master 106acbc): prerequisite, introduces NoMainThreadCheckComponentMainThreadExecutor (absent on release-2.2).
• FLINK-39914 (master 6d47db1, [FLINK-39914][runtime] Fix flaky TaskDeploymentDescriptorFactoryTest#testHybridVertexFinish caused by async TDD creation #28398): fix flaky TaskDeploymentDescriptorFactoryTest#testHybridVertexFinish.
• FLINK-39922 (master 6d106cd, [FLINK-39922][runtime/test] Fix the flaky test case AbstractAsyncRunnableStreamOperatorTest#testCheckpointDrain #28425): fix flaky AbstractAsyncRunnableStreamOperatorTest#testCheckpointDrain.
• FLINK-39921 (master a8d6a28, [FLINK-39921][runtime/tests] Fix the flaky test case ExecutionVertexCancelTest.testSendCancelAndReceiveFail #28449): fix flaky ExecutionVertexCancelTest.testSendCancelAndReceiveFail.
• FLINK-39929 (master 67eba45, [FLINK-39929][runtime/tests] Fix the flaky test case ExecutionTimeBasedSlowTaskDetectorTest due to unexepected ComponentMainThreadExecutor setting. #28434): fix flaky ExecutionTimeBasedSlowTaskDetectorTest.

Verifying this change

This change is already covered by existing tests:

• ExecutionVertexCancelTest + ExecutionTimeBasedSlowTaskDetectorTest: 4 runs, 20/20 green each run.
• TaskDeploymentDescriptorFactoryTest + AbstractAsyncRunnableStreamOperatorTest: green (36/36 combined run).
• flink-runtime test-compile verified at every individual commit boundary.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no (test-only)
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

---

Was generative AI tooling used to co-author this PR?

• Yes (Claude Code)

Generated-by: Claude Code (Claude Fable 5)

[flinkbot]

CI report:

• 1bec0e0 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[spuru9]

A little abmiguity: The PR title mentions only 2 items while it is cherrypicking 5. The packport to 2.3 contained 2 only.

[MartijnVisser]

A little abmiguity: The PR title mentions only 2 items while it is cherrypicking 5. The packport to 2.3 contained 2 only.

Good call, I'll fix this