Skip to content

[FLINK-40079][table] Reject PTF calls with sys-args if they are disabled#28675

Open
fhueske wants to merge 1 commit into
apache:masterfrom
confluentinc:fhueske-FLINK-40079-Fix-PTF-disableSystemArguments
Open

[FLINK-40079][table] Reject PTF calls with sys-args if they are disabled#28675
fhueske wants to merge 1 commit into
apache:masterfrom
confluentinc:fhueske-FLINK-40079-Fix-PTF-disableSystemArguments

Conversation

@fhueske

@fhueske fhueske commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

Add a check in SqlValidator to reject PTF calls with system-args (on_time, uid) if the function disabled them.
Before, system-args were either accepted but ignored (custom implementation like MLPredict) or the generic PTF handling framework throws an exception later (this part is untouched because the execution framework expects these args in the function signature even if not set).

Brief change log

  • add check to FlinkCalciteSqlValidator
  • add unit tests to ProcessTableFunctionTest
  • add unit tests for MLPredict function

Verifying this change

Added unit tests.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? n/A

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code 2.1.200 (Opus 4.8)

Add a check in SqlValidator to reject PTF calls with system-args (on_time, uid) if the function disabled them.

Generated-By: Claude Opus 4.8 (1M context)
@flinkbot

flinkbot commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

* <p>When a PTF sets {@code disableSystemArguments(true)}, the system arguments are not part of
* its signature. Enforcing this here covers all translation paths uniformly.
*/
private static void checkDisabledSystemArgs(SqlBasicCall call) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering why you choose this code location? Is SystemTypeInference not good enough? Changes to FlinkCalciteSqlValidator should be kept minimal.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a bunch of PTF argument checks in FlinkCalciteSqlValidator already and the check at this place is fairly straightforward.

If we want to perform the check at a later point, we would need to include system args in signatures of PTFs that disable them because Calcite drops unknown named arguments when it converts them into positional arguments. So without extending the signatures, we would not know if a function was called with system args or not.
That would mean that we need to refactor quite a bit and add more special case handling for PTFs with disabled system args.
We can move the actual check implementation as a static helper into SystemTypeInference but it's much easier to call it from FlinkCalciteSqlValidator than at a later time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants