fix(gemini): set IncludeServerSideToolInvocations when using built-in tools with function calling#762
Conversation
… tools with function calling When built-in tools (GoogleSearch, URLContext, CodeExecution, ComputerUse, FileSearch, GoogleSearchRetrieval, GoogleMaps) are used alongside function calling, the Gemini API requires IncludeServerSideToolInvocations to be enabled in the ToolConfig, otherwise it returns INVALID_ARGUMENT. This bumps google.golang.org/genai to v1.52.1 (field added in v1.51.0) and automatically sets the flag when any built-in tool is present alongside function declarations. Fixes cloudwego#761
|
cc @meguminnnnnnnnn @N3kox @JonXSnow — tagging for visibility as top contributors to the gemini component. |
|
@JonXSnow my apologies for the second direct ping, but I was hoping to get a review from you on this. This allows us to set labels on the vertex api calls for better telemetry. (please let me know if there is a contributors group / process for these tiny types of maintenance adjustments to conform to the changes in the vertex api) |
|
@jredl-va Thanks for the fix. I think this needs two more changes before merge.
Currently
So we should return a clear unsupported error for Could you also add regression tests for these two cases? |
…ide tool invocations AUTO function-calling mode is not supported when IncludeServerSideToolInvocations is enabled, so coerce AUTO/unspecified modes to VALIDATED when built-in tools are combined with function declarations. The flag is also rejected by the Vertex AI backend, so return a clear error instead of letting the request fail in the SDK. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@tangchaojun-bytedance thanks for the feedback! Both issues addressed with proper test coverage.
|
What type of PR is this?
fix: A bug fix
Check the PR title.
(Optional) More detailed description for this PR(en: English/zh: Chinese).
en:
When built-in tools (GoogleSearch, URLContext, CodeExecution, ComputerUse, FileSearch, GoogleSearchRetrieval, GoogleMaps) are used alongside function calling tools, the Gemini API requires
ToolConfig.IncludeServerSideToolInvocationsto be set totrue. Without this, the API returnsINVALID_ARGUMENT: Please enable tool_config.include_server_side_tool_invocations to use Built-in tools with Function calling.This PR:
google.golang.org/genaifrom v1.36.0 to v1.52.1 (theIncludeServerSideToolInvocationsfield was added in v1.51.0)IncludeServerSideToolInvocations = truein theToolConfigwhen any built-in tool is present alongside function declarations(Optional) Which issue(s) this PR fixes:
Fixes #761