feat(ai-proxy): rewrite Anthropic-to-OpenAI converter with whitelist body construction#13321
Open
nic-6443 wants to merge 14 commits intoapache:masterfrom
Open
feat(ai-proxy): rewrite Anthropic-to-OpenAI converter with whitelist body construction#13321nic-6443 wants to merge 14 commits intoapache:masterfrom
nic-6443 wants to merge 14 commits intoapache:masterfrom
Conversation
…body construction Replace core.table.clone() (blacklist approach) with whitelist body construction that builds the OpenAI request from scratch. This eliminates field leakage of unknown Anthropic fields (metadata, top_k, raw thinking, output_config, etc.) to upstream providers. Fixes: - Bug 1: Empty tools array no longer produces tools field - Bug 2: max_tokens now correctly maps to max_completion_tokens - Bug 3: output_config/output_format properly converts to response_format - Bug 4: Error responses now pass through as Anthropic error format New features: - Image and document (PDF) content block conversion - tool_choice conversion (auto/any/tool/none) - thinking config → reasoning_effort (low/medium/high thresholds) - thinking/redacted_thinking history blocks preserved - tool_result with multimodal content (text + image) - reasoning_content extraction in non-streaming responses - Streaming reasoning/thinking blocks with dynamic content indexing - cache_control preservation on content blocks and tool definitions - Billing header cch= stripping - Header conversion (x-api-key → Bearer, remove anthropic-*/x-stainless-*) - disable_parallel_tool_use → parallel_tool_calls=false - stop_sequences → stop conversion - Cached tokens handling (deduct from input_tokens) - cache_creation_input_tokens passthrough - Finish reason normalization (nil/empty/"null" not treated as stop) - Stream usage deferred flush pattern - Dynamic content_block index counter Tests expanded from 9 to 40 cases covering all conversion scenarios.
dosubot
Bot
added
size:XXL
Address code review finding: validate source.url is a non-empty string before using it, preventing invalid structures from being sent upstream. Add test coverage for URL source type including empty/nil edge cases.
There was a problem hiding this comment.
Pull request overview
Rewrites the Anthropic Messages → OpenAI Chat Completions converter to build outbound OpenAI request bodies via an explicit whitelist, preventing Anthropic-only fields from leaking upstream and expanding conversion coverage (tools, multimodal, reasoning/thinking, SSE behaviors).
Changes:
- Replaced clone-and-delete request conversion with whitelist body construction and expanded field mappings (tools/tool_choice, response_format, reasoning_effort, stop, etc.).
- Enhanced response + SSE conversion (reasoning/thinking blocks, cached token accounting, deferred usage flush, finish_reason normalization).
- Expanded integration/unit-style tests and added new OpenAI fixture responses to cover more scenarios.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
apisix/plugins/ai-protocols/converters/anthropic-messages-to-openai-chat.lua |
Major converter rewrite (request whitelist, header conversion, response/SSE conversion updates). |
apisix/plugins/ai-providers/base.lua |
Invokes protocol converter header transformation during upstream request build. |
t/plugin/ai-proxy-anthropic.t |
Expands test coverage from basic routing to extensive conversion and streaming behavior verification. |
t/fixtures/openai/chat-with-tool-calls.json |
New fixture for tool_calls response conversion coverage. |
t/fixtures/openai/chat-with-reasoning.json |
New fixture for reasoning_content → thinking conversion and cached token usage handling. |
t/fixtures/openai/chat-with-multiple-tool-calls.json |
New fixture for multi-tool_calls + text conversion ordering. |
t/fixtures/openai/chat-error.json |
New fixture for upstream error passthrough behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Add local pairs = pairs (luacheck compliance) - Strip all anthropic-* prefixed headers (not just version/beta) - Don't flatten single text block with cache_control to string - Drop thinking/redacted_thinking blocks from content (unsupported by OpenAI) - Remove variable shadowing in base.lua header conversion hook - Fix misleading comment about system block handling - Add test for arbitrary anthropic-* header removal
Remove the explicit elseif branch for thinking/redacted_thinking blocks and use a comment above the closing 'end' instead.
Whitespace-padded 'null' (e.g. ' null ') would bypass the check and be treated as a valid finish_reason, prematurely stopping the stream.
- TEST 2: use lookaheads instead of sequential matching (JSON key order is non-deterministic) - TEST 7,9,10: remove enclosing quotes from text assertions (values end with period before closing quote)
- input_tokens = max(0, prompt_tokens - cached) prevents negative values when cached_tokens exceeds prompt_tokens - convert_media_block returns nil when source.data is nil/empty instead of generating an empty data URL
JSON object keys like 'text' and 'type' can appear in any order within the same object. Use independent lookaheads instead of sequential matching within a single lookahead.
membphis
previously approved these changes
Apr 30, 2026
AlinsRan
previously approved these changes
Apr 30, 2026
…builtin tools, tool name sanitization, ping event, metadata.user_id, service_tier - Add stream_options.include_usage=true when stream=true (P0) - Strip cache_control from system, messages, and tools (P0) - Update thinking blocks comment to explain protocol limitation (P0) - Map metadata.user_id to OpenAI user field (P1) - Skip Anthropic built-in tools with debug log (P1) - Handle ping SSE event pass-through (P1) - Tool name truncation/sanitization with mapping restoration (P1) - Passthrough service_tier field (P1)
…unused test routes - Apply same sanitization to tool_choice.function.name as tool definitions so upstream receives consistent names - Disambiguate collisions when multiple tools sanitize to same name by appending numeric suffix (_2, _3, ...) - Remove unused route setup tests (TEST 15, TEST 31) that created admin routes never hit by HTTP requests - Add test coverage for collision and tool_choice consistency
Member
Author
|
CI failures are in unrelated tests not touched by this PR:
Our test shard ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR rewrites the Anthropic Messages → OpenAI Chat Completions converter using a whitelist body construction approach instead of the previous
core.table.clone()(blacklist) approach.The core issue: the old code cloned the entire Anthropic request and then tried to delete/rename fields. This meant any unhandled Anthropic-specific fields (metadata, top_k, raw thinking config, output_config, etc.) leaked through to the upstream OpenAI-compatible provider, causing unexpected behavior or errors.
The fix builds the outgoing OpenAI body from scratch (
openai_body = {}), explicitly converting only the fields that have OpenAI equivalents. This is the same approach used by other protocol bridges (litellm, higress).Changes
Bug fixes:
toolsfield in the forwarded requestmax_tokenscorrectly maps tomax_completion_tokens(never forwards rawmax_tokens)output_config/output_formatproperly converts toresponse_formatcache_controlfields stripped (OpenAI and compatible providers reject them)stream_options.include_usage=trueinjected when streaming (required for usage in stream chunks)New features:
tool_choiceconversion (auto/any/tool/none → OpenAI equivalents)thinkingconfig →reasoning_effort(low/medium/high by budget threshold)tool_resultwith multimodal content (text + image mixed)reasoning_content/reasoningextraction in non-streaming responses → thinking blockcch=stripping from system promptsx-api-key→ Bearer, remove anthropic-/x-stainless- headers)disable_parallel_tool_use→parallel_tool_calls=falsemetadata.user_id→ OpenAIuserfield mappingservice_tierpassthroughpingSSE event pass-through for long-lived streaming connectionsProtocol limitations (by design):
Tests: