Route modern OpenAI models through chat completions

[kayametehan]

Fixes #1580.

Overview

oaieval currently routes modern OpenAI chat/reasoning models like gpt-4o-mini through the non-chat completions path because is_chat_model() only recognizes older model families.

That causes requests such as:

• oaieval gpt-4o-mini ...

to fail with the completions endpoint error:

• This is a chat model and not supported in the v1/completions endpoint

What changed

• Extend is_chat_model() to recognize modern OpenAI chat/reasoning model families
• Cover current families including gpt-4o*, gpt-4.1*, gpt-4.5*, o1*, o3*, and o4*
• Add regression coverage for both chat-model detection and Registry.make_completion_fn() routing

Validation

I verified this change with focused checks before pushing:

• python -m pytest evals/registry_test.py -q
• direct Registry.make_completion_fn() checks for gpt-4o-mini, gpt-4.1-mini, o1-mini, o3-mini, o4-mini
• direct OpenAISolver checks for gpt-4o-mini and o3-mini
• git diff --check

[Copilot]

Pull request overview

Updates model routing so modern OpenAI chat/reasoning model families (e.g., gpt-4o*, gpt-4.1*, gpt-4.5*, o1*, o3*, o4*) are correctly identified as chat models and sent through the chat-completions path, preventing v1/completions endpoint errors.

Changes:

• Expand is_chat_model() to recognize additional modern model prefixes (gpt-4o, gpt-4.1, gpt-4.5, o1, o3, o4).
• Add regression tests covering new chat-model detection cases.
• Add a routing regression test ensuring Registry.make_completion_fn() returns an OpenAIChatCompletionFn for gpt-4o-mini.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
evals/registry.py	Extends chat-model detection to include modern OpenAI model families so routing uses chat completions.
evals/registry_test.py	Adds regression tests for new model-family detection and Registry.make_completion_fn() routing behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.