Repository navigation
[AI-517, AI-744] Add an OpenAI-compatible model provider sample for Baseten, Fireworks AI, and Together AI - #402
Merged
Conversation
… AI, and Together AI One Worker script covers all three providers: a provider table holds each endpoint's base URL, API key environment variable, and a default open-weight model with tool calling (gpt-oss-120b everywhere), and an OpenAICompatibleModelProvider turns an entry into a Chat Completions model with client retries off, so Temporal owns the retries. The Workflow names no model, so it runs unchanged on whichever provider the Worker was started with (--provider baseten|fireworks|together). Each provider entry carries its own snipsync marker so the Temporal docs page for that provider can quote it. Offline tests cover the provider table, the client configuration, and the Workflow with a scripted model.
DABH
changed the title
Review feedback: restarting the Worker to switch providers is not how a real deployment works. The model provider now holds every provider whose API key is set and routes `<provider>/<model>` names, such as `together/openai/gpt-oss-120b`, to the matching endpoint. The model name travels in the Workflow input, so Workflows on different providers run side by side on the same Worker, and the starter runs one Workflow per `--model` concurrently. Naming the model on the agent also lets the Agents SDK drop the default settings it keeps for OpenAI's default model, so the explicit `ModelSettings()` workaround is gone. A model name for a provider without a key, or with an unknown prefix, fails its Activity without retrying; the starter prints that root cause. Verified live on Fireworks AI alongside a keyless Together AI Workflow on the same Worker: the first completed through two model Activities, the second failed on its first attempt with the "set TOGETHER_API_KEY" error.
JasonSteving99
approved these changes
Oct 26, 2026
Contributor
Author
|
Reopening to re-trigger the CLA check, which did not post a status for the latest push. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
One sample in
openai_agents/model_providers/runs the same agent on Baseten, Fireworks AI, or Together AI, all of which serve open-weight models behind the OpenAI Chat Completions API:openai_compatible_providers.py: a provider table (base URL, API key env var, an example model with tool calling: gpt-oss-120b on all three) andOpenAICompatibleModelProvider, which builds a client for every provider whose API key is set and routes<provider>/<model>names such astogether/openai/gpt-oss-120bto the matching endpoint as anOpenAIChatCompletionsModelwithmax_retries=0, so the Activity's Retry Policy is the only retry policy. A name for an unknown prefix or a provider without a key fails the Activity without retrying.run_openai_compatible_worker.py: one Worker serves every configured provider and prints which ones at startup.workflows/openai_compatible_workflow.py: the model name travels in the Workflow input and goes on theAgent, so Workflows on different providers run side by side on the same Worker.run_openai_compatible_workflow.py --model ... [--model ...]: starts one Workflow per model concurrently and prints each result or root-cause failure.tests/openai_agents/model_providers_test.py: offline tests for the table, routing, the non-retryable failures (unit and through the Workflow), two concurrent Workflows on different providers sharing one Worker, and that no OpenAI-default model settings reach the provider.Each provider entry carries its own
@@@SNIPSTARTmarker (python-openai-agents-{baseten,fireworks,together}-provider), plus markers for the provider class, Worker wiring, and Workflow, so the matching Temporal docs pages can quote them.Testing
ruff,mypy, and the new tests pass locally. Run end to end against Fireworks AI with a Fireworks Workflow and a keyless Together Workflow on the same Worker: the first called the tool and answered through two completed model Activities, the second failed on its first attempt with the "Set TOGETHER_API_KEY" error. Baseten and Together AI were verified against their current docs (base URL, env var, model ID, tool calling) but not called.Related
main).