Repository navigation
feat: add openinfer provider with model mappings - #6
onkarbpatil wants to merge 6 commits into
Conversation
Adds OpenInfer (platform.openinfer.io) as an inference provider with mappings for Qwen 3.5 9B and Llama 3.2 1B Instruct, including new registry model entries for both. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Hi, I ahve added this PR to integrate OpenInfer's OICloud as an infernece provider service. I have the API key generated, but wasn't sure where to add or how to share. Please help with that. |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Hi, @madeburo, could we get an update on merging this PR? |
|
Hi @onkarbpatil Before we can merge, please address the following:
Please do not share the API key in this PR or commit it to the repository. API credentials can be handled separately through a private channel or repository secrets when telemetry integration is configured. |
Update the OpenInfer mappings with published rate-card pricing and extend coverage to two additional served models. Pricing (USD per 1M tokens, input/output): Qwen3.5 9B 0.15 / 0.18 Qwen3.5 27B 0.72 / 0.72 Llama3.2 1B 0.02 / 0.02 Gemma4 31B 0.52 / 0.75 Adds a canonical entry for Qwen 3.5 27B; Gemma 4 31B already exists upstream, so it needs only a mapping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Set context_window_override to the context length each model is actually served at on OpenInfer nodes: Qwen3.5 27B 32768 Llama3.2 1B 16384 Gemma4 31B 32768 Qwen3.5 9B is served at its full native 262144 window, so its override stays null. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Hi @madeburo I have added the pricing and context size. You can get the public API to access them here "https://console.openinfer.ai/usage" |
Gemma4 31B is served at 8192 tokens on OpenInfer nodes, not 32768. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Llama3.2 1B is served at 24000 tokens, not 16384. Qwen3.5 9B is served at 65536 rather than its full native 262144 window, so it now carries an explicit override instead of null. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Hi @madeburo pinging for update. |
|
@madeburo any update? |
|
@madeburo pinging for update again. |
|
@madeburo |
|
Hi @onkarbpatil, @chongyihu — thanks for the updates, The files validate against the current main and the model entries look correct. Before we can merge the provider, we still need publicly verifiable sources:
Please also update the PR description, which still mentions $0.00 pricing and two models. If it helps, we can merge the canonical model entries (Qwen 3.5 9B, Qwen 3.5 27B, Llama 3.2 1B) separately in the meantime. Just let us know. |
@madeburo Thank you! I will report back and let my team revise this. I will get back to you ASAP. |
|
Hi @madeburo |
madeburo
left a comment
There was a problem hiding this comment.
Thanks for following up. I re-reviewed the current head and tested a merge with the latest main: there were no conflicts, and local registry validation passed for all 587 files. The pricing and served-context overrides added in September are present. The remaining issues are public verification and consistency of the submission:
-
Public access. The Cloud page still presents a Request Access form and the expired “free through August 31, 2026” preview. The newer platform quickstart explicitly describes invite-only beta access, with users without an invitation landing on a Coming Soon page. Please clarify whether that applies to the hosted Cloud API in this PR or to a separate product, and provide the public self-serve signup/onboarding path for the submitted service. A registration form alone does not establish that a new user can obtain usable inference access without an invitation.
-
Public pricing and served limits. Please link an authoritative public rate card matching all four mappings, together with documentation for the served model IDs, 60 RPM / 100,000 TPM limits, and these provider-specific context windows: Qwen 3.5 9B — 65,536; Qwen 3.5 27B — 32,768; Llama 3.2 1B — 24,000; Gemma 4 31B — 8,192. Some model IDs now appear in the public platform documentation, but I could not verify the complete hosted catalog and the numerical prices/limits there. The previously supplied console usage page is not a public rate-card reference. The unauthenticated Models API returns 401, which is expected authentication behavior, not evidence of an outage. Unknown/unpublished RPM or TPM values can be null rather than asserted without a public reference.
-
Canonical API endpoint.
providers/openinfer.yamlspecifieshttps://api.openinfer.ai/v1, whereas the current platform quickstart documentshttps://platform.openinfer.io/v1. Please confirm which endpoint serves the hosted catalog and prices submitted here, and provide the applicable public API documentation. If these are separate services or both URLs remain supported, please explain that distinction rather than changing the URL without verification. -
Update the PR description. It still describes two models and $0.00 promotional pricing. The actual submission contains four paid provider-model mappings and three new canonical model definitions. Please update the summary, model list, pricing note, and public source links to describe the final submission.
Please do not post or commit an API key; public documentation is sufficient for these questions. Once the sources and access model are clarified and the metadata is aligned, we can re-review the provider addition. Thank you for helping make the entry independently verifiable.
Summary
Adds OpenInfer (openinfer.io) as an inference provider, with mappings for the two models it currently serves. Both models are new to the registry, so their canonical model entries are included.
Disclosure: I work at OpenInfer, so provider details (model catalog, rate limits) come from first-hand knowledge of the service.
Files added
providers/openinfer.yaml— OpenAI-compatible API athttps://api.openinfer.ai/v1, bearer auth, global routing across distributed CPU/GPU/NPU computemodels/qwen3-5-9b.yaml— Qwen 3.5 9B (Alibaba, Apache 2.0, 262K context, multimodal)models/llama-3-2-1b.yaml— Llama 3.2 1B Instruct (Meta, 128K context)mappings/openinfer/qwen3-5-9b.yaml— served as@oi/Qwen3.5-9Bmappings/openinfer/llama-3-2-1b.yaml— served as@oi/Llama-3.2-1B-InstructPricing note
OpenInfer Cloud is free through August 31, 2026 (see openinfer.io/cloud), so mappings list $0.00 input/output. We will submit an update with post-promo rates when the free period ends.
Validation
python validate_registry.pypasses — 403 files validated.🤖 Generated with Claude Code