Repository navigation
Description
Follow-up to #50 (activity + schemaEndpointId). Every listed model on GET /v1/models should carry the same facts OpenRouter publishes on its Models API standard, filled from this service’s sources — the bound request schema first, then the provider’s own listing/docs — with per-field provenance so a consumer can see why a value is there.
#58 lands the listing/docs slice for chat/text hosts. It skipped the thing we uniquely have: the generation schema the model is already bound to.
OpenRouter is compare-only. Never copy an OpenRouter value into another provider’s catalog, labeled or not, and not even to fill a null. Use it only as a discrepancy signal. If no first-party source states a fact, the field stays null.
Target shape
Mirror OpenRouter’s model object, in our camelCase catalog fields:
| OpenRouter | Us | Primary source |
|---|---|---|
context_length |
contextWindow |
listing / docs (schema almost never has this) |
top_provider.max_completion_tokens |
maxOutput |
listing / docs (max_tokens.maximum is a weak ceiling) |
architecture.input_modalities / output_modalities |
modalities |
bound schema content-parts or listing |
supported_parameters |
capabilities (string[] for chat/text) |
bound request schema properties, then listing flags |
default_parameters |
not stored yet | JSON Schema default on those properties |
pricing (+ overrides) |
pricing |
listing / pricing page / extras. Never the schema |
tokenizer / instruct_type |
not stored yet | docs, if we care later |
benchmarks |
out of scope | third party |
Chat/text capabilities use OpenRouter’s parameter names: tools, tool_choice, max_tokens, temperature, top_p, reasoning, include_reasoning, structured_outputs, response_format, stop, frequency_penalty, presence_penalty, seed (and top_k / reasoning_effort when the native schema has them).
Speech / image / video / world-model hosts keep host-native capability objects where those flags do not apply. Schema walk still fills modalities and the request properties that are on that model’s route.
Fill order (per field, never overwrite a stronger rung)
Strongest first. A weaker source only writes when the field (or flag) is still null.
| Rung | What it is | Self-heals? |
|---|---|---|
listing |
A field on the provider’s own /models row or extras JSON |
yes, every poll |
upstream-spec / generated |
A property on the bound generation request/response schema (schemaEndpointId → current input schema). Inherits that schema version’s derivation |
yes, every spec sync |
docs-derived |
Deterministic parse of a provider docs page (markdown table / bullet). Same fail-closed rule as #58 | yes, every poll (memoised) |
docs-extracted |
Extraction model over leftover prose, each value carrying the verbatim sentence | no |
generated specs that are OpenAI’s document with the host swapped (DeepSeek, Cerebras, DashScope, …) do not stamp OpenAI’s tools onto that host’s models. Only a spec the provider publishes (Groq Stainless, Anthropic, Gemini, xAI, BytePlus Ark, FAL per-model, …) counts as upstream-spec / honest generated.
Grain:
- Model-grained (FAL, Reactor, …): the schema is the model. Walking it is the whole
supported_parameters+ modalities answer. - Provider-grained, one family per route (Anthropic
v1/messages, Gemini:predictvs:generateContent): stamp the route’s properties onto every model bound to that route. This is how Anthropictools/tool_choice/temperatureget filled — they are on the messages schema, not the Models API row. - Provider-grained, mixed family on one route (OpenAI
chat/completions): schema is an upper bound. Activity +schemaEndpointIdalready keep whisper off chat flags. Per-model listing/docs overlay where a family on the same route does not honor a field (o-series vs gpt).
We store presence, not absence. If the schema has temperature and the model docs never mention it, the flag is present from upstream-spec. A later docs-extracted “this model ignores temperature” is a separate, explicit deny — do not invent denies.
Fine-grained provenance
Not one source on the row. Each stored fact names its winning rung, and capabilities is per flag.
Reuse the schema provenance idea (sourceUrl, sourceHash, fetchedAt, derivation) with extra locators:
interface FactSource {
derivation:
| 'listing'
| 'upstream-spec'
| 'generated'
| 'docs-derived'
| 'docs-extracted'
sourceUrl?: string
sourceHash?: string
fetchedAt?: number
/** Bound generation route, when the winner is the schema. */
endpointId?: string
/** JSON pointer or field name in the source document. */
path?: string
/** Verbatim sentence; required for `docs-extracted`. */
quote?: string
}
interface ModelFactSources {
contextWindow?: FactSource
maxOutput?: FactSource
modalities?: FactSource
pricing?: FactSource
/** Per OpenRouter parameter name (or per host-native key). */
capabilities?: Record<string, FactSource>
}Store as JSON on models (new column, e.g. fact_sources). The poller / schema-sync writes it next to the values. A flag that moved from docs-derived to upstream-spec is a real model.updated.
Expose
GET /v1/models— values only (list stays small).?provenance=1includesfactSources.GET /v1/models/{provider}/{id}— always includesfactSources.- Same
provenanceobject shape asGET /v1/schemas/...so agents already know how to read a trust ladder. _links.schemaalready points at the bound input schema;factSources.capabilities.tools.endpointIdshould matchschemaEndpointIdwhen the winner is the schema.
OpenRouter: compare only
Join native provider/rawId (and undated aliases) to OpenRouter’s id. The join produces one thing:
- Discrepancy. After first-party rungs, if OpenRouter has a different value (or has a flag we lack, or we have a flag they lack), record it. Do not change our value.
- On the detail payload:
discrepancies: [{ field, ours, openrouter, oursDerivation, openrouterId }]. - On the changes feed:
model.fact_mismatch(or a dedicated/v1/models/discrepancies) so it is watchable. - Typical: we have Anthropic
toolsfromv1/messagesand they agree; we lack Groktoolsfrom extras and they have them — that mismatch is the cue to walk xAI’s chat schema.
- On the detail payload:
No fill-nulls rung. A catalog that copied OpenRouter, even labeled, would cap accuracy at theirs and blur what the provider itself states. A catalog that shows the diff uses them as a checker. A null with an OpenRouter discrepancy is the honest answer when the provider publishes nothing.
What #58 already did
Listing/docs fill for openai, anthropic, gemini, grok, groq, jina, fireworks, mistral, moonshot, sambanova, hyperbolic, novita, cohere. Together already has context_length + pricing. OpenRouter is the reference catalog. Gemini schemaEndpointId is the bind the schema walk needs.
Still empty on purpose until this path: DeepSeek / Cerebras / DashScope / Perplexity (bare rows and generated OpenAI specs — schema walk is invalid until they have their own document). Grok capabilities: null is a #58 listing gap; the chat schema should fill tools / temperature / … for language models.
Speech/media (elevenlabs, fal, replicate, reactor, bfl, kling, …): schema walk is the right next fill for request properties and modalities; pricing stays listing/docs.
Implementation slices (each a PR)
fact_sourcescolumn + detail API (?provenance=1on the list). No new facts. Backfill listing-derived Fill contextWindow, maxOutput, modalities, and request-feature capabilities on grain=provider catalogs #58 values asderivation: listing(ordocs-derivedfor OpenAI pages / Grok context table).- Bound-schema walker — given
schemaEndpointId+ current input (and output) schema, emitsupported_parameters+ modalities. SkipgeneratedOpenAI-borrowed specs. Merge under the ladder (listing wins on conflict of presence from a per-model field; schema fills nulls). - OpenRouter compare — join, write
discrepancies, changes event. No writes to fact columns. - Pricing — listing/docs/
overrides; samefactSources.pricing. docs-extracted— leftover prose only (reasoning.mandatory, Gemini modalities, o-series temperature deny, bare-row host docs), quote required.
Fail closed stays: a docs parse of zero rows throws; a schema walk of a missing bind writes nothing (does not null listing facts).
Consumer
TanStack/ai#1308. Ids and activity already come from modelschemas. Facts still come from our OpenRouter catalog until first-party rungs cover the provider being synced. Where modelschemas has a null, the consumer decides whether to fall back. modelschemas never makes that choice for it.
Out of scope
- New adapters from
docs/providers-to-add.md. Day-one they still walk the bound schema and recordfactSources. - Any OpenRouter copy into another provider’s facts, labeled or unlabeled (decided on Fill maxOutput for Grok and Mistral; grok-4.7 is missing reasoning #75, 2026-09-26).
- Benchmarks / tokenizer unless a later issue asks.
Activity
feat(catalog): fill limits, modalities, pricing, capabilities on grai…
[-]Fill contextWindow, modalities, pricing, and request-feature capabilities on grain=provider catalogs[/-][+]Fill contextWindow, modalities, pricing, and request-feature capabilities on every catalog[/+][-]Fill contextWindow, modalities, pricing, and request-feature capabilities on every catalog[/-][+]Fill OpenRouter-shaped catalog facts from the bound schema, with per-field provenance[/+]tombeckenham commentedon Sep 14, 2026
Slice 5 (pricing as an OpenRouter-shaped blob) is superseded. A blob you cannot evaluate is the wrong target — see Openstory #1605 / #1606 rate cards.
New issues (no parent; attach to a milestone when one exists):
@modelschemas/rate-cardpackagePOST /v1/estimate