Skip to content

chore(catalog): sync from gateway — 4 added, 1 corrected, 0 retired - #415

Merged
guillaumegay13 merged 1 commit into
mainfrom
bot/catalog-sync
Oct 16, 2026
Merged

guillaumegay13 merged 1 commit into
mainfrom
bot/catalog-sync

Conversation

@guillaumegay13

Copy link
Copy Markdown
Collaborator

Automated sync between the Manifest gateway and this catalog (modelparams-agent).

How entries are verified: the local gateway loads this branch's catalog; for every declared param and test value the agent configures it through the gateway's model-params API, sends a request through the proxy pinned to the model, and checks the gateway applied the value upstream and got a 200 from that model. Params that fail are removed from the entry before it is filed.

This PR is rebuilt from main on every run and merged automatically once CI is green.

Added (4)

Entry Params verified Dropped Corrected Cloned from
fireworks/muse-spark-open.yaml max_tokens temperature top_p top_k presence_penalty frequency_penalty response_format.type — — fireworks/muse-glimmer-30b.yaml
alibaba/pre-mock_new_gateway_session_affinity.yaml max_completion_tokens temperature top_p extra_body.top_k extra_body.enable_thinking extra_body.thinking_budget — — alibaba/deepseek-v3.2.yaml
nvidia/nemotron-3.5-super-vl-120b-a12b.yaml temperature top_p max_tokens reasoning_effort reasoning_budget — — nvidia/nemotron-3.5-lightning-30b-a3b.yaml
nvidia/step-5-preview.yaml temperature top_p max_tokens frequency_penalty presence_penalty seed stop — nvidia/deepseek-v4-flash-0731.yaml
Evidence

models/fireworks/muse-spark-open.yaml

  • baseline: 200 from fireworks/accounts/fireworks/models/muse-spark-open (api_key)
  • max_tokens ✅ 1
  • temperature ✅ 0, 2
  • top_p ✅ 0, 1
  • top_k ✅ 1, 100
  • presence_penalty ✅ -2, 2
  • frequency_penalty ✅ -2, 2
  • response_format.type ✅ "text", "json_object"

models/alibaba/pre-mock_new_gateway_session_affinity.yaml

  • baseline: 200 from qwen/pre-mock_new_gateway_session_affinity (api_key)
  • max_completion_tokens ✅ 1
  • temperature ✅ 0, 1.9
  • top_p ✅ 0, 1
  • extra_body.top_k ✅ 20, 0
  • extra_body.enable_thinking ✅ true, false
  • extra_body.thinking_budget ✅ 1

models/nvidia/nemotron-3.5-super-vl-120b-a12b.yaml

  • baseline: 200 from nvidia/nvidia/nemotron-3.5-super-vl-120b-a12b (api_key)
  • temperature ✅ 1
  • top_p ✅ 0.95, 1
  • max_tokens ✅ 16384, 1
  • reasoning_effort ✅ "none", "low", "high"
  • reasoning_budget ✅ 16384, -1

models/nvidia/step-5-preview.yaml

  • baseline: 200 from nvidia/stepfun-ai/step-5-preview (api_key)
  • temperature ✅ 0.6, 0, 1
  • top_p ✅ 0.95, 1
  • max_tokens ✅ 4096, 1, 16384
  • frequency_penalty ✅ 0, -2, 2
  • presence_penalty ✅ 0, -2, 2
  • ❌ dropped seed: seed=18446744073709552000 → HTTP 400: Failed to deserialize the JSON body into the target type: invalid type: floating point 1.8446744073709552e+19, expected i64 at line 1 column 183
  • ❌ dropped stop: no testable value (string without default)

Corrected (1)

Existing entries re-verified through the gateway (daily rotation). A declared bound was refused by the vendor and one step inside was accepted; or enum values were refused by the vendor and the corrected list (refused values removed, values the vendor lists as supported added) was served to the gateway on a draft catalog and every value passed.

  • models/xai/grok-4.7.yaml
    • top_p: declared min 0 refused (HTTP 400: top_p must be positive but top_p = 0); 0.01 accepted → min corrected to 0.01

Needs a human: params now refused (12)

Re-verification found these declared params refused. Not removed — the param guard blocks removals and dropping a param is a maintainer's call.

  • models/anthropic/claude-opus-4-6.yaml top_p: top_p=1 → gateway did not apply it upstream (recorded undefined)
  • models/anthropic/claude-opus-4-6.yaml thinking.type thinking.budget_tokens: thinking.type="enabled" → HTTP 400: max_tokens must be greater than thinking.budget_tokens. Please consult our documentation at https://platform.claude.com/docs/en/build-with-claude/extended-t (same error for each)
  • models/anthropic/claude-sonnet-4-6.yaml top_p: top_p=1 → gateway did not apply it upstream (recorded undefined)
  • models/anthropic/claude-sonnet-4-6.yaml thinking.type thinking.budget_tokens: thinking.type="enabled" → HTTP 400: max_tokens must be greater than thinking.budget_tokens. Please consult our documentation at https://platform.claude.com/docs/en/build-with-claude/extended-t (same error for each)
  • models/mistral/magistral-medium-latest.yaml prompt_mode: prompt_mode="reasoning" → HTTP 400: Reasoning prompt mode is not enabled for this model
  • models/mistral/magistral-medium-latest.yaml response_format.type: response_format.type="json_schema" → HTTP 400: Schema response format type requires a json_schema
  • models/mistral/mistral-medium-latest.yaml response_format.type: response_format.type="json_schema" → HTTP 400: Schema response format type requires a json_schema
  • models/moonshot/kimi-k2.7-code.yaml response_format.type: response_format.type="json_schema" → HTTP 400: Invalid request: missing required parameter: 'response_format.json_schema'
  • models/nvidia/gpt-oss-20b.yaml seed: seed=18446744073709552000 → HTTP 400: 1 validation error: {'type': 'less_than_equal', 'loc': ('body', 'seed'), 'msg': 'Input should be less than or equal to 9223372036854775807', 'input': 1844674407
  • models/xai/grok-4.7.yaml response_format.type: response_format.type="json_schema" → HTTP 422: Upstream provider rejected the request

Every added param and every corrected bound was verified end-to-end through the gateway.
@vercel

vercel Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
modelparams.dev Ready Ready Preview Oct 16, 2026 5:39am UTC

Request Review

@github-actions github-actions Bot added the model Add a model that's missing label Oct 16, 2026
@guillaumegay13
guillaumegay13 merged commit 463d3ba into main Oct 16, 2026
13 checks passed
@guillaumegay13
guillaumegay13 deleted the bot/catalog-sync branch October 16, 2026 05:39

This branch was successfully deployed

1 active deployment
Preview — bd8d8204 Deployed Oct 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model Add a model that's missing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant