Repository navigation
RubyLLM2 W2-04: Port the Claude 5.5 and GPT-6 Bedrock registry entries to the 2.0 fork - #40
Conversation
The 2.0 registry has no bedrock entry for Claude Sonnet, Opus or Haiku 5.5, or for GPT-6 Sol, GPT-6 Luna, GPT-6.1 Sol or GPT-6.1 Terra, so Supernova cannot find the models it uses in production: no pricing, no capabilities and no context window. This ports upstream main's 26 entries for those models, without the thinking-off toggle on Haiku 5.5, and adds three GPT-6.1 Terra entries written from AppFolio's Bedrock agreement offer. The us. Haiku 4.5 entry moves to the global rate, which is what that profile bills and what the 1.16 fork already carries. openai.gpt-5.6-sol keeps its 2.0 rates, which match the October 2026 AWS bill; the spec pins them. Co-authored-by: Sam Boland <sam.boland@appfolio.com>
samuelboland
left a comment
There was a problem hiding this comment.
Verified: the 29 entries in models.json match the brief exactly and the only other change is the Haiku 4.5 us. price; spec passes under three seeds; validation, models and mantle specs green.
|
🤖 Supernova Code Review — View trace — via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com) |
There was a problem hiding this comment.
Reviewed the new registry_prices_spec.rb and spot-checked the added and changed models.json entries at the PR head: us.anthropic.claude-haiku-4-5-20251001-v1:0 (L19107), us.anthropic.claude-haiku-5-5 (L19200), us.anthropic.claude-opus-5-5 (L19900), openai.gpt-6-luna / gpt-6-sol / gpt-6.1-sol / gpt-6.1-terra (L16886–17271), and global.openai.gpt-6.1-sol / gpt-6.1-terra (L12796–12986). The prices, tiers, thresholds, effort values, and the missing Haiku 5.5 toggle all match what the PR description says. I also checked that Model#to_h returns fresh pricing hashes, so the metadata.cost.tiers rebuild example doesn't change the shared registry model.
The only finding is one nit, inline: the global Terra entry has no test.
— via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)
| %w[openai.gpt-6.1-terra us.openai.gpt-6.1-terra].each do |id| | ||
| it "prices #{id} at the US Terra rates with a 272,000 tier" do | ||
| text = bedrock_model(id).pricing.text_tokens | ||
|
|
There was a problem hiding this comment.
nit: global.openai.gpt-6.1-terra (models.json L12892) is also hand-written from the agreement offer (1 / 5 / 0.05 / 1.25, tier 2 / 7.5 / 0.1 / 2.5), but no spec checks it. Adding a matching example for the global rates would catch a typo in the only Terra entry this list leaves out.
What this does
Adds the bedrock registry entries that
lib/ruby_llm/models.jsononsupernova-2-0was missing for models Supernova runs in production: Claude Sonnet 5.5, Opus 5.5 and Haiku 5.5 (bare id plus theau.,eu.,global.,jp.andus.prefixes upstream has), and GPT-6 Sol, GPT-6 Luna, GPT-6.1 Sol and GPT-6.1 Terra (bare,us.andglobal.). Without them RubyLLM 2.0 cannot find these ids, so they have no pricing, capabilities or context window, and Supernova's thinking clamp returns'none'for them.29 entries, each inserted at the position the file's existing bedrock order puts it. The file is otherwise unchanged: a line diff against the base is 2,621 insertions and 8 deletions (the 8 are the Haiku 4.5 price lines below). GitHub's larger deletion count comes from how its diff lines up the inserted blocks.
main's, copied as is, except for the Haiku 5.5 change below.openai.andus.: 1.1 / 5.5 / 0.055 / 1.375, and 2.2 / 8.25 / 0.11 / 2.75 above 272,000 input tokens.global.: 1 / 5 / 0.05 / 1.25, and 2 / 7.5 / 0.1 / 2.5 above 272,000.nonethroughmax. Terra acceptsnone(checked live 2026-11-09); GPT-6.1 Sol does not.pdfinput is listed on the bare id only, where it was checked live.Differences from upstream
{"type":"toggle"}inmetadata.reasoning_options, onlyeffort(lowtomax). Upstream has the toggle; Sam decided on 14 Oct that Haiku 5.5 gets no thinking-off toggle.us.price. The existing entryus.anthropic.claude-haiku-4-5-20251001-v1:0changes from the regional rate onsupernova-2-0(1.1 / 5.5 / 0.11 / 1.375) to the global rate (1 / 5 / 0.1 / 1.25), in bothpricing.text_tokens.standardandmetadata.cost. It bills at about 94% of the regional rate, and the 1.16 fork has carried the global price since fork PR fix: correct Bedrock us. regional premium & cache-read rates #34. No other field or Haiku 4.5 entry changes.Not changed
openai.gpt-5.6-solis left as it is onsupernova-2-0: 4.4 / 22 / 0.44 / 5.5, with a 272,000 tier at 8.8 / 33 / 0.88 / 11. The 1.16 fork carries 5 / 30 / 0.5 / 6.25 with no tier, and that was wrong: the October 2026 AWS bill charged exactly 4.40 / 22.00 / 0.44 / 5.50. The new spec asserts the 2.0 values, so a later change to them is deliberate.No
metadata.endpointmarks are added to the GPT entries; RubyLLM2 W2-02 adds them.Tests
New
spec/ruby_llm/providers/bedrock/registry_prices_spec.rb(25 examples), reading the bundled registry throughRubyLLM.models.find(id, provider: 'bedrock'):us., Opus 5.5us., Sonnet 5.5us., Haiku 5.5us.).us.andglobal.: long-context rates and the 100,000 threshold, both from the pricing hash and rebuilt frommetadata.cost.tiersalone.reasoning_optionsis exactly effortlowtomax, with no toggle.us., Sonnet 5.5us., GPT-6 Sol, GPT-6 Luna, GPT-6.1 Sol and GPT-6.1 Terra resolve on bedrock with their context windows and max output tokens.us.): the US rates, the 272,000 tier, and effortnonethroughmax.openai.gpt-5.6-solkeeps its billed rates.Runs (with
LANG=C.UTF-8; the container's default US-ASCII locale cannot readmodels.json):spec/ruby_llm/models_json_validation_spec.rb,spec/ruby_llm/providers/bedrock/models_spec.rbandspec/ruby_llm/providers/bedrock/mantle_spec.rbtogether: 70 examples, 0 failures, 1 pending (the existing mantle Claude example that needs an AWS agreement).bundle exec rubocopon the new spec: no offenses.The whole suite was not run. No live calls or cassettes were needed.
This PR targets
supernova-2-0, notmain, which stays the 1.16 rollback line. It is not for upstreamcrmne/ruby_llm. Supernova moves to this branch later, in RubyLLM2 W3-01.