Package (Required)
Checked other resources
Example Code (Python)
from langchain_google_vertexai.model_garden import ChatAnthropicVertex
def run_test(llm, stream: bool):
prompt = "Road runner leaves A at 13:37 travelling 87 km/h toward B, 210 km away. When does he arrive? Answer with a single line giving the time in HH:mm format. No explanation."
if stream:
message = None
for chunk in llm.stream(prompt):
message = chunk if message is None else message + chunk
else:
message = llm.invoke(prompt)
print("Output:", message.text)
print("Usage:", message.usage_metadata)
llm_vertexai = ChatAnthropicVertex(
model_name="claude-sonnet-5",
location="eu",
stream_usage=True,
model_kwargs={"output_config": {"effort": "high"}},
)
print("---invoke---")
run_test(llm_vertexai, stream=False)
print("---stream---")
run_test(llm_vertexai, stream=True)
"""
> ---invoke---
> Output: 16:02
> Usage: {'input_tokens': 70, 'output_tokens': 188, 'total_tokens': 258, 'input_token_details': {'cache_read': 0, 'cache_creation': 0}}
> ---stream---
> Output: 16:02
> Usage: {'input_tokens': 70, 'output_tokens': 148, 'total_tokens': 218, 'input_token_details': {'cache_creation': 0, 'cache_read': 0}}
"""Error Message and Stack Trace (if applicable)
Description
Reasoning tokens (message.usage_metadata["output_token_details"]["reasoning"]) are not reported in ChatAnthropicVertex messages, even tough they are returned from the underlying SDK and reasoning/thinking has obviously taken place (see output token count in the MRE output vs. the actual text length).
Same bug in langchain-anthropic: langchain-ai/langchain#39249
Proposed fix: https://github.com/philemon-otera/langchain-google/tree/add-reasoning-token-count
Package (Required)
Checked other resources
Example Code (Python)
Error Message and Stack Trace (if applicable)
Description
Reasoning tokens (message.usage_metadata["output_token_details"]["reasoning"]) are not reported in ChatAnthropicVertex messages, even tough they are returned from the underlying SDK and reasoning/thinking has obviously taken place (see output token count in the MRE output vs. the actual text length).
Same bug in
langchain-anthropic: langchain-ai/langchain#39249Proposed fix: https://github.com/philemon-otera/langchain-google/tree/add-reasoning-token-count