Skip to content

test: compare start-lambda errors by code and message - #9455

Merged
licjun merged 1 commit into
aws:developfrom
licjun:fix/start-lambda-error-assertions
Nov 20, 2026
Merged

licjun merged 1 commit into
aws:developfrom
licjun:fix/start-lambda-error-assertions

Conversation

@licjun

@licjun licjun commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Which issue(s) does this change fix?

N/A. Fixes the local-start-lambda integration test failures on develop after the boto3 bump to 1.44.5 (https://github.com/aws/aws-sam-cli/actions/runs/44637882278).

Why is this change necessary?

botocore 1.44 defaults to the standard retry mode. The start-lambda test client sets max_attempts to 0, so a failed call now sets MaxAttemptsReached, and ClientError adds (reached max retries: 0) to its string:

An error occurred (NotImplemented) when calling the Invoke operation (reached max retries: 0): log-type: Tail is not supported. None is only supported.

The error-case tests compared str(ClientError) exactly, so 10 of them failed.

How does it address the issue?

The tests now compare response["Error"]["Code"] and response["Error"]["Message"] instead of the exception string. These are what the local Lambda service returns, so the tests no longer depend on how botocore formats the exception. The expected codes and messages are unchanged.

What side effects does this change have?

None. Test-only change.

Mandatory Checklist

PRs will only be reviewed after checklist is complete

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

botocore 1.44 defaults to the standard retry mode. With the test
client's max_attempts of 0, a failed call now sets MaxAttemptsReached,
and ClientError adds " (reached max retries: 0)" to its string. The
start-lambda error tests compared str(ClientError) exactly, so they
failed after the boto3 bump to 1.44.5.

Compare the Code and Message of the error response instead. These are
what the local Lambda service returns and do not depend on how botocore
formats the exception.
@licjun
licjun requested a review from a team as a code owner November 20, 2026 00:05
@aws-sam-tooling-bot

Copy link
Copy Markdown

🧪 Hands-on test: No problems observed in 4 scenarios

Commit f6e6890 · build LambdaToolingHandsOn-Docker:06bbffdd-5456-4bbb-b4e4-c5a3386d16c5. The plan's scenarios ran on an isolated build host; a problem is reported only if it reproduced at least twice. Passing scenarios show only that these checks behaved as expected, not that the change is correct.

Scenarios

Scenario Planned check Result Runner's note
S1 Main path: the 7 changed error-case tests in test_start_lambda.py pass with the PR's Code/Message assertions under boto… ✅ pass Head (boto3 1.44.5, botocore 1.44.7, Docker 29.9.2): all 7 targeted tests PASSED, none skipped, no reruns shown in output (7 passed in 15.47s, exit 0). Base, same setup with reruns disabled: all 7 FA…
S2 Main path for the CDK template: the 3 changed error-case tests in test_start_lambda_cdk.py pass with the PR. ✅ pass Head: all 3 CDK error-case tests PASSED (exit 0). The first head run happened in parallel with the base run and logged a Docker 409 'removal of container ... is already in progress' during teardown, …
S3 Check the PR's root-cause claim directly: str(ClientError) gains the retry suffix, while response Error Code and Messag… ✅ pass Ran samdev local start-lambda (head) on port 3010 with tests/integration/testdata/invoke/template.yml and called it directly with boto3 under retry modes None, legacy and standard (max_attempts 0). E…
S4 Check that the non-error tests in the same files, which the PR does not touch, still pass: the PR must not have disturb… ✅ pass Head: TestLambdaServiceWithInlineCode (incl. test_invoke_function_without_inline_code) and TestCDKLambdaService (incl. test_invoke_with_no_data, test_invoke_with_log_type_None, test_invoke_with_invoc…
Runner's notes in full
  • S1: Head (boto3 1.44.5, botocore 1.44.7, Docker 29.9.2): all 7 targeted tests PASSED, none skipped, no reruns shown in output (7 passed in 15.47s, exit 0). Base, same setup with reruns disabled: all 7 FAILED with AssertionError. Every actual string contains 'when calling the Invoke operation (reached max retries: 0):', e.g. 'An error occurred (NotImplemented) when calling the Invoke operation (reached max retries: 0): log-type: Tail is not supported. None is only supported.' (exit 1). This matches the PR's claim.
  • S2: Head: all 3 CDK error-case tests PASSED (exit 0). The first head run happened in parallel with the base run and logged a Docker 409 'removal of container ... is already in progress' during teardown, though all 3 tests still passed. That was a side effect of my concurrent runs. A rerun of head by itself with reruns disabled passed 3/3 cleanly. Base: all 3 FAILED with AssertionError, with actual strings containing '(reached max retries: 0)' (exit 1).
  • S3: Ran samdev local start-lambda (head) on port 3010 with tests/integration/testdata/invoke/template.yml and called it directly with boto3 under retry modes None, legacy and standard (max_attempts 0). Exit 0. Every call raised ClientError, and no call returned without an error. Code and Message matched the PR's expected values and were the same in all three modes: NotImplemented / 'log-type: Tail is not supported. None is only supported.', NotImplemented / 'invocation-type: DryRun is not supported. Only Event and RequestResponse are supported.', InvalidRequestContent / 'Could not parse request body into json: No JSON object could be decoded'. str(e) contains '(reached max retries: 0)' for None and standard but not for legacy. So the suffix comes from botocore's formatting, not from the local service.
  • S4: Head: TestLambdaServiceWithInlineCode (incl. test_invoke_function_without_inline_code) and TestCDKLambdaService (incl. test_invoke_with_no_data, test_invoke_with_log_type_None, test_invoke_with_invocation_type_RequestResponse, test_invoke_with_no_data_empty_response): 7 passed, none skipped, exit 0. Not run on base, because compare_base is false for this scenario.

🤖 Automated hands-on test — not a review approval · model claude-opus-5.5 · Kiro kiro-cli 2.33.1 · build LambdaToolingHandsOn-Docker:06bbffdd-5456-4bbb-b4e4-c5a3386d16c5 · commit f6e6890

History (1)
Commit Result When (UTC) Build
f6e6890 No problems observed 2026-11-20 00:12 LambdaToolingHandsOn-Docker:06bbffdd-5456-4bbb-b4e4-c5a3386d16c5

@licjun
licjun added this pull request to the merge queue Nov 20, 2026
Merged via the queue into aws:develop with commit 67cdea6 Nov 20, 2026
56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants