Skip to content

fix(logger): route log, warn, and fatal through buffer dispatch - #8871

Open
dhaiirryyaa wants to merge 3 commits into
aws-powertools:developfrom
dhaiirryyaa:fix/logger-log-fatal-warn-buffering
Open

dhaiirryyaa wants to merge 3 commits into
aws-powertools:developfrom
dhaiirryyaa:fix/logger-log-fatal-warn-buffering

Conversation

@dhaiirryyaa

@dhaiirryyaa dhaiirryyaa commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Issue number: closes #8827

Summary

Changes

  • Replaced duplicated buffer handling across logger methods with an internal _log() dispatcher.
  • Added log(self, level, msg, *args, **kwargs) to route through the buffer and auto-flush on error log.
  • Added warn = warning and fatal = critical aliases.
  • Updated buffer record frame depth to accurately preserve caller stack frames when dispatching through _log().

User experience

Before this change, calling logger.log(), logger.warn(), or logger.fatal() bypassed buffer routing and flush-on-error logic. Now they respect buffer_at_verbosity and flush buffered logs upon error or critical log levels.

Verification

  • Added regression tests in tests/functional/logger/required_dependencies/test_powertools_logger_buffer.py:
    • test_logger_log_method_buffers_and_flushes_on_error
    • test_logger_warn_and_fatal_aliases
  • All 31 buffer tests passed locally.
  • Verified with ruff check.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@dhaiirryyaa
dhaiirryyaa requested a review from a team as a code owner November 18, 2026 16:49
@dhaiirryyaa
dhaiirryyaa requested a review from svozza November 18, 2026 16:49
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 18, 2026
@dhaiirryyaa

Copy link
Copy Markdown
Contributor Author

Hi @leandrodamascena, this PR is ready for review. It routes log(), warn(), and fatal() through the buffer dispatcher and auto-flushes on error log when enabled, resolving #8827. All 31 functional buffer tests and SonarQube quality gates are passing. Please let me know if any adjustments are needed before merge. Thanks!

@Manishram-ai Manishram-ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up, the _log dispatcher reads much better than the three copies. I checked out the branch and ran the logger tests (180 passed), mypy and a few custom levels, and found two things, both with suggestions below. With both applied, tests/functional/logger and tests/unit/logger pass (207 with the extra test), and mypy, ty and ruff are clean.

Comment thread aws_lambda_powertools/logging/logger.py Outdated
# 1. Buffer is completely disabled - log right away
# 2. DEBUG is the maximum level of buffer, so, can't bypass if enabled
# 3. Store in buffer for potential later processing
level_num = logging._checkLevel(level)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make mypy fails on this line with Module has no attribute "_checkLevel" [attr-defined], since typeshed doesn't expose it. This does the same with public calls, and gives the same ValueError for a name it doesn't know, like "info":

Suggested change
level_num = logging._checkLevel(level)
level_num = logging.getLevelName(level) if isinstance(level, str) else level
if not isinstance(level_num, int):
raise ValueError(f"Unknown level: {level!r}")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied! Replaced logging._checkLevel with logging.getLevelName(level) if isinstance(level, str) else level and added the ValueError check for unknown levels.

Comment thread aws_lambda_powertools/logging/logger.py Outdated
Comment on lines +613 to +614
level_name = logging.getLevelName(level_num)
if _check_minimum_buffer_log_level(self._buffer_config.buffer_at_verbosity, level_name):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A custom level above buffer_at_verbosity ends up in the buffer here. getLevelName(25) returns "Level 25" (or "NOTICE" after addLevelName), _check_minimum_buffer_log_level doesn't know that name and returns False, so logger.log(25, "...") with the default buffer_at_verbosity="DEBUG" gets buffered while logger.info() prints right away. On develop that call goes straight to the stdlib logger and prints.

Comparing the numbers skips the name lookup, the same way _determine_log_level compares against buffer_at_verbosity:

Suggested change
level_name = logging.getLevelName(level_num)
if _check_minimum_buffer_log_level(self._buffer_config.buffer_at_verbosity, level_name):
if level_num > logging.getLevelName(self._buffer_config.buffer_at_verbosity):

With this, the _check_minimum_buffer_log_level import at the top of the file isn't used here anymore, so ruff will ask you to drop it. This is the test I used, it fails on the current head and passes with the change:

def test_logger_log_method_does_not_buffer_custom_level_above_threshold(stdout, service_name, monkeypatch):
    monkeypatch.setenv(constants.XRAY_TRACE_ID_ENV, "1-67c39786-5908a82a246fb67f3089263f")

    # GIVEN A logger that buffers DEBUG logs only
    logger_buffer_config = LoggerBufferConfig(max_bytes=10240, buffer_at_verbosity="DEBUG")
    logger = Logger(level="DEBUG", service=service_name, stream=stdout, buffer_config=logger_buffer_config)

    # WHEN Logging with a custom level between INFO and WARNING
    logger.log(25, "custom level via log()")

    # THEN Message should be output right away, like info()
    log = capture_multiple_logging_statements_output(stdout)
    assert len(log) == 1
    assert log[0]["message"] == "custom level via log()"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied! Switched to direct numeric comparison level_num > logging.getLevelName(self._buffer_config.buffer_at_verbosity) to support custom levels, dropped the unused _check_minimum_buffer_log_level import, and added your regression test. All 207 logger tests pass and mypy / ruff are clean. Thank you for the great suggestions!

@dhaiirryyaa

Copy link
Copy Markdown
Contributor Author

Thank you for the thorough review and suggestions @Manishram-ai!

Both items have been implemented in commit b7a31fd:

  1. Public Level Check: Replaced private logging._checkLevel with public logging.getLevelName() and type validation, resolving the mypy error.
  2. Custom Log Levels: Switched buffer verbosity check to numeric comparison (level_num > logging.getLevelName(...)), correctly handling custom integer log levels without name lookup failures.
  3. Cleaned Imports & Added Test: Dropped unused _check_minimum_buffer_log_level import and added the regression test for custom log levels above threshold.

All 207 tests in tests/functional/logger and tests/unit/logger pass cleanly, and both ruff and mypy checks are green.

@codecov

codecov Bot commented Nov 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.18%. Comparing base (19c0798) to head (b7a31fd).
⚠️ Report is 4 commits behind head on develop.

Files with missing lines Patch % Lines
aws_lambda_powertools/logging/logger.py 91.66% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #8871      +/-   ##
===========================================
- Coverage    97.20%   97.18%   -0.02%     
===========================================
  Files          327      327              
  Lines        16759    16754       -5     
  Branches      1523     1521       -2     
===========================================
- Hits         16290    16283       -7     
- Misses         328      329       +1     
- Partials       141      142       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sonarqubecloud

Copy link
Copy Markdown

@leandrodamascena

Copy link
Copy Markdown
Contributor

Thanks @dhaiirryyaa, this is the shape I asked for in the issue: one _log, log() on the same route, and the two aliases. One thing before I review properly: #8837 is landing ahead of you and it replaces the fixed frame depth in _create_buffer_record with a walk that honours stacklevel, so this will conflict. Once it's on develop, please rebase, drop the depth parameter and pass stacklevel through from _log to _add_log_record_to_buffer. Your stacklevel + 1 in each method already accounts for the extra frame, so the walk should land on the right caller. Ping me when it's up.

@leandrodamascena leandrodamascena added this to the 3.42.0 milestone Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logger size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: log(), fatal() and warn() bypass buffering and flush-on-error

3 participants