Skip to content

config: support log timeout action - #68964

Merged
ti-chi-bot[bot] merged 7 commits into
masterfrom
bb7133/log-timeout-action
Oct 13, 2026
Merged

ti-chi-bot[bot] merged 7 commits into
masterfrom
bb7133/log-timeout-action

Conversation

@bb7133

@bb7133 bb7133 commented Jun 4, 2026 •

Copy link
Copy Markdown
Member

Issue Number: close #68968

Problem Summary:

TiDB currently uses [log].timeout to detect stuck log writes and panic when the timeout is reached. This keeps the existing fail-fast behavior, but some deployments may prefer to keep TiDB serving traffic and discard later log writes after the timeout is observed.

What changed:

  • Add [log].timeout-action to TiDB config, defaulting to panic.
  • Accept valid values panic and discard, with case-insensitive validation/normalization.
  • Pass the setting through Log.ToLogConfig().
  • Update the enterprise audit submodule so audit logs also respect [log].timeout and [log].timeout-action.

Valid values:

  • panic: keep the existing behavior and panic on timeout. This is the default.
  • discard: after the timeout is observed, discard later log write/sync calls instead of panicking.

Dependencies:

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Manual test:

  • go test ./pkg/config -run 'TestLogConfig|TestLogTimeoutAction|TestConfig$' -count=1
  • go test --tags=intest ./pkg/extension/enterprise/audit -run 'TestAuditLoggerUsesGlobalLogTimeoutConfig|TestAuditLogger$' -count=1
  • git diff --check

Side effects:

  • panic remains the default and preserves existing behavior.
  • discard mode may lose log records after a timeout is observed. It cannot cancel a goroutine already blocked in the underlying OS/file write; it only prevents later log write/sync calls from continuing to block or panic.

Documentation:

  • New config item should be documented in the TiDB configuration reference.

Release note:

Add the `[log].timeout-action` configuration item to choose whether TiDB panics or discards later log writes when `[log].timeout` is reached.

Summary by CodeRabbit

  • New Features
    • Added log.timeout-action configuration option to control logging timeout behavior: panic (default) or discard (case-insensitive), with validation for invalid values.
  • Tests
    • Added coverage for default, configured, and invalid timeout-action values.
  • Chores
    • Updated logging-related dependencies and pinned artifacts.
    • Updated build system settings and the enterprise extension reference.

@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jun 4, 2026
@pantheon-ai

pantheon-ai Bot commented Jun 4, 2026 •

Copy link
Copy Markdown

@bb7133 I've received your pull request and will start the review. I'll conduct a thorough review covering code quality, potential issues, and implementation details.

⏳ This process typically takes 10-30 minutes depending on the complexity of the changes.

ℹ️ Learn more details on Pantheon AI.

@ti-chi-bot ti-chi-bot Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 4, 2026
@coderabbitai

coderabbitai Bot commented Jun 4, 2026 •

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a timeout-action field to the Log config struct with values panic (default) or discard. The field is validated case-insensitively in Config.Valid() and wired into the underlying zaplog.Config. The github.com/pingcap/log dependency is bumped in go.mod and DEPS.bzl. The enterprise submodule pointer is also updated.

Changes

Log Timeout Action Configuration Feature

Layer / File(s) Summary
Config field, defaults, and dependency pins
pkg/config/config.go, go.mod, DEPS.bzl
Adds TimeoutAction string field with toml:"timeout-action" / json:"timeout-action" tags to Log struct; sets default to zaplog.LogTimeoutActionPanic; bumps github.com/pingcap/log pseudo-version in go.mod and DEPS.bzl; removes // indirect marker from lumberjack.v2.
Validation and zaplog wiring
pkg/config/config.go
Lowercases log.timeout-action in Config.Valid(), maps "" and "panic" to panic, maps "discard" to discard, returns error for any other value; wires Log.TimeoutAction into zaplog.Config.TimeoutAction inside ToLogConfig().
Tests
pkg/config/config_test.go, pkg/config/BUILD.bazel
Reformats existing ToLogConfig() assertions to include TimeoutAction; adds TestLogTimeoutAction covering default value, normalization of enum and string input ("DISCARD"), and validation failure for "invalid"; increments test shard count from 37 to 38.

Enterprise Submodule Update

Layer / File(s) Summary
Enterprise submodule pointer update
pkg/extension/enterprise
Submodule reference updated to a new commit hash.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'config: support log timeout action' directly and concisely summarizes the main change—adding a new configuration option for controlling timeout action behavior.
Description check ✅ Passed The PR description comprehensively addresses all template requirements: issue number, problem summary, detailed explanation of changes, checklist items, side effects, documentation needs, and release notes.
Linked Issues check ✅ Passed All coding requirements from issue #68968 are met: new config field with validation, default value of 'panic', case-insensitive value handling, integration with Log.ToLogConfig(), and enterprise audit updates.
Out of Scope Changes check ✅ Passed All changes directly support the feature objective: config field, validation logic, test coverage, dependency updates (go.mod, DEPS.bzl), Bazel test shard count adjustment, and submodule pointer update.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch bb7133/log-timeout-action

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🐰 A rabbit hops where logs may stall,
With timeout-action set for all—
Say panic to crash, or discard to glide,
The lumberjack now walks by our side.
Modules bumped and submodules, too,
Every log write knows just what to do! 🌿

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot ti-chi-bot Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jun 4, 2026
@codecov

codecov Bot commented Jun 5, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 44.44444% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.8609%. Comparing base (7a3dacb) to head (8fb6773).
⚠️ Report is 23 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #68964        +/-   ##
================================================
- Coverage   76.3015%   71.8609%   -4.4406%     
================================================
  Files          2041       2118        +77     
  Lines        554276     604658     +50382     
================================================
+ Hits         422921     434513     +11592     
- Misses       130455     168249     +37794     
- Partials        900       1896       +996     
Flag Coverage Δ
integration 41.1896% <44.4444%> (+1.5179%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 58.8028% <ø> (ø)
parser ∅ <ø> (∅)
br 46.3866% <ø> (-16.3231%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@lance6716 lance6716 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems there's no test to verify the behaviour. Maybe inject some blocking to IO and check panic / discard really work

@tiprow

tiprow Bot commented Jun 14, 2026

Copy link
Copy Markdown

@bb7133: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
fast_test_tiprow ad61f32 link true /test fast_test_tiprow

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@bb7133

bb7133 commented Oct 8, 2026

Copy link
Copy Markdown
Member Author

/test check-dev

@ti-chi-bot ti-chi-bot Bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Oct 8, 2026
@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Oct 8, 2026
@ti-chi-bot

ti-chi-bot Bot commented Oct 8, 2026

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2026-10-08 00:55:28.375861623 +0000 UTC m=+260538.123092472: ☑️ agreed by lcwangchao.
  • 2026-10-08 02:11:50.521155878 +0000 UTC m=+265120.268386727: ☑️ agreed by YangKeao.

@ti-chi-bot

ti-chi-bot Bot commented Oct 10, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfzjywxk, lcwangchao, terry1purcell, YangKeao

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [YangKeao,cfzjywxk,lcwangchao,terry1purcell]
  • pkg/config/OWNERS [YangKeao,cfzjywxk,lcwangchao,terry1purcell]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the approved label Oct 10, 2026
@bb7133

bb7133 commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

/retest

@ti-chi-bot
ti-chi-bot Bot merged commit 9763d73 into master Oct 13, 2026
34 checks passed
@ti-chi-bot
ti-chi-bot Bot deleted the bb7133/log-timeout-action branch October 13, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support configurable action when log writes time out

7 participants