Skip to content

ci(timing): budget the job reads and publish a cut-short report (#1930) - #1931

Merged
JarryShaw merged 1 commit into
mainfrom
ci/1930-timing-report-cap
Oct 19, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
ci/1930-timing-report-cap

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

A full local --days 7 run of util/ci_timing.py took 16.6 min over 677 API calls, with no retries or rate limiting. That is longer than the job's 15-minute cap. Of that time, 908 s went on job reads, and Unit Tests' 301 reads alone took 734 s, at 2.4 s each, one at a time.

  • Job reads now run last, in an order that covers every sample even if reading stops part-way. They stop at --budget-minutes 30.
  • If reading stops early, the report is still written, with a note and a warning annotation.
  • Progress goes to stderr.
  • The step cap is 35 and the job cap is 40, both above the measured run.
  • The summary and upload steps run on always().

Tests cover the read order, both budgets still writing a report, the progress output and the nested caps. They fail on main.

Closes #1930

- util/ci_timing.py: read the listings, workflow files and JUnit artifacts
  first and the job reads last, interleaved so every prefix spans each
  sample; stop them at --budget-minutes (default 30) or --max-calls and
  still write the report, with a note and a warning annotation
- util/ci_timing.py: print a progress line per step to stderr, so a run
  killed by a cap shows how far it got; skip or abandon the runner-pool
  measurement once the budget is spent
- ci-timing.yml: give the report step a 35-minute cap and the job a
  40-minute one, above the measured 16.6-minute full run; add the summary
  and upload the report on always()
- docstring, workflow comments and workflows.rst carry the measured calls
  and where the time goes
- tests: the read order, both budgets writing a report, progress on
  stderr, and the nested caps

Closes #1930
@JarryShaw JarryShaw added ci Pull requests that change CI or workflow configuration (ci: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 19, 2026
@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 19, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at 7085b5e46. A Sonnet cross-review checked the profile logs against the stated phases: 677 calls, 993 s, 601 job reads taking 907.5 s, and no retries or sleeps. Runs whose jobs were never read leave the sample instead of counting as empty, and the budget clock starts when collect starts. The caps nest at 40/35/30, and the YAML parses with concurrency and permissions unchanged. Restoring main's script and workflow makes 8 tests fail. With the fix, test_ci_timing.py and the tier guard pass: 181 tests. The dated measurement in workflows.rst replaces the dated figure that was already there.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 90.07% (unit tier, Python 3.15, 7085b5e46, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 2 20 2 96.77%
pcapkit/const 19595 881 2618 867 92.11%
pcapkit/corekit 2504 70 792 36 96.54%
pcapkit/dumpkit 276 3 100 3 98.40%
pcapkit/foundation 2916 164 1016 34 93.59%
pcapkit/interface 138 7 52 5 93.68%
pcapkit/protocols 17411 224 4580 189 98.10%
pcapkit/toolkit 872 177 298 8 77.18%
pcapkit/utilities 522 2 158 4 99.12%
pcapkit/vendor 4431 2341 1018 156 43.64%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit 9ccd35d into main Oct 19, 2026
58 checks passed
@JarryShaw
JarryShaw deleted the ci/1930-timing-report-cap branch October 19, 2026 15:00
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 19, 2026
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Pull requests that change CI or workflow configuration (ci: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

ci(timing): the weekly CI timing report hits its 15-minute cap and has never uploaded a report

1 participant