Skip to content

ci: measure the round 4 CI changes together (do not merge) - #5788

Closed
ric-oliv wants to merge 5 commits into
mainfrom
ci/measure-combined-round4-changes
Closed

ric-oliv wants to merge 5 commits into
mainfrom
ci/measure-combined-round4-changes

Conversation

@ric-oliv

Copy link
Copy Markdown
Member

This draft is only here to measure CI with all four round 4 changes applied together. Please don't review or merge it. I'll close it once the runs finish.

It's main plus these PRs, applied in order:

Each PR shortens a different leg, so on its own a PR only moves the wall clock when it shortens whichever leg is slowest in that run. My model of the pipeline puts the combined wall clock at about 35 minutes, down from a median of 50.4 on main. This run checks that.

The only conflict was in the environment action, where #5782 and #5784 both gate the Android test app restore. I kept both checks, so the restore is skipped on Windows and whenever the solution filter leaves out Sentry.Android.AssemblyReader.Tests.

#skip-changelog

ric-oliv and others added 5 commits October 23, 2026 23:01
The app samples only check that they compile, but their Release builds trim,
AOT-compile and package each app. In the macOS leg that's about 13.7 of the
Build step's 28.3 task-minutes, and it's the whole tail of the step once the
Cocoa SDK comes from the cache. On Windows, the MAUI sample's Android build
is the last thing the Build step finishes.

The Android, iOS, Mac Catalyst, macOS and MAUI samples now build in a
separate job that starts with the .NET legs. It builds all of them on macOS,
and the MAUI sample on Windows, which is the only build of its Windows
target. The legs' solution filters no longer include them, and
Sentry-CI-Build-Samples-macOS.slnf lists them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Windows .NET legs spend about 12 minutes in Setup Environment and
Restore, against about 4.5 on Linux. On the Windows images, setup-dotnet
installs into C:\Program Files\dotnet, where the SDK installs workloads as
MSI packages, and NuGet extracts packages into the user profile on C:. The
workload step takes 5.6 minutes there, against about 2 on macOS and Linux,
which install more workloads.

This installs .NET and the NuGet cache next to the workspace, where
workloads install as plain files. On the x64 runners that also moves them to
D:, and on the arm64 runners they stay on C:, so the two legs separate the
install mode from the drive. Windows also stops restoring the Android test
app, which its solution filters exclude.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Setup Environment runs dotnet workload restore without a project, so it
evaluates every project in Sentry.slnx on every leg. The three Linux legs
without mobile targets install android, maui-android, maui-tizen, macos,
wasm-tools and wasm-tools-net9. Windows installs ios, maccatalyst and macos.
Every leg installs maui-tizen, which nothing targets, and macos, which only
the macOS sample needs.

The build legs now pass their solution filter, and the step restores the
workloads for the projects in it. dotnet workload restore doesn't accept a
solution filter, so the step passes its projects. The Android test app is
restored only when the filter includes the tests that build it, and the legs
without mobile targets skip Java and the Android SDK.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the build legs, the unit tests and the integration tests run one after
the other, though the integration tests only need the packages. On win-x64
the Test step takes 5.8 minutes and Integration test 14.

Outside the publishing build, Test now runs as a background step. The job
packs and runs the integration tests next to it, then waits for it before
uploading coverage. The macOS publishing build keeps the current order,
because its sparse checkout removes the test outputs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every solution build runs before.Sentry.sln.targets, which calls
git submodule update --init --recursive. With the tests in the background,
dotnet test and dotnet pack started in the same second on linux-x64, and
the second git call failed on the lock for a submodule's config file, so
the tests failed before they ran. Packing takes under a minute, so the
tests now start once it's done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.12%. Comparing base (ff0355e) to head (6bc6d24).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5788      +/-   ##
==========================================
- Coverage   76.13%   76.12%   -0.01%     
==========================================
  Files         528      528              
  Lines       19579    19579              
  Branches     3829     3829              
==========================================
- Hits        14906    14905       -1     
- Misses       3791     3792       +1     
  Partials      882      882              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: high PR risk score: high

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant