Skip to content

ci: run the macOS MAUI integration tests in their own job - #5797

Draft
ric-oliv wants to merge 2 commits into
mainfrom
ci/run-macos-maui-tests-in-own-job
Draft

ric-oliv wants to merge 2 commits into
mainfrom
ci/run-macos-maui-tests-in-own-job

Conversation

@ric-oliv

@ric-oliv ric-oliv commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

On macOS, the MAUI cases in cli.Tests.ps1 are the longest part of the integration tests. They build three Release apps and check what sentry-cli uploads for each:

Case #5788 #5793
Android build 186 s 244 s
iOS build 197 s 253 s
Mac Catalyst build 85 s 81 s
Whole integration test step 13.4 min 14.6 min

With the other open CI PRs applied, the macOS leg is the slowest leg in the build workflow at 32 to 37 minutes, so those 8 to 10 minutes are on the critical path.

This PR:

  • moves the MAUI (<framework>) block unchanged from cli.Tests.ps1 to a new cli-maui.Tests.ps1, with the same header,
  • skips that block when SKIP_MAUI_TESTS is true, which the build legs set only on macOS,
  • adds a MAUI integration test (macos) job that packs the six packages the block registers and runs cli-maui.Tests.ps1 with the same github-workflows action.

The Windows and Linux legs keep running the Android case, because it covers host-specific parts like the sentry-cli binary and Sentry.targets' Exec calls.

The new job tests packages it builds itself, not the ones the macOS leg publishes. For these cases that's equivalent. The macOS leg builds with CI_PUBLISHING_BUILD=true, which only adds the other desktop platforms' native libraries and sentry-cli binaries to the Sentry package. MAUI apps don't use the native libraries, and a macOS arm64 build always includes the Darwin arm64 sentry-cli.

Checked locally:

  • Pester discovery finds the same 39 tests, 36 in cli.Tests.ps1 and 3 in cli-maui.Tests.ps1, and skips the 3 MAUI cases when SKIP_MAUI_TESTS=true.
  • The moved block is identical to the original, line for line.
  • actionlint reports the same findings as on main.

On release branches the new job builds the Cocoa SDK from source, like the macOS leg does.

The first run's MAUI job failed in pack with NU5026. Packing Sentry.Maui while building it on macOS looks for its Windows target's .pri file, which macOS doesn't generate. The job now builds first and packs with --no-build, as the build legs do.

Measured on the second run, against 3 runs from the same evening without this PR:

Without this PR This PR
macOS leg 37.1 to 51.1 min 24.1
Its integration test step 12.3 to 15.5 3.7
MAUI integration test (macos) job 15.3
Android, iOS and Mac Catalyst cases 7.8 to 9.6 in the macOS leg 7.0 in the new job
Pester totals on the macOS leg 69 passed, 34 skipped 66 passed, 37 skipped
Pester totals in the new job 3 passed, 0 skipped

The 51.1 was a run whose Build step took 23.6 minutes, against 14.5 and 17.0 in the other two.

So the 3 MAUI cases moved from the macOS leg to the new job, and every other case still runs in the leg. The new job builds for 4.0 minutes, packs in 0.2 and finishes well before the macOS leg.

#skip-changelog

On macOS the MAUI cases in cli.Tests.ps1 build three Release apps, for
Android, iOS and Mac Catalyst. That takes 8 to 10 of the macOS leg's 13
to 15 integration test minutes, and the macOS leg is the slowest leg in
the build workflow.

The cases move unchanged to cli-maui.Tests.ps1. The macOS leg skips
them, and a new job on another Mac packs the six packages they use and
runs them. The other legs still run the Android case.

The new job's packages aren't the ones the macOS leg publishes. But
CI_PUBLISHING_BUILD only adds the other desktop platforms' native
libraries and sentry-cli binaries, which MAUI apps don't use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.13%. Comparing base (b41b96c) to head (9e33f11).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5797   +/-   ##
=======================================
  Coverage   76.13%   76.13%           
=======================================
  Files         528      528           
  Lines       19579    19579           
  Branches     3829     3829           
=======================================
  Hits        14906    14906           
  Misses       3791     3791           
  Partials      882      882           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

dotnet pack with a build failed with NU5026 on Sentry.Maui's Windows
target, looking for a .pri file that isn't generated on macOS. The
build legs build first and pack with --no-build, so this job does too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant