[ric-oliv]

Part 3 of 6 for #5806, based on #5808. The build legs built, ran the unit tests and then ran the integration tests, one step after another, and the app samples made their Build step longer. This PR moves both into jobs that run next to the legs, one commit each.

Build the app samples in their own job. The app samples only check that they compile, but their Release builds trim, AOT-compile and package each app. On macOS that was about 13.7 of the Build step's 28.3 task-minutes.

• A new Samples job builds all of them on macOS, the Android and MAUI samples on Linux, and the MAUI sample on Windows, which is the only build of its Windows target.
• The legs' solution filters drop them, regenerated from generate-solution-filters-config.yaml.
• So that msbuild.exe still builds an Android and a Windows app, the MSBuild job builds the MAUI sample after the solution filter. That takes about 3 minutes and the job still ends well before the slowest check.
• The only change in coverage is that win-arm64 no longer builds the MAUI sample. win-x64 builds the same targets.

Run the integration tests in their own jobs. A new integration-test matrix job runs next to the legs:

• On macOS it runs the MAUI cases, which build three Release apps and took 8 to 10 minutes of the macOS leg. They move unchanged from cli.Tests.ps1 to cli-maui.Tests.ps1, and the macOS leg skips them through SKIP_MAUI_TESTS.
• On linux-x64, win-x64 and win-arm64 it runs every integration test file, which those legs now skip.
• Each entry builds and packs only the packages its tests use. They aren't the packages the macOS leg publishes, but CI_PUBLISHING_BUILD only adds the other desktop platforms' native libraries and sentry-cli binaries, which these tests don't use.
• linux-arm64, linux-musl-arm64 and the macOS leg keep their integration tests, because they finish well before the slowest job.

On the stack's own three runs, #5812 with all 6 parts, the slowest check finished at 23.3, 25.5 and 24.3 minutes. On #5802, with the integration tests still in the legs, it was 32.4, and the win-arm64 leg was the slowest job at 31. On #5812, runs 1, 2 and 3:

	Leg	Integration job
win-x64	15.0, 16.4, 15.4 min	17.5, 17.5, 19.4
linux-x64	13.2, 13.7, 13.3	14.4, 13.6, 14.3
win-arm64	15.9, 16.7, 18.1	22.4, 21.0, 22.4
macOS	21.6, 16.0, 23.6	11.7, 24.5, 12.8, the MAUI cases

The macOS MAUI job depends a lot on the Mac it gets: on run 2 its build took 5.3 minutes against 2.5, and its tests 16.3 against 7.2. The three runs of the throwaway #5805 looked the same: legs 15.0, 11.2 and 17.9, integration jobs 18.5, 12.7 and 22.1. #5802 didn't have the other parts' later fixes either, so not all of those 9 minutes come from this PR.

The linux-x64, win-x64 and win-arm64 integration jobs each passed the same 69 tests and skipped the same 35 as their legs did before, and on all three runs of #5812. The first measurements of each change are in #5783 and #5797.

#skip-changelog

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.13%. Comparing base (7fa3c1e) to head (1db0ebb).

Additional details and impacted files

@@                    Coverage Diff                    @@
##           ci/speed-up-windows-jobs    #5809   +/-   ##
=========================================================
  Coverage                     76.12%   76.13%           
=========================================================
  Files                           528      528           
  Lines                         19579    19579           
  Branches                       3829     3829           
=========================================================
+ Hits                          14905    14906    +1     
- Misses                         3789     3791    +2     
+ Partials                        885      882    -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jamescrosswell]

The first 40 lines of code here is almost identical to

sentry-dotnet/.github/workflows/build.yml

Lines 645 to 695 in 1db0ebb

	- name: Checkout
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	submodules: recursive
	- name: Install .NET and NuGet packages on a fast drive
	if: runner.os == 'Windows'
	uses: ./.github/actions/windows-dotnet-drive
	- name: Setup Environment
	uses: ./.github/actions/environment
	with:
	projects: ${{ matrix.samples }}
	- name: Download sentry-native (${{ matrix.rid }})
	uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
	with:
	path: src/Sentry/Platforms/Native/sentry-native
	key: sentry-native-${{ matrix.rid }}-${{ hashFiles('scripts/build-sentry-native.ps1') }}-${{ hashFiles('.git/modules/modules/sentry-native/HEAD') }}
	fail-on-cache-miss: true
	enableCrossOsArchive: true
	# See build-sentry
	- name: Get Cocoa SDK cache key
	id: cocoa-key
	if: runner.os == 'macOS'
	uses: ./.github/actions/cocoa-cache-key
	- name: Restore Cocoa SDK
	id: cache-cocoa
	if: runner.os == 'macOS' && !startsWith(github.ref_name, 'release/')
	uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
	with:
	path: modules/sentry-cocoa/Carthage
	key: ${{ steps.cocoa-key.outputs.key }}
	- name: Build Native Dependencies
	uses: ./.github/actions/buildnative
	- name: Restore .NET Dependencies
	run: dotnet restore ${{ matrix.samples }} --nologo
	- name: Build
	run: dotnet build ${{ matrix.samples }} -c Release --no-restore --nologo -v:minimal -bl:build.binlog
	- name: Save Cocoa SDK
	if: ${{ !cancelled() && steps.cache-cocoa.outputs.cache-primary-key != '' && steps.cache-cocoa.outputs.cache-hit != 'true' && hashFiles('modules/sentry-cocoa/Carthage/.built-from-sha') != '' }}
	uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
	with:
	path: modules/sentry-cocoa/Carthage
	key: ${{ steps.cache-cocoa.outputs.cache-primary-key }}

Maybe as a follow up PR that could be refactored so fixes/changes don't have to be copy/pasted (and more importantly, so they don't fall out of sync).

[jamescrosswell]

These project lists are repeated four times, and three of them are identical. Could they come from generated solution filters instead, the way the build legs pick theirs with slnf:? In scripts/generate-solution-filters-config.yaml:

groupConfigs:
  integrationTestPackages:
    - "**/Sentry.csproj"
    - "**/Sentry.Extensions.Logging.csproj"
    - "**/Sentry.Maui.csproj"
    - "**/Sentry.Bindings.Android.csproj"
    - "**/Sentry.Android.AssemblyReader.csproj"
    - "**/Sentry.Bindings.Cocoa.csproj"

filterConfigs:
  - outputPath: Sentry-CI-IntegrationTest-macOS.slnf
    include:
      groups: ["integrationTestPackages"]
  - outputPath: Sentry-CI-IntegrationTest.slnf
    include:
      groups: ["integrationTestPackages"]
    exclude:
      groups: ["macOnly"]

Then each matrix entry only needs slnf: Sentry-CI-IntegrationTest[-macOS].slnf, and the job can use it the way the legs do:

      - uses: ./.github/actions/environment
        with:
          projects: ${{ matrix.slnf }}
      - run: dotnet build ${{ matrix.slnf }} -c Release --nologo -v:minimal
      - run: dotnet pack ${{ matrix.slnf }} -c Release --no-build --nologo

That keeps the list in one place, which test-solution-filters checks for drift, and it removes the PROJECTS env and both bash loops. Building then packing a filter with --no-build is what the macOS leg already does with Sentry.Maui, so it shouldn't hit the NU5026 problem.

[jamescrosswell]

This does make me a bit nervous.

The macOS leg now skips MAUI tests. They run in the new Integration test (macos) job on a different runner, which builds/packs its own packages.

That means that what we test isn't necessarily what we ship for MAUI anymore... GitHub can and do change what is installed on the runners between runs, so this isn't a hypothetical risk. I have had exactly the same code pass all the tests on one runner but fail on another, due to changes being rolled out progressively across the runners before.

I realise that's the crux of this build/performance improvement - it's also the reason we never explored doing this.

Note

We could do this... just worth being aware of the risks...