Add staged MAUI desktop benchmarks - #5298
Draft
LoopedBard3 wants to merge 1 commit into
Draft
Conversation
Stage a pinned MAUI source snapshot on the build agent, apply PerfLab MSBuild overlays without rewriting MAUI programs, and run Core, XAML, and Graphics as isolated Helix work items. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e89863e5-0ec5-4ca9-b7f9-ca7df0257e59
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
It introduces a new end-to-end Helix scenario with external repo staging and pipeline wiring where correctness depends on integration behavior that can’t be fully validated from the diff alone.
Pull request overview
Adds a new “staged MAUI desktop benchmarks” scenario that prepares a sparse MAUI checkout on the build agent, injects stable MSBuild imports/entrypoint overrides in the staged payload, and runs Core/XAML/Graphics BenchmarkDotNet suites as isolated Helix work items while recording MAUI source/toolchain metadata in PerfLab output.
Changes:
- Introduces
mauidesktopbenchmarksscenario payload preparation (pre.py) and Helix execution wrapper (test.py) plus an MSBuild overlay (props/targets + custom entrypoint). - Updates
RecommendedConfig.Createto (a) allow an optional explicitIToolchainand (b) omitMandatoryCategoryValidatorwhen no mandatory categories are provided; adds unit tests for this behavior. - Wires the new run kind into CI (new Helix project + pipeline jobs) and enables strict Pyright checking for the new scenario folder.
File summaries
| File | Description |
|---|---|
| src/tests/harness/BenchmarkDotNet.Extensions.Tests/RecommendedConfigTests.cs | Adds unit coverage for the “no mandatory categories => no validator” behavior. |
| src/scenarios/mauidesktopbenchmarks/test.py | Runs a staged MAUI suite in a Helix work item, forwarding BDN args and uploading expected artifacts. |
| src/scenarios/mauidesktopbenchmarks/pre.py | Prepares the sparse MAUI checkout + overlay, publishes the harness, and writes the source manifest into the payload. |
| src/scenarios/mauidesktopbenchmarks/overlay/PerfLabEntry.cs | Provides a stable benchmark entrypoint that selects toolchain mode and uses PerfLab’s recommended config/artifacts location. |
| src/scenarios/mauidesktopbenchmarks/overlay/MauiPerfLab.targets | Injects the PerfLab entrypoint + aligns BenchmarkDotNet package versions and references the published harness binaries. |
| src/scenarios/mauidesktopbenchmarks/overlay/MauiPerfLab.props | Forces desktop-only target framework selection by disabling non-Windows MAUI target frameworks. |
| src/harness/BenchmarkDotNet.Extensions/RecommendedConfig.cs | Adds optional toolchain support and avoids adding MandatoryCategoryValidator when mandatory categories are empty. |
| scripts/tests/test_maui_desktop_benchmarks.py | Adds targeted Python tests for framework->branch mapping and BDN argument shaping. |
| scripts/run_performance_job.py | Ensures binlogs produced by the new scenario’s prepare step are copied into CI artifacts. |
| pyrightconfig.json | Includes the new scenario directory as a strict Pyright execution environment. |
| eng/pipelines/sdk-perf-jobs.yml | Adds public and internal pipeline jobs for the new maui_desktop_benchmarks run kind. |
| eng/performance/maui_desktop_benchmarks.proj | Adds the Helix project wiring: prepare payload work item + 3 suite work items (core/xaml/graphics). |
Review details
- Files reviewed: 12/12 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
LoopedBard3
marked this pull request as draft
September 3, 2026 18:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is an alternative implementation to #5161 that keeps all changes in
dotnet/performancewhile avoiding runtime rewrites of MAUI benchmark entrypoints.dotnet/mauibranch to an exact SHA on the build agent and stages a sparse checkout in the correlation payloadProgram.csBenchmarkDotNet.Extensionsharness and aligns MAUI's benchmark projects to the performance repo's BDN versionbenchmarks_ci.pybuild, collection, and upload pathInProcessEmitbecause BDN's child reference gatherer otherwise rebuilds MAUI's incompatible legacy TFMsValidation
PreparePayloadWorkItemsMSBuild target against the currentnet11.0MAUI branchBenchmarkDotNet.Extensions.Tests