Skip to content

Add staged MAUI desktop benchmarks - #5298

Draft
LoopedBard3 wants to merge 1 commit into
dotnet:mainfrom
LoopedBard3:loopedbard3-maui-desktop-benchmarks-alternative
Draft

Add staged MAUI desktop benchmarks#5298
LoopedBard3 wants to merge 1 commit into
dotnet:mainfrom
LoopedBard3:loopedbard3-maui-desktop-benchmarks-alternative

Conversation

@LoopedBard3

Copy link
Copy Markdown
Member

Summary

This is an alternative implementation to #5161 that keeps all changes in dotnet/performance while avoiding runtime rewrites of MAUI benchmark entrypoints.

  • resolves the matching dotnet/maui branch to an exact SHA on the build agent and stages a sparse checkout in the correlation payload
  • injects stable MSBuild imports into the staged checkout, retaining every original Program.cs
  • publishes the existing BenchmarkDotNet.Extensions harness and aligns MAUI's benchmark projects to the performance repo's BDN version
  • runs Core, XAML, and Graphics as isolated Helix work items through the existing benchmarks_ci.py build, collection, and upload path
  • records the MAUI repository, branch, commit, and selected toolchain in PerfLab result metadata
  • uses the default out-of-process toolchain for Core and Graphics; XAML uses InProcessEmit because BDN's child reference gatherer otherwise rebuilds MAUI's incompatible legacy TFMs

Validation

  • exercised the real PreparePayloadWorkItems MSBuild target against the current net11.0 MAUI branch
  • executed representative Core, XAML, and Graphics benchmarks through simulated Helix work-item roots
  • verified public one-iteration BDN arguments are forwarded without the category filter that would exclude uncategorized MAUI benchmarks
  • ran all 46 BenchmarkDotNet.Extensions.Tests
  • ran the targeted Python tests and strict Pyright checks

Stage a pinned MAUI source snapshot on the build agent, apply PerfLab MSBuild overlays without rewriting MAUI programs, and run Core, XAML, and Graphics as isolated Helix work items.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e89863e5-0ec5-4ca9-b7f9-ca7df0257e59
Copilot AI lite review requested due to automatic review settings September 3, 2026 00:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It introduces a new end-to-end Helix scenario with external repo staging and pipeline wiring where correctness depends on integration behavior that can’t be fully validated from the diff alone.

Pull request overview

Adds a new “staged MAUI desktop benchmarks” scenario that prepares a sparse MAUI checkout on the build agent, injects stable MSBuild imports/entrypoint overrides in the staged payload, and runs Core/XAML/Graphics BenchmarkDotNet suites as isolated Helix work items while recording MAUI source/toolchain metadata in PerfLab output.

Changes:

  • Introduces mauidesktopbenchmarks scenario payload preparation (pre.py) and Helix execution wrapper (test.py) plus an MSBuild overlay (props/targets + custom entrypoint).
  • Updates RecommendedConfig.Create to (a) allow an optional explicit IToolchain and (b) omit MandatoryCategoryValidator when no mandatory categories are provided; adds unit tests for this behavior.
  • Wires the new run kind into CI (new Helix project + pipeline jobs) and enables strict Pyright checking for the new scenario folder.
File summaries
File Description
src/tests/harness/BenchmarkDotNet.Extensions.Tests/RecommendedConfigTests.cs Adds unit coverage for the “no mandatory categories => no validator” behavior.
src/scenarios/mauidesktopbenchmarks/test.py Runs a staged MAUI suite in a Helix work item, forwarding BDN args and uploading expected artifacts.
src/scenarios/mauidesktopbenchmarks/pre.py Prepares the sparse MAUI checkout + overlay, publishes the harness, and writes the source manifest into the payload.
src/scenarios/mauidesktopbenchmarks/overlay/PerfLabEntry.cs Provides a stable benchmark entrypoint that selects toolchain mode and uses PerfLab’s recommended config/artifacts location.
src/scenarios/mauidesktopbenchmarks/overlay/MauiPerfLab.targets Injects the PerfLab entrypoint + aligns BenchmarkDotNet package versions and references the published harness binaries.
src/scenarios/mauidesktopbenchmarks/overlay/MauiPerfLab.props Forces desktop-only target framework selection by disabling non-Windows MAUI target frameworks.
src/harness/BenchmarkDotNet.Extensions/RecommendedConfig.cs Adds optional toolchain support and avoids adding MandatoryCategoryValidator when mandatory categories are empty.
scripts/tests/test_maui_desktop_benchmarks.py Adds targeted Python tests for framework->branch mapping and BDN argument shaping.
scripts/run_performance_job.py Ensures binlogs produced by the new scenario’s prepare step are copied into CI artifacts.
pyrightconfig.json Includes the new scenario directory as a strict Pyright execution environment.
eng/pipelines/sdk-perf-jobs.yml Adds public and internal pipeline jobs for the new maui_desktop_benchmarks run kind.
eng/performance/maui_desktop_benchmarks.proj Adds the Helix project wiring: prepare payload work item + 3 suite work items (core/xaml/graphics).
Review details
  • Files reviewed: 12/12 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@LoopedBard3
LoopedBard3 marked this pull request as draft September 3, 2026 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants