Skip to content

Add Qwen3.5 NVFP4 V2 GB200 Dynamo-SGLang MTP / 新增 Qwen3.5 NVFP4 V2 GB200 Dynamo-SGLang MTP 配置 - #2594

Open
RohitNagraj wants to merge 3 commits into
mainfrom
qwen3.5-fp4-gb200-dynamo-sglang-mtp
Open

Add Qwen3.5 NVFP4 V2 GB200 Dynamo-SGLang MTP / 新增 Qwen3.5 NVFP4 V2 GB200 Dynamo-SGLang MTP 配置#2594
RohitNagraj wants to merge 3 commits into
mainfrom
qwen3.5-fp4-gb200-dynamo-sglang-mtp

Conversation

@RohitNagraj

Copy link
Copy Markdown
Collaborator

Description

Add a Qwen3.5-397B-A17B NVFP4 V2 GB200 Dynamo-SGLang disaggregated MTP configuration for the fixed-sequence 8k/1k scenario.

  • Register nine checked-in srt-slurm recipes covering TP and data-parallel-attention topologies.
  • Keep the master topology, container image, model alias, Mooncake KV-transfer metadata, and recipe inputs synchronized.
  • Enable chat-template formatting for every speculative-decoding benchmark point.

Validation:

  • Exact-key and filtered full-sweep matrix generation produced the expected nine points.
  • All nine recipes passed current NVIDIA/srt-slurm dry-run validation.
  • YAML parsing, launcher syntax, changelog newline, and master-to-recipe parity checks passed.

中文说明

新增 Qwen3.5-397B-A17B NVFP4 V2 在 GB200 上的 Dynamo-SGLang 分离式 MTP 配置,用于固定序列长度 8k/1k 场景。

  • 登记九个随仓库提交的 srt-slurm 配方,覆盖 TP 与数据并行注意力拓扑。
  • 保持主配置中的拓扑、容器镜像、模型别名、Mooncake KV 传输元数据与配方输入一致。
  • 所有投机解码测试点均启用聊天模板格式化。

验证:

  • 精确配置键与筛选后的完整 sweep 矩阵均生成预期的九个测试点。
  • 九个配方均通过当前 NVIDIA/srt-slurm 的 dry-run 验证。
  • YAML 解析、启动器语法、变更日志末尾换行及主配置与配方一致性检查均通过。

Related Issue

N/A

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have tested my changes locally
  • I have updated documentation if necessary
  • For every recipe addition, I appended a new entry to the physical end of perf-changelog.yaml without editing historical entries
  • An authorized maintainer will comment /reuse-sweep-run only after a final green sweep with passing evals

新增 Qwen3.5 FP4 GB200 Dynamo-SGLang MTP 配置。
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

1 similar comment
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

在性能变更日志中补充 PR #2594 链接。
将 PR #2594 链接补充到对应的 Qwen3.5 变更日志条目。
Comment thread perf-changelog.yaml Outdated
- "Add nine checked-in srt-slurm recipes covering TP and data-parallel-attention topologies"
- "Use chat-formatted benchmark inputs with Mooncake KV transfer"
- "Image: lmsysorg/sglang:nightly-dev-cu13-20260709-074bb928"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The new perf-changelog.yaml entry (line 5966) uses a placeholder pr-link: .../pull/XXX instead of a real PR number. Since this is PR #2594, it should read pull/2594, matching every other entry in the file.

Extended reasoning...

The last block appended to perf-changelog.yaml in this diff ends with:

pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX

XXX is a literal, unfilled placeholder rather than an actual PR number. Every other entry in the file follows the same schema and always points to its own real PR — for example the immediately preceding entries link to pull/2577, pull/2578, and pull/2580. This PR is #2594, so the correct value is pull/2594.

Why existing checks don't catch it: the PR description states that a 'changelog newline' check passed, but that only verifies the file ends with a trailing newline / is well-formed YAML — it does not validate that pr-link contains a real, resolvable PR number. There is no schema or CI validation cross-checking the pr-link field against the PR number that is actually opened, so a placeholder like XXX sails through untouched.

Proof / step-by-step:

  1. Open perf-changelog.yaml at the new entry (last block in the diff, starting at line ~5956).
  2. Read the pr-link field: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX.
  3. Compare against the file's own convention: scroll up to the entries directly above it (lines 5940-5955) — they use pull/2577, pull/2578, pull/2580, each matching the PR that introduced that entry.
  4. Note the current PR's actual number is 2594 (per PR metadata).
  5. Following the file's established convention, the link should be https://github.com/SemiAnalysisAI/InferenceX/pull/2594.

Impact: this is a documentation/metadata-only bug — the changelog is not consumed by any runtime code path, so nothing breaks functionally. However, anyone who later clicks the link while investigating this changelog entry (e.g., during a regression bisect or audit) will hit a 404 or an unrelated PR, making it harder to trace which PR introduced the entry. It should be fixed before merge since it is a one-line, unambiguous fix.

Fix: replace pull/XXX with pull/2594 in the new changelog block.

@github-actions

Copy link
Copy Markdown
Contributor

@kedarpotdar-nv

Copy link
Copy Markdown
Collaborator

Temporarily removing full sweep label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants