Update MiniMax-M3 NVFP4 B300 8k1k disaggregated Dynamo-vLLM sweep / 更新 MiniMax-M3 NVFP4 B300 8k1k 分离式 Dynamo-vLLM 扫描配置

[jasonlizhengjian]

Summary

• replace the MiniMax-M3 B300 FP4 8k1k DEP2-prefill recipes with TP1-prefill disaggregated recipes
• cover TP4, TP8, and DEP8 decode points from concurrency 1 through 1024
• colocate the TP4 decode topologies that fit on one B300 node
• use the exact tested vLLM nightly image (nightly-2dfaae752b4db0d43cfc0715c780e33be030d0f1)
• pin the new recipe path to srt-slurm c1fb6989fc5aca803b4ca0f2d17d8be85fad9732
• split the new 8k1k sweep from the existing 1k1k configuration so this submission does not rerun 1k1k
• preserve the proven 4P2D DEP2-prefill/TEP4-decode concurrency-4096 point as a separate legacy max-throughput configuration

8k1k sweep

Recipe	Concurrency
1P1D TP1/TP4	8, 16, 24, 32, 48, 64
1P1D TP1/TP8	1, 2, 4, 8, 16
2P1D TP1/TP4	128
3P1D TP1/TP4	256
5P2D TP1/DEP8	512
8P2D TP1/DEP8	768, 1024
4P2D DEP2/TEP4 (legacy max-throughput)	4096

Validation

• generated matrix contains exactly seven entries, all ISL 8192 / OSL 1024: six TP1 entries plus the preserved legacy max-throughput point
• all six new TP1 recipes pass srtctl dry-run; the legacy recipe is preserved unchanged from the previously validated submission
• YAML parsing and B300 launcher shell syntax pass
• 179 matrix generation and validation tests pass

中文说明

摘要

• 将 MiniMax-M3 B300 FP4 8k1k 的 DEP2 预填充 recipe（配方）替换为 TP1 预填充分离式 recipe
• 覆盖解码侧 TP4、TP8 和 DEP8 拓扑，并发范围为 1 到 1024
• 对可容纳在单个 B300 节点上的 TP4 解码拓扑启用预填充/解码共置
• 使用经过实际测试的 vLLM nightly 镜像（nightly-2dfaae752b4db0d43cfc0715c780e33be030d0f1）
• 将新 recipe 路径固定到 srt-slurm c1fb6989fc5aca803b4ca0f2d17d8be85fad9732
• 将新的 8k1k 扫描从现有 1k1k 配置中拆分，避免本次提交重新运行 1k1k
• 将经过验证的 4P2D DEP2 预填充/TEP4 解码、并发 4096 的性能点保留为独立的旧版最大吞吐量配置

8k1k 扫描

Recipe	并发数
1P1D TP1/TP4	8, 16, 24, 32, 48, 64
1P1D TP1/TP8	1, 2, 4, 8, 16
2P1D TP1/TP4	128
3P1D TP1/TP4	256
5P2D TP1/DEP8	512
8P2D TP1/DEP8	768, 1024
4P2D DEP2/TEP4（旧版最大吞吐量）	4096

验证

• 生成的矩阵恰好包含 7 个配置，全部为 ISL 8192 / OSL 1024：6 个 TP1 配置以及保留的旧版最大吞吐量配置
• 6 个新的 TP1 recipe 均通过 srtctl dry-run；旧版 recipe 与此前已验证的提交保持不变
• YAML 解析和 B300 启动器 shell 语法检查通过
• 179 项矩阵生成与验证测试全部通过

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[claude]

Claude finished @jasonlizhengjian's task in 4m 46s —— View job

---

Review: [DoNotMerge] Update 8k1k B300 Minimax Dynamo Disagg

• Gather context (diff vs origin/main, read changed files)
• Review recipe YAMLs (8k1k disagg recipes)
• Validate configs/nvidia-master.yaml changes (image accessibility, model prefixes, changelog pairing)
• Validate perf-changelog.yaml entry position
• Validate runners/launch_b300-nv.sh (enroot import pattern)
• Post review feedback

Summary

No blocking issues found. One 🟡 WARNING posted inline: the new launcher branch pins srt-slurm at c1fb6989 but skips the get_node_ip.sh patch (NVIDIA/srt-slurm#38) that the legacy minimaxm3 branch applies — please confirm the pinned commit already contains that fix, since srtctl dry-run would not surface a broken multi-node IP resolution path.

Everything else checked out:

• Master config ↔ recipe consistency: all six CONFIG_FILE paths exist, concurrency lists, worker counts, TP/EP/dp-attn per topology, and the container image all match between configs/nvidia-master.yaml and the recipe YAMLs.
• The 1k1k/8k1k split is clean — the legacy minimaxm3-fp4-b300-dynamo-vllm key retains only its isl: 1024 scenario and the new minimaxm3-fp4-b300-dynamo-vllm-8k1k-tp1 key holds only isl: 8192, so this submission won't rerun 1k1k.
• Colocation GPU math fits: the decode_nodes: 0 recipes place 5/6/7 GPUs on one 8-GPU node with allow_prefill_decode_colocation: true; 8P2D uses exactly 8 prefill GPUs on its own node.
• Launcher glob 8k1k/*-tp1-*.yaml matches all six new recipes and cannot capture the fp8 or 1k1k configs, and it correctly precedes the general minimaxm3 branch.
• perf-changelog.yaml entry is appended at the end of the file and paired with the master-config change.
• Image vllm/vllm-openai:nightly-2dfaae752b4db0d43cfc0715c780e33be030d0f1 is a public Docker Hub reference, and the existing enroot import docker:// flow in the launcher handles it.
• No dangling references to the deleted DEP2 fp4 recipes (remaining dep2-*-8k1k hits are the untouched fp8 directory).

Note: I could not run the pytest matrix suite or reach external APIs (Docker Hub / srt-slurm GitHub) from this sandbox — those checks are covered by CI and the inline comment respectively.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29297512174
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29297512174

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29298636780
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29298636780

[claude]

Claude finished @jasonlizhengjian's task in 4m 28s —— View job

---

Re-review: Update 8k1k B300 Minimax Dynamo Disagg

• Gather context (commits since last review, diff vs origin/main)
• Review new changes only (8840ef6b, 0c65a6cf, 5b0d83bd, 30be9347)
• Re-check master-config / perf-changelog / launcher rules on the new diff
• Post re-review feedback

Re-review: no new blocking issues

Four commits landed since my previous review (01:02 UTC): the --no-preflight bypass, the merge of main with the Dynamo bump, and the two commits adding the legacy-max-tput config. All checked out:

Preflight bypass (8840ef6b) — --no-preflight is scoped by the exact same glob (8k1k/*-tp1-*.yaml + framework/model/precision guards) as the pinned srt-slurm branch, so the bypass applies to precisely the recipe set that uses the newer revision with the login-host model.path check, and nothing else. The rationale comment explains why the login host can't stat /scratch-staged weights.

Dynamo bump (merge 0c65a6cf) — 1.3.0.dev20260614 → 1.3.0.dev20260713 is applied consistently: all six TP1 recipes' dynamo.version and the minimaxm3-fp4-b300-dynamo-vllm-8k1k-tp1 router pin match, and the changelog wording was updated to match. The legacy 1k1k key and its router pin are untouched. The merge also correctly kept this PR's changelog entry last in the file (main's new #2191/#2197/#2198 entries were inserted above it).

Legacy max-throughput point (5b0d83bd + 30be9347) — the net result is a faithful restoration:

• 4p2d-dep2-tep4-8k1k.yaml is byte-identical to origin/main's copy (empty diff), including the legacy image vllm-minimax-m3-perf-x86_64-13.0.1-8b00f41 and Dynamo 1.3.0.dev20260614.
• The new minimaxm3-fp4-b300-dynamo-vllm-8k1k-legacy-max-tput key reproduces main's conc-4096 search-space point exactly (4×DEP2 prefill / 2×TEP4 decode), with image and router pins matching the recipe.
• Launcher routing is correct by construction: 4p2d-dep2-tep4-8k1k.yaml does not match the *-tp1-* glob, so it falls through to the existing general minimaxm3 branch (sa-submission-q2-2026 checkout + the Sanitize srun output in node IP resolution NVIDIA/srt-slurm#38 get_node_ip.sh patch) and does not get --no-preflight — i.e., it runs exactly the way this point ran when it was originally submitted.
• The intermediate experimental 8p2d-tp1-tep4 recipe from 5b0d83bd was fully removed in 30be9347; no dangling references remain (all leftover dep2 hits are the untouched fp8/GB300 directories).

Repo rules re-verified on the new diff: master-config changes are paired with a perf-changelog.yaml entry listing both new config keys; the entry is appended at the very end of the file; all images are public Docker Hub references.

Still open from the previous review: the 🟡 inline warning on runners/launch_b300-nv.sh:84 — please confirm the pinned srt-slurm commit c1fb6989 already contains the NVIDIA/srt-slurm#38 get_node_ip.sh fix, since the TP1 branch (unlike the legacy branch) does not apply that patch. I couldn't verify ancestry from this sandbox (no external network access); if the sweep runs currently in flight bring up multi-node workers successfully, that empirically resolves it.

[claude]

🔴 The new changelog entry for PR #2184 is appended immediately after the prior pull/2198 entry with no blank separator line, unlike every other entry boundary in this file. utils/validate_perf_changelog.py (run in CI via run-sweep.yml) requires exactly one blank line before an appended - config-keys: entry and will raise ChangelogValidationError here, failing the changelog gate. Fix: insert one blank line before the new - config-keys: line at perf-changelog.yaml:4784.

Extended reasoning...

This PR appends a new perf-changelog entry (for the minimaxm3-fp4-b300-dynamo-vllm-8k1k-tp1 / -legacy-max-tput config keys) directly after the previous entry's pr-link: .../pull/2198 line, with no blank line separating the two entries. Every other entry boundary in perf-changelog.yaml has exactly one blank line before the next - config-keys: line — this new entry breaks that pattern.

This is enforced by a real CI gate, not just a style convention. utils/validate_perf_changelog.py::validate_raw_change() is invoked on every sweep PR via .github/workflows/run-sweep.yml:102. For an appended entry, it computes:

suffix = head_raw[len(base_raw):]
expected_start = (
    b"- config-keys:"
    if base_raw.endswith(b"\n\n")
    else b"\n- config-keys:"
)
if not suffix.startswith(expected_start):
    raise ChangelogValidationError(
        "new changelog entries must be separated from history by one "
        "empty line and appended at the end"
    )

Step-by-step proof:

1. The base file (before this PR) ends its final entry (PR fix: recover PR 2108 ingest via sweep reuse / 通过扫描复用恢复 PR 2108 的结果入库 #2198) with a single trailing \n — the validator itself forbids a base ending in \n\n via a separate check a few lines down (the final appended changelog entry must end with one newline), so base_raw.endswith(b"\n\n") is False.
2. Therefore expected_start = b"\n- config-keys:" — the validator requires the appended suffix to begin with a newline (i.e., a blank line) followed by - config-keys:.
3. Reading the actual file bytes around perf-changelog.yaml:4783-4784 confirms the diff appends - config-keys: immediately after pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2198\n with zero blank lines in between. So suffix begins with - config-keys:, not \n- config-keys:.
4. suffix.startswith(expected_start) evaluates to False, so validate_raw_change() raises ChangelogValidationError("new changelog entries must be separated from history by one empty line and appended at the end"), which exits non-zero and fails the changelog-gate step of the sweep pipeline.

One verifier raised a refutation arguing that utils/process_changelog.py's get_added_lines() (which extracts only +-prefixed diff lines and YAML-parses them) is indifferent to blank lines, and concluded the missing separator has "zero functional impact." That is true of process_changelog.py, but it is the wrong validator to look at — process_changelog.py handles ingesting changelog content into results, while utils/validate_perf_changelog.py is the distinct gate that CI runs against the raw file bytes to enforce formatting invariants (including the exact one-blank-line separator convention), and that gate does not tolerate this diff. The refutation's conclusion doesn't hold once the correct validator is traced.

Fix: insert a single blank line before the new - config-keys: line (i.e., between the pull/2198 pr-link line and the new entry), matching every other entry boundary in the file. This is a one-line fix but it currently produces a deterministic, CI-blocking failure — this PR's entire purpose is to trigger the 8k1k sweep via run-sweep.yml, and that pipeline step will fail before the sweep even runs.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29358158271
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=29358158271

[Ankur-singh]

/reuse-sweep-run 29358158271

[Ankur-singh]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Run Sweep #29358158271
• Verified that this PR passes evals. Run Sweep #29358158271
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.

Additional detail section:

• This is a multi-node disaggregated STP submission; the speculative-decoding, AgentX, and single-node recipe requirements are not applicable. / 本 PR 是多节点分离式 STP 提交；投机解码、AgentX 和单节点 recipe（配方）要求不适用。
• The new TP1 path pins srt-slurm c1fb6989, which includes the get_node_ip fix from 22d46ba9; all seven throughput and seven eval jobs passed. / 新的 TP1 路径固定到 srt-slurm c1fb6989，其中已包含 22d46ba9 引入的 get_node_ip 修复；7 个吞吐量任务和 7 个评估任务均已通过。

Signed: Ankur-singh

[Klaud-Cold]

✅✅✅ Verdict: PASS ✅✅✅

✅ Check 0 (CODEOWNER): PASS — Ankur-singh owns configs/nvidia-master.yaml (specific CODEOWNERS line); remaining paths are catch-all-only and covered.
✅ Check 1 (Passing sweep on in-PR commit): PASS — head 30be934 (matches pinned SHA) has all 7 multi-node 8k1k / and 7 multi-node eval / jobs success in run 29358158271; single-node lanes skipped as expected (PR has none).
✅ Check 2 (Evals pass): PASS — 7/7 GSM8K em_strict 0.951–0.961 vs 0.85 bar (agg_eval_all.json from that run); eval jobs ran the PR's exact images (vllm/vllm-openai:nightly-2dfaae752… for the 6 TP1 configs, vllm/vllm-openai:vllm-minimax-m3-perf-x86_64-13.0.1-8b00f41 for the legacy point).
➖ Check 3 (Recipe link): N/A — disaggregated/multi-node submission (srt-slurm-recipes/**, multinode: true, disagg: true); the recipe-link requirement applies to single-node recipes only.
✅ Check 4 (Reuse command): PASS — /reuse-sweep-run 29358158271 posted by Ankur-singh (COLLABORATOR).
✅ Check 5 (Latest checklist): PASS — every current-template item present and checked, with non-applicability explained in the detail section.
✅ Check 6 (Upstream image / engine-first): PASS — both images are from the upstream vllm/vllm-openai Docker Hub org, and single-node minimaxm3-fp4-b300-vllm (framework: vllm, runner: b300) already exists, satisfying engine-first ordering. Informational: the exact nightly-2dfaae752… tag has since been pruned from Docker Hub (vLLM's rolling nightly retention); the same-commit cu129-nightly-2dfaae752… tag remains and the run pulled the image successfully.
✅ Check 7 (No architecture hacks): PASS — no --hf-overrides/model-config edits; language-model-only is pre-existing and does not reduce text-generation FLOPs; fp8 kv-cache is precision-only with passing evals.
➖ Check 8 (Spec-decode chat templates): N/A — no speculative-decoding changes in this PR.
✅ Check 9 (No engine patches): PASS — no .patch/sed/heredoc engine edits; the pinned NVIDIA/srt-slurm clone, Dynamo install, and --no-preflight are harness/framework plumbing, not serving-stack modifications.
➖ Check 10 (Agentic golden AL): N/A — no agentic speculative-decoding changes.

[Klaud-Cold]

✅✅✅ Verdict: PASS ✅✅✅

✅ Check 0 (CODEOWNER): PASS — Ankur-singh owns configs/nvidia-master.yaml via its specific CODEOWNERS line; all other changed paths are catch-all-only and covered.
✅ Check 1 (Passing sweep on in-PR commit): PASS — commit 30be9347 (still in this PR) carries run 29358158271 with all seven multi-node 8k1k / and seven multi-node eval / check-runs success (executed, not skipped).
✅ Check 2 (Evals pass): PASS — aggregated eval artifact shows GSM8K em_strict 0.951–0.961 across all 7 configs, run on the same images as this PR's configs (vllm/vllm-openai:nightly-2dfaae752b… for the six TP1 recipes, vllm/vllm-openai:vllm-minimax-m3-perf-…-8b00f41 for the legacy max-throughput point).
➖ Check 3 (Recipe link): N/A — disaggregated/multi-node submission (srt-slurm recipes, master-config entries with multinode: true / disagg: true); the recipe-link requirement applies to single-node recipes only.
✅ Check 4 (Reuse command): PASS — /reuse-sweep-run 29358158271 posted by Ankur-singh (COLLABORATOR).
✅ Check 5 (Latest checklist): PASS — sign-off matches the current docs/PR_REVIEW_CHECKLIST.md item-for-item, all items checked.
✅ Check 6 (Upstream images / engine-first): PASS — both new master-config entries use upstream vllm/vllm-openai images, and vLLM-first is already satisfied for this model+SKU (minimaxm3-fp4-b300-vllm in configs/nvidia-master.yaml).
✅ Check 7 (No architecture hacks): PASS — no --hf-overrides or FLOPs-reducing overrides in the diff.
➖ Check 8 (Spec-decode chat templates): N/A — no speculative-decoding changes in this PR.
✅ Check 9 (No engine patches): PASS — no engine patching; the srt-slurm pin (c1fb6989), recipe copies, and --no-preflight are harness plumbing, and the pinned engine images run as shipped.
➖ Check 10 (Agentic golden AL): N/A — no agentic speculative-decoding changes.