Add DSV4 GB300 TRT disaggregated offline benchmark / 新增 DSV4 GB300 TRT 分离式离线基准测试

[Ankur-singh]

Summary

• Add the DSV4-Pro FP4 GB300 TensorRT-LLM disaggregated generation-only benchmark with 12 validated Pareto configurations.
• Include DEP32 MTP3 c512; exclude only DEP16 MTP1 c8192.
• Use the existing upstream gb300-nv runner settings for SLURM and shared storage.
• Preserve the bundled topologies and fixed acceptance-length decode-step metrics.
• Recover exact-full-batch measurements from the complete GEN worker log only when the timed slice contains none, with explicit runtime provenance.

Validation

• Focused offline adapter and result-processing tests: PASS (43 tests).
• Matrix-logic suite: PASS (180 tests).
• Python compilation and shell syntax checks: PASS.
• Matrix generation: exactly 12 jobs; c512 present exactly once; c8192 absent.
• Upstream GB300 runner-settings regression: PASS.
• Private-reference scan: PASS for tracked additions, commit messages, and this PR body.

Scope

• Base: offline-bench (55cc997dc14e1f9b8cbc3c7bfd8f449db4e839d4).
• No workflow changes.
• No perf-changelog.yaml changes.
• No partial-batch scoring.

中文说明

• 新增 DSV4-Pro FP4 GB300 TensorRT-LLM 分离式 gen-only 基准测试，共包含 12 项已验证的 Pareto 配置。
• 包含 DEP32 MTP3 c512，仅排除 DEP16 MTP1 c8192。
• 使用上游现有的 gb300-nv 运行器配置，包括 SLURM 与共享存储设置。
• 保留随代码提交的拓扑配置与固定平均接收长度的解码步指标。
• 仅当计时切片完全缺少完整批次样本时，才从完整 GEN worker 日志恢复结果，并明确记录运行时来源。

验证结果

• 离线适配器与结果处理专项测试：通过（43 项测试）。
• 矩阵逻辑测试：通过（180 项测试）。
• Python 编译与 Shell 语法检查：通过。
• 矩阵生成：共 12 个任务；c512 恰好出现一次，且不包含 c8192。
• 上游 GB300 运行器配置回归测试：通过。
• 私有来源引用检查：新增代码、提交信息和本 PR 描述均通过检查。

变更范围

• 基线：offline-bench (55cc997dc14e1f9b8cbc3c7bfd8f449db4e839d4)。
• 不修改工作流。
• 不修改 perf-changelog.yaml。
• 不对部分批次进行计分。

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

---

感谢你的贡献！对于 vLLM 与 SGLang，请确保你的 recipe 与官方 vLLM recipes 和/或 SGLang cookbook 保持一致

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

如果不一致，请先创建一个 PR，之后我们才能将你的单节点 PR 合并到 master 分支。让我们确保文档保持一流水准，使整个 ML 社区都能从你的辛勤工作中受益！谢谢

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。如果选择重新运行失败的任务，PR 作者有责任确保其最终通过。参见 GitHub 关于重新运行失败任务的文档：https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

一般而言，PR 作者应先向相应公司的 CODEOWNERS 请求审阅并获得 PR 批准，然后再请求核心维护者审阅。

如需更多帮助，PR 作者可通过 Slack 联系核心维护者。

[claude]

Beyond the reported nit, this run also looked into three other candidates raised during review and did not confirm them as bugs: the MoE backend differing between the dep32-c64 config (TRTLLM) and the other dep32 rows (MEGAMOE_DEEPGEMM) looks like a per-concurrency tuning choice rather than an inconsistency; the multinode tput_per_gpu formula dividing by total (CTX+GEN) GPUs for this disaggregated gen-only mode follows the same convention already used for other non-offline disagg rows in utils/process_result.py; and the full-GEN-worker-log fallback path in trt_disagg_gen_only.py (only used when the timed slice has zero exact-full-batch samples) was checked for scoring warmup-cohort iterations as timed results, but no scoring error was found.

Extended reasoning...

This is a first-pass review on a large (32-file) new multi-node benchmark addition that also touches shared utils/process_result.py logic (removing the prior offline+multinode rejection), so it warrants human review regardless of the inline nit. This note only records candidates that were examined and not confirmed as bugs during this run, so a future reviewer doesn't need to re-explore the same ground; it is not a correctness guarantee for the rest of the PR.

[claude]

🟡 The health-check loop in wait_server.sh uses curl -s -o /dev/null -w "%{http_code}" .../health but never passes --fail, so curl exits 0 for any completed HTTP response (2xx-5xx) and the captured status code is discarded to /dev/null — meaning a server that is up but returning a non-2xx status (e.g. 503 while the disaggregated CTX/GEN workers are still loading the 806GB checkpoint) is treated as healthy and the wait exits prematurely. The fix is a one-liner: add --fail (and drop the now-dead -w), matching the correct pattern already used in benchmarks/benchmark_lib.sh:176.

Extended reasoning...

The bug: wait_server.sh polls readiness with:

while ! curl -s -o /dev/null -w "%{http_code}" "http://${hostname}:${port}/health" > /dev/null 2>&1; do

The loop's exit condition is driven entirely by curl's own exit status via ! curl .... Without --fail, curl's documented behavior is to exit 0 for any HTTP response it successfully receives and parses — including 4xx and 5xx status codes. It only exits non-zero on a transport-level failure (connection refused, DNS failure, timeout). The -w "%{http_code}" output, which is the only place the actual status code is captured, is piped to stdout and then redirected to /dev/null, so it is never inspected. This is a strong signal the author intended a status-code check but the --fail flag (which is what actually turns a non-2xx response into a curl failure) was omitted, leaving the -w output dead code.

Trigger path: In disaggr_torch.slurm, this script is invoked as the 5_wait_server srun step right after the CTX/GEN worker and trtllm-serve disaggregated processes are launched in the background. The disaggregated frontend can bind its HTTP port and start answering /health before the CTX/GEN workers have finished loading the DeepSeek-V4-Pro checkpoint (documented elsewhere in this same PR as an ~806GB load). If /health returns anything other than 2xx while workers are still initializing (e.g. 503), the current loop treats that as success on the very first poll and 5_wait_server.log reports the server ready immediately.

Why nothing else catches it: The rest of the repo gets this right — benchmarks/benchmark_lib.sh:176 uses the correct idiom, curl --output /dev/null --silent --fail http://...:$port/health, which correctly keeps polling on any non-2xx response. This new wait_server.sh is the one place in the codebase that deviates from that pattern.

Impact: If the premature-ready case is hit, run_benchmark.sh (the next step, launched via client_cmds) starts issuing its UCX warmup and benchmark_serving requests against a server that isn't actually ready to serve. Whether this manifests as a hard failure or a silently-degraded first cohort depends on exactly how trtllm-serve's disaggregated router behaves before its workers finish loading (refuses connections outright vs. responds 503) — a detail not observable from this diff alone, which is why this is best treated as a robustness bug rather than a guaranteed on-merge failure.

Step-by-step proof:

1. Server process starts, binds port 8000, is added to the OS listen queue.
2. GET /health is now routable — trtllm-serve's HTTP layer answers, but returns HTTP 503 (or 404, or whatever the disagg router surfaces before backends attach) because CTX/GEN workers are still mid-checkpoint-load.
3. curl -s -o /dev/null -w "%{http_code}" http://host:8000/health runs. curl successfully completes the HTTP transaction and exits with status 0 — because no --fail flag was passed, curl does not treat 503 as a failure.
4. ! curl ... evaluates to ! 0, i.e. false. The while condition is false, so the loop body never executes and the script falls through immediately, printing "Server is healthy and ready to accept requests!".
5. 5_wait_server.log shows success within seconds of the workers starting, while the checkpoint load (which can take minutes) is still in progress in the background.
6. The client benchmark step begins hitting the not-actually-ready server.

The fix: curl -s -o /dev/null --fail "http://${hostname}:${port}/health" (dropping the now-unused -w). With --fail, curl exits non-zero on any HTTP response >= 400, so the loop correctly keeps polling until a genuine 2xx /health response — matching benchmarks/benchmark_lib.sh's established pattern in this repo.