Add robust eigh_v2 problem

[msaroufim]

Summary

• Adds a separate eigh_v2 linalg problem and leaderboard entry without changing the existing eigh implementation or rankings.
• Carries forward the important robustness ideas from the related eigh PRs: Improve eigh accuracy and benchmark balance #156 benchmark/accuracy balancing by @msaroufim, and eigh_py: reject physically-impossible benchmark times (roofline floor) #159/eigh_py: regenerate a fresh input each timed benchmark iteration #160/eigh_py: reject output-object deferral in the correctness check #161 roofline floor, fresh timed inputs, and output-deferral rejection by @robobryce.
• Does not carry the old profiler-capture cherry-pick from Wrap timed custom_kernel launches in a cudaProfiler capture range (qr_v2, eigh_py) #157 because profile mode is already present on main via Add Eigh profile mode #158.
• Adds block-diagonal correctness cases with varied block sizes, plus additional 512-shape benchmark distributions so precision choices need to respond to matrix quality rather than only public shape IDs.
• Simplifies the checker to the non-redundant hard gates: eigen-equation residual, eigenvalue error against torch.linalg.eigvalsh(A), and orthogonality. Reconstruction is skipped because it follows for square orthonormal Q when the eigen-equation holds.
• Keeps ranked runtime practical by making leaderboard mode use the same rechecked benchmark path instead of the previous 1000-repeat ranked loop.

Validation

• python3 -m py_compile problems/linalg/eigh_v2/eval.py problems/linalg/eigh_v2/reference.py problems/linalg/eigh_v2/task.py problems/linalg/eigh_v2/submissions/torch_eigh.py problems/linalg/eigh_v2/submissions/triton_diagonal_fast_path.py
• /Users/mark/Dev/kernelbot/.venv/bin/ruff check problems/linalg/eigh_v2
• git diff --check
• Local-only KernelBot debug setup against kernelbot_eigh_v2_debug on 127.0.0.1, with the local checkout registered through PROBLEM_DEV_DIR / PROBLEMS_REPO.
• Baseline torch_eigh.py local submissions on B200:
  • test: 41/41 pass
  • benchmark: 10/10 pass
  • real leaderboard submission after repeat-budget fix: pass, about 116s end-to-end locally; recorded phase durations included test at 7-10s, benchmark at 34-44s, and leaderboard at 44-48s.
• Adversarial local submissions:
  • Tensor subclass/output deferral failed in the evaluator with Q must be a plain torch.Tensor.
  • Cache/replay and harness timing patch attempts were rejected by KernelGuard on the normal local API.
  • With KernelGuard disabled only on a separate local debug API, cache/replay still failed evaluator correctness, and forged CUDA-event timing failed the new physical roofline floor.

Provenance

Resolved problem directory: problems/linalg/eigh_v2. Ranked/profile shapes come from eigh_v2/task.yml benchmarks:. Profile mode wraps the submitted kernel in the upstream custom_kernel NVTX region. Reference-kernels base used for this PR: origin/main at 4a1153e, with this branch at 9bcefc4.

[msaroufim]

Final local eigh_v2 audit

I finished a broader local audit and calibration pass for this PR. The changes described below are currently in the local eigh_v2 patch and still need to be committed/pushed to the PR branch.

Correctness model

The checker is now aligned with the invariant-based design described in Linear Algebra Kernels For The Age Of Research:

• Q.T @ Q ~= I
• A @ Q ~= Q @ diag(L)

It does not compare L against an independent torch.linalg.eigvalsh result.

For square orthonormal Q, these two invariants imply A ~= Q @ diag(L) @ Q.T and that the entries of L are eigenvalues. The checker therefore also skips the redundant reconstruction matmul. Sorted eigenvalues, finite plain FP32 outputs, expected shapes/layouts, and CUDA device placement remain output-contract checks.

Residuals are evaluated in FP64. The final dimension-scaled gates are:

• eigen equation: min(64 * n * eps32, 6e-3);
• orthogonality: min(25 * n * eps32, 2.5e-3).

Ascending order permits four cumulative FP32 ULPs at the returned spectrum's actual scale, including low-magnitude spectra.

Final problem design

• The existing eigh problem remains untouched; all changes are isolated to eigh_v2.
• Inputs follow torch.linalg.eigh(A, UPLO="L"): the diagonal and lower triangle are authoritative, and the ignored upper triangle may contain unrelated finite values.
• The suite uses only five workload dimensions:
  • batch=40, n=352
  • batch=640, n=512
  • batch=60, n=1024
  • batch=8, n=2048
  • batch=2, n=4096
• There are 28 correctness cases and 9 ranked cases. The ranked set contains one dense case at each dimension plus mixed, lower-triangle block, tridiagonal, and block-tridiagonal cases.
• Aggregate ranked batch * n^3 work is about 0.96x the corresponding QR-v2 suite.
• Mixed batches guarantee important numerical structures at seed-randomized positions, reducing the value of routing a public shape wholesale to a library fallback.

Added input families include tridiagonal matrices, variable-size/permuted block-tridiagonal matrices, lower-triangle block matrices with an independently generated amplified upper triangle, and heterogeneous mixed batches.

Three negative regressions cover averaging both stored triangles, a 1% eigenvalue shift that violates the eigen equation, and reversed eigenpairs. All remain rejected without an eigenvalue-reference oracle.

Benchmark and profile behavior

• Ranked timing uses fresh inputs and validates outputs after timed execution.
• Measurements accumulate up to 100 repeats or 2 seconds of kernel time.
• The physical floor is based on mandatory tensor traffic with a loose 0.1 factor; the unjustified generic n^3 FLOP floor was removed.
• Profile mode runs the submitted kernel inside the custom_kernel NVTX range.
• A fresh detached checkout was tested through the NCU service:
  • the default 10-launch capture passed and produced 10 real kernel records;
  • a separate 64-launch tridiagonal capture also passed;
  • both produced valid .ncu-rep artifacts.

No KernelBot or NCU-service source changes are part of this PR.

KernelBot validation

After removing the redundant eigvalsh oracle, the Torch reference passed:

• 28/28 public and secret correctness cases;
• 9/9 public and secret benchmark cases;
• 9/9 public and secret leaderboard cases.

A complete ranked submission took 142.3 seconds wall time (about 2m22s), down from 154.7s with the redundant oracle. The latest public/secret Torch leaderboard geometric means were 139.74 ms / 140.97 ms.

I also replayed ten distinct fast algorithm families from the existing Eigh leaderboard through the final invariant-only checker:

• 2/10 passed all 28 correctness cases;
• one of those also passed all 9 ranked cases and was historically the second-fastest family; its earlier final-suite measurement was 107.1 ms geometric mean;
• the other correctness-passing family still fails the ranked lower-triangle case with orthogonality residual 1.0;
• the remaining eight fail 3–18 correctness cases due to finite but incorrect eigen residuals, orthogonality, ordering, or triangle semantics;
• no replayed failure depended on the removed eigenvalue-reference oracle.

Thus only 1/10 is fully rankable as-is. The other failures should be addressed with semantic fixes, refinement, sorting, or reorthogonalization rather than another independent reference eigensolve.

Local validation

• Ruff: pass
• Python compilation: pass
• git diff --check: pass
• CPU baseline input-family checks: pass
• Three negative regressions: rejected
• Torch KernelBot test/benchmark/leaderboard: pass
• Ten-family invariant-only replay: complete
• NCU default and tridiagonal stress captures: pass

Resolved problem directory: problems/linalg/eigh_v2. Ranked/profile shapes come from task.yml benchmarks:. The PR head is currently 9bcefc424f25c10ad74bbdc9a05fa13e21391e63; the final audit changes above are the pending local diff on top of that SHA.

[brycelelbach]

The large matrix batch sizes are a bit small, it's hard to fully utilize the chip for them. It's probably too late to change it for this contest, but larger batch sizes would be nice in the future.