Nonlinear diff: skip evaluator rebuild when structure is unchanged

[yeabbratz]

NonLinearProgram._cache_evaluator! rebuilds the MOI.Nonlinear evaluator — AD tape
construction, Jacobian/Hessian sparsity detection, coloring — plus all the index mappings on
every forward_differentiate!/reverse_differentiate! call. But the Cache is a pure
function of the structure of the Form: parameter values do not live in it (they enter the
computation through the primal point / the Form's parameter storage), and the primal/dual
point lives in model.x/model.y/model.s. Rebuilding it on a model whose structure is
unchanged produces an identical evaluator and identical mappings, so the rebuild is pure
per-call overhead — paid, e.g., by every reverse-after-forward on the same solve, and by every
backward in a parametric training loop once the diff model is instantiated.

This PR makes _cache_evaluator! return the existing model.cache when there is one, and
invalidates the cache on every structural mutation of the model (add_variable(s),
add_constraint, objective function/sense changes; MOI.empty! already reset it). Inside
DiffOpt's own lifecycle the cache could in fact never go stale — the Form is populated
exactly once by MOI.copy_to, and any structural edit at the DiffOpt.Optimizer level
discards the whole diff model (diff = nothing) — but the invalidation hooks make the model
safe as a standalone MOI.ModelLike too.

Results are bit-identical to the rebuild path (same tapes, same sparsity order, same values in,
same FLOP order); only the per-differentiation setup cost is removed. No API change; behavior
is preserved by construction.

Tests

• test_evaluator_cache_reused_across_differentiations: repeated reverse (new seeds) and
  forward calls on one solve return the same Cache object, and the results are == (not
  ≈) to a fresh model differentiated once in the same final state.
• test_evaluator_cache_invalidated_on_structural_change: objective-sense set and
  add_variable on the inner model drop the cache; differentiation rebuilds it.
• test_evaluator_cache_after_public_structural_edit: adding a JuMP constraint after a
  backward rebuilds the diff model, and the post-edit gradient matches a fresh model built
  directly in the final state.
• Full test suite green locally (Julia 1.12.6).

Benchmark

Median of 7 (after warmup) of a repeated reverse_differentiate! on the same solve — the
diff model is already instantiated, so the evaluator rebuild is exactly the residual this PR
removes. Parameterized NLP with P parameters and P variables (script below), Ipopt, M4 Pro,
1 thread, Julia 1.12.6:

P	master	this PR
10	0.32 ms / 0.3 MiB	0.22 ms / 0.2 MiB
100	1.37 ms / 4.1 MiB	1.04 ms / 2.7 MiB
1000	44.9 ms / 196.0 MiB	38.8 ms / 181.9 MiB

At large P the remaining cost is the dense sensitivity contraction, which is what #369
removes — the two compose.

bench_persistent_evaluator.jl

# Reproducer for the two per-backward residuals the DiffOpt follow-on PRs remove
# (upstreaming of task 4.12, companion to jump-dev/DiffOpt.jl#369):
#
#   PR A ("evaluator rebuild"): `_cache_evaluator!` rebuilds the MOI.Nonlinear evaluator
#       (AD tapes, Jacobian/Hessian sparsity, coloring) on EVERY differentiation call,
#       even when the model structure is unchanged. Measured here as `rev_repeat`: a
#       second reverse_differentiate! on the SAME solve — under stock DiffOpt it pays
#       the full rebuild again; with PR A it reuses `model.cache`.
#
#   PR B ("diff-model rebuild"): `MOI.optimize!` unconditionally nulls `Optimizer.diff`,
#       so the first backward after every re-solve re-instantiates the diff model and
#       re-runs a full `MOI.copy_to`. Measured here as `rev_after_resolve`: a
#       Parameter-only re-solve followed by one reverse — with PR B's opt-in
#       `DiffOpt.PreserveDiffModel()` the diff model survives and only the
#       point-dependent state (parameter values + x/y/s) is refreshed.
#
# Run against a DiffOpt checkout (master, or either PR branch):
#
#   julia --project=<env-with-DiffOpt-devved> bench_persistent_evaluator.jl [P ...]
#
# where P is the parameter/variable count (default: 10 100 1000). The script
# auto-detects `DiffOpt.PreserveDiffModel` and adds the preserve column when present.
# Compare the tables across checkouts: the PR-A delta is `rev_repeat` (stock vs PR A
# branch); the PR-B delta is `rev_after_resolve` stock-vs-preserve on the PR B branch.

using JuMP
import DiffOpt
import Ipopt
import MathOptInterface as MOI
using Statistics

const REPS = 7

function build(P; preserve::Bool)
    model = DiffOpt.nonlinear_diff_model(Ipopt.Optimizer)
    set_silent(model)
    @variable(model, p[i = 1:P] in MOI.Parameter(1.0 + 0.2 * i / P))
    @variable(model, x[1:P] >= 0)
    @constraint(model, [i = 1:P], x[i] * (1 + 0.1 * sin(p[i])) >= p[i])
    @constraint(model, sum(x) >= 0.4 * P)
    @objective(
        model,
        Min,
        sum((x[i] - p[i])^2 for i in 1:P) + 0.01 * sum(x[i]^4 for i in 1:P)
    )
    if preserve
        MOI.set(model, DiffOpt.PreserveDiffModel(), true)
    end
    return model, p, x
end

function seed!(model, x, k)
    DiffOpt.empty_input_sensitivities!(model)
    for i in eachindex(x)
        DiffOpt.set_reverse_variable(model, x[i], sin(0.7 * i + k))
    end
    return
end

"""
Median seconds + allocated MiB of `f()` over REPS runs (after one warmup pair).
`setup()` runs untimed before every measurement — the solve lives there, so only the
differentiation call is measured.
"""
function timed(setup, f)
    setup()
    f()
    ts = Float64[]
    bs = Float64[]
    for _ in 1:REPS
        setup()
        GC.gc()
        stats = @timed f()
        push!(ts, stats.time)
        push!(bs, stats.bytes / 2^20)
    end
    return median(ts), median(bs)
end

function bench(P; preserve::Bool)
    model, p, x = build(P; preserve)
    optimize!(model)
    k = Ref(0)
    # first backward after a Parameter-only re-solve (solve itself untimed): pays the
    # diff-model instantiate + copy_to (+ evaluator build) under stock; only the
    # point refresh under PreserveDiffModel
    rev_after_resolve, alloc_resolve = timed(
        () -> begin
            k[] += 1
            for i in 1:P
                set_parameter_value(p[i], 1.0 + 0.2 * i / P + 0.1 * sin(k[]))
            end
            optimize!(model)
            seed!(model, x, k[])
        end,
        () -> DiffOpt.reverse_differentiate!(model),
    )
    # repeated backward on the SAME solve: under stock this still rebuilds the
    # evaluator every time (PR A's target); the diff model itself is reused
    rev_repeat, alloc_repeat = timed(
        () -> begin
            k[] += 1
            seed!(model, x, k[])
        end,
        () -> DiffOpt.reverse_differentiate!(model),
    )
    return rev_after_resolve, alloc_resolve, rev_repeat, alloc_repeat
end

function main()
    Ps = isempty(ARGS) ? [10, 100, 1000] : parse.(Int, ARGS)
    has_preserve = isdefined(DiffOpt, :PreserveDiffModel)
    println("DiffOpt at: ", pkgdir(DiffOpt))
    println("PreserveDiffModel available: ", has_preserve)
    println()
    header = "P      | rev_after_resolve      | rev_repeat"
    has_preserve && (header *= "             | rev_after_resolve (preserve)")
    println(header)
    println("-"^length(header))
    for P in Ps
        r1, a1, r2, a2 = bench(P; preserve = false)
        line = rpad(P, 6) * " | " *
               rpad(string(round(r1 * 1000; digits = 2), " ms  ",
                   round(a1; digits = 1), " MiB"), 22) * " | " *
               rpad(string(round(r2 * 1000; digits = 2), " ms  ",
                   round(a2; digits = 1), " MiB"), 22)
        if has_preserve
            r1p, a1p, _, _ = bench(P; preserve = true)
            line *= " | " * string(round(r1p * 1000; digits = 2), " ms  ",
                round(a1p; digits = 1), " MiB")
        end
        println(line)
    end
    return
end

main()

Context

Same residual-removal series as #369 (single adjoint solve for the nonlinear reverse). In a
downstream differentiable-ACOPF training loop, evaluator reuse was validated bitwise against
the rebuild path across a case14→case2000 ladder and composes with #369 and with keeping the
diff model across parameter-only re-solves (companion PreserveDiffModel PR) for ×12.6
cumulative per-backward at 2000-bus scale.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.93%. Comparing base (f3fdbd8) to head (5c2328a).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #370      +/-   ##
==========================================
+ Coverage   91.88%   91.93%   +0.05%     
==========================================
  Files          17       17              
  Lines        2722     2739      +17     
==========================================
+ Hits         2501     2518      +17     
  Misses        221      221              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.