docs: computation-model corrections + full make() reproducibility contract (incl. tripartite)

[dimitri-yatsenko]

Corrections and clarifications to the Computation Model explanation and the AutoPopulate spec, prompted by review of explanation/computation-model. Two files: explanation/computation-model.md and reference/specs/autopopulate.md.

Correctness fixes (Best Practices)

• Transaction guidance was wrong. The page recommended wrapping multi-row inserts in with dj.conn().transaction: inside make(). But populate() already opens a transaction per key, so a make()'s inserts are already atomic — and opening a nested transaction raises an error (DataJoint does not support nested transactions). Replaced with an explanation of the built-in atomicity plus a # WRONG example.
• Single-key testing antipattern. The page suggested testing by calling make() directly (T().make(key)), which bypasses populate() — runs outside the per-key transaction, skips job reservation and error capture, writes as an uncontrolled side effect. Now recommends T.populate(key, max_calls=1); notes a future no-insert debug mode as the only case where a direct make() call would be safe.

The make() reproducibility contract, surfaced and completed

• Five rules, not three. The "make() Contract" section listed only the three mechanical steps (receives/computes/inserts); it now enumerates the full five-rule reproducibility contract inline and links the normative spec (autopopulate.md §4.3).
• Tripartite phase contract (documented in both the explanation and, normatively, autopopulate.md §4.5, with a cross-reference from §4.3):
  • Simple rule up front — make_fetch only fetches, make_compute only computes, make_insert only inserts.
  • Precise boundaries: make_fetch must not insert and must be bitwise reproducible (it is re-run and hash-verified inside the transaction); make_compute must neither fetch nor insert but need not be deterministic — stochastic algorithms are allowed, as it runs once and is never re-verified; make_insert always runs inside the transaction, so it may also fetch/compute.
  • Consequence: make_fetch/make_compute are safely testable directly (no DB writes); only make_insert/populate() writes.
  • Framed as contract rules the author follows and a pipeline is validated against at review/deploy time — not runtime-enforced.

Terminology

• Reproducibility here means tracked derivation — every row is traceable to its declared inputs — not bitwise determinism. Stochastic computation is allowed. Reworded the "same output" claims accordingly, and used lineage/reproducibility rather than "provenance" in these framework docs.

References

• Added "See also" links (AutoPopulate / Cascade / Diagram specs, run-computations & distributed-computing how-tos, related concept pages).

Files: src/explanation/computation-model.md (+124/−13), src/reference/specs/autopopulate.md (+14/−1).

[dimitri-yatsenko]

Ready for review — resynced against main (clean, no conflicts; the autopopulate.md overlap with #195 auto-merged). Net change is two files: explanation/computation-model.md and reference/specs/autopopulate.md.

Scope (see the updated PR description for detail): the transaction + single-key-testing best-practice corrections, the full five-rule make() contract surfaced with the tripartite phase contract (now normative in autopopulate.md §4.5, make_fetch bitwise-reproducible / make_compute may be stochastic / make_insert always in-transaction), the reproducibility-≠-determinism framing, and the "See also" references.

[MilagrosMarin]

Thanks @dimitri-yatsenko — solid correctness pass. Verified against source:

• Transaction guidance. Connection.start_transaction() at connection.py:523 raises DataJointError("Nested connections are not supported.") when one's already open, and _populate_one opens one per key at autopopulate.py:655. The old "wrap in with dj.conn().transaction:" advice was outright broken; the # WRONG replacement is right.
• Testing pattern. populate(*restrictions, max_calls=1) accepts a dict positionally, and T().make(key) genuinely bypasses start_transaction, self._upstream construction, and the finally-block cleanup.
• Five-rule contract mirrors autopopulate.md §4.3.
• Tripartite phase contract matches the impl precisely: make_fetch runs outside (line 707) then inside (line 713) with deepdiff.DeepHash verification (lines 709+714); make_compute runs once outside (line 708), never re-verified; make_insert runs after start_transaction() (line 711) via gen.send() (line 717). "Bitwise reproducible for fetch, stochastic allowed for compute" is genuinely the shape of the guarantee, and stating it out loud is the clearest single win in this PR.

One pre-existing snag worth flagging (not this PR's regression): the tripartite worked example in computation-model.md shows def make_compute(self, key, fetched) with body (raw_signal,) = fetched, but the impl at autopopulate.py:305 calls self.make_compute(key, *fetched_data). So make_compute receives raw_signal, not the tuple — the destructuring breaks. Same shape for make_insert. Worth a small follow-up to either drop the tuple wrapping in the example returns or align the impl's unpacking.

Approving.

[dimitri-yatsenko]

Thanks @MilagrosMarin — great catch. Fixed the tripartite example to match the implementation' calling convention (make_compute(key, *fetched_data), make_insert(key, *computed_result) per autopopulate.py:283-311):

• Worked example: make_compute(self, key, raw_signal) and make_insert(self, key, avg) now receive the unpacked tuple (no more (raw_signal,) = fetched, and make_insert no longer takes a fetched param it never receives).
• Fixed the "How It Works" pseudocode and the safe-testing snippet to show *fetched / *computed.
• Corrected the same signatures in the phase-contract bullets (both computation-model.md and autopopulate.md §4.5).
• While there: autopopulate.md §4.5's example returned bare scalars where the impl *-unpacks — wrapped those returns in tuples and fixed the make_fetch docstring ("runs outside the transaction; re-run and hash-verified").

Re-requesting review.

[MilagrosMarin]

LGTM! Thanks