Summary
Add a complete, executed worked example of the three-part (tripartite) make — make_fetch / make_compute / make_insert — as a tutorial. Today the pattern is documented in prose (Computation Model, autopopulate.md §4.5) but there is no end-to-end, runnable example.
Why
- The calling convention is a footgun. The framework invokes
make_compute(key, *fetched_data) and make_insert(key, *computed_result) (the returned tuples are unpacked; make_insert receives only the computed result, not fetched). This is easy to get wrong — the docs themselves shipped an incorrect example recently. A copy-pasteable, correct template is the best antidote.
- The semantics are subtle.
make_fetch runs outside the transaction and is re-run and hash-verified inside it (so it must be bitwise-reproducible); make_compute runs once and is never re-verified (so it may be stochastic); make_insert runs inside the transaction. These boundaries are hard to grasp abstractly and land better as concrete, running code.
- Real need. It's the recommended pattern for long-running computations (minutes–hours) that can't hold a transaction open — a common ML / signal-processing case.
What it should show
- All three phases with the correct signatures and tuple return/unpack convention.
- The re-fetch/verify behavior, ideally including the failure mode: inputs change mid-compute → rollback, no insert.
- A
make_compute that is legitimately non-deterministic (e.g. a seeded vs. unseeded stochastic step), to make "reproducibility = tracked derivation, not bitwise determinism" concrete.
- How
populate(make_kwargs=...) forwards to make_fetch.
Recommended form
- Executed, not prose — a tutorial notebook run in CI (or a doctest), so it stays correct as the API evolves. A non-executed snippet would drift out of sync again (exactly the failure we just fixed).
- Placement: Tutorials → Advanced (e.g. "Long computations with the three-part make"), or How-To → Computation. Pair with, and cross-link,
autopopulate.md §4.5 and the Computation Model.
Notes
- Scope/reach is modest (most pipelines use plain
make(); tripartite is advanced), but the value for its audience is high and the reliability win — a tested example that can't drift — is the main argument.
- Simulating a genuinely "long" computation cheaply in CI is the one awkwardness; a trivial compute with a comment noting where the real work would go is acceptable.
Targeted for a future release, not v2.3.1.
Summary
Add a complete, executed worked example of the three-part (tripartite)
make—make_fetch/make_compute/make_insert— as a tutorial. Today the pattern is documented in prose (Computation Model,autopopulate.md§4.5) but there is no end-to-end, runnable example.Why
make_compute(key, *fetched_data)andmake_insert(key, *computed_result)(the returned tuples are unpacked;make_insertreceives only the computed result, notfetched). This is easy to get wrong — the docs themselves shipped an incorrect example recently. A copy-pasteable, correct template is the best antidote.make_fetchruns outside the transaction and is re-run and hash-verified inside it (so it must be bitwise-reproducible);make_computeruns once and is never re-verified (so it may be stochastic);make_insertruns inside the transaction. These boundaries are hard to grasp abstractly and land better as concrete, running code.What it should show
make_computethat is legitimately non-deterministic (e.g. a seeded vs. unseeded stochastic step), to make "reproducibility = tracked derivation, not bitwise determinism" concrete.populate(make_kwargs=...)forwards tomake_fetch.Recommended form
autopopulate.md§4.5 and the Computation Model.Notes
make(); tripartite is advanced), but the value for its audience is high and the reliability win — a tested example that can't drift — is the main argument.Targeted for a future release, not v2.3.1.