fix: rely on DB library to purge project cache after config updates#12749
Closed
ChiragAgg5k wants to merge 1 commit into
Closed
fix: rely on DB library to purge project cache after config updates#12749ChiragAgg5k wants to merge 1 commit into
ChiragAgg5k wants to merge 1 commit into
Conversation
The manual `purgeCachedDocument('projects', ...)` calls added after
project-config updates (#12654) worked around a stale read-after-write:
`updateDocument()`'s post-commit cache purge ran before the real COMMIT
when the write was nested inside an endpoint-level withTransaction(), so
a concurrent reader could re-cache the pre-commit project.
utopia-php/database now defers cache purges until the outermost
transaction commits (utopia-php/database#910), which fixes the race at
the source and makes these manual purges redundant. Remove them from the
endpoints that write the project document directly.
Kept the purges in Platforms/Keys/DevKeys/Webhooks endpoints: those write
child collections but invalidate the project's subquery-populated cache,
which the library cannot purge automatically.
Points composer at the dev branch pending the library release + tag.
✨ Benchmark resultsComparing
Per-scenario breakdown & investigation detailsMetrics below reflect the current branch (after). Δ P95 compares against the base.
Top API waits (after)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes the manual
purgeCachedDocument('projects', ...)band-aids added in #12654 and fixes the underlying stale read-after-write at its source inutopia-php/database.Background
Project config endpoints write the project document inside an endpoint-level
withTransaction()(for lost-update safety) and are read back from the cached project document.updateDocument()'s "purge again after commit" guard only fired after the realCOMMITwhen the write owned the outermost transaction. Nested in a caller's transaction, that purge ran while still inside the transaction — before commit — so a concurrent reader could re-cache the pre-commit project. #12654 patched around it by purging the project cache again in each endpoint.The proper fix lives in the DB library: utopia-php/database#910 defers every in-transaction cache purge until the outermost transaction commits. That closes the race for all callers, so the manual purges here are redundant.
Reproduced (real MariaDB + Redis, 16 concurrent readers, single writer): 40.6% → 0% stale reads after the library fix.
Changes
purgeCachedDocument('projects', ...)from the 15 endpoints that write the project document directly (AuthMethods, OAuth2, Protocols, SMTP, Services, and 10 Policies).subQuery*-populated cache, which the library cannot purge automatically (not a DB relationship).composer.json/composer.lockpointutopia-php/databaseat the fix branch via a VCS repo.Blocked on / merge order
composer.jsonfromdev-fix/defer-cache-purge-until-outermost-commit as 6.1.0to the tagged release and re-lockKept as draft until the library release is out — it must not merge against a dev branch.