perf(make_idempotent): introduce row lock to improve 2PC for idempotent atomic writes#2214
Merged
empiredan merged 24 commits intoapache:masterfrom Apr 14, 2025
Merged
Conversation
15 tasks
acelyc111
approved these changes
Apr 13, 2025
foreverneverer
approved these changes
Apr 14, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#2197
Previously in #2198, we implemented
idempotence for each atomic write by blocking the entire mutation queue until the 2PC
pipeline was drained. However, this significantly affects performance since the pipeline
is stalled and all write requests get stuck in the mutation queue.
To address this performance issue, we introduce a row-lock mechanism: each hash key
and the highest decree currently in the 2PC phase are recorded in a hash table. For each
atomic write request, if the maximum decree associated with its hash key has not yet been
applied to the storage engine, the request is blocked in the mutation queue. Otherwise,
the hash key is considered unlocked and the request can proceed into the 2PC phase at
any time.
To avoid the performance overhead of deserialization, we directly use the
partition_hash(an unsigned 64-bit integer) from the client instead of the hash key. This also makes memory
usage more predictable, as the
partition_hashhas a fixed size.Additionally, to mitigate the performance impact caused by frequent insertions and deletions
in the row-lock hash table, we introduce an LRU strategy: keys are only evicted when the hash
table exceeds a certain size threshold, and only the least recently used keys with no active
usage are removed.
Give a concrete example to illustrate how atomic write requests are handled after introducing
row locks. Suppose a client issues an
incrrequest to a primary replica. If the primary replicahas been configured to make all atomic write requests idempotent, then:
appended to the mutation queue.
locked(i.e. the maximum decree associated with the hash key has not been applied to the
storage engine).
engine, and create a single put request to store the final value 101.
plogand broadcast tosecondary replicas.