Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AOF Replay using a Background Task at the Replica #783

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

AOF Replay using a Background Task at the Replica #783

wants to merge 14 commits into from

Conversation

vazois
Copy link
Contributor

@vazois vazois commented Nov 7, 2024 •
edited
Loading

This PR separates AOF enqueue operation from replay. It contains the following

  • Implementation of an IBulkLogConsumer that performs the AOF replay at the replica side.
  • Adds throttling of enqueue operations when AOF replay lags beyond a specific amount (i.e. throttle if TailAddress - ReplicationOffset >ReplicationMaxLag)
  • Ensure re-initialization due to address skipping happens safely while replay consumer is reading from the log.

PS: Fixed a bug related to LRANGE which was incorrectly calling RMWObjectStoreOperationWithOutput instead of ReadObjectStoreOperationWithOutput

@vazois vazois force-pushed the vazois/aof-background-process branch 2 times, most recently from 74ba15b to e0f28f1 Compare November 8, 2024 19:50
@vazois vazois marked this pull request as ready for review November 8, 2024 20:02
@vazois vazois requested a review from badrishc November 8, 2024 20:02
@vazois vazois force-pushed the vazois/aof-background-process branch from e0f28f1 to 6880d53 Compare November 8, 2024 20:11
@TalZaccai TalZaccai self-requested a review November 12, 2024 19:13
@vazois vazois force-pushed the vazois/aof-background-process branch 4 times, most recently from 7baae01 to 4ea6b48 Compare November 14, 2024 01:53
@vazois vazois force-pushed the vazois/aof-background-process branch from e645410 to 5fe06ca Compare November 15, 2024 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@badrishc badrishc badrishc left review comments

@TalZaccai TalZaccai Awaiting requested review from TalZaccai

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vazois @badrishc