Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: flaky rln test #3173

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

chore: flaky rln test #3173

wants to merge 1 commit into from

Conversation

darshankabariya
Copy link
Contributor

Close #3095

Fix flaky behavior in RLN relay nullifier test.

Copy link

github-actions bot commented Nov 8, 2024

You can find the image built from this PR at

quay.io/wakuorg/nwaku-pr:3173

Built from 1d28d6f

@gabrielmer
Copy link
Contributor

Thank you! Small question, why do we need to attempt many times until we get the value that we want?

Isn't there a way to make it more deterministic?

@darshankabariya
Copy link
Contributor Author

darshankabariya commented Nov 8, 2024

Thank you! Small question, why do we need to attempt many times until we get the value that we want?

Isn't there a way to make it more deterministic?

The nullifier log works like a capped queue with a maximum size of 4 epochs. When a new message arrives in epoch 5, the log needs to both remove the oldest message (from epoch 1) and add the new message. The flaky test occurs because we're checking the log length immediately after this operation - sometimes we catch it when it's only removed the old message (length = 3) but hasn't yet added the new one (should be 4). Instead of using a fixed wait time, we can make the test more reliable by actively waiting until the log reaches its expected length of 4, confirming both the removal of the old message and addition of the new one is complete. or we can do like as you suggest more deterministict, wait little longer like ( 10000 milies ). that also help get rid this flakiness. it's kind of fixed waiting but yeah it's more straight forword. whatever you suggest ? 🤟🏻

@gabrielmer
Copy link
Contributor

The nullifier log works like a capped queue with a maximum size of 4 epochs. When a new message arrives in epoch 5, the log needs to both remove the oldest message (from epoch 1) and add the new message. The flaky test occurs because we're checking the log length immediately after this operation - sometimes we catch it when it's only removed the old message (length = 3) but hasn't yet added the new one (should be 4)

Got it, thank you! Makes sense, yes it's better as you did then! Better to catch earlier and don't wait more time than needed.

Last question then. If the queue has size 4 and a message is removed and the new one is added - isn't it possible for the queue to be size 4 and pass the test even before the message was even sent?

In other words, how do we now that this check is not done before we remove the message and pass without having done any operation?

@darshankabariya
Copy link
Contributor Author

darshankabariya commented Nov 8, 2024

The nullifier log works like a capped queue with a maximum size of 4 epochs. When a new message arrives in epoch 5, the log needs to both remove the oldest message (from epoch 1) and add the new message. The flaky test occurs because we're checking the log length immediately after this operation - sometimes we catch it when it's only removed the old message (length = 3) but hasn't yet added the new one (should be 4)

Got it, thank you! Makes sense, yes it's better as you did then! Better to catch earlier and don't wait more time than needed.

Last question then. If the queue has size 4 and a message is removed and the new one is added - isn't it possible for the queue to be size 4 and pass the test even before the message was even sent?

In other words, how do we now that this check is not done before we remove the message and pass without having done any operation?

Excellent observation! I agree – since the pre-check has already done in epoch 4, we just need to focus on the post-check for the subsequent epoch.

image

Copy link
Contributor

@gabrielmer gabrielmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: flaky RLN test
2 participants