Randomly my badge goes back to silver from gold?

[DefinetlyNotAI]

The badge randomly downgraded to silver, and when I checked the dashboard, it shows everything is 100% complete, and nothing is wrong.

Here is the project,
https://www.bestpractices.dev/en/projects/9451?criteria_level=2

Here is the repo,
https://github.com/DefinetlyNotAI/Logicytics

[david-a-wheeler]

You're right, that's wrong.

I think what's happening is that the badge image is cached at the CDN, and for some reason the cached value is not being removed when there's an update. We need to figure out WHY that happened.

Did you recently complete things to earn gold?

I will investigate regardless. Thanks for the report!

[DefinetlyNotAI]

Funny part was, I completed everything 2 days ago, GitHub didn't update the badge until 3 hours before I opened the ticket, so I refreshed the page, and its back to silver - I thought I was the problem as in I was hallucinating the gold status 💀

[DefinetlyNotAI]

And now its back to gold - dunno if this helped or anything, but thanks anyways :)

[DefinetlyNotAI]

Wait, no - In github it shows gold but the openssf website shows silver???

[david-a-wheeler]

We use Fastly as our CDN. Fastly distributes cached data out quickly (as pitched by the name) to many locations. There may be an inconsistency. Another option is that slightly different request names are being interpreted as different cached results.

There are 2 big problems in Computer Science: Caching, naming, and off-by-one errors. Looks like we hit item #1. For scale we need caching.

Are there different URLs that are being requested when you see different results? If you have any other data that might give me a lead, please let me know.

Sorry for the trouble. The fix will probably be not too hard, but the trick will be to track it down.

[david-a-wheeler]

Don't close it yet. I want to track this down.

It'll fix itself eventually, because cached data has a limited lifetime. But I'd like to fix it.

[DefinetlyNotAI]

We use Fastly as our CDN. Fastly distributes cached data out quickly (as pitched by the name) to many locations. There may be an inconsistency. Another option is that slightly different request names are being interpreted as different cached results.

There are 2 big problems in Computer Science: Caching, naming, and off-by-one errors. Looks like we hit item #1. For scale we need caching.

Are there different URLs that are being requested when you see different results? If you have any other data that might give me a lead, please let me know.

Sorry for the trouble. The fix will probably be not too hard, but the trick will be to track it down.

Welp, I guess when a programmer finds something out of his ability to fix rn - he jumps to observing it 😅

Ehh, i dont really worry if it bounces from silver to gold and reverse, its not like I am depending my computer security on it - good luck on your bug-hunt - I have a feel you may need it

[david-a-wheeler]

Welp, I guess when a programmer finds something out of his ability to fix rn - he jumps to observing it 😅

Fair enough, you certainly aren't wrong :-). But I really would like to fix this.

Ehh, i dont really worry if it bounces from silver to gold and reverse, its not like I am depending my computer security on it - good luck on your bug-hunt - I have a feel you may need it

Thanks.... I think?!?

My fear is that this is caused by a race condition that's outside our direct control. When our site sends information to Fastly, they're necessarily via different TCP/IP streams. Inside the site I can use various locking mechanisms to enforce ordering, but nothing about TCP/IP requires that one stream is faster/slower than another. Fastly may also not necessarily order things as we expect.

I do have a sledgehammer. When an image updates, we could send a few "clear cache" values early on. Typically badges change a lot (while people edit their entry), but then they stay consistent for a long period of time afterwards (especially once they earn a badge). We could schedule them in the database, instead of in memory, to ensure they were remembered and sent. Or we could just periodically send "clear cache" messages for recent changes. I hate to use the sledgehammer, but that would eliminate many problems.

[DefinetlyNotAI]

Ooh, at least it seems you have a plan!

Most bugs issue's are actually finding them - it seems you narrowed it down to caching problem's that may or may not be out of ur control - I would like to help, so if you need anything from me, do say

[david-a-wheeler]

Thanks so much! If I can fix it on our end, I'd rather actually fix it. I'm going to see if I can find an actual bug to fix. Thanks for the offer. I'll have to figure out what to ask first :-).

[DefinetlyNotAI]

Well, at least I can say I am the only project with 300% silver

[DefinetlyNotAI]

This may help you, as it must have somthing to do with caching, as I have been in silver for almost 4 months, and in the end it does recognise I am 300% but doesnt get the proper badge for it, which is pretty weird, I have thought it was linked as in only the percent actually mattered and the Badge was then shown based on the percent