cloudflare protection

[adultviews20]

Hi i read alot about anti-blocking features and human-like fingerprints at your website.
but finally i cannot pass entrance protection by cloudflare
https://www.x-kom.pl/g-2/c/159-laptopy-notebooki-ultrabooki.html?page=3
is it possible to pass it ? thanks

[adultviews20]

hi again, i try alot of different system as pupputeer + stealth mode, playwright, selelenium, selenium undetectable driver, also try to change alot of kind headers and browser args, nothing help, but finally i found solution https://github.com/zfcsoftware/youtube_lessons_resources,
if you interesting maybe you can upgrade your system with that info.

[barjin]

Hello and thank you for your interest in Crawlee!

Indeed, Cloudflare Bot Protection is a thing we keep seeing more and more often nowadays. From what we understand, Cloudflare actually allows its users to set custom rules for triggering the Bot Protection - so in some cases, Crawlee can pass just fine, but get blocked in others (if the webmaster decides to set the screening really strict).

In its screening evasion attempts, Crawlee actually does many of the things you've mentioned (switching up headers, browser launch arguments, etc.) automatically - so doing it again manually might actually decrease the anti-blocking performance. We're hoping to come up with a more systemic solution in the upcoming year, as it becomes more and more apparent that the tricks Crawlee is using now are becoming outdated - bot protection and anti-blocking is a never-ending arms race. And well, having our cards on the table as an open-source project doesn't really help :)

Either way, thank you for sharing that project with us - we've gone through the codebase and it doesn't seem to do anything more than Crawlee does - that just shows how random (or crazy good) the Cloudflare tool sometimes is. We'll keep it in mind for any rewrites of the anti-blocking tooling though.

I'll now close this discussion, but feel free to share any extra findings if you come across something. Thanks!