waiting forever for the Cloudflare check

[Alfred]

For some pages the Cloudfare check takes extremly long (5 minutes or sometimes forever). I am trying to access them from a linux desktop with chromium or a recent version of firefox (115.3 ESR).

The worst case is a local bus company, and I am accessing it from the IP of a UK broadband provider

https://www.oxfordbus.co.uk

The check is inmediate if I access it on firefox or chrome on an Android phone using the same IP over WIFI.

Any suggestions?
perhaps making Cloudfare believe that I am using firefox on a mobile phone?

[Jim Jackson]

Post by Alfred
For some pages the Cloudfare check takes extremly long (5 minutes or
sometimes forever). I am trying to access them from a linux desktop
with chromium or a recent version of firefox (115.3 ESR).
The worst case is a local bus company, and I am accessing it from the
IP of a UK broadband provider
https://www.oxfordbus.co.uk
The check is inmediate if I access it on firefox or chrome on an
Android phone using the same IP over WIFI.
Any suggestions?
perhaps making Cloudfare believe that I am using firefox on a mobile phone?

Just looked. Worked ok for me on Firefox 102.5.0esr with Ublock Origin
on a Devuan (chimaera) Linux Desktop.

It's the first time ever that I have seen this cloudflare thing and had
to declare I wasn't a robot!!!! How prevalent is it? I've tried deleting
cookies etc and NOT managed to get it to ask again. Anyone explain?

[John McCue]

Follow-ups trimmed to comp.misc

In comp.misc Jim Jackson <***@franjam.org.uk> wrote:
<snip>

Post by Jim Jackson
It's the first time ever that I have seen this cloudflare thing
and had to declare I wasn't a robot!!!!

Lucky you, I have been getting that from them for over a year.
Sometimes I am graced with a captha.

Post by Jim Jackson
How prevalent is it? I've tried deleting cookies etc and NOT managed
to get it to ask again. Anyone explain?

I do not know how cloudflare works, but if I get it I just
bail, thinking going to that WEB Site is not worth my time.

Now the interesting thing, I never get Cloudflare when I log
into my Bank's site. That alone tells be Cloudflare is
doing something I believe is something you would not want
them to do. I almost suspect Cloudflare is examining your
browser cache and maybe cookies. Yes I put my tinfoil hat
on when Cloudflare prompts me :)

[Scott Dorsey]

Post by John McCue
Now the interesting thing, I never get Cloudflare when I log
into my Bank's site. That alone tells be Cloudflare is
doing something I believe is something you would not want
them to do. I almost suspect Cloudflare is examining your
browser cache and maybe cookies. Yes I put my tinfoil hat
on when Cloudflare prompts me :)

Does your bank use Cloudflare for caching at all? If they don't use it,
you won't have any issues with it.

Cloudflare is likely setting cookies in your browser in order to keep track
of connections, since you might have nultiple cloudflare-cached websites
open at the same time.

Cloudflare is not examining your browser cache, it is in part replacing it.
--scott

[John McCue]

followups trimed to only: comp.misc

In comp.misc Scott Dorsey <***@panix.com> wrote:
<snip>

Post by Scott Dorsey
Does your bank use Cloudflare for caching at all? If they don't use it,
you won't have any issues with it.

I do not do any real on-line banking, I only login once per
month to get the statement.

Post by Scott Dorsey
Cloudflare is likely setting cookies in your browser in order to keep track
of connections, since you might have nultiple cloudflare-cached websites
open at the same time.
Cloudflare is not examining your browser cache, it is in part replacing it.

You are probably correct, but the real question is "How can
we prove it ?". Luckily OpenBSD has pledge(2) and unveil(2),
so what ever Cloudflare examine is very limited :)

Post by Scott Dorsey
--scott

[Blue-Maned_Hawk]

Post by Alfred
For some pages the Cloudfare check takes extremly long (5 minutes or
sometimes forever). I am trying to access them from a linux desktop
with chromium or a recent version of firefox (115.3 ESR).
<snip/>
Any suggestions?

**Manually** wiggle your **mouse** across the **contents of the page**—
**do not** use a script or numpad pointerkeys, as this will be detected as
illegitimate.

Post by Alfred
perhaps making Cloudfare believe that I am using firefox on a mobile phone?

I'ven't needed such measures drastic, but you could try using a user-agent
spoofer extension and see if that helps.

[Alfred]

UPDATE:

Running firefox with default settings solved the problem (or at
minimum improves it notably).

The problem is complex. I tried bisecting my prefs.js file, but
it's hard to diagnose the exact setting that originated the problem.

* It seems to be affected by more than one of the settings.

* The result of the cloudfare check is stochastic, not deterministic.

* cloudfare keeps some fuzzy memory of the check that it did one minute ago.

* Different websites using cloudfare have different levels of security.

* Cloudfare is very exigent with the browser being up-to-date. A 2 years old
version of Chrome with default settings is not able to load the page.

These are likely suspects to be contributing to the problem:

user_pref("network.dns.disablePrefetch", true);
user_pref("network.dns.disablePrefetchFromHTTPS", true);
user_pref("network.predictor.enabled", false);
user_pref("network.captive-portal-service.enabled", false);

[Alfred]

Post by Alfred
user_pref("network.dns.disablePrefetch", true);
user_pref("network.dns.disablePrefetchFromHTTPS", true);
user_pref("network.predictor.enabled", false);
user_pref("network.captive-portal-service.enabled", false);

It also seems that Cloudfare wants to force users to be fingerprinted,
as I also had to remove these in prefs.js/user.js:

user_pref("network.trr.mode", 0);

// 2410: disable User Timing API
// https://trac.torproject.org/projects/tor/ticket/16336
user_pref("dom.enable_user_timing", false);

// 2411: disable resource/navigation timing
user_pref("dom.enable_resource_timing", false);

// 2412: disable timing attacks - javascript performance fingerprinting
// https://wiki.mozilla.org/Security/Reviews/Firefox/NavigationTimingAPI
user_pref("dom.enable_performance", false);

PS: Another website that I have found to be most difficult to connect to is
https://uk.indeed.com

[Oregonian Haruspex]

Post by Alfred
For some pages the Cloudfare check takes extremly long (5 minutes or
sometimes forever). I am trying to access them from a linux desktop with
chromium or a recent version of firefox (115.3 ESR).
The worst case is a local bus company, and I am accessing it from the IP
of a UK broadband provider
https://www.oxfordbus.co.uk
The check is inmediate if I access it on firefox or chrome on an Android
phone using the same IP over WIFI.
Any suggestions?
perhaps making Cloudfare believe that I am using firefox on a mobile phone?

I browse with Tor Browser mostly and half the web’s like this now. It’s
extremely shitty.

[immibis]

Post by Oregonian Haruspex
I browse with Tor Browser mostly and half the web’s like this now. It’s
extremely shitty.

What if we (and by "we" I mean "whoever is interested, motivated, and
capable of defending themself from copyright lawsuits, so probably not
me") made a specialized outproxy to bypass site-specific garbage?

For the specific case of Cloudflare there is FlareSolverr, which I think
just runs headless Chrome to take care of these CAPTCHAs (as Cloudflare
seems to want to know if you're a browser, not if you're a human like
Google does); just using an IP address classed as "residential" and
caching to avoid duplicate queries would also avoid a lot of CAPTCHAs.

Specific tweaks could be written to remove ads, cookie popups, and so on.

[Stefan Ram]

Post by immibis
What if we (and by "we" I mean "whoever is interested, motivated, and
capable of defending themself from copyright lawsuits, so probably not
me") made a specialized outproxy to bypass site-specific garbage?

It seems that the Cloudflare check sometimes runs into an
endless loop. If it has not finished after few seconds,
it is possible that it ran into such a loop.

I can't remember the details now, but IIRC, sometimes, such
an endless loop happened when an "http" address was used.
It did continue, sometimes, after the "http" was replaced by
"https". This would make some sense since Cloudflare says that
it is checking that the connection is "secure". However, good
software of course in such as case would explain to the user
what's wrong and not just go into an endless loop!

If there are any MIDM proxies (software which decodes and
re-encodes TLS data between the browser and the server),
it might also be worth a try to bypass those proxies.

[immibis]

Post by Stefan Ram
If there are any MIDM proxies (software which decodes and
re-encodes TLS data between the browser and the server),
it might also be worth a try to bypass those proxies.

I think you might have missed the point of my suggestion. Sites that are
full of garbage on the HTML would *deliberately* be proxied in a way
that would not preserve TLS encryption, in order to remove the garbage.

And there's no point trying to bypass a site's load balancer. That's
within the purview of the site operator, and he's entitled to use one.

[Stefan Ram]

Post by immibis

Post by Stefan Ram
If there are any MIDM proxies (software which decodes and
re-encodes TLS data between the browser and the server),
it might also be worth a try to bypass those proxies.

I think you might have missed the point of my suggestion.

Yes. Sorry! It was the first time I saw the subject line
"waiting forever for the Cloudflare check", and I responded
more to that subject line than to your contribution.
I should have clarified this from the outset!

[Scott Dorsey]

Post by immibis

Post by Stefan Ram
If there are any MIDM proxies (software which decodes and
re-encodes TLS data between the browser and the server),
it might also be worth a try to bypass those proxies.

I think you might have missed the point of my suggestion. Sites that are
full of garbage on the HTML would *deliberately* be proxied in a way
that would not preserve TLS encryption, in order to remove the garbage.

What do you mean by "sites that are full of garbage on the HTML?"

And cloudflare is ALREADY acting as a proxy, why do you want another layer
of stuff to go wrong?

Post by immibis
And there's no point trying to bypass a site's load balancer. That's
within the purview of the site operator, and he's entitled to use one.

The whole point of using cloudflare is that you don't need load balancing
anymore, the cloudflare proxies cache your data for you.
--scott

[immibis]

Post by Scott Dorsey

Post by immibis

Post by Stefan Ram
If there are any MIDM proxies (software which decodes and
re-encodes TLS data between the browser and the server),
it might also be worth a try to bypass those proxies.

I think you might have missed the point of my suggestion. Sites that are
full of garbage on the HTML would *deliberately* be proxied in a way
that would not preserve TLS encryption, in order to remove the garbage.

What do you mean by "sites that are full of garbage on the HTML?"

Sorry, I have a habit of not proofreading thoroughly. Blame social
media. I probably meant to write "in", but it still isn't phrased well
with that correction.

Post by Scott Dorsey
And cloudflare is ALREADY acting as a proxy, why do you want another layer
of stuff to go wrong?

The EFF uses the term Adversarial Interoperability:
https://www.eff.org/deeplinks/2019/10/adversarial-interoperability

Stuff is ALREADY going wrong, and Cloudflare is partially the cause.
This happens for deep political reasons, and we can't prevent that
without some kind of revolution. You will never convince Cloudflare to
stop blocking Tor users because its actual paying customers want them to
be blocked. Maybe you can pay Cloudflare more money than all its other
paying customers - I doubt it.

What one *can* do (in principle, with enough work) is evade the
filtering by force. Adblockers are an instance of this. The server
injects ads into the page you request, and the client extension deletes
the ads before the page is rendered. You're asking why we need to add an
extra layer - rather than not sending the ads to begin with, I presume.
I hope it's obvious why that solution is impossible, given the current
state of society. Another instance is 12ft.io, which aims to remove
"free article limits" from newspaper sites.

Any such system requires continual upkeep to stay ahead of protection
employed by websites. Adblockers evolved to spoof ad playback in
response to the recent ad-blocker-blocking on YouTube, and 12ft.io seems
to be broken now.

Post by Scott Dorsey

Post by immibis
And there's no point trying to bypass a site's load balancer. That's
within the purview of the site operator, and he's entitled to use one.

The whole point of using cloudflare is that you don't need load balancing
anymore, the cloudflare proxies cache your data for you.

Yes, I agree with you. The article I replied to suggested it.