If you're not sure, put everything behind Cloudflare and don't expose your origin at all. Proxy the API requests through workers or at least shield them behind page rules.
Implement some basic rate limiting by IP so you don't get your Google Maps API DoSed. Block China and Russia altogether unless you expect customers from there (sadly, many bots & drive-by scans originate there). Sanitize your inputs, especially if you have any that will reach one of your own endpoints like for a database lookup (and look into SQL injection prevention in general). Use prepared statements in PHP if you use that for DB access. Not sure about Python.
You can read OWASP guidelines for other best practices (https://owasp.org/www-project-top-ten/) or ask ChatGPT to summarize. But realistically, Cloudflare takes care of so much that it seems a bit foolhardy to try to DIY it these days...
If it were me doing this, I wouldn't self-host anything at all, and just use managed services all the way down, including the DBs. A lot less maintenance that way, especially for solo devs. Lets you focus on the business logic instead of trying to reinvent your own secure little nano cloud. It takes serious manpower to stay on top of the latest vulnerabilities and zero-days, and IMO it's not worth spending your limited time on that when the big clouds can do it much more cheaply and much more thoroughly... it's a full-time job in and of itself, and you still probably wouldn't keep up with all the latest attacks =/
Of course you end up learning less this way because other professionals do all the hard work for you. But unless you want to become a backend/security professional yourself and REALLY dive deep into this stuff, I don't think just having basic security skills is going to do you much good anyway, since it takes all of 30 seconds to spin up a pre-hardened cloud host these days, usually for free, and they will have much more exhaustive coverage. Just my 2c.
If you're not sure, put everything behind Cloudflare and don't expose your origin at all.
While I very much understand where this sentiment comes from. Please do not blindly recommend CF.
Cloudflare seems invisible for gullible users, but is unusable and hostile to humans.
I use a VPN to a static IP by Hetzner, not to hide my true identity. But because I have to, my current living situation has my (only available) internet running through a corporate network, packet filtering/logging and all. (Yes this is all legal and I am grateful).
But still to remain any kind of privacy I have to use a VPN. My public IP is registered directly to my full name and has not changed in 3 years.
I also try and limit the amount of unnecessary data my browser transmits.
The combination of those has CF absolutely convinced that I am a existential threat to any site they so honorably "protect".
I simply cannot use ANY site with the default CF configuration. And no, I'm not the only one. This is a very common problem among humans that don't want to share everything about them to pass a human verification.
Cloudflare is the cancer of the Internet. They protect and enable criminals, only to sell the solution later. All the while, ridiculing humans into giving up more and more data in the name of safety. They trick users with promises of "Securing the connection" when they are just matching the browser to their database to sell another page visit. The internet used to be a free and open connection to the world, cloudflare has build a panopticon of surveillance and false security and they are being praised for it.
I've seen this criticism a lot here in HN, and it's something that's always concerned me.
There's a CloudFlare "essentially off" option that I've always hoped would make a difference when it comes to that. I always set it to that when setting websites up with CloudFlare, in hopes that it makes a difference.
That way I can still make use of the CDN and all the other features of CloudFlare without actually bugging visitors.
Would you be willing to load one of my websites[0] and let me know if "essentially off" actually works for you? If it does, great, but if it doesn't, I'll at least be aware that CF is a problem no matter what setting you put it at.
Unfortunately, it's not so much a "blind" suggestion, but a cost-benefit thing. For many sites/businesses, Cloudflare is a conscious decision because it's worth the tradeoff to the site owner, even if it incurs a few false positives (i.e. blocks a few legitimate, privacy-conscious users).
Yes, it sucks that a few (very few, in my experience) real users might get affected, but that's outweighed by the thousands if not millions of other useless bot visits that would otherwise get through. None of the small orgs I've worked for had the time or personnel to manually filter through those otherwise... it's just too much.
That said, whenever I could, I would happily tweak the rules or make an IP whitelist exception for real users who emailed us complaining they couldn't access something because of Cloudflare, but that only ever happened one or twice as far as I can remember.
--------------
The combination of those has CF absolutely convinced that I am a existential threat to any site they so honorably "protect".
I'm sure you know this, but CF isn't a targeted attack towards you. Your usage patterns are just different from most people's, and unfortunately gets treated as a bot because it looks like one. You can email the site operators to ask for an exception, or... frankly... probably they'd just rather lose you as a customer than deal with making the website work for you :(
If the alternative is to either spend 10x more time on securing the website manually, or loosen security such that it impacts all their other customers... it's usually a no-brainer to choose to just live with the false positives instead and deal with them on a case-by-case basis as they come in.
Cloudflare is the cancer of the Internet. They protect and enable criminals, only to sell the solution later. All the while, ridiculing humans into giving up more and more data in the name of safety.
I think our experiences have been different in this regard. IMO they are one of the most useful service providers on the Web, not just for WAF stuff but also their excellent CDN and serverless products, etc. You don't have to agree, but they didn't become this big by offering a bad product... probably most site operators would value overall server stability more than an atypical user's needs.
Considering that you suggest managed services, what’s a good version of the cloudflare tunnels and access, with the same features except that it does not terminate the TLS?
Those typically require custom client side code, for a website you have the requirement that a web browser must be able to connect to it using TLS. Or maybe I'm not getting what your suggestion is - Access is supposed to intercept the connection and display a custom authentication page, with requests not reaching your server at all until they are actually authenticated.
The reverse proxies sometimes support TLS pass through (see Traefik). If the reverse proxy puts an authentication page in front, sure, the TLS pass through may not work. But it could work if all you need from Cloudflare is its firewalls, restricting the IP range, hiding your IP, rate limiting, DDoS mitigation, not having to open port in internal servers, etc.
CloudFlare has some TCP proxying features, but most of what you actually get from adopting CF (or any CDN) requires decrypting traffic because most of the features depend on understanding the HTTP requests.
Comments
If you're not sure, put everything behind Cloudflare and don't expose your origin at all. Proxy the API requests through workers or at least shield them behind page rules.
Implement some basic rate limiting by IP so you don't get your Google Maps API DoSed. Block China and Russia altogether unless you expect customers from there (sadly, many bots & drive-by scans originate there). Sanitize your inputs, especially if you have any that will reach one of your own endpoints like for a database lookup (and look into SQL injection prevention in general). Use prepared statements in PHP if you use that for DB access. Not sure about Python.
You can read OWASP guidelines for other best practices (https://owasp.org/www-project-top-ten/) or ask ChatGPT to summarize. But realistically, Cloudflare takes care of so much that it seems a bit foolhardy to try to DIY it these days...
If it were me doing this, I wouldn't self-host anything at all, and just use managed services all the way down, including the DBs. A lot less maintenance that way, especially for solo devs. Lets you focus on the business logic instead of trying to reinvent your own secure little nano cloud. It takes serious manpower to stay on top of the latest vulnerabilities and zero-days, and IMO it's not worth spending your limited time on that when the big clouds can do it much more cheaply and much more thoroughly... it's a full-time job in and of itself, and you still probably wouldn't keep up with all the latest attacks =/
Of course you end up learning less this way because other professionals do all the hard work for you. But unless you want to become a backend/security professional yourself and REALLY dive deep into this stuff, I don't think just having basic security skills is going to do you much good anyway, since it takes all of 30 seconds to spin up a pre-hardened cloud host these days, usually for free, and they will have much more exhaustive coverage. Just my 2c.
While I very much understand where this sentiment comes from. Please do not blindly recommend CF.
Cloudflare seems invisible for gullible users, but is unusable and hostile to humans.
I use a VPN to a static IP by Hetzner, not to hide my true identity. But because I have to, my current living situation has my (only available) internet running through a corporate network, packet filtering/logging and all. (Yes this is all legal and I am grateful).
But still to remain any kind of privacy I have to use a VPN. My public IP is registered directly to my full name and has not changed in 3 years.
I also try and limit the amount of unnecessary data my browser transmits.
The combination of those has CF absolutely convinced that I am a existential threat to any site they so honorably "protect".
I simply cannot use ANY site with the default CF configuration. And no, I'm not the only one. This is a very common problem among humans that don't want to share everything about them to pass a human verification.
Cloudflare is the cancer of the Internet. They protect and enable criminals, only to sell the solution later. All the while, ridiculing humans into giving up more and more data in the name of safety. They trick users with promises of "Securing the connection" when they are just matching the browser to their database to sell another page visit. The internet used to be a free and open connection to the world, cloudflare has build a panopticon of surveillance and false security and they are being praised for it.
I've seen this criticism a lot here in HN, and it's something that's always concerned me.
There's a CloudFlare "essentially off" option that I've always hoped would make a difference when it comes to that. I always set it to that when setting websites up with CloudFlare, in hopes that it makes a difference.
That way I can still make use of the CDN and all the other features of CloudFlare without actually bugging visitors.
Would you be willing to load one of my websites[0] and let me know if "essentially off" actually works for you? If it does, great, but if it doesn't, I'll at least be aware that CF is a problem no matter what setting you put it at.
[0]: https://pocketarc.com
Unfortunately, it's not so much a "blind" suggestion, but a cost-benefit thing. For many sites/businesses, Cloudflare is a conscious decision because it's worth the tradeoff to the site owner, even if it incurs a few false positives (i.e. blocks a few legitimate, privacy-conscious users).
Yes, it sucks that a few (very few, in my experience) real users might get affected, but that's outweighed by the thousands if not millions of other useless bot visits that would otherwise get through. None of the small orgs I've worked for had the time or personnel to manually filter through those otherwise... it's just too much.
That said, whenever I could, I would happily tweak the rules or make an IP whitelist exception for real users who emailed us complaining they couldn't access something because of Cloudflare, but that only ever happened one or twice as far as I can remember.
--------------
I'm sure you know this, but CF isn't a targeted attack towards you. Your usage patterns are just different from most people's, and unfortunately gets treated as a bot because it looks like one. You can email the site operators to ask for an exception, or... frankly... probably they'd just rather lose you as a customer than deal with making the website work for you :(
If the alternative is to either spend 10x more time on securing the website manually, or loosen security such that it impacts all their other customers... it's usually a no-brainer to choose to just live with the false positives instead and deal with them on a case-by-case basis as they come in.
I think our experiences have been different in this regard. IMO they are one of the most useful service providers on the Web, not just for WAF stuff but also their excellent CDN and serverless products, etc. You don't have to agree, but they didn't become this big by offering a bad product... probably most site operators would value overall server stability more than an atypical user's needs.
Considering that you suggest managed services, what’s a good version of the cloudflare tunnels and access, with the same features except that it does not terminate the TLS?
That doesn't exist, for it to work it has to terminate TLS. You can't do something like Access without decrypting the connection.
Presumably there are other ways to tunnel encrypted traffic (SSH, VPN protocols, etc.?) that don't necessarily rely on TLS?
Those typically require custom client side code, for a website you have the requirement that a web browser must be able to connect to it using TLS. Or maybe I'm not getting what your suggestion is - Access is supposed to intercept the connection and display a custom authentication page, with requests not reaching your server at all until they are actually authenticated.
The reverse proxies sometimes support TLS pass through (see Traefik). If the reverse proxy puts an authentication page in front, sure, the TLS pass through may not work. But it could work if all you need from Cloudflare is its firewalls, restricting the IP range, hiding your IP, rate limiting, DDoS mitigation, not having to open port in internal servers, etc.
CloudFlare has some TCP proxying features, but most of what you actually get from adopting CF (or any CDN) requires decrypting traffic because most of the features depend on understanding the HTTP requests.
Sorry, I don't know. That's not a use case I'm personally familiar with. Maybe others have ideas?