Yeah, I implement PoW on my Edge nodes as well, which connect back to my self-hosted infra over Wireguard tunnels. The PoW works very similarly to Anubis, but practically speaking, they don't really work. Whole swarths of botnets, including botnets that utilise residential proxies (thousands of comprised desktops in poorer countries) just solve the PoW(s) and get through anyway at massive scale.
The only effective way I've found to combat "unwanted traffic" (bots, botnets, crawlers, search engines, etc) is to a) make request/responses as cheap as possible (heavy caching, rate limiting, etc) and b) measure cohorts, fingerprints and behaviour and block patterns that can't possibly look like "normal human traffic" (tricky to get right, but this is to a large extent what Cloudflare is doing I believe, and I've had great success with this technique and building out an OSS Caddy module for this...)
Yeah I feel static site generators and CDNs are the way. I've been using those for years and it surprises me how people still setup LAMP servers to this day
I had recently done some work where I had rehash a bunch of insecure hashes for security reasons. It required brute forcing about 1 trillion SHA-256 hashes to achieve this. Was the first time I had actually ever used Hashcat, very cool piece of software.
I later had a shower thought about Anubis and it's proof-of-work solution to tackle web scrapers. I thought... What if I cracked the hashes out of the browser? Here is my write up of what I discovered.
Yeah, I implement PoW on my Edge nodes as well, which connect back to my self-hosted infra over Wireguard tunnels. The PoW works very similarly to Anubis, but practically speaking, they don't really work. Whole swarths of botnets, including botnets that utilise residential proxies (thousands of comprised desktops in poorer countries) just solve the PoW(s) and get through anyway at massive scale.
The only effective way I've found to combat "unwanted traffic" (bots, botnets, crawlers, search engines, etc) is to a) make request/responses as cheap as possible (heavy caching, rate limiting, etc) and b) measure cohorts, fingerprints and behaviour and block patterns that can't possibly look like "normal human traffic" (tricky to get right, but this is to a large extent what Cloudflare is doing I believe, and I've had great success with this technique and building out an OSS Caddy module for this...)
Yeah I feel static site generators and CDNs are the way. I've been using those for years and it surprises me how people still setup LAMP servers to this day
I had recently done some work where I had rehash a bunch of insecure hashes for security reasons. It required brute forcing about 1 trillion SHA-256 hashes to achieve this. Was the first time I had actually ever used Hashcat, very cool piece of software. I later had a shower thought about Anubis and it's proof-of-work solution to tackle web scrapers. I thought... What if I cracked the hashes out of the browser? Here is my write up of what I discovered.