2026-01-30
Google's dual-purpose crawler creates an unfair AI advantage. To protect publishers and foster competition, the UK’s Competition and Markets Authority must mandate crawler separation for search and AI....
Continue reading »
2025-10-30
The world is adopting anonymous credentials for digital privacy, but these systems are vulnerable to quantum computers. This post explores the cryptographic challenges and promising research paths toward building new, quantum-resistant credentials from the ground up....
2025-10-24
Cloudflare is partnering with Visa and Mastercard to help secure the future of agentic commerce....
2025-09-24
We are proposing—as starting points—responsible AI bot principles that emphasize transparency, accountability, and respect for content access and use preferences....
2025-09-03
How do we embrace the power of AI without losing control? That was one of our big themes for AI Week 2025. Check out all of the products, partnerships, and features we announced....
August 28, 2025 2:00 PM
Cloudflare now lets websites and bot creators use Web Bot Auth to segment agents from verified bots, making it easier for customers to allow or disallow the many types of user and partner directed....
August 28, 2025 2:00 PM
Cloudflare launches AI Crawl Control (formerly AI Audit) and introduces easily customizable 402 HTTP responses....
August 04, 2025 1:00 PM
Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites....
July 01, 2025 10:01 AM
It’s Content Independence Day: Cloudflare, along with a majority of the world's leading publishers and AI companies, is changing the default to block AI crawlers unless they pay creators for content....
July 01, 2025 10:00 AM
Pay per crawl is a new feature to allow content creators to charge AI crawlers for access to their content. ...
July 01, 2025 10:00 AM
From May 2024 to May 2025, crawler traffic rose 18%, with GPTBot growing 305% and Googlebot 96%....
July 01, 2025 10:00 AM
Bots can start authenticating to Cloudflare using public key cryptography, preventing them from being spoofed and allowing origins to have confidence in their identity....
May 15, 2025 1:00 PM
Bots now browse like humans. We're proposing bots use cryptographic signatures so that website owners can verify their identity. Explanations and demonstration code can be found within the post....
March 19, 2025 1:00 PM
How Cloudflare uses generative AI to slow down, confuse, and waste the resources of AI Crawlers and other bots that don’t respect “no crawl” directives....
December 23, 2024 2:01 PM
Cloudflare observed a 4x increase in bot-related traffic on Black Friday in 2024. 29% of all traffic on our network on Black Friday was Grinch Bots wreaking holiday havoc....
September 27, 2024 1:00 PM
This year for Cloudflare’s birthday, we’ve extended our AI Assistant capabilities to help you build new WAF rules, added new AI bot & crawler traffic insights to Radar, and given customers new AI bot ...
September 23, 2024 1:00 PM
Cloudflare customers on any plan can now audit and control how AI models access the content on their site. ...
July 03, 2024 1:00 PM
To help preserve a safe Internet for content creators, we’ve just launched a brand new “easy button” to block all AI bots. It’s available for all customers, including those on our free tier...
