Cloudflare today announced a new feature, which is the ability to prevent AI bots from browsing your web, as well as prevent these bots from "stealing" web content for the purpose of training their respective language models.
For information, many web developers are ready to use robots.txt for this purpose. However, there are some developers who do not respect it, and violate in stealing content. Through this latest Cloudflare offering, it is expected to help overcome that.
For Cloudflare users, you just need to go to the Security > Bots page, and then activate the "AI Scrapers & Crawlers" feature. This feature is also offered to users of Cloudflare's free plan.
Tags
CYBER LIFE