For media organizations and publishers looking to block ChatGPT and other products from crawling and accessing their content, there is a solution that is very easy to put in place.
Some major companies have announced this intent recently and just last week, the UK's Independent Publishers Alliance warned its members to block ChatGPT and bot crawling on their websites as soon as possible. If you feel similarly about protecting your intellectual property from AI tool training on your content, here's your solution.
Simply add this exact text to your robots.txt settings:
<code>User-agent: CCBot</code>
<code>Disallow: /</code>
<code>User-agent: GPTBot</code>
<code>Disallow: /</code>
<code>User-agent: ChatGPT-User</code>
<code>Disallow: /</code>
<code>User-agent: anthropic-ai</code>
<code>Disallow: /</code>
<code>User-agent: cohere-ai</code>
<code>Disallow: /</code>
<code>User-agent: Google-Extended</code>
<code>Disallow: /</code>
Drop us a line if you need help taking the next step here.
