Robots.txt - Jim Nielsen’s Blog
I realized why I hadn’t yet added any rules to my
robots.txt: I have zero faith in it.
Link tags: robots.txt
3
Documentation for GPTBot - OpenAI API
Now that the horse has bolted—and ransacked the web—you can shut the barn door:
To disallow GPTBot to access your site you can add the GPTBot to your site’s robots.txt:
User-agent: GPTBot Disallow: /
http://google.com/robots.txt
User-agent: zombies Disallow: /brains