Disallow AI-bots to crawl site

bonny · Nov 18, 2024 · a8812bc · a8812bc
1 parent d6aa59f
commit a8812bc
Showing 1 changed file with 9 additions and 0 deletions.
diff --git a/public/robots.txt b/public/robots.txt
@@ -1,2 +1,11 @@
 User-agent: *
 Disallow: /pixel?
+
+User-agent: ClaudeBot
+Disallow: /
+
+User-agent: GPTBot
+Disallow: /
+
+User-agent: Google-Extended
+Disallow: /