Merge pull request #198 from sablier-labs/chore/robots

chore: add robots.txt
sablier-labs · Nov 26, 2024 · 7f4fef7 · 7f4fef7
2 parents c1847d4 + 70ea3e8
commit 7f4fef7
Showing 1 changed file with 3 additions and 0 deletions.
diff --git a/static/robots.txt b/static/robots.txt
@@ -0,0 +1,3 @@
+# Allow everything to be crawled except Cloudflare's /cdn-cgi/
+User-agent: *
+Disallow: /cdn-cgi/