From 2166f73bd8a1370f5b872569fe0afafe6fb2b778 Mon Sep 17 00:00:00 2001 From: z0x Date: Tue, 4 Feb 2025 23:43:41 -0500 Subject: [PATCH] fix(package): fix robots.txt --- src/pages/robots.txt.ts | 5 ----- 1 file changed, 5 deletions(-) diff --git a/src/pages/robots.txt.ts b/src/pages/robots.txt.ts index 7d33aa5..8fa2c2f 100644 --- a/src/pages/robots.txt.ts +++ b/src/pages/robots.txt.ts @@ -47,11 +47,6 @@ User-agent: VelenPublicWebCrawler User-agent: Webzio-Extended User-agent: YouBot Disallow: / -DisallowAITraining: / - -# Block any non-specified AI crawlers -User-Agent: * -DisallowAITraining: /within Sitemap: ${sitemapURL.href} `