z0x/blog.z0x.ca
Code Actions

fix(package): fix robots.txt

Browse source
This commit is contained in:
z0x 2025-02-04 23:43:41 -05:00
parent a0c1eed0f1
commit 2166f73bd8
1 changed files with 0 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
src/pages/robots.txt.ts
View file

@ -47,11 +47,6 @@ User-agent: VelenPublicWebCrawler
User-agent: Webzio-Extended
User-agent: YouBot
Disallow: /
DisallowAITraining: /
# Block any non-specified AI crawlers
User-Agent: *
DisallowAITraining: /within
Sitemap: ${sitemapURL.href}
`