You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

13 lines
233 B
Plaintext

User-Agent: *
# blocking some nasty "SEO" bots down here:
User-agent: AhrefsBot
Disallow: /
User-agent: PetalBot
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: barkrowler
Disallow: /
Sitemap: {{ "sitemap.xml" | absURL }}