crapaud-fou
/
crazy-toads.github.io
2
0
Fork 0
Code Issues Pull Requests Releases Activity
crazy-toads.github.io/robots.txt

59 lines
933 B
Plaintext
Raw Permalink Normal View History

install lanyon-plus theme
2017-08-08 19:53:28 +00:00
User-agent: Googlebot
Disallow: /assets/
Disallow: /CNAME
Disallow: /*.md$
Disallow: /files/
User-agent: Googlebot-Image
Disallow: /assets/
Disallow: /CNAME
Disallow: /*.md$
Disallow: /files/
User-agent: Twitterbot
Disallow: /assets/
Disallow: /CNAME
Disallow: /*.md$
Disallow: /files/
User-agent: Bingbot
Disallow: /assets/
Disallow: /CNAME
Disallow: /*.md$
Disallow: /files/
User-agent: DuckDuckBot
Disallow: /assets/
Disallow: /CNAME
Disallow: /*.md$
Disallow: /files/
User-agent: ia_archiver
Disallow: /assets/
Disallow: /CNAME
Disallow: /*.md$
Disallow: /files/
User-agent: Applebot
Disallow: /assets/
Disallow: /CNAME
Disallow: /*.md$
Disallow: /files/
User-agent: facebookexternalhit
Disallow: /assets/
Disallow: /CNAME
Disallow: /*.md$
Disallow: /files/
User-agent: Facebot
Disallow: /assets/
Disallow: /CNAME
Disallow: /*.md$
Disallow: /files/
User-agent: *
Disallow: /
cleanup
2017-09-05 20:13:56 +00:00
Sitemap: http://crapaud-fou.org/sitemap.xml