# robots.txt, Mon 10/06 2019 # robots.txt for http://www.mactale.com/ # author: keith orbit # ------------------------------------------------------ User-agent: * Disallow: /css/ Disallow: /scss/ Disallow: /js/ Disallow: /nav/ User-agent: Baiduspider Disallow: /* User-agent: HaoSouSpider Disallow: /* User-agent: Sogou web spider Disallow: /* User-agent: Sogou inst spider Disallow: /* User-agent: Sogou spider2 Disallow: /* User-agent: Twitterbot Disallow: * Allow: /images Allow: /article Sitemap: https://www.mactale.com/sitemap.xml