User-agent: ia_archiver Disallow: / User-agent: wget Disallow: / User-Agent: * Disallow: /wp-admin/ # A capture bot, downloads gazillions of pages with no public benefit # http://www.webreaper.net/ User-agent: WebReaper Disallow: / # BEGIN XML-SITEMAP-PLUGIN Sitemap: http://weblog.oxidiser.nl/sitemap.xml.gz # END XML-SITEMAP-PLUGIN