# see http://www.robotstxt.org/orig.html for documentation User-agent: MJ12bot Disallow: / User-agent: Yandex Disallow: /?*restoreRequest=* User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: * Disallow: /?*restoreRequest=* Disallow: /EX/ User-agent: bingbot Crawl-delay: 1 User-agent: msnbot Crawl-delay: 1 User-agent: msnbot-media Crawl-delay: 1