# www.robotstxt.org/ # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 User-agent: * Disallow: /admin Allow: / Disallow: /interns/search*?* Disallow: /interns/search/*?* User-agent: AhrefsBot User-agent: MJ12bot User-agent: Yandex User-agent: spbot User-agent: ezooms User-agent: sistrix User-agent: Mail.RU Disallow: /