# last updated 3/25/2016 User-agent: Slurp Crawl-Delay: 20 User-agent: stress-agent Disallow: / User-agent: MSIECrawler Disallow: / User-agent: Microsoft URL Control Disallow: / # Chinese search bot User-agent: Baiduspider Crawl-delay: 2 # Chinese search bot User-agent: baiduspider Crawl-delay: 2 # Russian search bot User-agent: Yandex Crawl-delay: 2 # Russian search bot User-agent: YandexBot Crawl-delay: 2 # Russian search bot User-agent: Yandex* Crawl-delay: 2 # User-agent: EtaoSpider Disallow: / # Spider for www.picsearch.com User-agent: psbot Disallow: / # http://www.linkdex.com/en-us/about/bots/ User-agent: linkdexbot Disallow: / # http://cloudservermarket.com/spider.html User-agent: CloudServerMarketSpider Disallow: / # Czech bot: https://www.seznam.cz/ User-agent: SeznamBot Crawl-delay: 2 SITEMAP: https://www.connection.com/index.xml.gz User-agent: * Disallow: /IPA/Account/ Disallow: /IPA/Admin/ Disallow: /IPA/Campaigns/ Disallow: /IPA/Content/Global/ Disallow: /IPA/Content/HelpAdmin/ Disallow: /IPA/Content/Help/ Disallow: /IPA/Content/HelpPop/ Disallow: /IPA/Content/Error/ Disallow: /IPA/Content/Int/ Disallow: /IPA/Nav/ Disallow: /IPA/PM/BrandsTiles/ Disallow: /IPA/PM/Promos/ Disallow: /IPA/SHOP/Checkout/ Disallow: /IPA/Shop/Offline/ Disallow: /IPA/Shop/Lists/ Disallow: /IPA/Shop/Product/Compare Disallow: /IPA/Shop/Product/ImageBrowser Disallow: /IPA/Shop/Product/WarrantyPop Disallow: /IPA/Shop/Product/Search*