User-agent: * Allow: / #Baiduspider User-agent: Baiduspider Disallow: / user-agent: AhrefsBot disallow: / User-agent: MJ12bot Disallow: / User-agent: CCBot Disallow: / User-agent: msnbot Disallow: / User-agent: bingbot Disallow: / User-agent: YandexBot Disallow: / User-agent: Fatbot Disallow: / user-agent: SEOkicks-Robot Disallow: / User-agent: BLEXBot Disallow: / User-agent: Slurp Disallow: / User-agent: ccbot Disallow: / User-agent: turnitinbot Disallow: / User-agent: discobot Disallow: / User-agent: linkdex Disallow: / User-agent: linkdex.com Disallow: / User-agent: linkdex.com/v2.0 Disallow: / User-agent: PiplBot Disallow: / User-agent: oBot Disallow: / User-agent: ICCrawler - iCjobs Disallow: / User-agent: Yandex Disallow: / User-agent: OpenindexSpider Disallow: / User-agent: Ezooms Disallow: / User-agent: Ezooms/1.0 Disallow: / User-agent: Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com) Disallow: / User-agent: Yeti Disallow: / User-agent: Yeti/1.0 Disallow: / User-agent: Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/) Disallow: / User-agent: findlinks Disallow: / User-agent: findlinks/2.0.2 Disallow: / User-agent: findlinks/2.0.2 (+http://wortschatz.uni-leipzig.de/findlinks/) Disallow: / User-agent: brandwatch Disallow: / User-agent: magpie Disallow: / User-agent: magpie-crawler Disallow: / User-agent: magpie-crawler/1.1 Disallow: / User-agent: magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net) Disallow: / User-agent: Unister Disallow: / User-agent: unister Disallow: / User-agent: unisterbot Disallow: / User-agent: UnisterBot Disallow: / User-agent: UnisterBot (Mozilla/5.0 compatible; crawler@unister-gmbh.de) Disallow: / User-agent: pixray Disallow: / User-agent: Pixray Disallow: / User-agent: pixray-seeker Disallow: / User-agent: Pixray-Seeker Disallow: / User-agent: Pixray-Seeker/1.1 Disallow: / User-agent: Pixray-Seeker/1.1 (Pixray-Seeker; crawler@pixray.com) Disallow: / User-agent: Screaming Frog SEO Spider/1,80 Disallow: / User-agent: CatchBot Disallow: / User-agent: Dropcatcher.de Disallow: / User-agent: CligooRobot/1.0 Disallow: / User-agent: cityreview Disallow: / User-agent: sistrix Disallow: / User-agent: spbot Disallow: / User-agent: dotbot Disallow: / User-agent: SurveyBot Disallow: / User-agent: Eurobot Disallow: / User-agent: MJ12bot Disallow: / User-agent: WebmasterCoffee Disallow: / User-agent: AboutUsBot Disallow: / User-agent: BizInformation Disallow: / User-agent: 008 Disallow: / User-agent: WebImages 0.3 Disallow: / User-agent: panscient.com Disallow: / User-agent: CazoodleBot Disallow: / User-agent: Python-urllib/2.1 Disallow: / User-agent: ia_archiver Disallow: / User-agent: VadixBot Disallow: / User-agent: Jim Disallow: / User-agent: Gigamega.bot/1.0 Disallow: / User-agent: EnaBot/1.2 Disallow: / User-agent: AhrefsBot Disallow: / User-agent: BacklinkCrawler Disallow: / User-agent: Sogou Disallow: / User-agent: XoviBot Disallow: /