User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /readme.html Disallow: /license.txt #Add this if you want to stop Alexa from indexing your site. User-agent: ia_archiver Disallow: / #Add this to stop Duggmirror User-agent: duggmirror Disallow: / #Add this to stop MJ12bot User-agent: MJ12bot Disallow: / #Add this to stop Netseer User-agent: netseer Disallow: / #Add this to stop Blekko User-agent: ScoutJet Disallow: / #Add this to stop SurveyBot User-agent: SurveyBot Disallow: / #Add this to stop ShopWiki User-agent: ShopWiki Disallow: /