User-agent: * Disallow: /kickersoppad/ Disallow: /_notes/ Disallow: /bb/ Disallow: /div/ Disallow: /img/ Disallow: /img2/ Disallow: /protest/ Disallow: /loco/ #idiot-bot; generates tons of 404 and malformed urls User-agent: wget Disallow: / User-agent: ExtractorPro Disallow: /