# All robots will spider the domain User-agent: msnbot Disallow: /cgi-bin/links/