# Robots.txt file fives a directive to web crawlers on what not to search # The format is User-agent: Agent Names ( * means all agents not otherwise explicitly mentioned ) # Disallow: directory or filename # Note: Disallow is the only directive currently supported # If the directive ends with / this will indicate directories only # however if it does not end with / then it will imply directories and files that # begin with that name. E.G. a directive of /help will ignore /help/helpme.htm and /helpme.htm # whereas /help/ will only ignore /help/helpme.htm User-agent: * Disallow: /_vti_cnf/ Disallow: /_vti_log/ Disallow: /_vti_pvt/ Disallow: /cgi-bin/ Disallow: /bin/ Disallow: /XMLSearch.aspx # Ignore any searches as this will perform a search Disallow: /Results.aspx # Ignore any searches as this will perform a search Disallow: /Router.aspx # Ignore any searches as this will perform a search Disallow: /Configs/ Disallow: /include/