# Robots Exclusion File # http://www.robotstxt.org/wc/norobots.html # All robots will spider the domain User-agent: *