robots.txt

This is where you communicate with search engines about indexing your site, pages and resources. You can allow or disallow search engines from indexing specific resources or even single out specific search engines in your robots.txt file.