Control how search engine bots crawl and index your website.
Point search engines to your XML sitemap
Configure crawling rules for specific bots
