What is Robots Meta Tag
Crawlers do not freely index your site, they need instructions. That’s the use of robots meta tag in Magento 2. While robots.txt file defines rules to index the whole site, the robots meta tag tells crawlers to index or not specific pages of your store. The robots meta tag is placed at the head section of a page. And in a default Magento 2 page, it looks like this: <meta name=”robots” content=”INDEX,FOLLOW” />
Why Robots Meta Tag is important for SEO in Magento 2
Not all of your content needs to be indexed and shown in Google. To get in the high ranking, you need a wise instruction for the bots about what NOT to index. If all of your content is crawled, the performance of SEO will not be good for sure. A few examples of what you should restrict from crawlers? 404 not found pages, duplicate content, out date product pages and more.
How Robots Meta Tag feature of BSS Magento 2 SEO Extension can help you
By default, admins have to change the “INDEX, FOLLOW” by fixing the source code of the pages manually. But with our SEO module settings, you can complete your need of changing the robots meta tag for any pages within a few clicks at the backend.
There are 3 combinations you can set up for with our robots meta tag:
- NOINDEX, NOFOLLOW: Disallows indexing and following of current page and links in that page.
- NOINDEX, FOLLOW: Disallows indexing but allows crawlers to follow the links.
- INDEX, NOFOLLOW: Allows crawlers to index the current pages but blocks it from following the links.
And here is how easily you can configure:
- Choose the robots meta tag for specific pages, categories or folder
Just fill in the URL pattern of the page you want to set up. Choose the tag combination you want. Click “Save” then flush cache to complete. Admins can have as many pages changed as you want without any seek for developers.
- Set NOINDEX, FOLLOW for a list of regular pages
We select a collection of pages that we think should be set to noindex, follow for a better SEO performance such as checkout, CMS page, customer account pages, etc. You can choose the page based on your preference.