The User Agent tells the server who and what type of user is requesting the pages. Many sites use bot or spider detection tools so they don’t waste resources to people trying to seal content or do malicious things. In most cases they allow Google to access freely without any limitations.

Action: Choose which User Agent you prefer us to use. We can can request pages as either GoogleBot or with our own named bot names Hreflang Builder.

Default Setting: GoogleBot

By default we are using GoogleBot, by there are cases where the sites specifically block requests that are not the real GoogleBot. In this case we suggest using the Hreflang Builder bot spider ID and make sure you add “HreflangBuilder/0.1 +https://www.hreflangbuilder.com/bot.html” to your whitelist.

By whitelisting our bot, those cases where your DevOps team will block everyone from crawling their sites, except our Hreflang Builder allowing us to do the diagnostic checks.

August 1, 2019

Select User Agent

The User Agent tells the server who and what type of user is requesting the pages. Many sites use bot or spider detection tools so they don’t waste resources to people trying to seal content or do malicious things. In most cases they allow Google to access freely without any […]
July 3, 2019

Disable or Enable Strip index.html in URLs

This option allows you to remove the index.html from any country home page or folder to match other countries that do not use them. For example, if you have an older site in Bulgaria that still has the home page as mysite.bg.index.html that will also work as mysite.bg then we […]
July 2, 2019

Disable or Enable HTTPS Checking

By default this feature is turned ON (unchecked) so that if the system encounters a http version of a URL it will automatically check for an https version. If we find an https version we will remove the http and replace it with the https. When should this be enabled? […]
July 2, 2019

Enable Mapping Tag Names

This is one of our advanced mapping features that lets you give us to match all pages that have a unique product ID even if the URL’s are localized or have a different structure across markets. This ID needs to be contained in the page as a Meta Tag or […]