How To: Change your User Agent to GoogleBot or SchemaBot

This support document describes the process of loading a page as a different user agent. The two most common bots you will want to emulate will likely GoogleBot and/or SchemaBot. The article assumes you'll be using Chrome, but most browsers should have a similar functionality. For more information on Robots Only deployment, review our article: Robots Only Deployment.

TABLE OF CONTENTS

• Scenarios for Rendering a Page as a Specific User Agent
  • SchemaBot
  • GoogleBot
• How To Load a WebPage as a Different User Agent
  • Step 1: Open Developer Tools and Select Network Conditions
  • Step 2: Select your preferred User Agent (Typically GoogleBot) and Disable Caching
  • Step 3: Reload the Page
• Tools for Frequent Switchers

Scenarios for Rendering a Page as a Specific User Agent

SchemaBot

Loading a page as SchemaBot can be useful for the following scenarios:

• Confirming deployment when using Robots Only
  A critical part of the Schema App process is confirming that JSON-LD is being loaded on your page. Viewing a page as SchemaBot can be one method of confirmation
• Troubleshooting unexpected Analyzer or Crawler functionality
  Schema App's Analyzer tool and Crawler deployment functionality require SchemaBot to be allowlisted.  Viewing the page as SchemaBot can help you assess whether the user agent is automatically blocked from accessing the page.
• Validating what is being rendered for SchemaBot
  There might be a discrepency in what's rendered for a user versus for SchemaBot. One strategy for troubleshooting highlighter targeting is to load the page as SchemaBot would see it. There may be additional differences in how pages load in SchemaBot's headless chrome rendering environment versus a standard browser, but imitating SchemaBot is a recommended initial troubleshooting step.

GoogleBot

Loading a page as Googlebot can be useful for the following scenarios:

• Confirming deployment when using Robots Only
  A critical part of the Schema App process is confirming that JSON-LD is being loaded on your page. Viewing a page as Googlebot can be one method of confirmation.
• Using a dynamic rendering solution, e.g. Prerender.io
  As Google Search docs describe, dynamic rendering means switching between client-side rendered and pre-rendered content for specific user agents. This means the JavaScript content is loaded on a server side and only the results (rendered) are sent to the client. Viewing the page as Googlebot may be necessary to confirm Google has access to the full contents of the page URL.
• Validating what is being rendered for Googlebot
  There might be a discrepency in what's rendered for a user versus for a bot. One strategy for troubleshooting this is loading the page as Googlebot sees it.

How To Load a WebPage as a Different User Agent

If you want to test locally in your browser it is possible to alter the user agent for these purposes. 

Step 1: Open Developer Tools and Select Network Conditions

In Google Chrome, open the developer tools (CMD|CTRL+SHIFT+I) and navigate to the Hamburger Menu on the right side of the screen. Select "More Tools" > "Network Conditions". 

Step 2: Select your preferred User Agent (Typically GoogleBot) and Disable Caching

Once you click on Network Conditions, panel should appear at the very bottom of your window. Under "Network conditions" you can set the user agent. For most cases choosing Googlebot will show you the results. You can test your own user agents as well by selecting custom and pasting the user agent into the text field.

The SchemaBot user agent is:  Mozilla/5.0 (compatible; SchemaBot/1.2; +https://www.schemaapp.com/bot/)

Note: Be sure to select the "Disable Cachine" tickbox, to ensure that the page loads a fresh version as your desired user agent.

Step 3: Reload the Page

Refresh the page to ensure your user agent changes are reflected.

Note: If you close the Dev Tools panel, your user agent will revert to the browser default.

Tools for Frequent Switchers

If you find yourself switching User Agent often, you may want to install a plugin that gives you quicker access.  A plugin like User-Agent Switcher for Chrome makes it easier to change agent at the click of a button:

However, you do have to do a little extra initial legwork to set it up with the specifics of the agents you want to spoof.  So this tool is only worthwhile if you think you'll be switching agent more than once.

If you do go down this route, please check out the list of google bot User Agents for details you can copy and paste into your User-Agent Switcher plugin configuration.