Google Updates Gemini and Vertex AI Docs

Google has recently updated its guidance regarding the Google-Extended user agent, offering clarification on its role in the broader Google ecosystem. According to the new documentation, Google-Extended is not considered a ranking signal, meaning it does not influence the position of a website in search results. This clarification is important for publishers concerned about how their content might be utilised by Google’s AI technologies.

The user agent in question is primarily used by publishers to manage how their data is handled by Google Gemini and Vertex AI systems. Specifically, it gives publishers control over whether their data can be used for AI training or for grounding AI-generated answers. By using Google-Extended, publishers can decide whether their content should contribute to improving the AI’s understanding of the web or if they prefer to keep it isolated from such processes.

This update aims to give publishers greater transparency and control over their data, allowing them to make more informed decisions about its usage. Importantly, Google reassured that this process does not affect search inclusion. Content from publishers using the Google-Extended user agent will still be indexed and included in search results, regardless of whether they opt to allow their data to be used for training or grounding purposes.

Updated Guidance

Google has recently updated its guidance regarding the Google-Extended user agent, responding to feedback from publishers to enhance clarity and offer more specific details. The goal of this update is to help publishers better understand the role of Google-Extended and how it interacts with Google’s AI technologies, including the Gemini Apps and Vertex AI generative APIs.

In the previous version of the documentation, Google-Extended was described as a standalone product token that publishers could use to control whether their websites would assist in improving the Gemini and Vertex AI models. Specifically, it allowed publishers to decide if their sites’ data would contribute to training these generative AI systems. Additionally, the guidance explained that if a publisher opted to disallow Google-Extended, their pages would not be used in the grounding process within Vertex AI. This meant that pages with Google-Extended blocked would not be included in the data sets used to improve Google’s search and AI capabilities.

The newly updated documentation goes further in providing a more thorough explanation of what the Google-Extended user agent is designed to do and the impact of blocking it. The revised version clarifies the operational procedures of the user agent, making it easier for publishers to understand exactly how their data may be used or excluded. It also offers more detailed information on the consequences of blocking Google-Extended, giving publishers better insight into how this action would affect their participation in Google’s broader AI training and grounding efforts.

This update is part of Google’s ongoing effort to be more transparent with publishers about how their data is used, especially when it comes to generative AI. By providing a clearer and more accessible breakdown of these processes, Google aims to empower publishers to make informed decisions about their data and how they engage with AI technologies. This change reflects a growing trend in the industry to ensure that content creators are aware of how their information may be leveraged for AI development, and it ensures that they have more control over how their data is utilised.

Google-Extended Is Not A Ranking Signal

Google has updated its guidance regarding Google-Extended to clarify that it is not used as a ranking signal for Google Search. This update is important as it directly addresses concerns about how data might impact a website’s visibility or position in search results. Allowing Google-Extended to use a website’s data for grounding Gemini AI answers will not have any bearing on the site’s ranking within Google Search.

Grounding refers to the process of using web data and knowledge base data to enhance the answers provided by a large language model. This helps to ensure that the responses are accurate, up-to-date, and factual, minimising the risk of “hallucinations” or fabricated information that AI models may otherwise generate.

The previous version of the documentation did not specifically mention the absence of a ranking signal. It simply stated: “Google-Extended does not impact a site’s inclusion or ranking in Google Search.” In contrast, the updated version explicitly clarifies that Google-Extended does not affect a website’s ranking, stating: “Google-Extended does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search.”

This new wording aligns with other existing guidance provided by Google, reinforcing the distinction between Google-Extended and Google Search. For example, Google Search Central previously clarified that Google-Extended does not offer a way to control how website information is displayed in Google Search. Instead, publishers should use other methods, such as robots.txt or other robot controls, to manage how their content appears in search results.

Takeaways

Google-Extended Documentation Update
Google has updated its Google-Extended documentation to provide a clearer explanation of its purpose and impact. The update aims to make it easier for publishers to understand how Google-Extended works and what it controls.

Separation From Ranking Signals
One of the key points clarified in the updated guidance is that Google-Extended has no effect on a website’s inclusion in Google Search. It also explicitly states that Google-Extended is not used as a ranking signal. This ensures that publishers can block their content from being used in AI training or grounding without worrying about any impact on their Search rankings.

Internal Use By AI Models
The update also reinforces that Google-Extended controls whether a site’s content can be used for training and grounding Gemini models. This helps prevent misunderstandings about how site data is utilised for AI purposes while ensuring the integrity of Search results remains unaffected.

Consistency Across Documentation
The new language in the Google-Extended documentation now aligns with other longstanding guidance across Google’s documentation. This consistency reinforces that Google-Extended is entirely separate from tools used to manage search visibility and ranking, ensuring clarity for webmasters and publishers.

Google’s update underscores that while publishers can choose to block their content from being used for AI training and grounding, it will not affect their site’s indexing or ranking in Google Search. This ensures that the control over content usage is independent of search performance.