Question
What is Google-Extended?
Google-Extended is a robots.txt token, introduced by Google, that lets you control whether your content is used to train and ground Gemini models — without affecting Google Search crawling or ranking, which Googlebot still handles.
Why it exists
Site owners wanted a way to stay in Google Search while opting out of generative AI training. Google-Extended separates those two consents into different robots.txt tokens so you do not have to trade ranking for AI control.
Key facts
- It is a policy token, not a distinct crawler with its own IPs or user-agent string.
- Blocking it has no effect on Google Search indexing, ranking, or normal crawling by Googlebot.
- It governs Gemini app and Vertex/Gemini API training and grounding use of your content.
How to set it
User-agent: Google-Extended
Disallow: / # opt out of Gemini training/grounding
# (omit or use Allow: / to opt in)Related pages: should I allow Google-Extended, AI crawler user agents.