Google · Training

Google-Extended

This robots.txt token controls content use. It has no separate HTTP user agent or crawler visits.

Download AiolaAI crawler directory

Key facts

Robots.txt token: Google-Extended
Training: Scraped for model training or dataset development.
This robots.txt token controls content use. It has no separate HTTP user agent or crawler visits.

What Google-Extended is

Google-Extended is a robots.txt product control for Gemini training and grounding, with no separate HTTP User-Agent. It does not control Google Search inclusion or ranking.

In Aiola's taxonomy, Training means scraped for model training or dataset development.

Google-Extended robots.txt token

This robots.txt token controls content use. It has no separate HTTP user agent or crawler visits.

Robots.txt token
Google-Extended

Official IP ranges

This robots.txt token controls content use. It has no separate HTTP user agent or crawler visits.

Request verification for Google-Extended

This robots.txt token controls content use. It has no separate HTTP user agent or crawler visits.

Control Google-Extended with robots.txt

Google documents this token for robots.txt rules. This robots.txt token controls content use.

Allow Google-Extended
User-agent: Google-Extended
Allow: /
Block Google-Extended
User-agent: Google-Extended
Disallow: /

Read official crawler documentation

Aiola Analytics

Track AI crawler visits on your site

This robots.txt token controls content use. It has no separate HTTP user agent or crawler visits.

Download AiolaSet up bot tracking
Aiola Analytics showing AI crawler visits by bot and page

Common questions

Build, ship and fix in one app

Errors, user feedback, GitHub and analytics live next to your coding agents in one desktop app. Every issue becomes a task an agent can take.

Download Aiola