← AI crawler directory

xAI

Grok-DeepSearch

AI answersNo fetched rangesVerified July 2026

What it is

Grok-DeepSearch fetches sources while a Grok user runs DeepSearch. Aiola classifies it as AI answers because access happens on demand.

In Aiola's taxonomy, AI answers means fetched to answer a user's question now.

User agent

Look for the match token inside the complete HTTP User-Agent header. Tokens can be spoofed, so use the network checks below for authentication.

Full example User-Agent
Grok-DeepSearch
Case-insensitive match token
Grok-DeepSearch

Official IP ranges

xAI has not published official IP ranges that Aiola successfully fetched for Grok-DeepSearch. Aiola does not list cloud-provider ranges or community guesses as if they authenticated this crawler.

Verify authenticity

Start with the source IP recorded by your trusted edge or server, not an untrusted forwarded header.

Reverse- and forward-DNS double lookup
host REQUEST_IP
# Confirm the hostname ends in an official suffix
host RETURNED_HOSTNAME
# The forward lookup must return REQUEST_IP

For production checks, test against every current prefix in the vendor feed. Re-fetch feeds regularly: a July 2026 snapshot is evidence of publication, not a permanent firewall list.

Control it with robots.txt

Grok-DeepSearch is user-triggered. xAI's controls may differ from automated crawling, so do not assume a robots.txt block is an authentication boundary.

Allow Grok-DeepSearch
User-agent: Grok-DeepSearch
Allow: /
Block Grok-DeepSearch
User-agent: Grok-DeepSearch
Disallow: /

No official crawler documentation URL is listed because Aiola could not verify one confidently.

Track it

Aiola tracks this crawler on your site — crawls, pages, trends. See when Grok-DeepSearch arrives, which URLs it requests, and how activity changes over time.

Explore Aiola Analytics

FAQ

What is Grok-DeepSearch?

Grok-DeepSearch fetches sources while a Grok user runs DeepSearch. Aiola classifies it as AI answers because access happens on demand.

What user-agent token identifies Grok-DeepSearch?

Match the case-insensitive token “Grok-DeepSearch” in the User-Agent header. A matching header alone does not authenticate the sender.

Can I verify Grok-DeepSearch by IP address?

No official Grok-DeepSearch CIDR snapshot was successfully fetched for this verification date. Do not treat an arbitrary cloud IP as proof of identity.

Can robots.txt block Grok-DeepSearch?

Grok-DeepSearch is user-triggered. xAI's controls may differ from automated crawling, so do not assume a robots.txt block is an authentication boundary. Use the specific “Grok-DeepSearch” group when you want a rule for this identity without changing rules for every crawler.

Related