xAI
xAI-SearchBot
What it is
xAI-SearchBot discovers pages for xAI search retrieval used by Grok. Aiola classifies it as indexing because it supports a searchable retrieval index.
In Aiola's taxonomy, Indexing means crawled for ai or search indexing.
User agent
Look for the match token inside the complete HTTP User-Agent header. Tokens can be spoofed, so use the network checks below for authentication.
xAI-SearchBotxAI-SearchBotOfficial IP ranges
xAI has not published official IP ranges that Aiola successfully fetched for xAI-SearchBot. Aiola does not list cloud-provider ranges or community guesses as if they authenticated this crawler.
Verify authenticity
Start with the source IP recorded by your trusted edge or server, not an untrusted forwarded header.
host REQUEST_IP
# Confirm the hostname ends in an official suffix
host RETURNED_HOSTNAME
# The forward lookup must return REQUEST_IPFor production checks, test against every current prefix in the vendor feed. Re-fetch feeds regularly: a July 2026 snapshot is evidence of publication, not a permanent firewall list.
Control it with robots.txt
xAI does not provide crawler documentation that Aiola could confidently verify for this token. The rules below express your preference, but vendor compliance is unconfirmed.
User-agent: xAI-SearchBot
Allow: /User-agent: xAI-SearchBot
Disallow: /No official crawler documentation URL is listed because Aiola could not verify one confidently.
Track it
Aiola tracks this crawler on your site — crawls, pages, trends. See when xAI-SearchBot arrives, which URLs it requests, and how activity changes over time.
Explore Aiola AnalyticsFAQ
What is xAI-SearchBot?
xAI-SearchBot discovers pages for xAI search retrieval used by Grok. Aiola classifies it as indexing because it supports a searchable retrieval index.
What user-agent token identifies xAI-SearchBot?
Match the case-insensitive token “xAI-SearchBot” in the User-Agent header. A matching header alone does not authenticate the sender.
Can I verify xAI-SearchBot by IP address?
No official xAI-SearchBot CIDR snapshot was successfully fetched for this verification date. Do not treat an arbitrary cloud IP as proof of identity.
Can robots.txt block xAI-SearchBot?
xAI does not provide crawler documentation that Aiola could confidently verify for this token. The rules below express your preference, but vendor compliance is unconfirmed. Use the specific “xAI-SearchBot” group when you want a rule for this identity without changing rules for every crawler.